From patchwork Fri Apr 11 09:16:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B95F8C36010 for ; Fri, 11 Apr 2025 09:17:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F5E6280196; Fri, 11 Apr 2025 05:17:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A5CD280190; Fri, 11 Apr 2025 05:17:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F7D5280196; Fri, 11 Apr 2025 05:17:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1768D280190 for ; Fri, 11 Apr 2025 05:17:03 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9A30FC2235 for ; Fri, 11 Apr 2025 09:17:03 +0000 (UTC) X-FDA: 83321208726.13.D681901 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id DA33F80008 for ; Fri, 11 Apr 2025 09:17:01 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363022; a=rsa-sha256; cv=none; b=IIL1V2+PSVEEYN3/WBsqxycBtBu1CvmUNHcB6+Y+jr0c4nvCDKlhRk0NIG8b3duCwoGT5e UoRO16TWvwIXLXipU62TyVqOZc2+KhOBLtHX7d+W2aKrH0pAVTDPK8Qpi0XVJtuGsaVk7P ZLViFdgLM14K6X96bODQrer+YL1ACLU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363022; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0TInbeeHsJ1w6rsCRtJk/Jeb1cP06TeabMr1jT8HfEs=; b=mawJaQ4WhTcCYQQbtVvkjPvRNeyvWnpO8GVcTel4T6tCXbpWfoe9xlTC/vmHQGYihWh7Z5 EYrU99h3WbPrKbPzozut17FStsrnZXMTNPjhxDBYle6qGU2khbQukPsY5MiQYuV1d0Pzm/ ck0Wtx9GLi9hWM2BsqWNLghJE23TdoE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 081411596; Fri, 11 Apr 2025 02:17:01 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 763A83F6A8; Fri, 11 Apr 2025 02:16:57 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 01/18] mm: Introduce kpkeys Date: Fri, 11 Apr 2025 10:16:14 +0100 Message-ID: <20250411091631.954228-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: DA33F80008 X-Stat-Signature: 9xk44r4fu1cwikwiwcza9dxohhuzswnb X-HE-Tag: 1744363021-796632 X-HE-Meta: U2FsdGVkX189CW7V9HF97yHGqOnpmLyiAcp5DGpfoZNuhcSUItFtCqVW/QAj6lJJqhAR1dv5CTBWXpp9ENHJTuUQuMqfp5RIAeHTxzcJw0jUt2PNS2niynS9ns78unz6ClQy84CT7MZ7NcLZkydPSxO2H9zzincd35z39ShyFO2/DB5exZlZjKVqlAzkhnyv0K3iuEDQOaGrO4ycyqps4Ph9ez5Fqbxz+4RLlP8hNYukAH3Iggq2e6o4858WKp0cs9AWoGAy3rIju9rwwm7T0R/JqR6PZQb0X3VCg1d7tO9x728bAI/gl1MQcz9R/S+auQHRJTX9mbTJ5494tBILsojRGRyBha1vennSgz/xqenKyE+NIRtXQXALVcZkoKi4sssf67LdpbuRJ/nOL+0LC8mGaeZQuo9B0R+5BpKzBivZGWDVnl/YVQTjYFRPRAa4Duf31iij/EMryVgIXKrwGJMsSNKuT4kWwbDLg5BYPkCXWc6nGE9DC3hJ9spiOxRKgaJg4D5Y9Jq/lEpYb0AM8G4OH7RwU3rYrvPvKzYr6XDSL029qaTWk3jRR6jTD6l8JMdq9YqzmBXCrYCe/OuWfBQKwe1kuxKItONEbKYxqsA6Tma7nFomxdO7YqggqQ8BaVWFP0d6Ioe3U8S0WaS79/3VbZ/2K0EXG7SPTc1ailBE/YzUDM7W6nq6k9YeE7TfFaaG+JQpY4HA2NmkbrY3YVOQBWV9vUlPJ3e8yAp0TGGBr348Biy/RmtdXZUaDsOUOPZpJIr6swNRn8TmzterfntqqSAMeINaSUmKU6HFkPoN7mEsJT8sZ7b98ItQJpBZlQx9+b178bChLl5bEExggU2tCOWf2shqWr3/rx4bbonOVsTv79n2wHlntz73BBLn4xxYFN28az94DNJNR10+XAwLnHykVmOMVQDbHdYwnMn//Qu7kgrO4joirkgWu1Anh7HpkWhvAAtOfyfy8Mg khPZ6jaw us48hYEufwSrD6rviJIOUKqu+BJ7W0S+bcpavqd7mf1eN8Pbpk+V4Nf1LCygcKsMDGLb9gEPRGvUwYletoEU+qV/AE+uRzePz55Bc1bVyH3rp7PgEXOmCryKtRCRDcwqsHkjRqFAjvXT6Z1U+i4Lmds6Wxnq5vkpMYxkjbNni7ujN+2lKlrq53PzWwCurwZTF1T/N X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys is a simple framework to enable the use of protection keys (pkeys) to harden the kernel itself. This patch introduces the basic API in : a couple of functions to set and restore the pkey register and macros to define guard objects. kpkeys introduces a new concept on top of pkeys: the kpkeys level. Each level is associated to a set of permissions for the pkeys managed by the kpkeys framework. kpkeys_set_level(lvl) sets those permissions according to lvl, and returns the original pkey register, to be later restored by kpkeys_restore_pkey_reg(). To start with, only KPKEYS_LVL_DEFAULT is available, which is meant to grant RW access to KPKEYS_PKEY_DEFAULT (i.e. all memory since this is the only available pkey for now). Because each architecture implementing pkeys uses a different representation for the pkey register, and may reserve certain pkeys for specific uses, support for kpkeys must be explicitly indicated by selecting ARCH_HAS_KPKEYS and defining the following functions in , in addition to the macros provided in : - arch_kpkeys_set_level() - arch_kpkeys_restore_pkey_reg() - arch_kpkeys_enabled() Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 17 ++++++ include/linux/kpkeys.h | 113 +++++++++++++++++++++++++++++++++++ mm/Kconfig | 2 + 3 files changed, 132 insertions(+) create mode 100644 include/asm-generic/kpkeys.h create mode 100644 include/linux/kpkeys.h diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h new file mode 100644 index 000000000000..ab819f157d6a --- /dev/null +++ b/include/asm-generic/kpkeys.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_GENERIC_KPKEYS_H +#define __ASM_GENERIC_KPKEYS_H + +#ifndef KPKEYS_PKEY_DEFAULT +#define KPKEYS_PKEY_DEFAULT 0 +#endif + +/* + * Represents a pkey register value that cannot be used, typically disabling + * access to all keys. + */ +#ifndef KPKEYS_PKEY_REG_INVAL +#define KPKEYS_PKEY_REG_INVAL 0 +#endif + +#endif /* __ASM_GENERIC_KPKEYS_H */ diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h new file mode 100644 index 000000000000..faa6e2615798 --- /dev/null +++ b/include/linux/kpkeys.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _LINUX_KPKEYS_H +#define _LINUX_KPKEYS_H + +#include +#include + +#define KPKEYS_LVL_DEFAULT 0 + +#define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT + +#define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ + __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ + DEFINE_CLASS(name, u64, \ + restore_pkey_reg, set_level, set_arg); \ + static inline void *class_##name##_lock_ptr(u64 *_T) \ + { return _T; } + +/** + * KPKEYS_GUARD_NOOP() - define a guard type that does nothing + * @name: the name of the guard type + * @cond_arg: an argument specification (optional) + * + * Define a guard type that does nothing, useful to match a real guard type + * that is defined under an #ifdef. @cond_arg may optionally be passed to match + * a guard defined using KPKEYS_GUARD_COND(). + */ +#define KPKEYS_GUARD_NOOP(name, ...) \ + __KPKEYS_GUARD(name, 0, (void)_T, ##__VA_ARGS__, void) + +#ifdef CONFIG_ARCH_HAS_KPKEYS + +#include + +/** + * KPKEYS_GUARD_COND() - define a guard type that conditionally switches to + * a given kpkeys level + * @name: the name of the guard type + * @level: the kpkeys level to switch to + * @cond: an expression that is evaluated as condition + * @cond_arg: an argument specification for the condition (optional) + * + * Define a guard type that switches to @level if @cond evaluates to true, and + * does nothing otherwise. @cond_arg may be specified to give access to a + * caller-defined argument to @cond. + */ +#define KPKEYS_GUARD_COND(name, level, cond, ...) \ + __KPKEYS_GUARD(name, \ + cond ? kpkeys_set_level(level) \ + : KPKEYS_PKEY_REG_INVAL, \ + kpkeys_restore_pkey_reg(_T), \ + ##__VA_ARGS__, void) + +/** + * KPKEYS_GUARD() - define a guard type that switches to a given kpkeys level + * if kpkeys are enabled + * @name: the name of the guard type + * @level: the kpkeys level to switch to + * + * Define a guard type that switches to @level if the system supports kpkeys. + */ +#define KPKEYS_GUARD(name, level) \ + KPKEYS_GUARD_COND(name, level, arch_kpkeys_enabled()) + +/** + * kpkeys_set_level() - switch kpkeys level + * @level: the level to switch to + * + * Switches the kpkeys level to the specified value. @level must be a + * compile-time constant. The arch-specific pkey register will be updated + * accordingly, and the original value returned. + * + * Return: the original pkey register value if the register was written to, or + * KPKEYS_PKEY_REG_INVAL otherwise (no write to the register was + * required). + */ +static __always_inline u64 kpkeys_set_level(int level) +{ + BUILD_BUG_ON_MSG(!__builtin_constant_p(level), + "kpkeys_set_level() only takes constant levels"); + BUILD_BUG_ON_MSG(level < KPKEYS_LVL_MIN || level > KPKEYS_LVL_MAX, + "Invalid level passed to kpkeys_set_level()"); + + return arch_kpkeys_set_level(level); +} + +/** + * kpkeys_restore_pkey_reg() - restores a pkey register value + * @pkey_reg: the pkey register value to restore + * + * This function is meant to be passed the value returned by kpkeys_set_level(), + * in order to restore the pkey register to its original value (thus restoring + * the original kpkeys level). + */ +static __always_inline void kpkeys_restore_pkey_reg(u64 pkey_reg) +{ + if (pkey_reg != KPKEYS_PKEY_REG_INVAL) + arch_kpkeys_restore_pkey_reg(pkey_reg); +} + +#else /* CONFIG_ARCH_HAS_KPKEYS */ + +#include + +static inline bool arch_kpkeys_enabled(void) +{ + return false; +} + +#endif /* CONFIG_ARCH_HAS_KPKEYS */ + +#endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index e113f713b493..819ef5b70695 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1131,6 +1131,8 @@ config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS bool +config ARCH_HAS_KPKEYS + bool config ARCH_USES_PG_ARCH_2 bool From patchwork Fri Apr 11 09:16:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0A3EC36010 for ; Fri, 11 Apr 2025 09:17:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7862F280197; Fri, 11 Apr 2025 05:17:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E950280190; Fri, 11 Apr 2025 05:17:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B122280197; Fri, 11 Apr 2025 05:17:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3E24D280190 for ; Fri, 11 Apr 2025 05:17:07 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D4828162285 for ; Fri, 11 Apr 2025 09:17:07 +0000 (UTC) X-FDA: 83321208894.03.D9697C9 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id 49D08A0008 for ; Fri, 11 Apr 2025 09:17:06 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363026; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DAvN/ymhOC9RH1764Pqdo1P8xtkdW4aJF6g43VRuQIg=; b=1CA5uy9uriL2I8M0v8hfFJTzwyXOqqbFIJKHrw2wMZwATm0anNzvZO0acT+7DRFay0TQwZ fzoX84Wie84CXO+FqzBPLy9Kpuk0HX95wJtHpc1SGZKVxrvaLpw5McDqWvW5ZMiqXNS17c fVvX9mFGIEuKZNVgyYPRkoAk86jTepg= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363026; a=rsa-sha256; cv=none; b=QW/kV123coGzivpDRmcltb7G/8Q669Ek2XKcNFQiFQmGTy73hPkZyWWM7Uad6ZRVhFXjge ujzjqOViguki+oG9vrU8ToVFcp5H8js2yguRQkMcmTGnK+rGRQ/3lhnZTjIIlGcy7T83QK EORcaOxPHwmhlSS8elaS6/rKHtTG+QE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 19C221692; Fri, 11 Apr 2025 02:17:05 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 87FBC3F6A8; Fri, 11 Apr 2025 02:17:01 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 02/18] set_memory: Introduce set_memory_pkey() stub Date: Fri, 11 Apr 2025 10:16:15 +0100 Message-ID: <20250411091631.954228-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 49D08A0008 X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: f59xcaiixongi49k8knctmqh8aq111hy X-HE-Tag: 1744363026-472925 X-HE-Meta: U2FsdGVkX19kkqH+NLJp9/zx+8mN5TNS4VXgsOoY55bEf4tUqCjLFOycSOp8JK5IJ6Yxs3ur/Zo/E4iCFxfbR3OH1msXcU+VEUEQUaoO/iTlOZf7/TtfN/geVjmeby+PFLeYc6ijHno86JSC64CezfEn70dMw/6mkMOc3HpZ3Q8d1dHFL5xC0tkjheXFLClT92lh2KlSKF/xTxeLdiOACvxHvzU2XX7eQ/3eYnNZVBew3fozgPCy0pwrN9AdO+GPXZhAkSkjaIbxi8+APOxEj9g1I7oEoslL4RaMOfvI66CFiOPJ5PjgZl4nV9nvO/bq87gYovS/MFuWOnUYk2+bqnqKPsQnpIVKuAoB81RRRthd5zSbex14nOX2ZUsh2MK92i/CLHPq6nQFzG0eoY1sQ+JIMlBWO8u0iGBnkbfgwKWO81Ce2RUkV4os4KGhAnbpid3ZEodmWr4TjPxdIXlqtAklLrzhg8V1kqGGbfODZgYu0D9xJE2WRJaHmqzx63DbmLjYx28E87sU3Napory9oNQUYWYaL7RpILhKsd6CMl4shuNuOV5q3dtSlQk6dLcfFcoyXHQVfGDwUOmfw4cTKK2pUfW4qseWQiSNMysvMJLFD4ZT4GAVMqCFzBN7kiqFzT1Os9zmkp97KEp5TVWGmW37jWCvayP3Z29sYxCymGl0ED+M5jq4NTizhHw8S1+rMGy2hKl4a6sNeR3kduWHx4xVt0bHT6N9/PWXa83xlwGwf7Hdt5bJ23CqpPBdbTm7d8CrvG44p6PHC+Tn44LZm5f014UR9FLN9SkJoxOaNTSXnPM7PRW/yUPIT98pdbBR9tp2MnOlEswz9REHwN3AopWYE3sBmB6G0i8Wj9LlNMboq1IfAsX0S++41KCSaHNOBgnO+LyGFlkyBm56SojuFR4CSCHKjPjkJa0RO1zC72b9XRCH1TfFO8BL4R2wyZcrwgmWCBXHCmjStbQfBKX lj+jxzlh oOKdGxBbIlEbZIHv/S+2Zwvv0KM5u9vFhCznXSdZ7Cxyy2v/KeUetO24SmWDUJmc+HDtadUdUKakqG+H3t2bGTdCIKupMJfPA7Mx8hlYLpUucJ96qGinVcaaf4MOL8w1CuDo0fH9MDydEDhBjQm163Uh5bMJDa29JoF+dyWwP9ZTXIl8tlycgK992f3pKTIiuyo6jhJE+7TWgjC8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new function, set_memory_pkey(), which sets the protection key (pkey) of pages in the specified linear mapping range. Architectures implementing kernel pkeys (kpkeys) must provide a suitable implementation; an empty stub is added as fallback. Signed-off-by: Kevin Brodsky --- include/linux/set_memory.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 3030d9245f5a..7b3a8bfde3c6 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -84,4 +84,11 @@ static inline int set_memory_decrypted(unsigned long addr, int numpages) } #endif /* CONFIG_ARCH_HAS_MEM_ENCRYPT */ +#ifndef CONFIG_ARCH_HAS_KPKEYS +static inline int set_memory_pkey(unsigned long addr, int numpages, int pkey) +{ + return 0; +} +#endif + #endif /* _LINUX_SET_MEMORY_H_ */ From patchwork Fri Apr 11 09:16:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBA9AC369A2 for ; Fri, 11 Apr 2025 09:17:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 76B01280198; Fri, 11 Apr 2025 05:17:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 718D4280190; Fri, 11 Apr 2025 05:17:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E096280198; Fri, 11 Apr 2025 05:17:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3FA4B280190 for ; Fri, 11 Apr 2025 05:17:11 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D84BBB0F40 for ; Fri, 11 Apr 2025 09:17:11 +0000 (UTC) X-FDA: 83321209062.28.D264F9E Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id 5563DC000E for ; Fri, 11 Apr 2025 09:17:10 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363030; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4bn2iTHRxV4Y+JziL3tQkHFaUTnfOTGLKiy637DrfqM=; b=a8DTr6I8/rltZSVsYU5J9vS/VR19gnXCzpIN7pPpKVuWxAowQmXtI/d8Q7gT9NjwG7pEzi r/RNBiHtMlERN65Gt2xjHlD1AdsN4MMVF1J8+WaQTSIO2eDT9rbu9LvZqeWwUrjWZNIArE 7w/jPdHDaAyREQKLpC8GpCFKzLVrOaQ= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363030; a=rsa-sha256; cv=none; b=GFlf1Eb7CAQskpJySrcU42LSkgeYNN3SOCMa/aAk5GPOsGjiX7suiw/NJVdLoakSn8bdgM nMGbV0+dYHE2oZb+RRS+PfckavSv4XMKACMCc1nGzL8OqeBcIOxJbo9WS3y1AhyqO2sugH q2JODHyv4t29LR82u/z6kaq7nFvlybg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C2C7106F; Fri, 11 Apr 2025 02:17:09 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9A1553F6A8; Fri, 11 Apr 2025 02:17:05 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 03/18] arm64: mm: Enable overlays for all EL1 indirect permissions Date: Fri, 11 Apr 2025 10:16:16 +0100 Message-ID: <20250411091631.954228-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 5563DC000E X-Stat-Signature: e6skqarsxw7wd3yop48p1npqp7dscwcy X-HE-Tag: 1744363030-132171 X-HE-Meta: U2FsdGVkX18aF8wk2xEMx/m66xWHHikz3pY3vnI+OaU3Qi4GQw5fbkYWH5LQbw56zIfG8d+HKWsneVGwCSiMmV9bGelK+QuzXyomqlg1kdgyEdbNWQvtvjXOfU6w6dmEFoX0LXPa8Q3MVd9dQH11JR3kEqUxWUwsKEJ12MwO1BT96xE2Xeourb+lA6EjMRZy46x8I+9ZfUkBXCETvXrEsccreK83zgUvIcJCn6TPUn8Sh+L/7ukIBjqcpZ6B569SKpNfaDoJAk8G2ctHU47+J1R9un+oswS7MWuPoFbW4Vz2BLJPmpdgNpAaZwVfI3WRxleO252yK9yAV821xFy+KFC760DL6iTQvn4SvFA72v3tb0g4bBVKpiT9Hth0yfhZi5rq+CBVkJi2YpcPDne42yooigeDw6Hc0GtG1RDlBjLS1uNKG2tOEKUrh5kgNVwicksQt1tmjtw8lTAwc0d/19QgaMBO9daItMq3ryBlbyOA0SOQpa5uKNFCwcA6TjxenQFt4FW8/i8D5eDf6lZfJoYXyb40uD0NQCM4J+gklZ7FVKsgyYtW+nv5rnQeTPYdftkUhPOb6QJAmVy/L3fSOWwbJMTZSHD2TqX6QId4yZIhzSZ6aUEdIokut6DlHWyX5M3yB34daNwWzWhisHcpbKfr9y91r/uLOOS/4fhjKzsZ1EI8CsUmZoAUubKwNIIM3mE0rxpG+naGc+h+A1kh5QtZ66TC41Pv4L5reehxgHfDqGHSaxQkVewqJeEup/g2Y1YrfpDk3K+XQtfOGx3+k7mX8slkLWGvGIjX4tupAh+7qrYCDrZ2cLXwt+fRmEQ1WilmdlADrlfqPOYGf1/1b25+LCCxC4mPAqvQ6A1A52vXgIDo1yHunZTrl+kyL1cFAM6mLxRtyVYkE/mImqtjkVldx6X+bVPA9NP64ozezeyGiBVUD9pHevQYJxW0jszQJl7w9DKPWddQp/tUHy3 aA0Ua7Eu TLkHopXc7vsld3q8FlXIKe85b+pHc5HhBIuO1LDvCIh++K3Ze7tPkByzk0NjUlipqi61KjU94kSyG3nbDx0XJT0S+H7l2I9w0oT08o6tU7nQEFyBAdeV1cAAfGZsDW0AYv38dsdvn1/4irr5mt6gsf+kQvQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation of using POE inside the kernel, enable "Overlay applied" for all stage 1 base permissions in PIR_EL1. This ensures that the permissions set in POR_EL1 affect all kernel mappings. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable-prot.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 7830d031742e..c863d8e51281 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -181,13 +181,13 @@ static inline bool __pure lpa2_is_enabled(void) PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS), PIE_NONE_O) | \ PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS_RO), PIE_NONE_O) | \ PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_EXECONLY), PIE_NONE_O) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_READONLY), PIE_R) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_SHARED), PIE_RW) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL_ROX), PIE_RX) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL_EXEC), PIE_RWX) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL_RO), PIE_R) | \ - PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL), PIE_RW)) + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_READONLY), PIE_R_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_SHARED), PIE_RW_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL_ROX), PIE_RX_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL_EXEC), PIE_RWX_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL_RO), PIE_R_O) | \ + PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_KERNEL), PIE_RW_O)) #endif /* __ASM_PGTABLE_PROT_H */ From patchwork Fri Apr 11 09:16:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA8A0C36010 for ; Fri, 11 Apr 2025 09:17:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E611280199; Fri, 11 Apr 2025 05:17:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BA2B280190; Fri, 11 Apr 2025 05:17:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D220280199; Fri, 11 Apr 2025 05:17:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6FCBF280190 for ; Fri, 11 Apr 2025 05:17:15 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D5E1EC21F1 for ; Fri, 11 Apr 2025 09:17:15 +0000 (UTC) X-FDA: 83321209230.12.CCC840B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf21.hostedemail.com (Postfix) with ESMTP id 36DCB1C0007 for ; Fri, 11 Apr 2025 09:17:14 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf21.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363034; a=rsa-sha256; cv=none; b=L2eEG+CkGPo3k0jrTJ5hF6T3G3AK/GVAiwY1gImuyanwo6LIlU7b6hGMFYNi1jtB/1mU6g aJnNMBmSqbdkmDSEtbPbl9TI0yJPgjsZYHfmnf1f66GzVCNrJg93gA6KkmxhumE9DS8Qb1 CfSsBXnSBJL7lMAg2zTvAuNuuUZUyXE= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf21.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AUz8ih9TJh8rve7uDxN9yr0bb8Udhpj9bSc6wxAckKg=; b=3AKp/0qR7mFPwx0t02UzNDZrRfzfXnlrB07VjaYtqRlF7updCyu+QLK4SfLTJFaFDUST0S 3eEFgXOEAl1bnsbeRCHXln+j9zfGAsDiP7xL/ZSuDHBBsfCEqZwSISz9dWhdoN2KePCfsT 3hGNhag2/cy7CK7YYMdPY8TpyGwEK+I= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3F8CD1596; Fri, 11 Apr 2025 02:17:13 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AC79A3F6A8; Fri, 11 Apr 2025 02:17:09 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 04/18] arm64: Introduce por_elx_set_pkey_perms() helper Date: Fri, 11 Apr 2025 10:16:17 +0100 Message-ID: <20250411091631.954228-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 36DCB1C0007 X-Stat-Signature: ya5ttfuu3fxkieers8icdy33hw86a346 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1744363034-506582 X-HE-Meta: U2FsdGVkX18WCrJnppFFBCVsCb+gS14SaEJN4T5Td7I+jn8lTXYHirofiDzl7GRZ29bNhj6bxB7FDT6KVuEvcPfD5TM3is+M6z8C0Q9bTwgaWsUMHvZaJBJdm+y/VyJ8uhr2zu4YOJY7xbT+50+83xElOd7L48PL0+dVAt0nHTfMOMPyLAxWE9vlM0t/QV1ABvRSgq9NAe2nGyyNWx90+RAyopXcjwKFMVKcGa056eg3sKzuE+SqUe0vgFzMJSFCO+2rjg62KLGdNRU4ujN0sIZbWmGHM55umYOv6pPrrvNNpdAuNj2+mBqzBbfS7NafrlutO+kVb8orSDcxFiCm3FvSozIJrBdJIfCWRyi1QwUbeDl/lw8rAFQjqltZbLrwk+pGMCfScuFv5XXtTsWJ7jbWQi9x2Ry3jN9C8NE9cjA4oAJTInePxI1KypFhq8ADcVHjUBl9V5JsN5eN0Fd1zvH5+hWiTMPeO1AFsPeuMDyO5nTvZDORevmLa1XZDygLrC5UGARfjiK5SBYuzxOgFrCZP/lNg0DQRUVQ46iWtEMheuXUJ9nBncYBUXEFDW8hau426LslPaDHaNPjdFbdQPDsx+1M+kT7Vzo+QEbFAIrqpNBca4YlmKNOHEOhze/UckSsSLZBykKVDeI19HwafzDJIvMS5UNARV1tSne1rboMT+94c7NrTRkuaWtfgOduCs7lHaY20WlD5z46jlhmMKqKTDTfYRlagF0NA8260Xal525dUESG9OmBM6t1bSohVJxwxGEPB01D1Z3CoUrrSv87d1F4P4ERtjbdjAQQ0TrYAdIzRyomuoVtEPf8uLn5T41dvnE26gyzoq4ZCqr+MhBadtRg3pE5L/lt5g3cdXV/gxBqP5AyY84ymAvFezYoFooH1gFbO5wgtdRZNTNLzcsTYuOgI4/4BBk/dFUu+ZpHgGcOMtdJ9gJhr9CHkkZMjQ91mG26DV55w3MX5nB Qma/oSkt fQfKScCAhMrG+v5oOlx/q62GB6raItv7/TUYk4lRF/haPq1wcvLbS6RkqO+W3Dg72paf9HnvxIfqJ/kUggEOgmTvpSfToaonTehkacvqwK2yf3x1daeAV//nWoC6sdeIGVxAPLuXti8CHLSVZ3J5hkAMV/cn63dWrbZPjEKrxceKqF5sFxsVcQnHZmlNcxcwPKaGQvLkbVTl3g4k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a helper that sets the permissions of a given pkey (POIndex) in the POR_ELx format, and make use of it in arch_set_user_pkey_access(). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/por.h | 7 +++++++ arch/arm64/mm/mmu.c | 26 ++++++++++---------------- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/por.h b/arch/arm64/include/asm/por.h index d913d5b529e4..bffb4d2b1246 100644 --- a/arch/arm64/include/asm/por.h +++ b/arch/arm64/include/asm/por.h @@ -31,4 +31,11 @@ static inline bool por_elx_allows_exec(u64 por, u8 pkey) return perm & POE_X; } +static inline u64 por_elx_set_pkey_perms(u64 por, u8 pkey, u64 perms) +{ + u64 shift = POR_ELx_PERM_SHIFT(pkey); + + return (por & ~(POE_MASK << shift)) | (perms << shift); +} + #endif /* _ASM_ARM64_POR_H */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 8fcf59ba39db..89a331e699d2 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1578,8 +1578,8 @@ void __cpu_replace_ttbr1(pgd_t *pgdp, bool cnp) #ifdef CONFIG_ARCH_HAS_PKEYS int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long init_val) { - u64 new_por; - u64 old_por; + u64 new_perms; + u64 por; if (!system_supports_poe()) return -ENOSPC; @@ -1593,25 +1593,19 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long i return -EINVAL; /* Set the bits we need in POR: */ - new_por = POE_RWX; + new_perms = POE_RWX; if (init_val & PKEY_DISABLE_WRITE) - new_por &= ~POE_W; + new_perms &= ~POE_W; if (init_val & PKEY_DISABLE_ACCESS) - new_por &= ~POE_RW; + new_perms &= ~POE_RW; if (init_val & PKEY_DISABLE_READ) - new_por &= ~POE_R; + new_perms &= ~POE_R; if (init_val & PKEY_DISABLE_EXECUTE) - new_por &= ~POE_X; + new_perms &= ~POE_X; - /* Shift the bits in to the correct place in POR for pkey: */ - new_por = POR_ELx_PERM_PREP(pkey, new_por); - - /* Get old POR and mask off any old bits in place: */ - old_por = read_sysreg_s(SYS_POR_EL0); - old_por &= ~(POE_MASK << POR_ELx_PERM_SHIFT(pkey)); - - /* Write old part along with new part: */ - write_sysreg_s(old_por | new_por, SYS_POR_EL0); + por = read_sysreg_s(SYS_POR_EL0); + por = por_elx_set_pkey_perms(por, pkey, new_perms); + write_sysreg_s(por, SYS_POR_EL0); return 0; } From patchwork Fri Apr 11 09:16:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03099C36010 for ; Fri, 11 Apr 2025 09:17:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A3772800D9; Fri, 11 Apr 2025 05:17:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DBE72800C0; Fri, 11 Apr 2025 05:17:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 756182800D9; Fri, 11 Apr 2025 05:17:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4F8142800C0 for ; Fri, 11 Apr 2025 05:17:19 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B31F2121E0E for ; Fri, 11 Apr 2025 09:17:19 +0000 (UTC) X-FDA: 83321209398.09.CF6E186 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 2FE9A8000A for ; Fri, 11 Apr 2025 09:17:18 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363038; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aJoDwb/u3Nzlv4jn8+XZaODNbWbOa/Nrb+B9eqkeeBc=; b=ntnXNsG7G6+Hjzn1llDD0osIpK6Cjh0sbuYSKQhjfHeZ8+PbAnJ5/mqXbGWRPzV2ta5+8C q2FJcTPIHcyssNCCGtjAZzdSCkyDV3bQVNJz6efq1uleCaxwNqGEOTdT8b7G57+zwh9RnR BSA04/gAtUhCDchClfDyC5BQyMZtYqk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363038; a=rsa-sha256; cv=none; b=bpb1UXpgU+shXMFgfzfCJaK0TVko5Ao+cxq/kISMEk7uRNhaUBVtzAe9mAiVs+FhRPlkcG CJPrAN+HcO2hNrkwyrYfltik0iA2Y/PwM+kA0WnskOLslnPjh0G9gk7ITs57fDJhto5SRS DARIIWuU7lcSRUWk1BXKU+OOQpoKhxg= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5339C106F; Fri, 11 Apr 2025 02:17:17 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BFD4B3F6A8; Fri, 11 Apr 2025 02:17:13 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 05/18] arm64: Implement asm/kpkeys.h using POE Date: Fri, 11 Apr 2025 10:16:18 +0100 Message-ID: <20250411091631.954228-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2FE9A8000A X-Stat-Signature: s7b9pgcowsf7imp61aejd4s5n8rg3ejf X-HE-Tag: 1744363037-353716 X-HE-Meta: U2FsdGVkX18Wpqtl6WVZSyXIbyqFgBFtXyM0QCkO36awy/oMhcJhJNkYTpjfDW2AhbpNW53J+SCm2Jq6Vw2O8jVcQ6i7lD4bbmlXhNkKeuyoyxKDkv8gZ8WbLtEFoqdFioS2ZP5Ek0AsahNx/coNL8789L9L3DZaTiI3aFzzOyqvIMCUqiPeV073gHR9q2A+W/S9E7pD3nq+IrIgGYNiOyIVs9qYHXHK8aoI9c/z1d2arQkRjwzwKsHFNdU97D0B03SJfXRIHbpRHaY9ZPAnKEvOn6PmQskAD8MgkrYx5seYzWA/+wWAii4+M0EoYgurFkmpU6TmIhiZr9Kp6W3AaX6B48gM6tHI8Ccki7B+QRxPFBrEyb2bB2DBXpof4wPtncbLLgHOXPwCIqUymKfLf8ye3LuilBpceej5J7mTzCpsCTRhHiwK+W2W2exqOqk1rWjD4pxdt6W+nKMHnG6jvnSQVUoBvi/wQKcIJDDYIJagKspVMvk32kF4yDZ9sGAbcqKmRmdA9SNF6Fra6vZelny15A8PyAiNPRxcKem4r94O/JPPgPnIlbn9xT4qUhtw1nJlOvAQA6AOkf7lfp2JCSLK/rg+ZRwIMlbzyYvzZhGdiaKFfIxirc7uu/Sp5uwVsdy0H5yQH72Fz01mW4oN7iID6iC+Gkms+nK00SgdrEd+yIGRjFWNunH8zZr3wqVuaxa0P8ry6DwgjqxylnD+qvVeik7Pwy7FCS69KLgeunEOj5YZ7iv/07iY7D9cDwstCD4rJmTR6Oox0u/reV2zClhXCroXhAkgK5yo+yiuttqXw2m1OtyXwMtGvlHyVKGNRMBkq5T8kpEV9CHCGsvTb3dkbcBmMxV5uy8COS6k6tupaIlA6/WpFD91prPQcztW8r8cHTMN+1LCGJXM89megAWbvFMTqtX+fHTb739GAOC+clktpCUw7VbEmdJ9ff3VnprHm6VPAVo2UANoygS /PMbUj0D 4qTRFoOz32hKV4J4tIAUhZL4nYed0kN6DJO/+XvjNB6+QdRRShdDo+4/V6PxBdRP7ZqloyNJRRRAXhylsGiXtTnRdLFQJ9SPgxzRDdb3PgDapAeMDwoA5Blf7N83Khx5uLNvCqUip3HFHG/qG0uY15d3ZYZcMubsi7fIR0TKLhZXG/e4f/V66HI3/c3bf2i33BA8V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Implement the kpkeys interface if CONFIG_ARM64_POE is enabled. The permissions for KPKEYS_PKEY_DEFAULT (pkey 0) are set to RWX as this pkey is also used for code mappings. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 49 +++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 arch/arm64/include/asm/kpkeys.h diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h new file mode 100644 index 000000000000..3b0ab5e7dd22 --- /dev/null +++ b/arch/arm64/include/asm/kpkeys.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_KPKEYS_H +#define __ASM_KPKEYS_H + +#include +#include +#include + +#include + +static inline bool arch_kpkeys_enabled(void) +{ + return system_supports_poe(); +} + +#ifdef CONFIG_ARM64_POE + +static inline u64 por_set_kpkeys_level(u64 por, int level) +{ + por = por_elx_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RWX); + + return por; +} + +static __always_inline void __kpkeys_set_pkey_reg_nosync(u64 pkey_reg) +{ + write_sysreg_s(pkey_reg, SYS_POR_EL1); +} + +static __always_inline int arch_kpkeys_set_level(int level) +{ + u64 prev_por = read_sysreg_s(SYS_POR_EL1); + u64 new_por = por_set_kpkeys_level(prev_por, level); + + __kpkeys_set_pkey_reg_nosync(new_por); + isb(); + + return prev_por; +} + +static __always_inline void arch_kpkeys_restore_pkey_reg(u64 pkey_reg) +{ + __kpkeys_set_pkey_reg_nosync(pkey_reg); + isb(); +} + +#endif /* CONFIG_ARM64_POE */ + +#endif /* __ASM_KPKEYS_H */ From patchwork Fri Apr 11 09:16:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7719C36010 for ; Fri, 11 Apr 2025 09:17:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 850302800DC; Fri, 11 Apr 2025 05:17:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B0242800C0; Fri, 11 Apr 2025 05:17:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62AEA2800DC; Fri, 11 Apr 2025 05:17:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 41A632800C0 for ; Fri, 11 Apr 2025 05:17:24 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 97EF081BF7 for ; Fri, 11 Apr 2025 09:17:24 +0000 (UTC) X-FDA: 83321209608.26.704FE31 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 423F4120005 for ; Fri, 11 Apr 2025 09:17:22 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363042; a=rsa-sha256; cv=none; b=dMgwL0xyueVBY4T5v/xVtP9dTQuPjr1kGNK+6gkwnfwRFJ0XIzJ1571SYQVoG0Xpsgxzn9 YfdtAnfbxp/YTBoaintJh/Xavlv2aQw1WBp3PUXv84lON+hN/RoK24jlkgqzyzQ5oXyDWx VhZJeuRnBvt5TxU0qWVXDHpEfBnItQI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363042; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oHokXrrIxgFozOdPdKGv7pm7boYyoVv2BPENAFW6e/0=; b=NcvlWK3gpH0jlXbqdYdPPkI7Fp0oqGZxszJiOGtu+Q84ZMnXqFZ92V4ALW5kvXVv2ZKgOU V28RbhElFqwcf6SwTU5Jz1CyedUW2P1J5nyUM8wo21bOC198CWm0A6YQCJeiA4wJOBwkv4 44IQX0TkhNi6mzNzk3p9nY/glefoZeA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 65F051692; Fri, 11 Apr 2025 02:17:21 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D39293F6A8; Fri, 11 Apr 2025 02:17:17 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 06/18] arm64: set_memory: Implement set_memory_pkey() Date: Fri, 11 Apr 2025 10:16:19 +0100 Message-ID: <20250411091631.954228-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 423F4120005 X-Stat-Signature: tgu4edrczi3dyj8g9cwdqi9jbg7h5otr X-Rspam-User: X-HE-Tag: 1744363042-449213 X-HE-Meta: U2FsdGVkX1+sT37ScV5zJSogvjJvJNZCe8uTHkplAxCZQva4DpV7PzKZW0ggWogLjCn3RuiT0WoHJJKyG0uIj23/EhDMhGrvPBVu0S2l0QnEWi0SVqiR7FiOlLWy+KL6z99RBxiXLWkEz3ADfJJSIyoX184/LK9gFJmwRlLyheSvolzaeONBnXcZ9ZLz6AEdknUhNhF6D/oW7/T+DuvJmQoNL5Q36g6J3m7I0BHaAtvefGkbMV0XJoMaAADw2IsCQdSydmIKRHqPi6VDEzKndOfo2vtEcmwBy9yio7bPQYIORRTnL/vaxelQqmp3UOtkjH5Ld7Lb1XLanRT1K/cf942FN7i/UGhx9aJOmOv8bD9Gmzypueh3q3jKZL/00kZLDrIt5vcLkqtuzOQV+zjmfKZTqHZasdtndw1f0YxZwC5mj8jxF+S+gjJc4pvDnkkOmugZQOHFSNElZcH7jtzAGUesWuXJZ/N1quHmyOF3P/OTVTIEz/zkzYPG/L2i09UppZl16Gx6ivr4ko9+7NrZNWXC6mu9WlJ7jxVeXGuk9wFjhc0wIvrSdwMXclwUb1M1qFDD5W9ihOSNrud0eHiwZ8nsLYzpANp4YTbBUjPSWc/b0V63iraZs1p4/5C5LD1RG5psA47FiygSdzQGJ8sV4Eb4M25Yo18GCRbq/ymodyHG1zvu/nspfq9PmC4Mj1oN9hzCC5NRUzNZVLGWWqWylY22wfdGja6DpN+KiB9v8vGEUC4oybDALirxKiJtIGrcswEAlGsQL8ji8S/ETm99kLkLXNVYJqgVzPQpHsbvsiB70A5gFm6rJZQQO+vMihwREpr+cav6DtpgHe8Ty+rK9ujjlTsCmZMqgKi1f/PqF703m8Nleez0lQOyO3IijMR8UKicCbGB/2N0Lp74zKgEAwqBjO4veDvB06UmZ46CXrJ1k/MoCLsT8sB6EUtDl0AULEaZMUnECaq6iZrnon0 hfiDgLYR T0WeLMLKR6la08odZZFKEuXJxVMdZ3+vu51UG+2e7eITu6utbPfW2Up4evZXsNf4kVUhwBNfLQ+U/YILceYV6ha9UIJiDTQbAGp3DCtsjffy4aMlwzViZr3fMYSAZL2oX3hkpI2jWK6AXAyRlDrcpn9LBYbpscKxeJJWfLTJfoOIupvjZufoBdG97XT2XarUHJ/h6hsWgxm57agvtbXcUIpIybzASo7uMipxArRCVDviMvH4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Implement set_memory_pkey() using POE if supported. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/set_memory.h | 4 ++++ arch/arm64/mm/pageattr.c | 25 +++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 90f61b17275e..b6cd6de34abf 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -19,4 +19,8 @@ bool kernel_page_present(struct page *page); int set_memory_encrypted(unsigned long addr, int numpages); int set_memory_decrypted(unsigned long addr, int numpages); +#ifdef CONFIG_ARCH_HAS_KPKEYS +int set_memory_pkey(unsigned long addr, int numpages, int pkey); +#endif + #endif /* _ASM_ARM64_SET_MEMORY_H */ diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 39fd1f7ff02a..9721a74adbe2 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -292,6 +293,30 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid) return set_memory_valid(addr, nr, valid); } +#ifdef CONFIG_ARCH_HAS_KPKEYS +int set_memory_pkey(unsigned long addr, int numpages, int pkey) +{ + unsigned long set_prot = 0; + + if (!system_supports_poe()) + return 0; + + if (!__is_lm_address(addr)) + return -EINVAL; + + if (pkey >= arch_max_pkey()) + return -EINVAL; + + set_prot |= pkey & BIT(0) ? PTE_PO_IDX_0 : 0; + set_prot |= pkey & BIT(1) ? PTE_PO_IDX_1 : 0; + set_prot |= pkey & BIT(2) ? PTE_PO_IDX_2 : 0; + + return __change_memory_common(addr, PAGE_SIZE * numpages, + __pgprot(set_prot), + __pgprot(PTE_PO_IDX_MASK)); +} +#endif + #ifdef CONFIG_DEBUG_PAGEALLOC /* * This is - apart from the return value - doing the same From patchwork Fri Apr 11 09:16:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29D39C36010 for ; Fri, 11 Apr 2025 09:17:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C016128019C; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8A6728019B; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93E6128019C; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 708DD28019B for ; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E0723818C0 for ; Fri, 11 Apr 2025 09:17:27 +0000 (UTC) X-FDA: 83321209734.07.5EDA388 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP id 5136DC0002 for ; Fri, 11 Apr 2025 09:17:26 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf22.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363046; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WSd0yigUChVpRcAhfJwe0J3l1gFAvojA/sDA8zvgUxg=; b=dn5f0Xz115Wc2wb3N9Z4sn7PmlukcRv8BHQnHjeDjlcOybp/BfNgiHG2tR2hdKLJp1u5h5 +FsHONJovXhaU21DWIIg5KxTssEDcTgar0px8iWcKZ2LYgyzoiUZeHXKsica0HhSX7IhwV qmWqvUfCuzG3e7x0HvWj5eCIJgH0cD0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363046; a=rsa-sha256; cv=none; b=aTUHP2elyi2HOmfE6+eAneUdGiwbgPB7RiNX2emDjHVUOJwdhFpkCGnOjvuysm/L6DiKq/ cVk71h83rqJQsRfw+FcciI8PpqVNzaWpO3b6fLrdOXPIIpujYxSrWegASDO4GrQD4lkmM3 Wn3SY30wmF8MrKpiE4Xiy2RbwQlf5LU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf22.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 783311596; Fri, 11 Apr 2025 02:17:25 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E5EB13F6A8; Fri, 11 Apr 2025 02:17:21 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 07/18] arm64: Reset POR_EL1 on exception entry Date: Fri, 11 Apr 2025 10:16:20 +0100 Message-ID: <20250411091631.954228-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5136DC0002 X-Stat-Signature: 3d845ds569zmh4p4s6eead83yrjd8jwz X-Rspam-User: X-HE-Tag: 1744363046-555879 X-HE-Meta: U2FsdGVkX1/GuRrH5u9OUsQRSobpygPFGFXK0KZcW2Few+RAKo0NegrYu0uvMC6CSz2z61qafNcVDqiuyco8FbXvP99Yrxme7fIy9QW59l4NZCjYWdnbZq7WsoEmIjGRzU0NtqrMw6wyQgbV//u8iaiyOKdKsGZrLFeH5STFjw1gxYajRv5QFJo+DKajoppV1GfhrWI2wpOCftRF8NHWB4W9JkftT073XqbsjlycFY41099MIdEQyk1vwSu6hKVko+7RDa0VTiufvsPvl3S/fzlSv4OzDk8HI+BLF0VIqkja3Z+rjDW5V+qOx6xOJ8SOKISYQLzh/8WTKGYhd0VXA9HqzK1mUVX9yO7HStYbJ6KtDk2WALh1TTHC2ZmLmUtZwejVExRhvRzj3QjUNJlJkN9dvK175v4sxphnQ/BzqplVk/PuhpqTVX0B5w67YKRO16/k/t5689AnG1imcYZ896f32+luJaPdFIraDNRRtahXmrSVDlc4oZVE7jIMhRlFrWK2AzAoxhVpg0Ij3hZDjOpozoa2FAfwoURCCljSb/CjtyX9e3br8PR1rEnoVnjVJEDdqmk+WFqJ20Eae9Id815q8k36IpGCf1TzdkTNzNPcYSOfPN9Om1rw1qlKDzN/Vu16kweTaZ5bDntCruvgRHXMb+2rC9fMI/fhBdBmOPHLAqIbeSldQT0C74wPwtkDbR8RAsXZzERGF0/KBuJvxCKv4mERgJdWSRM517SFktlQhK1zJbgezi1ZXgDxenUsvPEQddzciSPts21Qso+4WBs40opKSAcVuxJu9NEri9tbN8tnwHNCNTi/PBf7elUinZECbUtBC59Ls9r3zW/G+mD9YeveAIKDbLoy9KExYEux40Fx/I5OXznYDquTTtIfcouJVHubUZfsAtkV3tgkuaLqE7oh17wePFfVS7Zie624iWIu8Iwo4vUpw8B8caiQ1oQxKEACU0NuC+LijVW f/4mZPkl QsiQjEew4vVLRAxPnG8nMCK0vPZllLhsOtsJxVoyr7jVtpEfS8WYp0VimEJBSrosakrEjvvAwx+KXExSsgWcqRfi8+wbVGOzACs1BspwWg+GBMjZxpNKxf6y6mW4N6wTyrZyoXZ1d9twrkfJC9/uhrpu2ThdQhrn2E7gt+bzTEQ5jixWFZpjN9skQyaKm2HgFBImbrXGAVHki0Ss= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: POR_EL1 will be modified, through the kpkeys framework, in order to grant temporary RW access to certain keys. If an exception occurs in the middle of a "critical section" where POR_EL1 is set to a privileged value, it is preferable to reset it to its default value upon taking the exception to minimise the amount of code running at higher kpkeys level. This patch implements the reset of POR_EL1 on exception entry, storing the original value in a new pt_regs field and restoring on exception return. To avoid an expensive ISB, the register is only reset if the interrupted value isn't the default. No check is made on the return path as an ISB occurs anyway as part of ERET. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 10 ++++++++++ arch/arm64/include/asm/por.h | 4 ++++ arch/arm64/include/asm/ptrace.h | 4 ++++ arch/arm64/kernel/asm-offsets.c | 3 +++ arch/arm64/kernel/entry.S | 24 +++++++++++++++++++++++- 5 files changed, 44 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index 3b0ab5e7dd22..79ae33388088 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -8,6 +8,14 @@ #include +/* + * Equivalent to por_set_kpkeys_level(0, KPKEYS_LVL_DEFAULT), but can also be + * used in assembly. + */ +#define POR_EL1_INIT POR_ELx_PERM_PREP(KPKEYS_PKEY_DEFAULT, POE_RWX) + +#ifndef __ASSEMBLY__ + static inline bool arch_kpkeys_enabled(void) { return system_supports_poe(); @@ -46,4 +54,6 @@ static __always_inline void arch_kpkeys_restore_pkey_reg(u64 pkey_reg) #endif /* CONFIG_ARM64_POE */ +#endif /* __ASSEMBLY__ */ + #endif /* __ASM_KPKEYS_H */ diff --git a/arch/arm64/include/asm/por.h b/arch/arm64/include/asm/por.h index bffb4d2b1246..58dce4b8021b 100644 --- a/arch/arm64/include/asm/por.h +++ b/arch/arm64/include/asm/por.h @@ -10,6 +10,8 @@ #define POR_EL0_INIT POR_ELx_PERM_PREP(0, POE_RWX) +#ifndef __ASSEMBLY__ + static inline bool por_elx_allows_read(u64 por, u8 pkey) { u8 perm = POR_ELx_PERM_GET(pkey, por); @@ -38,4 +40,6 @@ static inline u64 por_elx_set_pkey_perms(u64 por, u8 pkey, u64 perms) return (por & ~(POE_MASK << shift)) | (perms << shift); } +#endif /* __ASSEMBLY__ */ + #endif /* _ASM_ARM64_POR_H */ diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 47ff8654c5ec..e907df4225d4 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -166,6 +166,10 @@ struct pt_regs { u64 orig_x0; s32 syscallno; u32 pmr; +#ifdef CONFIG_ARM64_POE + u64 por_el1; + u64 __unused; +#endif u64 sdei_ttbr1; struct frame_record_meta stackframe; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index eb1a840e4110..a69c8ba0e085 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -75,6 +75,9 @@ int main(void) DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno)); DEFINE(S_SDEI_TTBR1, offsetof(struct pt_regs, sdei_ttbr1)); DEFINE(S_PMR, offsetof(struct pt_regs, pmr)); +#ifdef CONFIG_ARM64_POE + DEFINE(S_POR_EL1, offsetof(struct pt_regs, por_el1)); +#endif DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); DEFINE(S_STACKFRAME_TYPE, offsetof(struct pt_regs, stackframe.type)); DEFINE(PT_REGS_SIZE, sizeof(struct pt_regs)); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 5ae2a34b50bd..bc042eabd730 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -279,6 +280,19 @@ alternative_else_nop_endif .else add x21, sp, #PT_REGS_SIZE get_current_task tsk +#ifdef CONFIG_ARM64_POE +alternative_if_not ARM64_HAS_S1POE + b 1f +alternative_else_nop_endif + mrs_s x0, SYS_POR_EL1 + str x0, [sp, #S_POR_EL1] + mov x1, #POR_EL1_INIT + cmp x0, x1 + b.eq 1f + msr_s SYS_POR_EL1, x1 + isb +1: +#endif /* CONFIG_ARM64_POE */ .endif /* \el == 0 */ mrs x22, elr_el1 mrs x23, spsr_el1 @@ -409,7 +423,15 @@ alternative_else_nop_endif mte_set_user_gcr tsk, x0, x1 apply_ssbd 0, x0, x1 - .endif + .else +#ifdef CONFIG_ARM64_POE +alternative_if ARM64_HAS_S1POE + ldr x0, [sp, #S_POR_EL1] + msr_s SYS_POR_EL1, x0 + /* No explicit ISB; we rely on ERET */ +alternative_else_nop_endif +#endif /* CONFIG_ARM64_POE */ + .endif /* \el == 0 */ msr elr_el1, x21 // set up the return data msr spsr_el1, x22 From patchwork Fri Apr 11 09:16:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047906 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 499E3C369A2 for ; Fri, 11 Apr 2025 09:17:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF80928019D; Fri, 11 Apr 2025 05:17:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C89F528019B; Fri, 11 Apr 2025 05:17:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A361628019D; Fri, 11 Apr 2025 05:17:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8177C28019B for ; Fri, 11 Apr 2025 05:17:31 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 194AD1421DF for ; Fri, 11 Apr 2025 09:17:32 +0000 (UTC) X-FDA: 83321209944.18.83FF72A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf06.hostedemail.com (Postfix) with ESMTP id 833BB180005 for ; Fri, 11 Apr 2025 09:17:30 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363050; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kfkzdaySghlKYvqTb+hEmk4OPnsFL6Vuh0dxNa6E2UQ=; b=0G1GhdLj7Ax3oUzdUhPFXwDnHkoxfs8fa4yiiFcQ5C4E8cOt3vEmozLB5aMxN1YWCn5Ist El+I7rB9Yh7QG9rPY55GAamA9/YCpSn9Tw8hrOdYpHxZZJPuZiiN9yq/cPl8zOzouxZCZb x8XU4PQG5ZL1mk65N1K8DRXuLjDE5x4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363050; a=rsa-sha256; cv=none; b=Oa8GGHFUPPSDn2PaDHc+D4Pkqb7OfjPkquvrCRNk9Vu+NHKp7KeoDHGq/x9/omjDlYwc/R u82j/vxdV8Tq+Aqw2U/k4fpn/Gb7LYkfevb1SYkYlRFKXB83xKGLVFJjf6ik44Pczetmyt 7Sm+fNHumU/Gh3zTGwK4VWdhnnl+Mj0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B014106F; Fri, 11 Apr 2025 02:17:29 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 042073F6A8; Fri, 11 Apr 2025 02:17:25 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 08/18] arm64: Context-switch POR_EL1 Date: Fri, 11 Apr 2025 10:16:21 +0100 Message-ID: <20250411091631.954228-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 833BB180005 X-Stat-Signature: u5aoen9dmj84ke6aau7ht6mc3de8p7jw X-Rspam-User: X-HE-Tag: 1744363050-574735 X-HE-Meta: U2FsdGVkX1+FqAbsHTUEENZz18xeyfkIu6bme94osnS1jIxjp7tSDE6z9k2p1pJQiYks/Mr2JgFq2nVwvST6A1SAXTD2Spm3//Dp6j9tD0epGToMAGRWpjFAW94f3PnIWtWPgAlUEvz+5xrPktFEwBp9gLMONNLWjfPEIDc6bDJLdyseykEx8Rl7FKNbN5sNhTIVR8hBcBLsQvasJRXWfedEJT7up+PuU6hKiVd0sZNnsLEcRJ+R/v0xawzdBScu253t5toElcbNZedZ8ZWM2Ua4c3HK1IobryEzYwyhF4cY+uSI+7FikLSsuibqN1kbT+NS0GADHYWQGuU041vRoGrw9fz1hEpUO+taAsCUlzm34QE10aVr7JGmsBP3NEfgUaqcANl1dzc3KPRvK7GD0PonASf98ts2AdZ/7uPTgz0xnSU15IK/pDEYkGF6a8RaOZd4fVfhhaqFdI6Mqt9wTaI8HK4MeL2GqeViFf97gS6xvFdsRMIBfUqWSmyqzXkGT2geEedtwDW5WOejsnK9Abo/cDQrb4U+BUBkUV0KSFgQOol669a1BmV8UsHKj8cquwsP0Jr4wQtBKl9vzs4d7/Ottt+LyUyO8Dhp4GwLK+9oql7QwutsWQFWuFt1xZTTF3ZfYoNy3xYGl4V8AwJOxDiAD1QUs6R5A8fo4HuruCuIG10cV133xyQzLxdAo9l0kNfD+5Xu3CCQDiL9Ac2y9FY6+wXURSOt2WqW0zS5oXEN/8JSar5dNgUuX4kp/i+o/Fjxn94uL96QaIv6q4WTVCu1bBu2a+IgtQ9I6qzi6OSppWXwEuReEZ4l43HX+JM0Di/qjn7GyTkrEoIBOl5HzhdrkTNdORqm3VgxQHT8Wf0meJlzUWencGT/5KTKQaiBcYsV13AwLbO6tadP0IBtef5Fsv1BEZGCYMnAq/UXnB4CXV1nHXIC6Tw/vfAeoFDJ9PPaZ4c3P3UzV47ukXZ YprBwDes rApjvwxf+qS8fMiqY3RjFVOD1rOTTZCaOd8yoXDxah1fUu7X5Uaq/QVHHyzzvFqkd3HHDTMzusG2ONkM9n+8OLBNXk9GvWbMTxrGGy8Ip4OtxjRmcy8IMDLg0dRG0rUZgpI1+h8csfV+M3SmiTVw+YRef6m7iOlW+RmYBsZYYhiE/eERB3SrUSdDNYfziY9K2nzPKqjhxYFp0gLU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: POR_EL1 is about to be used by the kpkeys framework, modifying it for (typically small) sections of code. If an exception occurs during that window and scheduling occurs, we must ensure that POR_EL1 is context-switched as needed (saving the old value and restoring the new one). An ISB is needed to ensure the write takes effect, so we skip it if the new value is the same as the old, like for POR_EL0. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/kernel/process.c | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 1bf1a3b16e88..0afaf96ca699 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -185,6 +185,7 @@ struct thread_struct { u64 svcr; u64 tpidr2_el0; u64 por_el0; + u64 por_el1; #ifdef CONFIG_ARM64_GCS unsigned int gcs_el0_mode; unsigned int gcs_el0_locked; diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 45a55fe81788..1f38c29b0c95 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -418,6 +418,9 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) ptrauth_thread_init_kernel(p); + if (system_supports_poe()) + p->thread.por_el1 = read_sysreg_s(SYS_POR_EL1); + if (likely(!args->fn)) { *childregs = *current_pt_regs(); childregs->regs[0] = 0; @@ -639,6 +642,12 @@ static void permission_overlay_switch(struct task_struct *next) if (current->thread.por_el0 != next->thread.por_el0) { write_sysreg_s(next->thread.por_el0, SYS_POR_EL0); } + + current->thread.por_el1 = read_sysreg_s(SYS_POR_EL1); + if (current->thread.por_el1 != next->thread.por_el1) { + write_sysreg_s(next->thread.por_el1, SYS_POR_EL1); + isb(); + } } /* From patchwork Fri Apr 11 09:16:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE6F8C369A2 for ; Fri, 11 Apr 2025 09:17:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 952FA28019E; Fri, 11 Apr 2025 05:17:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9085B28019B; Fri, 11 Apr 2025 05:17:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CC0D28019E; Fri, 11 Apr 2025 05:17:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5FC8328019B for ; Fri, 11 Apr 2025 05:17:35 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 101E4C21FA for ; Fri, 11 Apr 2025 09:17:36 +0000 (UTC) X-FDA: 83321210112.09.DFC9352 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 89BAC120012 for ; Fri, 11 Apr 2025 09:17:34 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363054; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D6OJ6eTJLaGqk3wgOaTb1V4BCC/cGWqmQ6ZcX7KPIXg=; b=N3CoScoE9cvoX53oAade0HdhjTaV7OxU/bzxSPvXUBeSs14uj4qDNWLdfBFnIAPxJjNzf0 gvUX2jN4LG+6nECte8lPbEj224Eg9gATyq3/tvgbRqD/qhn3ivNdfATHDinzOnsHHzSh5+ UuEQhg9tgpBVXXFVf+rDK6RMP8EArrY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363054; a=rsa-sha256; cv=none; b=C4B2TAQ5xiBa0qYasXWe9WSkw4wlYM4Hbb7m94hBNMVafRzOgePoGcWbd/uktWb2y9DxfK 9oC5Rs0g49A0iazY31odaDw8kh+9i4H2UHwTeRgI32+fNWLeJBoKpBMrWFgsrl5CFpXGw4 K6A56/BEUnyhY4HIyyBX8KS757vn8iI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A4C361713; Fri, 11 Apr 2025 02:17:33 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1681D3F6A8; Fri, 11 Apr 2025 02:17:29 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 09/18] arm64: Enable kpkeys Date: Fri, 11 Apr 2025 10:16:22 +0100 Message-ID: <20250411091631.954228-10-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 89BAC120012 X-Stat-Signature: 9mexj419ynwje88yxayoz51i6t7jkg68 X-Rspam-User: X-HE-Tag: 1744363054-749702 X-HE-Meta: U2FsdGVkX1/OvpbxKpIePwKPeL+dMb9idCSZe5miw+BZk4UriwviK7iJmcRDFvre7gYCDBd0xPt0Ypjw+shZ8lTSgGlmFZWRl+3D/iFlXZ8k9VJoIGt+eOY8s4UA/tbmE2MMkVncHTmIxRKgw8hOTmTNeyuRi1HzX39MDqRjzQCblM9StIrmfvCdXFY9V8vWcf9V/XyYXBUWEbzsPBvEaUadrwV+aW/Y8NUwwkQw4r52HeWeEJfJmQdF2L6qryHZu1jmax/ukNNE6wEIC5bL9VUUzZQWPWbdaN8o7UhAvH5GpCod1zGh2w/W9wA8ah72jb3KIgqVJprKCI88T9IUhnD2fzrkFkev3YucSbRXqf/sHGUSqN1Yua8ngzMh5HZDUnQgUW206bv9zTsKHCZQ+KM1xikvGDi20joAufJfl0Gb/iPIRV+0huthhbrnUvjGW8vblLKabNwuOWtc0eFkqZ/oAkQ+gvlRF4UX07ddSPdcBasCdeVxosh35EU3E1O6AvBcCbWKPiyjCCZMKmLku7cS7y01rnQVtik/BPMmbdoiEzXp9YaaXcjCr3OcuYWe+2ulph8SIxiQXfoXlQqtO4cBxNZBDhNHASJ287MuNVc8jkdPDs7TVhkrwBKRMYWdP0ln3ASF6tl8kzJk2AVgiKAFSLL1viO+cIpJGoHG8Ys4W5kmMe/mN9JO8d0FsaGzeluaWLlMLcA/PCB7Uxzh12FpGQdzeYIXkHh4uXdXkGJMl9KdrtdiObu7wPmdtB/q+UZpr952wTnf6jU76K25DaW/RQ0r/goJGy9cxEc6vKUIljddBcBwixbkRoGFQe43PCEdAnkXA31YGS4QHlSQwm7MZOnKlJJvWkgS88+maZyCTrR/xiiIOWv2W2cJ9ketAHYzTuNcKbBYU/oP7OKtV6x1cV3uA5xx5+LCy75787FOTk6Dy97vjC39NHuT/2A7R87/rDpKDO+Md5HnY07 urD0ruGk fShe0tY0Qbxd8S4bS18Z+W1tfgsAloTQU9s5/KfUcRC8qYnlQIyb/sDOQDrVQ9bzHUQuyKY4U2URdqWqH6u5kzSrplvCgbMNzsIZp+Jvx82GzMszh99/Rc7zdCAShxsKv1kuXRZmeGuRS2ihVK6WUatUX460XzWjOyruFq+q/bv1QPKL8UDKuwL9qKj2IqglucS/ytN9q/wGrFSPVIM/S1ZHOy4uWE19DZJMGcgyU4psrdF0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is the final step to enable kpkeys on arm64. We enable POE at EL1 by setting TCR2_EL1.POE, and initialise POR_EL1 to the default value, enabling access to the default pkey/POIndex (0). An ISB is added so that POE restrictions are enforced immediately. Having done this, we can now select ARCH_HAS_KPKEYS if ARM64_POE is enabled. Signed-off-by: Kevin Brodsky --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/cpufeature.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a182295e6f08..5f81db8134da 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2198,6 +2198,7 @@ config ARM64_POE def_bool y select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS + select ARCH_HAS_KPKEYS help The Permission Overlay Extension is used to implement Memory Protection Keys. Memory Protection Keys provides a mechanism for diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 9c4d6d552b25..2043f04c26ef 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -76,6 +76,7 @@ #include #include #include +#include #include #include @@ -2409,8 +2410,10 @@ static void cpu_enable_mops(const struct arm64_cpu_capabilities *__unused) #ifdef CONFIG_ARM64_POE static void cpu_enable_poe(const struct arm64_cpu_capabilities *__unused) { - sysreg_clear_set(REG_TCR2_EL1, 0, TCR2_EL1_E0POE); + write_sysreg_s(POR_EL1_INIT, SYS_POR_EL1); + sysreg_clear_set(REG_TCR2_EL1, 0, TCR2_EL1_E0POE | TCR2_EL1_POE); sysreg_clear_set(CPACR_EL1, 0, CPACR_EL1_E0POE); + isb(); } #endif From patchwork Fri Apr 11 09:16:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 252C3C36010 for ; Fri, 11 Apr 2025 09:17:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5B6F28019F; Fri, 11 Apr 2025 05:17:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B0D4828019B; Fri, 11 Apr 2025 05:17:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D3D128019F; Fri, 11 Apr 2025 05:17:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7D17228019B for ; Fri, 11 Apr 2025 05:17:39 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2D82D162285 for ; Fri, 11 Apr 2025 09:17:40 +0000 (UTC) X-FDA: 83321210280.15.BA9A7B4 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 90113180006 for ; Fri, 11 Apr 2025 09:17:38 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HDIDxyDyfX0gdKCGHJVPMPRL70ks6utR2npUV1XvMpc=; b=2EkqGVj3x4zitkZJRCk31lCQByZKOsEjqe/Le1dkxegtTjOOMJ3Vw+xDUbWzba+NFXyB3J 9AOCD90hLExDlubuzmn+TVEvB4fQCJSjfbD5yv6wRla+eLv+WzPQcd1jr9L3jM5n4tLX3f NaBDNB2Cl6NdvviXJ6dtbmwS9YteijQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363058; a=rsa-sha256; cv=none; b=r15mEFoNKiS/AYVSt1v+GsEWlMDTvkPtL+UxYQDsxg0kqzeTqCaj+5cyIL4RN7tu+Wh/YK Im70cRKqCT6scXeA0mAFMf9gtNHDKsHe9GOgQQ4Meu6lVQJgztwpjMSEbwhR/J2sW10PV5 K/MrurvWevhLMEif0ePDbEk0Gjnccq4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AD61A1596; Fri, 11 Apr 2025 02:17:37 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 286563F6A8; Fri, 11 Apr 2025 02:17:34 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 10/18] mm: Introduce kernel_pgtables_set_pkey() Date: Fri, 11 Apr 2025 10:16:23 +0100 Message-ID: <20250411091631.954228-11-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 90113180006 X-Stat-Signature: irkcwao8qip16dchcndrrn7a6ou4bgco X-HE-Tag: 1744363058-389307 X-HE-Meta: U2FsdGVkX1+VnCF/371kbHF98JNNFjC19svUSnN4MVMIsH21yPDLqMUYDF7AdscmaRA+Nu4XL/XMKwnKwSO0HAkgm2YrS42w2PsEIayKQfSAFDKoy8+7SW7it0dbfT9ZPxiDoen2HGgHBd8FShAJc6sXoN0aQE3b+dPPIDGaMdVan5MBjY8dzC901DQtMQ4RirsDHRrrvDlfFbrhXg0AVW4TTkBQcvvPpizR/DLgiWU1P2Eq7FIhvjbQgr69tClzPKmW5rbEpo6CgeRKetmdlbO1CDwjAuvPZXnErN0sklI5XqxLb+vAa8+bZQGPZU6Mu3rvggFSAWp0h/hP+BH9XKnm6QJolSPrPydrvt5Sm8orXBUc/gXHldyjzXwDGZzzB6UNKN23ir3YWuH9PuqOzN6DjT6D5XU2UMFvUTlPBRjmNy7bFL+gPmaLgBArJOTEw1iP5GxwrQ2oM47PYGy/i2fP7giTkJDfh2a84A/KnHZh9afso8JXWHyjd6WWhbI7KQfy3K5PRssmQlfbPH9hwcPV5Q+AvlVD8qb2eX+wZZrTFfhC5MzS3GUE6Wq1lFAVLLleOWKL+D8VzJBpm4yMv40AE+mVyrORJgDT2GNXY3wn+MpY9+PWLNj1Kg47H+9VOQBuNZJ6yKE91i/cXAJx5dso2pt3LFz+hMKiry5LRzVunxiKgfFjidYNpDkjm/3tE7BLjX9/5+RayIxxA8xezcy+Qow90RhgrnGk5YoDtoWwgsc/qG0BHjLoI2qeMX2NEFOB1tYLGycEzyET9KOhAZqQmyLiAlqBcal7bpHj6ZUXHhLADI7POgOQAD1YUaeCHSq4KoldFLj7icDiJagowxZ0fcPAKXQtC2eM8dPknn1Gk76OH6D7B2DAkLy9BFPKKdF5ujYOpTre/5ATrJVbDpuM9/8kiw1FJgFaSDDpdVJq0sAvYggrtHcPiIUohxWsZHn9QK1+sadngr8DZ2q /g5B3ZDa zDFkJfcZGaISQ76nibYzIOniYcQ0AwEIyrFphI7X9PeYIz9MxwI98AoOB1oiCRsAqneWQLKWNqu3D/1YwBldh9sen7dTcobSw16SspRc3Lk5KcEDYreWd1BbFdClKldaKu/nefXF/4pWh2PCdZkFbL+TTXmY81FpKcmSKc6PBMdqX63mAhT8BBVwAVm0Mp+gCe4NbKJmCQD6EHJY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kernel_pgtables_set_pkey() allows setting the pkey of all page table pages in swapper_pg_dir, recursively. This will be needed by kpkeys_hardened_pgtables, as it relies on all PTPs being mapped with a non-default pkey. Those initial kernel page tables cannot practically be assigned a non-default pkey right when they are allocated, so mutating them during (early) boot is required. Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 2 + mm/memory.c | 137 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index ef420f4dc72c..dd1b918dc294 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4240,6 +4240,8 @@ int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *st int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); +int kernel_pgtables_set_pkey(int pkey); + /* * mseal of userspace process's system mappings. diff --git a/mm/memory.c b/mm/memory.c index 2d8c265fc7d6..37c2bb35faea 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -76,6 +76,8 @@ #include #include #include +#include +#include #include @@ -7376,3 +7378,138 @@ void vma_pgtable_walk_end(struct vm_area_struct *vma) if (is_vm_hugetlb_page(vma)) hugetlb_vma_unlock_read(vma); } + +static int __init set_page_pkey(void *p, int pkey) +{ + unsigned long addr = (unsigned long)p; + + /* + * swapper_pg_dir itself will be made read-only by mark_rodata_ro() + * so there is no point in changing its pkey. + */ + if (p == swapper_pg_dir) + return 0; + + return set_memory_pkey(addr, 1, pkey); +} + +static int __init set_pkey_pte(pmd_t *pmd, int pkey) +{ + pte_t *pte; + int err; + + pte = pte_offset_kernel(pmd, 0); + err = set_page_pkey(pte, pkey); + + return err; +} + +static int __init set_pkey_pmd(pud_t *pud, int pkey) +{ + pmd_t *pmd; + int i, err = 0; + + pmd = pmd_offset(pud, 0); + + err = set_page_pkey(pmd, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (pmd_none(pmd[i]) || pmd_bad(pmd[i]) || pmd_leaf(pmd[i])) + continue; + err = set_pkey_pte(&pmd[i], pkey); + if (err) + break; + } + + return err; +} + +static int __init set_pkey_pud(p4d_t *p4d, int pkey) +{ + pud_t *pud; + int i, err = 0; + + if (mm_pmd_folded(&init_mm)) + return set_pkey_pmd((pud_t *)p4d, pkey); + + pud = pud_offset(p4d, 0); + + err = set_page_pkey(pud, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_PUD; i++) { + if (pud_none(pud[i]) || pud_bad(pud[i]) || pud_leaf(pud[i])) + continue; + err = set_pkey_pmd(&pud[i], pkey); + if (err) + break; + } + + return err; +} + +static int __init set_pkey_p4d(pgd_t *pgd, int pkey) +{ + p4d_t *p4d; + int i, err = 0; + + if (mm_pud_folded(&init_mm)) + return set_pkey_pud((p4d_t *)pgd, pkey); + + p4d = p4d_offset(pgd, 0); + + err = set_page_pkey(p4d, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_P4D; i++) { + if (p4d_none(p4d[i]) || p4d_bad(p4d[i]) || p4d_leaf(p4d[i])) + continue; + err = set_pkey_pud(&p4d[i], pkey); + if (err) + break; + } + + return err; +} + +/** + * kernel_pgtables_set_pkey - set pkey for all kernel page table pages + * @pkey: pkey to set the page table pages to + * + * Walks swapper_pg_dir setting the protection key of every page table page (at + * all levels) to @pkey. swapper_pg_dir itself is left untouched as it is + * expected to be mapped read-only by mark_rodata_ro(). + * + * No-op if the architecture does not support kpkeys. + */ +int __init kernel_pgtables_set_pkey(int pkey) +{ + pgd_t *pgd = swapper_pg_dir; + int i, err = 0; + + if (!arch_kpkeys_enabled()) + return 0; + + spin_lock(&init_mm.page_table_lock); + + if (mm_p4d_folded(&init_mm)) { + err = set_pkey_p4d(pgd, pkey); + goto out; + } + + for (i = 0; i < PTRS_PER_PGD; i++) { + if (pgd_none(pgd[i]) || pgd_bad(pgd[i]) || pgd_leaf(pgd[i])) + continue; + err = set_pkey_p4d(&pgd[i], pkey); + if (err) + break; + } + +out: + spin_unlock(&init_mm.page_table_lock); + return err; +} From patchwork Fri Apr 11 09:16:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F67BC36010 for ; Fri, 11 Apr 2025 09:17:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34F582801A0; Fri, 11 Apr 2025 05:17:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FEE528019B; Fri, 11 Apr 2025 05:17:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C6AA2801A0; Fri, 11 Apr 2025 05:17:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F291928019B for ; Fri, 11 Apr 2025 05:17:43 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A868FBD03F for ; Fri, 11 Apr 2025 09:17:44 +0000 (UTC) X-FDA: 83321210448.09.787BF55 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 16FC3180005 for ; Fri, 11 Apr 2025 09:17:42 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf16.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363063; a=rsa-sha256; cv=none; b=He36x22V4PRBhT5GGxBIW29giFNmk7kd78oeZx07ZeDWAYAdSCQybkywGz4bWkQGRpjzH5 yvEawI7zq4nLNYZ2vGFJsubWDk+FXc6xcoK8ewRKOTIm6yfIRcKD7Jli0hYfIw6dtMaEN1 JEQa1Cf4++H7IXZ1nGdNiQMORWoNi4A= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf16.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu1673XrmeD4DkkrL7shuinMre40KndBvHpM+lv2ifE=; b=DOToJzxbjg2xn4iV/99csavnwV2M9N1tHjv8rFOa0LFM5pUiTXTyswO5Kd3RKHQqQEHXIm gIpuxFIeptyjNbT4SFlvxjgjW50fDft7COurkpH5OWbF1DX1N6MNNkXwWtzmX0obITby34 owIUuwrafl970L/qK4W3MshWtFm0K6Y= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1DC4D106F; Fri, 11 Apr 2025 02:17:42 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8C2003F6A8; Fri, 11 Apr 2025 02:17:38 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 11/18] mm: Introduce kpkeys_hardened_pgtables Date: Fri, 11 Apr 2025 10:16:24 +0100 Message-ID: <20250411091631.954228-12-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 16FC3180005 X-Stat-Signature: amsed351j4sqbkfrnpjwkyfinnc5xjfm X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1744363062-589821 X-HE-Meta: U2FsdGVkX1+AkiFRgcxprb+QnyZ3cKYPxSTpmHshVtmotn079cMgaWCeSa6o5C/sjZlnEJIOLGY358yx1dUO+WaVpi8BDnMuizO34hCLWAtibTyCGfvv9mIirpdOdXHBMnvZQH8tEGKMwPIyuiZUfSSa2q9fJW0W9q/xCyI86J0u+N2rfls5kzea+iW/oB9mFdrgNV9k1AjSXJQoSnB84DSx0H7TvEe766w1YJyZwix21m3BEwtDbBLJevpFDbIxmSICrVUayMRfcyhg5nWmhJ5dVLicvIx3ijbIL5IoeRFuu2NcP6rzNCmEdwn5//bKzVLq9TorpdCElBDsJLjIKkGWq4ezPvzmEMytNnnexLzTvWwk6Gt+598ILlTXnupUnoa8917rGOBO1CBJbC2P8XS8VhNxE8Gf4Bml/hQNCh1AkbDRZFHHnBCBqDzk/XRD4Ut/TZVuItby8d6ViBH3Z0KLAjR8x573E3x8esaJFdMNqqaBWeas4oIPhYKNA9TAQVsbY5kdVEYaAR5Yogu0GXvzuqW15BJOtRhKfNGe5qiBzAZOhhTuKRKq8DajnGeu8+T1wclHiaujKY9zQzxUvYY1qAETdmnKL4hIgKGdPuyRw2wTQWguIKsfXbgoXVGd0Phnv8DYmocpBP2YUUAcaLqV5QhiachNiWMEakrLjNNVi41pcsJ47QdfAW55tYw7+j3w4ll0FhMGlGBr+igfhATRMgBjDlOHKtRCBNuurfQIYBWhUcMnaw7Y2Pgukq+NK/CZiptHj345S9V216AUrk4WkWxFTLecT/Ert8mhOeS/28jVa1V8kAQmm/PmMrXhLtecr75ZoYwu1H4u3HuuMVXAMckVFVVL8mAEjXQfKJI52A3KpYgPy328Ti9MyTapYA/rpo07mAkRfDIu8oQOfbkOUtWYoResvM1/rIpiarRIuuS9wQkL0Zbrf4+2dhFBpBuRrZh8PADWwSnoHjn kZAN/hcr UenN6o16zVnboq6MNEfy8SIGa1HkKYrPkX5aIednZMTemrqif6R+cLeh5AG3I5YQJHqsrarF8xJNZm+vHY1KQG6T20DGUm6oLlnssfV+tyhz/6lXu8gda1OSEtHFt91ApwO/D/jCqfUMpCPGMc11K73oby+F/QxwoElrLLfQUSP59cccXP5MS/OnKyWqEUQNlG7pCYuwzV986SXg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys_hardened_pgtables is a hardening feature based on kpkeys. It aims to prevent the corruption of page tables by: 1. mapping all page table pages, both kernel and user, with a privileged pkey (KPKEYS_PKEY_PGTABLES), and 2. granting write access to that pkey only when running at a higher kpkeys level (KPKEYS_LVL_PGTABLES). The feature is exposed as CONFIG_KPKEYS_HARDENED_PGTABLES; it requires explicit architecture opt-in by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES, since much of the page table handling is arch-specific. This patch introduces an API to modify the PTPs' pkey. Because this API is going to be called from low-level pgtable helpers, it must be inactive on boot and explicitly switched on if and when kpkeys become available. A static key is used for that purpose; it is the responsibility of each architecture supporting kpkeys_hardened_pgtables to call kpkeys_hardened_pgtables_enable() as early as possible to switch on that static key. The initial kernel page tables are also walked to set their pkey, since they have already been allocated at that point. Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 4 +++ include/linux/kpkeys.h | 46 ++++++++++++++++++++++++++++++++++- mm/Kconfig | 3 +++ mm/Makefile | 1 + mm/kpkeys_hardened_pgtables.c | 44 +++++++++++++++++++++++++++++++++ security/Kconfig.hardening | 12 +++++++++ 6 files changed, 109 insertions(+), 1 deletion(-) create mode 100644 mm/kpkeys_hardened_pgtables.c diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h index ab819f157d6a..cec92334a9f3 100644 --- a/include/asm-generic/kpkeys.h +++ b/include/asm-generic/kpkeys.h @@ -2,6 +2,10 @@ #ifndef __ASM_GENERIC_KPKEYS_H #define __ASM_GENERIC_KPKEYS_H +#ifndef KPKEYS_PKEY_PGTABLES +#define KPKEYS_PKEY_PGTABLES 1 +#endif + #ifndef KPKEYS_PKEY_DEFAULT #define KPKEYS_PKEY_DEFAULT 0 #endif diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index faa6e2615798..5f4b096374ba 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -4,11 +4,15 @@ #include #include +#include + +struct folio; #define KPKEYS_LVL_DEFAULT 0 +#define KPKEYS_LVL_PGTABLES 1 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT -#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_PGTABLES #define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ @@ -110,4 +114,44 @@ static inline bool arch_kpkeys_enabled(void) #endif /* CONFIG_ARCH_HAS_KPKEYS */ +#ifdef CONFIG_KPKEYS_HARDENED_PGTABLES + +DECLARE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_key); + +static inline bool kpkeys_hardened_pgtables_enabled(void) +{ + return static_branch_unlikely(&kpkeys_hardened_pgtables_key); +} + +int kpkeys_protect_pgtable_memory(struct folio *folio); +int kpkeys_unprotect_pgtable_memory(struct folio *folio); + +/* + * Enables kpkeys_hardened_pgtables and switches existing kernel page tables to + * a privileged pkey (KPKEYS_PKEY_PGTABLES). + * + * Should be called as early as possible by architecture code, after (k)pkeys + * are initialised and before any user task is spawned. + */ +void kpkeys_hardened_pgtables_enable(void); + +#else /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + +static inline bool kpkeys_hardened_pgtables_enabled(void) +{ + return false; +} + +static inline int kpkeys_protect_pgtable_memory(struct folio *folio) +{ + return 0; +} +static inline int kpkeys_unprotect_pgtable_memory(struct folio *folio) +{ + return 0; +} +static inline void kpkeys_hardened_pgtables_enable(void) {} + +#endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + #endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 819ef5b70695..ae1cb209a1ac 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1133,6 +1133,9 @@ config ARCH_HAS_PKEYS bool config ARCH_HAS_KPKEYS bool +# ARCH_HAS_KPKEYS must be selected when selecting this option +config ARCH_HAS_KPKEYS_HARDENED_PGTABLES + bool config ARCH_USES_PG_ARCH_2 bool diff --git a/mm/Makefile b/mm/Makefile index e7f6bbf8ae5f..0f30cc85c6f1 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -148,3 +148,4 @@ obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o diff --git a/mm/kpkeys_hardened_pgtables.c b/mm/kpkeys_hardened_pgtables.c new file mode 100644 index 000000000000..931fa97bc8a7 --- /dev/null +++ b/mm/kpkeys_hardened_pgtables.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +DEFINE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_key); + +int kpkeys_protect_pgtable_memory(struct folio *folio) +{ + unsigned long addr = (unsigned long)folio_address(folio); + unsigned int order = folio_order(folio); + int ret = 0; + + if (kpkeys_hardened_pgtables_enabled()) + ret = set_memory_pkey(addr, 1 << order, KPKEYS_PKEY_PGTABLES); + + WARN_ON(ret); + return ret; +} + +int kpkeys_unprotect_pgtable_memory(struct folio *folio) +{ + unsigned long addr = (unsigned long)folio_address(folio); + unsigned int order = folio_order(folio); + int ret = 0; + + if (kpkeys_hardened_pgtables_enabled()) + ret = set_memory_pkey(addr, 1 << order, KPKEYS_PKEY_DEFAULT); + + WARN_ON(ret); + return ret; +} + +void __init kpkeys_hardened_pgtables_enable(void) +{ + int ret; + + if (!arch_kpkeys_enabled()) + return; + + static_branch_enable(&kpkeys_hardened_pgtables_key); + ret = kernel_pgtables_set_pkey(KPKEYS_PKEY_PGTABLES); + WARN_ON(ret); +} diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index c17366ce8224..c2b0987768ca 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -334,6 +334,18 @@ config BUG_ON_DATA_CORRUPTION If unsure, say N. +config KPKEYS_HARDENED_PGTABLES + bool "Harden page tables using kernel pkeys" + depends on ARCH_HAS_KPKEYS_HARDENED_PGTABLES + help + This option makes all page tables mostly read-only by + allocating them with a non-default protection key (pkey) and + only enabling write access to that pkey in routines that are + expected to write to page table entries. + + This option has no effect if the system does not support + kernel pkeys. + endmenu config CC_HAS_RANDSTRUCT From patchwork Fri Apr 11 09:16:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9324C36010 for ; Fri, 11 Apr 2025 09:17:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F3392801A1; Fri, 11 Apr 2025 05:17:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A2FC28019B; Fri, 11 Apr 2025 05:17:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41DD22801A1; Fri, 11 Apr 2025 05:17:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2199C28019B for ; Fri, 11 Apr 2025 05:17:48 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BC110142236 for ; Fri, 11 Apr 2025 09:17:48 +0000 (UTC) X-FDA: 83321210616.28.882100C Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 2909340009 for ; Fri, 11 Apr 2025 09:17:46 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nv2oqsrFF3oYfOvHVkuk73o0k7HCEGakiogCU9B9JNg=; b=BL5Km1kDnGn2ee0dBiEMZ+9nMVmT0q+tn0rc7xgR8qo4HspAwBU0cNhuTfhvqutT0q6AQ2 pjZfw4TXJqpNYISoCSG4gRu3bEuoj7Xpda39reYm2PzKd3BQRrM7cmYRPva0pbECVA0glf mDr53nv0ST2cdrEwe0QQDAkBOpP8m90= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363067; a=rsa-sha256; cv=none; b=QF5GvlkwoUseThVJT18OVI40E2BKoGw9frVwm3iXpMUnJ9YD84SDcAVOHPb2tPF7HOUSAJ vPG92NG5oy6hKAC2tPHlaCDtluN5aqsip/gHxXjDHZ5dzcERIZoTNt/NOv32tez3IWOXHn bM2i5oud8fOdASnVp0lnqzUh+mS37fg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 30AA31596; Fri, 11 Apr 2025 02:17:46 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9E54D3F6A8; Fri, 11 Apr 2025 02:17:42 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 12/18] mm: Allow __pagetable_ctor() to fail Date: Fri, 11 Apr 2025 10:16:25 +0100 Message-ID: <20250411091631.954228-13-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: 1sgx6tw5omgrf5affp4j4hiqxjgo5g3a X-Rspam-User: X-Rspamd-Queue-Id: 2909340009 X-Rspamd-Server: rspam08 X-HE-Tag: 1744363066-517676 X-HE-Meta: U2FsdGVkX1/4fTu7t5h2Rmt5kQJSSlcyvKFI24sowwqXoyX3zeCZu3xpVaRf07MKkUhHreR42LmtEtgQt4FsJOpdpu0e4pNdlpGAsDFxy4zPrbikofsL2Z8bpgdxXEH/WyOtBTINcT04tk/Asmqf339oPGwfHCMuQnNLRyNP4BfQKFWxSLZzxr24qumic2C7KDLENj6ZX24cuWTEriejpNPp3ff7nFKUPJZmmyjqATZPQ2jG6zPekJmn4Y2eX9D4U5xiKC31zkjXEi/rBJ+NfLnLmJbqIYLenT64bKKDYLzjte2qZfSwDpT5HEAJaVodWyx9E0KUXzPy5o+C1rrMQSaeUV78XQKHfuf6SOe1Ea8PUCNtB0MmH+czS1eqRpKzpS2nUmrXp/yaA3jH2WGtLPL9tHXW56A/sq88HMKvdM422319jgsbRPU6NNIoZ2UwxwX4G0SG6Iaf5dHmufGTGCkh9gkxpfBJZSweGD7xxGJQ3KFakVdGpJF2crX/21FjKAzmGX8kgBYjIfxpkRv4oisbgr2VG9ZKMEY5uqj2EPtd7nB0zwKgu19I5EZ94gbMEDh7TuittC8MVUW7JSL3Kx+uYXyxi2E4r/saZLrHOpYK7Uo0LCKMlJlWJI1BKbDwYODNdQZctEV7zAeiB78dUKkdeiL1YulJGSaJoDzGr2QauXaCzYGP2ebtf+QvEkzxe/3MfZ3M56FnAxsEIlOfuc4+kcOLKCRBO0wUs4yf2EDvlykWs5k8ML+kcJ63Gz6fWiBtetpJ6Y6BoVz7bfdgsngud7Yc60BuvrY2jMy5i1LYzZsh9N0Esto2KvviSlhjZq1ptO77tfFcBaB50yUvD89aiByDyUgoGteJCHN61S6aeSCCVP1WqZ+OXbHsfg5qHLmnhXW2WCA4SZReEkoTx3N9eLTv9CBeSmIUQiscfREwMFs1EjdXp2qQTGcnUC/SDxlhS9+1gOfoGN7mT/L z94ejgo5 tm5KqsW5vMIdZynvUQ51FwslQsIbMYgZPogjVJ7mBuxQTvZ7JV1cGw4S71FI0RxHTR1YukCZZMcWFNXnieMQuBHSQDe+YQpIcrUh5Dx6U32/HK6eKJC2l6Cj4gy8gWlpjTgJaKIQxXLKf1cewsQUu8M3WexZLYWXVnDuXbUZMCNErcQ4oE+jaAcXS3plsDb9Q+NP+cSRBJOQ3PG4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation for adding construction hooks (that may fail) to __pagetable_ctor(), make __pagetable_ctor() return a bool, propagate it to pagetable_*_ctor() and handle failure in the generic {pud,p4d,pgd}_alloc. Signed-off-by: Kevin Brodsky --- include/asm-generic/pgalloc.h | 15 ++++++++++++--- include/linux/mm.h | 21 ++++++++++----------- 2 files changed, 22 insertions(+), 14 deletions(-) diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 3c8ec3bfea44..3e184f3ca37a 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -178,7 +178,10 @@ static inline pud_t *__pud_alloc_one_noprof(struct mm_struct *mm, unsigned long if (!ptdesc) return NULL; - pagetable_pud_ctor(ptdesc); + if (!pagetable_pud_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } #define __pud_alloc_one(...) alloc_hooks(__pud_alloc_one_noprof(__VA_ARGS__)) @@ -232,7 +235,10 @@ static inline p4d_t *__p4d_alloc_one_noprof(struct mm_struct *mm, unsigned long if (!ptdesc) return NULL; - pagetable_p4d_ctor(ptdesc); + if (!pagetable_p4d_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } #define __p4d_alloc_one(...) alloc_hooks(__p4d_alloc_one_noprof(__VA_ARGS__)) @@ -276,7 +282,10 @@ static inline pgd_t *__pgd_alloc_noprof(struct mm_struct *mm, unsigned int order if (!ptdesc) return NULL; - pagetable_pgd_ctor(ptdesc); + if (!pagetable_pgd_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } #define __pgd_alloc(...) alloc_hooks(__pgd_alloc_noprof(__VA_ARGS__)) diff --git a/include/linux/mm.h b/include/linux/mm.h index dd1b918dc294..43e35d41b850 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3077,12 +3077,13 @@ static inline bool ptlock_init(struct ptdesc *ptdesc) { return true; } static inline void ptlock_free(struct ptdesc *ptdesc) {} #endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ -static inline void __pagetable_ctor(struct ptdesc *ptdesc) +static inline bool __pagetable_ctor(struct ptdesc *ptdesc) { struct folio *folio = ptdesc_folio(ptdesc); __folio_set_pgtable(folio); lruvec_stat_add_folio(folio, NR_PAGETABLE); + return true; } static inline void pagetable_dtor(struct ptdesc *ptdesc) @@ -3105,8 +3106,7 @@ static inline bool pagetable_pte_ctor(struct mm_struct *mm, { if (mm != &init_mm && !ptlock_init(ptdesc)) return false; - __pagetable_ctor(ptdesc); - return true; + return __pagetable_ctor(ptdesc); } pte_t *___pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp); @@ -3213,8 +3213,7 @@ static inline bool pagetable_pmd_ctor(struct mm_struct *mm, if (mm != &init_mm && !pmd_ptlock_init(ptdesc)) return false; ptdesc_pmd_pts_init(ptdesc); - __pagetable_ctor(ptdesc); - return true; + return __pagetable_ctor(ptdesc); } /* @@ -3236,19 +3235,19 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud) return ptl; } -static inline void pagetable_pud_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pud_ctor(struct ptdesc *ptdesc) { - __pagetable_ctor(ptdesc); + return __pagetable_ctor(ptdesc); } -static inline void pagetable_p4d_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_p4d_ctor(struct ptdesc *ptdesc) { - __pagetable_ctor(ptdesc); + return __pagetable_ctor(ptdesc); } -static inline void pagetable_pgd_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pgd_ctor(struct ptdesc *ptdesc) { - __pagetable_ctor(ptdesc); + return __pagetable_ctor(ptdesc); } extern void __init pagecache_init(void); From patchwork Fri Apr 11 09:16:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047911 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2EDBC36010 for ; Fri, 11 Apr 2025 09:17:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38E3E2801A2; Fri, 11 Apr 2025 05:17:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3221128019B; Fri, 11 Apr 2025 05:17:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B5692801A2; Fri, 11 Apr 2025 05:17:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ECC3928019B for ; Fri, 11 Apr 2025 05:17:51 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A30111CC0CE for ; Fri, 11 Apr 2025 09:17:52 +0000 (UTC) X-FDA: 83321210784.12.6F2145E Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 2148E2000D for ; Fri, 11 Apr 2025 09:17:50 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363071; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dt/XoONL9y6fQB1g9UJTjYyCBLEHEeGxmo0qFMBni9E=; b=7doQoDlziae46FX8WLywmwU1iKp8fWkf7d6BP9XaHCGivoL8hsjFg/rS1JpStdBPPrXkOK fYM6Y30U/TjP9NeZQMwMhhtvCoGMy/1IPil61dxhwCWCOZK3UhxHOPFeD2Pdd1cRqjTtTe hjkN4kxo7h0q/0vGydpk0PYU7E9Aoks= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363071; a=rsa-sha256; cv=none; b=HahEwUn7m2+gOSgWYtUSSSN/KqzWjAFf2JYDsN5w0/sUblgejuBJWOQz4rIkqtjW4HWKQ1 I4jGfHfybjdOh/xCErTGVQF0H5rfAMRNMuBeVxWC0o+yxCdPL1K56FYTYH0sxr6kYh47mc WSnRO1Jsqz1U31p6YKpKIKAUoVgox9c= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 44125106F; Fri, 11 Apr 2025 02:17:50 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B0DF23F6A8; Fri, 11 Apr 2025 02:17:46 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 13/18] mm: Map page tables with privileged pkey Date: Fri, 11 Apr 2025 10:16:26 +0100 Message-ID: <20250411091631.954228-14-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 2148E2000D X-Stat-Signature: xfjdnsbhs6xoti95u65z5yo3q5i3jek5 X-Rspam-User: X-HE-Tag: 1744363070-484594 X-HE-Meta: U2FsdGVkX19yk1CCnH8QLXHcovGvuteYTFh40YM+BZtRk4utfC5UBK/gE/w4RfP3eBq0MXtB5qKv0z28V40q/uZbL1yvuZWWhFspzjswTnKpYooFVhzItKXbJyxFXsyrqLnKP58ntsRIwTq97i6DtoHeDyaQM6Zz/omTjM9JNX+JW9YHmcJ9Lhf6JPJKjIQbgrO5qG8n5nnINuMY0iohlxXpqf2TLINgpOoJcZaD1kpZWpovnVM5gENCTXIh8HhV7wCdZ3eEGjoNsYSMGQVyHNA1oyU/BsYUihoJuf7We3ssYT+cJu2JEmvz6/otCzp2cFV1FAq5Rp147ZWw54GCt7EfUBMEoSyUZ8lwWrxnnwj58cMsLyXY34EHMY+wLDj50BDe8D8VVXIhWcQ+yWWeE8faLf2zJVeVMBxXopsjVMY/8JEGgqwFAxP/U9FwBVAczp2g8LvxtkEXz0i/+pB5skG9xHO/WUgyYt6tW+cBKMgHbWk2UPaGiuo5zw9GrF7w3V0+ADOgeqz4F6SVpIA4RAXm1RD9mM5b/cnQSfxRltsvgkEDSGoHDgsf89AjjVRa7QiJXKNyrOOnCUgdhfMzvhN8o5cAozs9J5TqbLtwJjMFMsrYQQx6S2/cfdu7L7s7Yxttes7BHIrekoNel0to2v/vMTERlqY5dkZzcCfjV4tUHINLUFs3zStKkRd1qwHQx+fdKJL0mSyo11zNlQk0lqGlNpCE8W29qfMSbf+IoXifoHus9plr/+mrzM0xVxJI7yqrTYQ2v+WcnYN5/+ix3BjNU7ZuwiyZ5m2MAAk5GxkY4a+ihgUZ7MrwNgW0AHsziTLeiuomYTiBg4EI7FzJGEpdEuTGInHn6PPFYZbpZ700F5ZhS3JeIvg9+VXEvnDNux2Bt1rwhaSGpzBBBXdyomW7yC5D7mh2do5uJPQPthAnYJxrCTatdIoRes2uWFybDjxwyxQtVccmz/yDRzY /M+PMbyX TPlo5WfXU7+BBQHGVnepDmz03kxvaOXzzFXuyEBeacoj4p27KqDjPDQxx/0vgzld/2LijkDj85pwUvW9XYNhhy1APYuAVsloXkXfaI9vX0RiTjhqnwj6FOsfLx6m9Ta/8FWFxTLyiTm1SWCPbq/atHuMcHkdhPFhsE6pVTBjNGtedIl5mNnKN5iTTgDAoo7DxeveQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, map allocated page table pages using a privileged pkey (KPKEYS_PKEY_PGTABLES), so that page tables can only be written under guard(kpkeys_hardened_pgtables). This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 43e35d41b850..411707dabe65 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -33,6 +33,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -3083,6 +3084,8 @@ static inline bool __pagetable_ctor(struct ptdesc *ptdesc) __folio_set_pgtable(folio); lruvec_stat_add_folio(folio, NR_PAGETABLE); + if (kpkeys_protect_pgtable_memory(folio)) + return false; return true; } @@ -3093,6 +3096,7 @@ static inline void pagetable_dtor(struct ptdesc *ptdesc) ptlock_free(ptdesc); __folio_clear_pgtable(folio); lruvec_stat_sub_folio(folio, NR_PAGETABLE); + kpkeys_unprotect_pgtable_memory(folio); } static inline void pagetable_dtor_free(struct ptdesc *ptdesc) From patchwork Fri Apr 11 09:16:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047912 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2E12C369A2 for ; Fri, 11 Apr 2025 09:17:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 67D8A2801A3; Fri, 11 Apr 2025 05:17:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 62D7E28019B; Fri, 11 Apr 2025 05:17:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CF682801A3; Fri, 11 Apr 2025 05:17:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2A79A28019B for ; Fri, 11 Apr 2025 05:17:56 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C4CA65C1EC for ; Fri, 11 Apr 2025 09:17:56 +0000 (UTC) X-FDA: 83321210952.25.3C61C91 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf23.hostedemail.com (Postfix) with ESMTP id 3D283140008 for ; Fri, 11 Apr 2025 09:17:55 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf23.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363075; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cd8SVz56UFDTvkQjpDWLbjdDftI6n7YlnUbxT3vQe+A=; b=JUSCtYjgh/23CqQw7X0t1aXZ9sZ0QCi4ifQE/iotYEELcA+TlBWS0jfQ4lkV5cpDNqti1W WsIoc/kP6p/f0dA5FNuMMyZDpVH1S9vVEqjKiwDlMJjcLPdgN8LFu42Il0bq4kuAlPf/m9 iCgTkH7rHFX0rrkuGwcupqJuigaxyNc= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf23.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363075; a=rsa-sha256; cv=none; b=t62odN0dh196wXWWvqURH2qZyECGieoX7KV6w9YIAuZLcuGbjbVUZp4wHVS7GOhSmaH8bE EcsGIBHlqQfmyvR9NGFmlsaPIBrpAhtCz+7fXWya8oRMp2hqs9jk+a52e9GzIh+HLejeDS Sw1rTBvHCHXanBlmPlw8AQ23Vshf+8k= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5604C1692; Fri, 11 Apr 2025 02:17:54 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C41DE3F6A8; Fri, 11 Apr 2025 02:17:50 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 14/18] arm64: kpkeys: Support KPKEYS_LVL_PGTABLES Date: Fri, 11 Apr 2025 10:16:27 +0100 Message-ID: <20250411091631.954228-15-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Stat-Signature: mk567tkss7t9hkostj17fr9wqd8d3gjk X-Rspam-User: X-Rspamd-Queue-Id: 3D283140008 X-HE-Tag: 1744363075-413181 X-HE-Meta: U2FsdGVkX19Acn8Dc/Sz5lKgAyWTf+U8eIko0xX/JjnKWb2X9W0NNsiRhTqHy9iMB6M5SC06wGYqXe4zcM+KZPdhxSLZbIJH0ww0rmC/7cSpk6n2/S6MF0AG/vl+kB//uLjyrTkGKvMdS2vZomuR+cQWS1sT4S+JDb7t+ZxE1BfT8rTt60sQ1pD10E55xnxEQ+STQNouMGotMd0HxLJuacyLMUFJMAXrFVMdBIJCqJTJv8eFOHzZvt53eJC7eXr73s9V9ZTSp7pjhgxPsXFXE78s/S+BJROh0Dcv3IrlIb29qPAQWV4BSc4nNgeWr1ko0fX2ykmbv+lgJf8dYv5iDCP+MTh4Az2FSVs45FDSFhkVqOHMrVXWj9REzfusxV7nu0MzLYDp76mt1nZ51qd2NMUUqyPqBjWscDcJELSQnfe/pxPNLIDTwg9jADifzSrggv2uZRrI3K2RSCdXZDB7wmipjaOP5u3hmaYkpCT5Y7LFZ+uALhYAJttzYQqRbqWfsQVrotr7zZvmDk8Q8j95M6HnBVWgAHvEhaUuLVTiRNQTw64nGX6f04/CwaXlQeHrYdM0/4o9gme6WTY7tK16GjVXCyL2fg+XG4HMn+KL+it2SIOxpeSEJs2F5GrPz1QqoJ3LTxegzB99PbjwIT+I2OcuQNbDOCg4cRJKPxeTu91PHF1a/N1LaiWDTDi72hSsbN95KrFnp29YdtVryG0aPtrpdxx/07v+1h+2pjqOJngFE4mFNbJoMB8cC5fS2LU2n8uYaYrZ9Vqx8qIXglWkQZtEcSvDjy6n3vovp+7Opmhk/SdYv0zVQ4bm2YUo5R0bE5H6ggNRlNTIg7bCiwN+jDTolcy31dEn41VtubmraO30u3K7hb/ROYzyPuv4ywYH9U/EpNXb3kkWLtb8Cn0ugDJgvVjfwSnDIr6t0zkv50/D0hm4oYoLJBNydpPxarDPXnkvEGOm4sH0YwxmKpp Vr97plf9 7q6kHp09baO7PAirQJ4g+c7z8Vm7935tCtma6x/1pOQIn+C9txRfyRlqXSu64s0GBHL5VTGG6Ak3CMapYzoWPS2VYVf81NrzbvJS7iebJ/HJpsNLa1buiKKOg2iMv9IGTgB+3lI2U7umm6kQ0J4lbqbkUE4LP4iSNyrg69yu+MDUYqEXbmYsB5sPEMjbVK8NLWny3Z3xqc9dL41o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Enable RW access to KPKEYS_PKEY_PGTABLES (used to map page table pages) if switching to KPKEYS_LVL_PGTABLES, otherwise only grant RO access. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index 79ae33388088..64d6e22740ec 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -12,7 +12,8 @@ * Equivalent to por_set_kpkeys_level(0, KPKEYS_LVL_DEFAULT), but can also be * used in assembly. */ -#define POR_EL1_INIT POR_ELx_PERM_PREP(KPKEYS_PKEY_DEFAULT, POE_RWX) +#define POR_EL1_INIT (POR_ELx_PERM_PREP(KPKEYS_PKEY_DEFAULT, POE_RWX) | \ + POR_ELx_PERM_PREP(KPKEYS_PKEY_PGTABLES, POE_R)) #ifndef __ASSEMBLY__ @@ -26,6 +27,8 @@ static inline bool arch_kpkeys_enabled(void) static inline u64 por_set_kpkeys_level(u64 por, int level) { por = por_elx_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RWX); + por = por_elx_set_pkey_perms(por, KPKEYS_PKEY_PGTABLES, + level == KPKEYS_LVL_PGTABLES ? POE_RW : POE_R); return por; } From patchwork Fri Apr 11 09:16:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F146C36010 for ; Fri, 11 Apr 2025 09:18:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9F922801A4; Fri, 11 Apr 2025 05:18:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4D5128019B; Fri, 11 Apr 2025 05:18:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF66F2801A4; Fri, 11 Apr 2025 05:18:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AF6E028019B for ; Fri, 11 Apr 2025 05:18:00 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5C35914223E for ; Fri, 11 Apr 2025 09:18:01 +0000 (UTC) X-FDA: 83321211162.24.1B15950 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP id BC5B740007 for ; Fri, 11 Apr 2025 09:17:59 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363079; a=rsa-sha256; cv=none; b=FZkGzGli1QEX6foxi0j22irGPgz3wiT2P/bJaY2SwL4ZANA/PnpYIwBW+5qw4pAxxiP4Ox L5oQ5GTnj8QCOjuiGK2H/mar4oJFqqh9k8cDNcFBm4YOcB0y9H7oHaTgoTu8z6KUg8Z4Sc iBNSMzgh0UPmFJnCJbHK2i4v2jXg62E= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363079; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kfWEv7Ptu52vQV052WCDfhDg7Nqo7t04Slp2nei36Js=; b=S5Pey7vteu7u7efOOUp8zv44iiLIP/PFu9UsBvzwwVqdGJwVF4ZmG+WXxE7DQgpNAdOrXN j2ThHt1b9dp7jYQntnMo7aEoN6OTgYITgXsenmzzJ5rOeknDQXV+gLMvzFiqVVZm3vyV7g 1jgBUURsZawUoGpX5ge+Q2My3ekWmDk= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9EFFC106F; Fri, 11 Apr 2025 02:17:58 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D6AEE3F6A8; Fri, 11 Apr 2025 02:17:54 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 15/18] arm64: mm: Guard page table writes with kpkeys Date: Fri, 11 Apr 2025 10:16:28 +0100 Message-ID: <20250411091631.954228-16-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: BC5B740007 X-Stat-Signature: 1x3o6oktbnrsjkotzayxsgi7jf8rw5r5 X-HE-Tag: 1744363079-315661 X-HE-Meta: U2FsdGVkX19UrhRg8fHAvMkaLuFL7YIbbpQO0zB7Mc+y3FOnXxld23vZQHcha7xg1NuvUAZBvRYUN4OVu/zaSr+Pw/3DavMJG1I7ucjQDKXn8b9mp9Pm8TmSjZw7dYEbMoq/ePJn758i51Ztxsjt+iumjXWzZ/ZcsAJUloSjkfqt+V0e3NYP8zT4CNDiYjmhZxpV34Bo0Awk2SK8Wbbe5TXjClxkSZw2QsFEjvNsapWPgkS7k67lO4M2LN7icw03/jD+UPhajJyii6AjUnyaUl+AE2o4XMC9Ez9dPVwtrq3YCDsADx9Y+tr1XBQ591hEcIsHfmobRdZG9ycurcPVwBuNOzZnIXTIxineBOFRNELwRbwTNyYXHK5tk1ES/viOygJ8B201XTNAvjCteFOuBD3+Ns8caHeojaqAyeKTcyr33KcbwhSDa9WEWap01RpiPOpxOHqEkN31HAYKoNCs3ZRbfp/01CWbUt26s9mn65/habKWSDq5dbBnT2FVfpHi+qz3Zmf75RagIOTtUHBnhYK1fDEk2JK/Uk9DAaA5gp2cj0fQHPhoGrN1haoXZbBlsTPxshiW6r06zF5i4t8oIXKG2wtlQr+q+6dpMZdWIrPWSfbJ0YjBG8Xh0n3MNTANrClB5H/bKojzWWvGdbTGNGfM5ipV5ee5aat/wKM6KuZIns5iZMB6jyS8Lx7QCSF7vdc7lTB9ixbDV6/lIbp4na4yyPWitZjTYW3HAmHdYfL2qtehMKIBYbNv4tRU+gyWF5SFUdwqiBA3Bjx2AJRgLIGMIwkECQRrQytU/MfrFT+CAGhFl02zuxcY72AihwnsR7vy9ygbIxf9i/7Dhr4FMRambKOF6cKZr+hNf5N1IUUHoYCr0+BKL8vIWLxqbp9mmc+FBm4Qznpcnl5hPnMADPg74tUcqsOtsDQ1PgiN7rhuHbJssBnFyi/lTgRbn87IQUkO19ROZpFXhm4X5ze wvQQNyGF zUWk/d5pQx12rrFWeRofmp5eXswRsspKDyo6td393JPuIHEfZ4SgjGh5DRKQ7p9g+ZU97QygmdHaxGkw1/k3G/WwrzDcIBlS0pUISIz+6bz2WkGVECWfXYhBJCnsC82KDa6HvC/XSCXcoNOlDlGkwVu3Fj0q/J10eDyfm6wGA9/dDTQ9EGU2Ylhr8RBVvcz6ecfw1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, page tables (both user and kernel) are mapped with a privileged pkey in the linear mapping. As a result, they can only be written at an elevated kpkeys level. Introduce a kpkeys guard that sets POR_EL1 appropriately to allow writing to page tables, and use this guard wherever necessary. The scope is kept as small as possible, so that POR_EL1 is quickly reset to its default value. Where atomics are involved, the guard's scope encompasses the whole loop to avoid switching POR_EL1 unnecessarily. This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable.h | 22 +++++++++++++++++++++- arch/arm64/mm/fault.c | 2 ++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 6c780c428a6d..7929b79cd6b1 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -39,6 +39,14 @@ #include #include #include +#include + +#ifdef CONFIG_KPKEYS_HARDENED_PGTABLES +KPKEYS_GUARD_COND(kpkeys_hardened_pgtables, KPKEYS_LVL_PGTABLES, + kpkeys_hardened_pgtables_enabled()) +#else +KPKEYS_GUARD_NOOP(kpkeys_hardened_pgtables) +#endif static inline void emit_pte_barriers(void) { @@ -366,6 +374,7 @@ static inline pte_t pte_clear_uffd_wp(pte_t pte) static inline void __set_pte_nosync(pte_t *ptep, pte_t pte) { + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*ptep, pte); } @@ -843,6 +852,7 @@ static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) } #endif /* __PAGETABLE_PMD_FOLDED */ + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pmdp, pmd); if (pmd_valid(pmd)) @@ -909,6 +919,7 @@ static inline void set_pud(pud_t *pudp, pud_t pud) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pudp, pud); if (pud_valid(pud)) @@ -990,6 +1001,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*p4dp, p4d); queue_pte_barriers(); } @@ -1118,6 +1130,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pgdp, pgd); queue_pte_barriers(); } @@ -1317,6 +1330,7 @@ static inline int __ptep_test_and_clear_young(struct vm_area_struct *vma, { pte_t old_pte, pte; + guard(kpkeys_hardened_pgtables)(); pte = __ptep_get(ptep); do { old_pte = pte; @@ -1363,7 +1377,10 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma, static inline pte_t ptep_get_and_clear_anysz(struct mm_struct *mm, pte_t *ptep, unsigned long pgsize) { - pte_t pte = __pte(xchg_relaxed(&pte_val(*ptep), 0)); + pte_t pte; + + scoped_guard(kpkeys_hardened_pgtables) + pte = __pte(xchg_relaxed(&pte_val(*ptep), 0)); switch (pgsize) { case PAGE_SIZE: @@ -1434,6 +1451,7 @@ static inline void ___ptep_set_wrprotect(struct mm_struct *mm, { pte_t old_pte; + guard(kpkeys_hardened_pgtables)(); do { old_pte = pte; pte = pte_wrprotect(pte); @@ -1467,6 +1485,7 @@ static inline void __clear_young_dirty_pte(struct vm_area_struct *vma, { pte_t old_pte; + guard(kpkeys_hardened_pgtables)(); do { old_pte = pte; @@ -1514,6 +1533,7 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp, pmd_t pmd) { page_table_check_pmd_set(vma->vm_mm, pmdp, pmd); + guard(kpkeys_hardened_pgtables)(); return __pmd(xchg_relaxed(&pmd_val(*pmdp), pmd_val(pmd))); } #endif diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ec0a337891dd..5681553e2db9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -220,6 +220,8 @@ int __ptep_set_access_flags(struct vm_area_struct *vma, if (pte_same(pte, entry)) return 0; + guard(kpkeys_hardened_pgtables)(); + /* only preserve the access flags and write permission */ pte_val(entry) &= PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; From patchwork Fri Apr 11 09:16:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0135C36010 for ; Fri, 11 Apr 2025 09:18:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A5B92801A5; Fri, 11 Apr 2025 05:18:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 755A928019B; Fri, 11 Apr 2025 05:18:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F2C12801A5; Fri, 11 Apr 2025 05:18:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3CE0828019B for ; Fri, 11 Apr 2025 05:18:04 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E093E5C4F8 for ; Fri, 11 Apr 2025 09:18:04 +0000 (UTC) X-FDA: 83321211288.11.5C7EE8E Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 5AB1C40007 for ; Fri, 11 Apr 2025 09:18:03 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363083; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OWDvh2bFfbHkJemGgaNc6HxyvRtDNFqCF7qScL2ZdLs=; b=XndA3D4nYNhOTvx6HzgUd/Hc9jHX9E65SIRRL2j4o1LmTctyJnSRCt2oXEbYmhEc/PlRkU NfFHcoh2BvlJeFSE4Q0wjXHyg7G7aO3KBOiLbeEWWz4Hmox96jD48HoSVoWNvzkGsEgVHy mLMeFhvHvTxIc7BTCINqX6qebS6SR3Y= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363083; a=rsa-sha256; cv=none; b=Dq7RnvaW7aIoA5HB5fvnKNXpDKOMzHWoOBfh6wrIp/2qENaNLhLRLq+xEAPFMy+yLV7GUl 4y8gvjBoSzJv/wL/lLb359yLAMDoLZgwq1bjp6BbCVdxuoYEw96brZ4U4QNgZ75fCdLShy 3bu8d6BqBONNzwqevDLAMe2j0ArCPMg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7BCCF1596; Fri, 11 Apr 2025 02:18:02 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E90163F792; Fri, 11 Apr 2025 02:17:58 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 16/18] arm64: Enable kpkeys_hardened_pgtables support Date: Fri, 11 Apr 2025 10:16:29 +0100 Message-ID: <20250411091631.954228-17-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 5AB1C40007 X-Stat-Signature: joteh5i6mn1m1oj77sp9zpbikycpfshq X-Rspam-User: X-HE-Tag: 1744363083-724665 X-HE-Meta: U2FsdGVkX1+OcHfFJr69/cnNX+Z8nzkSeS3RWEPZFZvWBfZ/DcuGpRfeaEyLyvXY0q6eMqLnaFQoW9eao/hfn63FdpHoM4TWPsEUy0FZu0tH4B4DdCwVReql9bou/gzfBXzFD176Tawb4quBqC83l6Al/04liaqQrbbJsPnqpVmdNEE6pkRM9QVOP1smzBmN2brw4iWpLEArSyiaXtbVvJ6wUzuyn8drLAobcAXleF/2XUfzBMyhP+lY+cAz4dV01iInaEtjcTkJ+/gopWXmZDhKEBN/VofwTlI8riDHpQPwlbQhshToW+RMRQR+IFkEON4Juuu1RkUERXsXvG0gXP4WGOuvZ3HiprLnl7DS3Xie0GlcJqOh1RYUaNpRrdcyoIWYvf7ektcV1r/7+fR2beX3GaJgiLct2aL03GCBTN/iQjLvXLqhIDH8ACr6W26K6XT7BBZOK6vcx9/SYiyOpsTeCDSnNrbWuZPm7SKNYSC/uMr5Zg9bFGoCsGvFh6Rp2bqO9GJAZ/Rc0VCLNd2f97v9Ajlamsdiv/kc9BTXBaBvL4jBilPUhRsK5XV3aX5bcT6WEg7eVb5VlDXfq8x1CXmKw3m1ULok9aPKbZZRfnKa4qGowWV5kh8dAgyCKZ2mcTfrwhfQoQAZc9j5nvIF81v47VDkPF09WGyCH8yThf5AL398puEBGtTabLld4UMaR8p/UGi+/3S+9uXkxcbcgbKWQkJBGNo6ME/Id9aOsmZ2z6tY+9XD3rByeMEd278jxQimA8yOg/ST7qmevS44d5rZ0ouuf2ul3mV4qVeBzBT/K9z9ap7SgohIkfJ53FL4TPUL9yUbyL1Ku09IOYegj5NQuMTxJsLjOC+28oac5VuA+7Dk2Fsk5sPrWE1fMzJg97zLASf39WDv6bGrpSLiQknhdkab8kY2RbI807/etxiZVlt7MnE+CFBfwyN6LvpbAaittuoKoX0I278jLS3 nnQiH5kV H6QDhJ9FYywCNS/jNDJPKf7QIJQXOKsfo93AoDRAHiZkA353tBWck4bS6WFWcdnkhr4zWsWN2G4Lgu7c5pDePwzqumbjSpgV1oTVkaa5WPsRM+ugmDNygXYlLEzaUgYMAc84QWrT2KT0bWPiHL8vg4HJpvzW5LN/oPaZowX1P7eAkB8Y2zwpOXACj+otikGZRfbl1G9jEIMb7iEDQgxJPE+95frAg0+rk9hkFW82WGz9LpPvRePPxeCqjlPrrYTpcMWnxCfx7+OnZ9efQMgJAK/hjros5Fkxt0F7H X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys_hardened_pgtables should be enabled as early as possible (if selected). It does however require kpkeys being available, which means on arm64 POE being detected and enabled. POE is a boot feature, so calling kpkeys_hardened_pgtables_enable() just after setup_boot_cpu_features() in smp_prepare_boot_cpu() is the best we can do. With that done, all the bits are in place and we can advertise support for kpkeys_hardened_pgtables by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES if ARM64_POE is enabled. Signed-off-by: Kevin Brodsky --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/smp.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 5f81db8134da..3d90b895e2ef 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2199,6 +2199,7 @@ config ARM64_POE select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS select ARCH_HAS_KPKEYS + select ARCH_HAS_KPKEYS_HARDENED_PGTABLES help The Permission Overlay Extension is used to implement Memory Protection Keys. Memory Protection Keys provides a mechanism for diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 3b3f6b56e733..074cab55f9db 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -468,6 +469,7 @@ void __init smp_prepare_boot_cpu(void) if (system_uses_irq_prio_masking()) init_gic_priority_masking(); + kpkeys_hardened_pgtables_enable(); kasan_init_hw_tags(); /* Init percpu seeds for random tags after cpus are set up. */ kasan_init_sw_tags(); From patchwork Fri Apr 11 09:16:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23879C36010 for ; Fri, 11 Apr 2025 09:18:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A6CF72801A6; Fri, 11 Apr 2025 05:18:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1D8F28019B; Fri, 11 Apr 2025 05:18:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BF842801A6; Fri, 11 Apr 2025 05:18:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 69C2128019B for ; Fri, 11 Apr 2025 05:18:08 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2154AB0F40 for ; Fri, 11 Apr 2025 09:18:09 +0000 (UTC) X-FDA: 83321211498.20.E7FA00C Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP id 7A03040008 for ; Fri, 11 Apr 2025 09:18:07 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363087; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bdMqiFDP4vUIg95dX44Zi6HrmWZHm2nf87Jx6pv15M8=; b=ngY61SUfR1NBdG8cZH6l9ZYqz2MVgCK1ZL15K9aYSHnVOOzadLh268rXXOk98JZVDZnUmj 7fKgroGbTrhKdvwNyd7oCREyJMFtuwTLowfwpOt26dohWXylEYijm6IE5zPy/1yO8vnIQV NpOoNB76w26UrkNqwf7+bur3JFHt/kE= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363087; a=rsa-sha256; cv=none; b=qeLfKqrL5RTh/+ts/X0AZhrJXg1qkucMzC+MmvgPuXPwpWrR1OhrDOzDt//aIypSk8aO4I TMjuUl3JkpuhqyGRcdIu9Ntrkm3JMdV7jO2DN4U3PNdSGJsZpgYZ5qDSkLbY5cznDFRnel /Y1Agb9RHsvdwmX5mVCKJIBET98yXLk= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8E785106F; Fri, 11 Apr 2025 02:18:06 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 07A993F6A8; Fri, 11 Apr 2025 02:18:02 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 17/18] mm: Add basic tests for kpkeys_hardened_pgtables Date: Fri, 11 Apr 2025 10:16:30 +0100 Message-ID: <20250411091631.954228-18-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 7A03040008 X-Stat-Signature: p37e9csdud53ckftxyxqm6z4bmzxr38z X-Rspam-User: X-HE-Tag: 1744363087-410357 X-HE-Meta: U2FsdGVkX18tIUbczjXLPRXHovavYQQvnH2zxpF7CIhmm3wVAXgVwpfcNthedrVU2QXVZMnl6Dgfm5PQ7c4BHDH/o5Z2fFk4dIWw8jptQ9UcctmiTcUG2wasUmw1oiCQRbB9doTW1ipQY24OMLeToC4VCo7eyhDmbVs5C+ZORogdounPudQFzJn9pkheCGQ/Nsb4qsI87yulAUomoWJGT3B/TFIonDP9QLPz8CkUeoP1APvKt9tC6fQ4a48qeKTeMqfh5KEXCUOMPHSd7TvylTohnPkziQjspf9Nkd5D9lfJsAmZ1B1Xq8EP4IN4SeIrmlQPIWTDVTJgmvtACBKOkzA3RA6Zo6k2WMVk2KTU998FCGEuqq59BV6VgEg8AOKyQWJBREJMAOuttrGNG8+kZe56LF4nmoNlBQjjOp+RJBxFARE+INLaB7LMPdDPO/l+qEpjaQQpOisNb5ulI0o/lOfa1uLfUhoc8ArDvM/zXXSVEGgKJoWaU6d4IJPCkhfVi3FS8TblFPgR7uOsTBJJb3hgjfpBSz+zdaWgB6yFKgCoq5jao7+Kq2fgtXJ1ASCRNlCswxIQy34Lc5iJg9WUVrNBNmHnJPuLFD9c5kw3iYci8mcH2Ba8zNuSSikIah4RhFNZweXdHFsDydrL3h5bLRkkgPEqCU+xkGqNu7mvPm/hDWmvM3kjK52TavrOMWiFSwE1d37f1dWJdOF59DPYm9kuUR1J51sgpLa98l+FqtGyNd+/zIqlPUkw3MS4s0f/qXXvlnXCSKpQ77RE8WPU4ThyYNae9uEtJrCn0rRg3vWQfAgGEsbO25R3uCEG5D0bRlllB5f5sI3451J7yC5AqHgk1DLJZvmQERMWLjdeQA9OJJYdPlJOONB7LQrrQ1o5TKW+tR7fB8kffzpGagIefJJqBu0xSoXgbqYHsOZYeiDS+Gu92NAqLBUKwufei55OyrDiy5HDDRoFtmG11Jc +XCYoz3v b84B/tqVusLqZqkukYghXkwlqdDdQcPrDa0YbjQBRS7AYyCNh3Xwz0RrpGYWzIRScHsPxB8bIRSqHaWy74h1w1LZKYkD4bII1kEw24pft8ZR51S+I3DK+prwUbJOdds6NEErW3JBdTXoVvE2r4e6z124YNmAPSx2RDhqT68ALf2Gor4cU46TyZL5oJqVyXha8s1Wf6nzh9Tc+DzM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add basic tests for the kpkeys_hardened_pgtables feature: try to perform a direct write to kernel and user page table entries and ensure it fails. Signed-off-by: Kevin Brodsky --- mm/Makefile | 1 + mm/tests/kpkeys_hardened_pgtables_kunit.c | 97 +++++++++++++++++++++++ security/Kconfig.hardening | 12 +++ 3 files changed, 110 insertions(+) create mode 100644 mm/tests/kpkeys_hardened_pgtables_kunit.c diff --git a/mm/Makefile b/mm/Makefile index 0f30cc85c6f1..778df78c65d5 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -149,3 +149,4 @@ obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES_KUNIT_TEST) += tests/kpkeys_hardened_pgtables_kunit.o diff --git a/mm/tests/kpkeys_hardened_pgtables_kunit.c b/mm/tests/kpkeys_hardened_pgtables_kunit.c new file mode 100644 index 000000000000..6e29721262b2 --- /dev/null +++ b/mm/tests/kpkeys_hardened_pgtables_kunit.c @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include +#include + +KUNIT_DEFINE_ACTION_WRAPPER(vfree_wrapper, vfree, const void *); + +static void write_linear_map_pte(struct kunit *test) +{ + pte_t *ptep; + pte_t pte; + int ret; + + if (!arch_kpkeys_enabled()) + kunit_skip(test, "kpkeys are not supported"); + + /* + * The choice of address is mostly arbitrary - we just need something + * that falls in the linear mapping, such as the address of a global + * variable. + */ + ptep = virt_to_kpte((unsigned long)&init_mm); + KUNIT_ASSERT_NOT_NULL_MSG(test, ptep, "Failed to get PTE"); + + pte = ptep_get(ptep); + pte = set_pte_bit(pte, __pgprot(PTE_WRITE)); + ret = copy_to_kernel_nofault(ptep, &pte, sizeof(pte)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PTE write wasn't prevented"); +} + +static void write_kernel_vmalloc_pte(struct kunit *test) +{ + void *mem; + pte_t *ptep; + pte_t pte; + int ret; + + if (!arch_kpkeys_enabled()) + kunit_skip(test, "kpkeys are not supported"); + + mem = vmalloc(PAGE_SIZE); + KUNIT_ASSERT_NOT_NULL(test, mem); + ret = kunit_add_action_or_reset(test, vfree_wrapper, mem); + KUNIT_ASSERT_EQ(test, ret, 0); + + ptep = virt_to_kpte((unsigned long)mem); + KUNIT_ASSERT_NOT_NULL_MSG(test, ptep, "Failed to get PTE"); + + pte = ptep_get(ptep); + pte = set_pte_bit(pte, __pgprot(PTE_WRITE)); + ret = copy_to_kernel_nofault(ptep, &pte, sizeof(pte)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PTE write wasn't prevented"); +} + +static void write_user_pmd(struct kunit *test) +{ + pmd_t *pmdp; + pmd_t pmd; + unsigned long uaddr; + int ret; + + if (!arch_kpkeys_enabled()) + kunit_skip(test, "kpkeys are not supported"); + + uaddr = kunit_vm_mmap(test, NULL, 0, PAGE_SIZE, PROT_READ, + MAP_ANONYMOUS | MAP_PRIVATE | MAP_POPULATE, 0); + KUNIT_ASSERT_NE_MSG(test, uaddr, 0, "Could not create userspace mm"); + + /* We passed MAP_POPULATE so a PMD should already be allocated */ + pmdp = pmd_off(current->mm, uaddr); + KUNIT_ASSERT_NOT_NULL_MSG(test, pmdp, "Failed to get PMD"); + + pmd = pmdp_get(pmdp); + pmd = set_pmd_bit(pmd, __pgprot(PROT_SECT_NORMAL)); + ret = copy_to_kernel_nofault(pmdp, &pmd, sizeof(pmd)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PMD write wasn't prevented"); +} + +static struct kunit_case kpkeys_hardened_pgtables_test_cases[] = { + KUNIT_CASE(write_linear_map_pte), + KUNIT_CASE(write_kernel_vmalloc_pte), + KUNIT_CASE(write_user_pmd), + {} +}; + +static struct kunit_suite kpkeys_hardened_pgtables_test_suite = { + .name = "Hardened pgtables using kpkeys", + .test_cases = kpkeys_hardened_pgtables_test_cases, +}; +kunit_test_suite(kpkeys_hardened_pgtables_test_suite); + +MODULE_DESCRIPTION("Tests for the kpkeys_hardened_pgtables feature"); +MODULE_LICENSE("GPL"); diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index c2b0987768ca..cc3458510e97 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -346,6 +346,18 @@ config KPKEYS_HARDENED_PGTABLES This option has no effect if the system does not support kernel pkeys. +config KPKEYS_HARDENED_PGTABLES_KUNIT_TEST + tristate "KUnit tests for kpkeys_hardened_pgtables" if !KUNIT_ALL_TESTS + depends on KPKEYS_HARDENED_PGTABLES + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable this option to check that the kpkeys_hardened_pgtables feature + functions as intended, i.e. prevents arbitrary writes to user and + kernel page tables. + + If unsure, say N. + endmenu config CC_HAS_RANDSTRUCT From patchwork Fri Apr 11 09:16:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46350C36010 for ; Fri, 11 Apr 2025 09:18:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DAE5B2801A7; Fri, 11 Apr 2025 05:18:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D5C6428019B; Fri, 11 Apr 2025 05:18:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD4262801A7; Fri, 11 Apr 2025 05:18:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9A82928019B for ; Fri, 11 Apr 2025 05:18:12 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 43F0F82281 for ; Fri, 11 Apr 2025 09:18:13 +0000 (UTC) X-FDA: 83321211666.25.052D4D2 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf21.hostedemail.com (Postfix) with ESMTP id A5E511C0007 for ; Fri, 11 Apr 2025 09:18:11 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363091; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LvMNmUJw5YHbXOAt8Uq/tq9yqI5eyvYGL9Ei7Fvb7sQ=; b=7T1KCVE9Id6oOE3xzxEEADOHpAekEfeoeGGEGuTR4MbLwkDrCW1r8chLL2AV0zrMdeY6vh fz62dvaemFpeBOX2El3K1R18dEdsoIm72YaAuaZl6u8SfO9oALV73zMAMCZ3xeaSZ+SLR8 QL64VZFPF1sHxWvSumgu2x9xp2ilgA0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363091; a=rsa-sha256; cv=none; b=WkbgaaSx4QJNtSqDm3RJfki6274PvQ0387ETIbYymOFOP+MIyCGF6nbh9Z82XNbJT4cVZA a7w7qluw4JRPOUao0eMih7YSdm79x0bEzWLzzIRgdTqvMQPM9eZw0IB0RM8wyGiUdCF8qM qJbUZrFnpVJSjWt2I3Uxo6X9aIUWZeY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A04EC1596; Fri, 11 Apr 2025 02:18:10 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1A69D3F6A8; Fri, 11 Apr 2025 02:18:06 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 18/18] arm64: mm: Batch kpkeys level switches Date: Fri, 11 Apr 2025 10:16:31 +0100 Message-ID: <20250411091631.954228-19-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: 5bdbre3nwjgdspxi3qiausdra3q6cfuw X-Rspam-User: X-Rspamd-Queue-Id: A5E511C0007 X-Rspamd-Server: rspam08 X-HE-Tag: 1744363091-79388 X-HE-Meta: U2FsdGVkX1+HtKM16tMBrtnK9CJKpiP1anX3DPJpycdj1x4FC96CMd28DAjhF5DhEiAgU8Rctoqc3pBeYfRUQHHfw3wVpq916fjGLr8Hx4Z0LQTPDRGPrZ3KxbnOcAbMlry+A0t2VQ8V/zGXMeGfE3PJxL/6aSqA54B69S732HuayBy4iJxSWx9aQ8e5e7+ZVAGr4FtzB9P8Y40eB8yQP0boVCmMLRyzxz5s5e5j8ncC/cJQUMqg0GFCoArzlA8AL22GDQ77HZ+hItDJ+yvayXFuTd+Of5RJ6iEzB7Rz8RldrLGnA+UOPy9z685NNWnbsu/BimLq27304KehR9Q2JtHB4wjgvSzOkCYR9AtQRostus85B5QyKph9KTjnVjKp768v6yrNje6lAtE5U1i8MuqEYfXDZuivq7iPkhctfkD7YqjjMNqW9WPBH8oCpJDWI5p3MZqWII+evoTtMkBQd558yP/dmOaR9SXYUjIek1vEtgMAencbVeSalwIJ7HDCe38TjYHkOYKEOgVz2zm4EPuj+b9qV9QhGbAlxUZoRQxJuedN380gc5KJOh8FtZ66RZNK98PrrteZh3aMv5grRIRtekOE615P2V/OwfVwWd2j5rgTdZ+GmphLpCy4FSjvLHpsnWZYu6yuXUme+nXo87ptFWlwLuYyT9uyUN194Ho7iYWUm51ozRbHZAxAMKOWfcNqo530CUs0AC8hPb9Bf9bYip4wF4a3d4jBpJjHl02ef+a1q1h16Nlo2wg/7vcKIB+xvRYImkJejYdOsRnuG+r420hOUDfQW9oja7NNgYRaVm3S+phGjZJRUdsi7NzFj0V1H4mYLxIW6fA5I048UxkqRV+OIy1PE4kPqF5QgoS62nnDI7bZsLxKuh4g+UAlEyO2a0mUIhDnHJ8VZOkavCbmKPypXADyEHI+j2S4MA3TThvvopFIVmPIIN5+Fd7bXjNbcd3DSYF6SznvRkj D806cpyr 8JZhRfK/mbRVRZ7LRUxRnrrzwtzVTXa03ToQtv1DqW/eJAOW88+xREFGC/aFneZQFv9rDAnkldSv/9GsErZ3dOV68WaBrROtupqfRTcYvvUkAYq3XhnG/bIo1MT7FIBbQX8DHBWKg8YFSWnwp8djqEU7jLE/9PMrDByIiSCaN40QEpZ4KWm+miQpe0g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The kpkeys_hardened_pgtables feature currently switches kpkeys level in every helper that writes to page tables, such as set_pte(). With kpkeys implemented using POE, this entails a pair of ISBs whenever such helper is called. A simple way to reduce this overhead is to make use of the lazy_mmu mode, which has recently been adopted on arm64 to batch barriers (DSB/ISB) when updating kernel pgtables [1]. Reusing the TIF_LAZY_MMU flag introduced by this series, we amend the kpkeys_hardened_pgtables guard so that no level switch (i.e. POR_EL1 update) is issued while that flag is set. Instead, we switch to KPKEYS_LVL_PGTABLES when entering lazy_mmu mode, and restore the previous level when exiting it. Restoring the previous kpkeys level requires storing the original value of POR_EL1 somewhere. This is a full 64-bit value so we cannot simply use a TIF flag, but since lazy_mmu sections cannot nest, some sort of thread-local variable would do the trick. There is no straightforward way to reuse current->thread.por_el1 for that purpose - this is where the current value of POR_EL1 is stored on a context switch, i.e. the value corresponding to KPKEYS_LVL_PGTABLES inside a lazy_mmu section. Instead, we add a new member to thread_struct to hold that value temporarily. This isn't optimal as that member is unused outside of lazy_mmu sections, but it is the simplest option. A further optimisation this patch makes is to merge the ISBs when exiting lazy_mmu mode. That is, if an ISB is going to be issued by emit_pte_barriers() because kernel pgtables were modified in the lazy_mmu section, we skip the ISB after restoring POR_EL1. This is done by checking TIF_LAZY_MMU_PENDING and ensuring that POR_EL1 is restored before emit_pte_barriers() is called. Checking TIF_LAZY_MMU flag in all pgtable writers is currently overkill, as lazy_mmu sections are only used at the lowest level of page tables. In other words, set_pgd() (for instance) will never be called with TIF_LAZY_MMU set. However, such higher-level helpers are called relatively infrequently and the overhead of checking a TIF flag is low. The flag is therefore checked in all cases for simplicity's sake, just like in [1]. [1] https://lore.kernel.org/linux-mm/20250304150444.3788920-1-ryan.roberts@arm.com/ Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable.h | 37 +++++++++++++++++++++++++++++- arch/arm64/include/asm/processor.h | 1 + 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7929b79cd6b1..61dee76be515 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -43,11 +43,40 @@ #ifdef CONFIG_KPKEYS_HARDENED_PGTABLES KPKEYS_GUARD_COND(kpkeys_hardened_pgtables, KPKEYS_LVL_PGTABLES, - kpkeys_hardened_pgtables_enabled()) + kpkeys_hardened_pgtables_enabled() && + !test_thread_flag(TIF_LAZY_MMU)) #else KPKEYS_GUARD_NOOP(kpkeys_hardened_pgtables) #endif +static void kpkeys_lazy_mmu_enter(void) +{ + if (!kpkeys_hardened_pgtables_enabled()) + return; + + current->thread.por_el1_lazy_mmu = kpkeys_set_level(KPKEYS_LVL_PGTABLES); +} + +static void kpkeys_lazy_mmu_exit(void) +{ + u64 saved_por_el1; + + if (!kpkeys_hardened_pgtables_enabled()) + return; + + saved_por_el1 = current->thread.por_el1_lazy_mmu; + + /* + * We skip any barrier if TIF_LAZY_MMU_PENDING is set: + * emit_pte_barriers() will issue an ISB just after this function + * returns. + */ + if (test_thread_flag(TIF_LAZY_MMU_PENDING)) + __kpkeys_set_pkey_reg_nosync(saved_por_el1); + else + arch_kpkeys_restore_pkey_reg(saved_por_el1); +} + static inline void emit_pte_barriers(void) { /* @@ -83,6 +112,7 @@ static inline void arch_enter_lazy_mmu_mode(void) VM_WARN_ON(test_thread_flag(TIF_LAZY_MMU)); set_thread_flag(TIF_LAZY_MMU); + kpkeys_lazy_mmu_enter(); } static inline void arch_flush_lazy_mmu_mode(void) @@ -93,6 +123,11 @@ static inline void arch_flush_lazy_mmu_mode(void) static inline void arch_leave_lazy_mmu_mode(void) { + /* + * The ordering should be preserved to allow kpkeys_lazy_mmu_exit() + * to skip any barrier when TIF_LAZY_MMU_PENDING is set. + */ + kpkeys_lazy_mmu_exit(); arch_flush_lazy_mmu_mode(); clear_thread_flag(TIF_LAZY_MMU); } diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 0afaf96ca699..14a4b483098d 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -186,6 +186,7 @@ struct thread_struct { u64 tpidr2_el0; u64 por_el0; u64 por_el1; + u64 por_el1_lazy_mmu; #ifdef CONFIG_ARM64_GCS unsigned int gcs_el0_mode; unsigned int gcs_el0_locked;