From patchwork Mon Feb 3 10:18:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01B71C02192 for ; Mon, 3 Feb 2025 10:19:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B2F4280010; Mon, 3 Feb 2025 05:19:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 86434280002; Mon, 3 Feb 2025 05:19:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70351280010; Mon, 3 Feb 2025 05:19:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 51A87280002 for ; Mon, 3 Feb 2025 05:19:56 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 444484C821 for ; Mon, 3 Feb 2025 10:19:48 +0000 (UTC) X-FDA: 83078237256.25.D077614 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id A7B2CA0003 for ; Mon, 3 Feb 2025 10:19:46 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738577986; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fSKs8pcmNTnnytgS8+O5jxAzIy+XDlinRu8X5YqLwhU=; b=IcYjWHZLS1/O6G2QbMo/eCdiuRciy+Jy/Ypsfh5KM7ZCDJ/+Za+FFlRuhGQu8Ky7+B0BFG TJIl81Zg2eO2z1oLpVqy3td3qWPfjunJlXzjlXUCi0xiPEFqRHi6Xfax+vh1IN3rm7FxRr HpsV3il5wF2Qai0czM8p/rHDCBazdBU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738577986; a=rsa-sha256; cv=none; b=kjCMHYc9mk1rbUbsx5i0Mrwu6CNCnReiR5CQ23gqyVGkiX0KKjjBBDJHUidocPmP27uFib PQPW3shrHO42xkLaTuFy+bz89Rn2YBHgydofgEVyMt9XPrioL8qcu2FK3OQoPqykXyjyH8 MkpUBVw+4FokzKKrn8QUq4yXwwfvclk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 50A281476; Mon, 3 Feb 2025 02:20:10 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5E69A3F63F; Mon, 3 Feb 2025 02:19:42 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 01/15] mm: Introduce kpkeys Date: Mon, 3 Feb 2025 10:18:25 +0000 Message-ID: <20250203101839.1223008-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A7B2CA0003 X-Stat-Signature: wmn4cmxoez97t8zwcwip6ei96d7j3oga X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1738577986-857202 X-HE-Meta: U2FsdGVkX1+AR/Sbsj/pjmXEw7/yxEIHsQXnY9EL2DSr72pd7EQc8IJG6PHq9g9AqPPW+9f5Zx++dC5rireJaW32sfPxtFy56SdsseNQf4yODeQT4CNIhvCQ8ZOnvm2MYxeqOaiZQy5KRUrYySK89j3IdDfrudhO3wbIyCcjdZAASWxW/5XEy1oE3Wg57xbGPDHOYs8NM5VgIAeiPMvNgKxP/eVM3xXVojqDJ7dAijQci6P3U6rRJkNVreGSlNTN6lQ4d/4gNbj0RFml58Ql0sFwmQxkh5N4g0nhJnEsdgzlOAPC++a5l79AEocjJuXqhqaKJEjRc9ePLXWdc78JSDwDQtlWWXuYqcVO/ss8aJSGtwN5XI/mmSm6Ys6H+8i3zE9hNiHQHXqIJ9jFxNKF3uxYSZPzkIUZJnr30H+e2Yjq/Nl/ABlzV8clu2fA8x1AOnkDhmaMvJWbPPypp1JGEgMOFRppFyt0zUjR5DJoucbkN8zk343vxGBMCMsqMcKdAitC3cRFIv+jD8VLcV/27zI07ElaVawBQFyQwBIGWWuLocHokoNJKJYYrqh3DnEBj0InzN9J2SjBJz0P/oRgjgzlk3xkXZJGo/2/4LcoVy2zqDo3EsAGAkJ7gN19Bd8nweO81zrWUty3CVOu5djbriQ0o9XCKlLHBWx5orjHMnBKnckEBWBIJ1txgTziIjf7wQgHCzCi4RL7OHCMgjm6YOSIMVU7oxDiK1pPHFGJSwvq6jTm6EIop9tyH/94TIODJfKVJo0EZYdPpPOEeyEgBoUUOsVkIfKnLQjcQyWIG+OQobpfFzrfTrT1pUu29f7TzyU0gxZhE6CJ2VYTP0fouW8Ji+r7nxGUWpwgI967Pl6vT+75ibFupKePui9r+NCIOP3J+es1PeO1DbGn3KmjU6mUAcdkcbN61nHgS8TVubv5lJpoe0QsBsFwQiuapccpiMfvXlaMFCAGlVoEnDb we3JQ0O8 /NSW5OEwasgJM76qvMfNcQhj70gzO96h7vdnIBhj3TLwiG+mSUQvKhIMd3KrEPx7bymqb6gpDr25nlEJCf7DZFrsITn6m5P32sgPjUnAwkdoVcsHIumu8jBwjtIw6PU1vvt6PmTFO7rKCWigBSQZhTfoD3xG/MP1UMD/vAhBXZR0NRI0SO7T5z1CiriPchteoPtcQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys is a simple framework to enable the use of protection keys (pkeys) to harden the kernel itself. This patch introduces the basic API in : a couple of functions to set and restore the pkey register and macros to define guard objects. kpkeys introduces a new concept on top of pkeys: the kpkeys level. Each level is associated to a set of permissions for the pkeys managed by the kpkeys framework. kpkeys_set_level(lvl) sets those permissions according to lvl, and returns the original pkey register, to be later restored by kpkeys_restore_pkey_reg(). To start with, only KPKEYS_LVL_DEFAULT is available, which is meant to grant RW access to KPKEYS_PKEY_DEFAULT (i.e. all memory since this is the only available pkey for now). Because each architecture implementing pkeys uses a different representation for the pkey register, and may reserve certain pkeys for specific uses, support for kpkeys must be explicitly indicated by selecting ARCH_HAS_KPKEYS and defining the following functions in , in addition to the macros provided in : - arch_kpkeys_set_level() - arch_kpkeys_restore_pkey_reg() - arch_kpkeys_enabled() Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 17 ++++++ include/linux/kpkeys.h | 113 +++++++++++++++++++++++++++++++++++ mm/Kconfig | 2 + 3 files changed, 132 insertions(+) create mode 100644 include/asm-generic/kpkeys.h create mode 100644 include/linux/kpkeys.h diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h new file mode 100644 index 000000000000..ab819f157d6a --- /dev/null +++ b/include/asm-generic/kpkeys.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_GENERIC_KPKEYS_H +#define __ASM_GENERIC_KPKEYS_H + +#ifndef KPKEYS_PKEY_DEFAULT +#define KPKEYS_PKEY_DEFAULT 0 +#endif + +/* + * Represents a pkey register value that cannot be used, typically disabling + * access to all keys. + */ +#ifndef KPKEYS_PKEY_REG_INVAL +#define KPKEYS_PKEY_REG_INVAL 0 +#endif + +#endif /* __ASM_GENERIC_KPKEYS_H */ diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h new file mode 100644 index 000000000000..62f897c65658 --- /dev/null +++ b/include/linux/kpkeys.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _LINUX_KPKEYS_H +#define _LINUX_KPKEYS_H + +#include +#include + +#define KPKEYS_LVL_DEFAULT 0 + +#define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT + +#define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ + __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ + DEFINE_CLASS(name, u64, \ + restore_pkey_reg, set_level, set_arg); \ + static inline void *class_##name##_lock_ptr(u64 *_T) \ + { return _T; } + +/** + * KPKEYS_GUARD_NOOP() - define a guard type that does nothing + * @name: the name of the guard type + * @cond_arg: an argument specification (optional) + * + * Define a guard type that does nothing, useful to match a real guard type + * that is defined under an #ifdef. @cond_arg may optionally be passed to match + * a guard defined using KPKEYS_GUARD_COND(). + */ +#define KPKEYS_GUARD_NOOP(name, ...) \ + __KPKEYS_GUARD(name, 0, (void)_T, ##__VA_ARGS__, void) + +#ifdef CONFIG_ARCH_HAS_KPKEYS + +#include + +/** + * KPKEYS_GUARD_COND() - define a guard type that conditionally switches to + * a given kpkeys level + * @name: the name of the guard type + * @level: the kpkeys level to switch to + * @cond: an expression that is evaluated as condition + * @cond_arg: an argument specification for the condition (optional) + * + * Define a guard type that switches to @level if @cond evaluates to true, and + * does nothing otherwise. @cond_arg may be specified to give access to a + * caller-defined argument to @cond. + */ +#define KPKEYS_GUARD_COND(name, level, cond, ...) \ + __KPKEYS_GUARD(name, \ + cond ? kpkeys_set_level(level) \ + : KPKEYS_PKEY_REG_INVAL, \ + kpkeys_restore_pkey_reg(_T), \ + ##__VA_ARGS__, void) + +/** + * KPKEYS_GUARD() - define a guard type that switches to a given kpkeys level + * if kpkeys are enabled + * @name: the name of the guard type + * @level: the kpkeys level to switch to + * + * Define a guard type that switches to @level if the system supports kpkeys. + */ +#define KPKEYS_GUARD(name, level) \ + KPKEYS_GUARD_COND(name, level, arch_kpkeys_enabled()) + +/** + * kpkeys_set_level() - switch kpkeys level + * @level: the level to switch to + * + * Switches the kpkeys level to the specified value. @level must be a + * compile-time constant. The arch-specific pkey register will be updated + * accordingly, and the original value returned. + * + * Return: the original pkey register value if the register was written to, or + * KPKEYS_PKEY_REG_INVAL otherwise (no write to the register was + * required). + */ +static inline u64 kpkeys_set_level(int level) +{ + BUILD_BUG_ON_MSG(!__builtin_constant_p(level), + "kpkeys_set_level() only takes constant levels"); + BUILD_BUG_ON_MSG(level < KPKEYS_LVL_MIN || level > KPKEYS_LVL_MAX, + "Invalid level passed to kpkeys_set_level()"); + + return arch_kpkeys_set_level(level); +} + +/** + * kpkeys_restore_pkey_reg() - restores a pkey register value + * @pkey_reg: the pkey register value to restore + * + * This function is meant to be passed the value returned by kpkeys_set_level(), + * in order to restore the pkey register to its original value (thus restoring + * the original kpkeys level). + */ +static inline void kpkeys_restore_pkey_reg(u64 pkey_reg) +{ + if (pkey_reg != KPKEYS_PKEY_REG_INVAL) + arch_kpkeys_restore_pkey_reg(pkey_reg); +} + +#else /* CONFIG_ARCH_HAS_KPKEYS */ + +#include + +static inline bool arch_kpkeys_enabled(void) +{ + return false; +} + +#endif /* CONFIG_ARCH_HAS_KPKEYS */ + +#endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 1b501db06417..71edc478f111 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1147,6 +1147,8 @@ config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS bool +config ARCH_HAS_KPKEYS + bool config ARCH_USES_PG_ARCH_2 bool From patchwork Mon Feb 3 10:18:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22773C02196 for ; Mon, 3 Feb 2025 10:20:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E0E1280012; Mon, 3 Feb 2025 05:20:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 69C3B280013; Mon, 3 Feb 2025 05:20:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 507D3280012; Mon, 3 Feb 2025 05:20:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 35BDD280002 for ; Mon, 3 Feb 2025 05:20:12 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 14FB74CB0A for ; Mon, 3 Feb 2025 10:19:52 +0000 (UTC) X-FDA: 83078237424.25.E0B35E3 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf18.hostedemail.com (Postfix) with ESMTP id 5ECA31C000C for ; Mon, 3 Feb 2025 10:19:50 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf18.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738577990; a=rsa-sha256; cv=none; b=ZQlxfSUUDOoXvI0keKOTdBrSBOmVannVCb8sDzDo/xJc3y+GJW8Uw/Nc/aoklxPMBS5HX/ Bla8wu/J9lvGrRP9WefZ6QG0wi/19oD/DuTOajLZfHXIcC6kj+w+mh5a2P/27ijufrvPtv 7f7rTBIWyUrWk9pzTpN6bBOqQVxkayY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf18.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738577990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DAvN/ymhOC9RH1764Pqdo1P8xtkdW4aJF6g43VRuQIg=; b=Bh3H69YCeZxdDk6bRthYhq3cZYh3cBxce83v4pTjyzzUdnhOj9eL5+tEL6zbF5N9kyZ7es a4RlxwA09r5fgXT7kKUde7Ee92qFQO2ULvj55494YhMr2liBqAQJ199iVFJe4zBjqTjH/6 iccjgWqLYTb1ErZuaF5DixPU/o89O64= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 249791682; Mon, 3 Feb 2025 02:20:14 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3298A3F63F; Mon, 3 Feb 2025 02:19:46 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 02/15] set_memory: Introduce set_memory_pkey() stub Date: Mon, 3 Feb 2025 10:18:26 +0000 Message-ID: <20250203101839.1223008-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5ECA31C000C X-Stat-Signature: 8ysmea9mixwcjofnqgajxwpgbm5djhwd X-Rspam-User: X-HE-Tag: 1738577990-131407 X-HE-Meta: U2FsdGVkX1+f7Zt6J2YLAdqk2IjeqMK3eO5yQjqSMdHP+4ySbaUqvUfKwMGutsbm96kTVMF1qHFYTuzT47Mc+l8qUIFHFPAPGAD2GI5eHFYlHrLdYmk/Tp9fn+s1WXSGMKloB21GOURxZQS7JCmnVAK7I8LgoQneNdGdnYEl5nps5eK6BaHZx0kwkGrYDJlaUtz4BV3ra/+/9i2q66TPcUHxLvYd+2lp/qWA64XTqor+1U4UEUc/DpxfN2YhHqwW//RhXEbTFNDXEVWHepo7tmC2Tfh9bJ/DmssXPRQgc9P5CpphZg3r/UV9UYPPxVcos5mmjoIawGPYgDd/zSltxllxHfnq/J6e+YEQMvEBdJmCbIGDc2soPk/9su/X26qRuk9H0s8yshhB6pBQbKS5UZBFAqkwTfL9cazoYoE1oZhsnxn9Cc7TlUqHJ7j66VO6nfW0NOKVRX6vzjCkJo4goov3kVu/7Ze3xNEvU8WmrfYhZ1W+HLrV5euHv+dfo7EbxVA1zaJeNTvZItrw2i8HgbNNa5w9lEzpA8hgBQYFqyZnWCl+AKtVv4A7ZoM8qD8qSig+QS+g1x6JTxcRG/t13shw1Vq0+XDoPs6GWbOvE+M6VgISmjRLiMwNUBV/L7Z1rmAZncUGGlbk7MXtNixYDd2WpNj77noHAb3DS5Alkx0OFB/oOOw0jclK3QrHVz9ZaKzOR8KcXIHnC2Yz7cNOPr5XrnS6mwamESiPqYkILo8CEKDVrxSv1+eZA7xFS78qYEHuaqVkhcyGFszBQsLdDBgJ+3VzZWY0I4Zn/qH2TDLmjB63yZzB8RsZjJZfWY9IrvvqzyMzVP6xLbE/MNSYIoDcQpGLYF0hvCcRZcpEPrONpWJyUyGo2OwV3ssUUMKFPkzmsAosLd437junpEGVe89O9FhmJ8WCNB+IzaowwyRr4Xv3VhT6bQZ48hXtnBfRbHYDyFzLlgcM/JypDRU N3gy0Igp /kPIspxZTZFb1wH3ZJoz6ybVv9cCYHqbnT6B55K0mEFd2FOoDc8ELRoQ0KCXV5k7RKUa1vdmm2R0JbKaD7sXCPVhmeuVSdo83Ki0fzvE+KrbhxuDjo9Fz2a0L817kAvGxLkCovGKInnpe0D1epg+h49Obrh1fp3H/L85XG0YnSTe1aLI8+zOpYO8I06JcuWVT3yffOt+frgIdaTk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new function, set_memory_pkey(), which sets the protection key (pkey) of pages in the specified linear mapping range. Architectures implementing kernel pkeys (kpkeys) must provide a suitable implementation; an empty stub is added as fallback. Signed-off-by: Kevin Brodsky --- include/linux/set_memory.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 3030d9245f5a..7b3a8bfde3c6 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -84,4 +84,11 @@ static inline int set_memory_decrypted(unsigned long addr, int numpages) } #endif /* CONFIG_ARCH_HAS_MEM_ENCRYPT */ +#ifndef CONFIG_ARCH_HAS_KPKEYS +static inline int set_memory_pkey(unsigned long addr, int numpages, int pkey) +{ + return 0; +} +#endif + #endif /* _LINUX_SET_MEMORY_H_ */ From patchwork Mon Feb 3 10:18:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B554C02192 for ; Mon, 3 Feb 2025 10:20:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A87D1280011; Mon, 3 Feb 2025 05:20:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A39FC280002; Mon, 3 Feb 2025 05:20:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D863280011; Mon, 3 Feb 2025 05:20:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 71DA2280002 for ; Mon, 3 Feb 2025 05:20:11 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EA023C2687 for ; Mon, 3 Feb 2025 10:19:55 +0000 (UTC) X-FDA: 83078237550.01.E5EED50 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf18.hostedemail.com (Postfix) with ESMTP id 5CA4E1C0007 for ; Mon, 3 Feb 2025 10:19:54 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738577994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j8fuaCOyRTWXJGQpe+LY3cOq0r0pHJ9P9FMQOSh2CCw=; b=44z2iAxREtWH3L6iJl8aympKU1UBs1l4DmrHgBDP1pZm3/bnxD6r4zByjDpMHx6zv7SQdY RAXwmfSID/H+TKQMiopIEM7kaSHH5Zr1HlPw4pQWgQl/PH5v4WnlZ5AKquQ92xzPgRW6w+ 2i9rsHN88r79MHxfzCeslVAhWnkdahI= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738577994; a=rsa-sha256; cv=none; b=a5iBsIrAEq6Tk6tvzHIUKU0E90RiNeLZK7GlyxJ1JNKzcOBLxtImEtL/m9DECmYofRUgoW tMM6PdDZVM8OG5czz6QvJ5PCxBZaG9ORT5z3+MGGl1K67ChVY0QMn9kX1it9AICbgfIYuD NAYOoKY4KdBC0UvlmvZZXdVRU+Lc+4E= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EBC3B1A32; Mon, 3 Feb 2025 02:20:17 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 066F33F63F; Mon, 3 Feb 2025 02:19:49 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 03/15] arm64: mm: Enable overlays for all EL1 indirect permissions Date: Mon, 3 Feb 2025 10:18:27 +0000 Message-ID: <20250203101839.1223008-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5CA4E1C0007 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: eop718qt5wibns3tfn1kjqxtxrxe34ry X-HE-Tag: 1738577994-126575 X-HE-Meta: U2FsdGVkX18hdco4RAqAFyoI9jqrJudROcVAqnqh63oQJ+lGNYsugQntDYwdBGimD7fMbdA4GjYpM1gX6Jd57PufIfK5by/j0xABKTEVkB79DtGBx24QArZSkNinmLfR+AelS6E9reMJWHSgr8WBI7KmyV7ccqiGT4+/C/wNIEqVXiNmZg57r2IU7v6hepEmqiq3kmxrSfbLkQwScw03xF0dl0R9oN+i+EidlbywewGbRknWF0mz00HsxgdfNItmvI6YumwaP6yh9llXJO0qMsW0c16TagK/AQe2V7e7CqGIOzbrIoS8YGEexPQ9xc19IehozjubHxvGz+CZf/hxb7WEg3EZPWTSJVT/8Mq2u6sTENih8gwaZ5FuvRSoUMdllewYbxuTYhCy71e/O/KrWg6mCkjDIiOwzV3J31EqUwS424xY1xLpi6zT2qVb08bN87A7VAtBTtADLW95ZiG7W3DncizrhpK97/LS5fis9P/P/5NeHwu2bNZ09Dn1jn7Q5C5UwpjLMiHuw/mQzGCW1qEMPPr6svgUTZ+6isflTRdvKzd3ztPqwT74lOexTRfARURRYEnjB9DWeahtSSyYz/aU5lWKvEW8RLttQUrqLoy+lEq+vXFlEnZlBXE3Fidekd6BYyiU1Pvb7CZ6PPNnR49/UE6mg5vBUcP3cQnby0xG9LaAAnwrYIue3YtK9ynLXd/yWmqcvgtybHl57GiSo8e9PEzaqQRlH9xVOO68dDvKG+4Q99pqtrqcPEqLAmrHpSrhu1bh1fCoStGtHl9EeQTQhCsnCmYG/WTlmK0VmmDkiAbUnSzUwJiFWlD73nRAmJKgs1LL4/xQ/3T5SZPwpn36UQHs/4gkqfD9aENSYMSce99PG+Q2WeJiRnMl7kNANBg9vt6U4D0PdO53mPMc7BS7JHG58wSI9fXNNmzErKGyGlbp3Jrd9OYFVj1wJIWPSOaUQ/l/2jWvjaot/T6 rzdJNUGw Yp7CuQsT8Ni1wHuM03LdXvIJnRdhP8plx8cU4IztV085AcPzJYmTQ2NBaE0XmrUBY/QgBMsyQSqjmlPyMGFuy8k17VWVwSO/Rb/iaiA9e+En8DlMk3cIpxP6iQg0HKkLN1iunOsbR5ZQrImbJ6hatoMqeDQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation of using POE inside the kernel, enable "Overlay applied" for all stage 1 base permissions in PIR_EL1. This ensures that the permissions set in POR_EL1 affect all kernel mappings. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable-prot.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index a95f1f77bb39..7c0c30460900 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -181,13 +181,13 @@ static inline bool __pure lpa2_is_enabled(void) PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_NONE_O) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY), PIE_R) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_ROX), PIE_RX) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_EXEC), PIE_RWX) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_RO), PIE_R) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL), PIE_RW)) + PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY), PIE_R_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_ROX), PIE_RX_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_EXEC), PIE_RWX_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_RO), PIE_R_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL), PIE_RW_O)) #endif /* __ASM_PGTABLE_PROT_H */ From patchwork Mon Feb 3 10:18:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A10F0C02192 for ; Mon, 3 Feb 2025 10:20:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E6FD280014; Mon, 3 Feb 2025 05:20:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 27094280002; Mon, 3 Feb 2025 05:20:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C265280014; Mon, 3 Feb 2025 05:20:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D6DE3280002 for ; Mon, 3 Feb 2025 05:20:25 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 04AAF1225F4 for ; Mon, 3 Feb 2025 10:19:59 +0000 (UTC) X-FDA: 83078237760.08.4385ACE Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 4C3D640009 for ; Mon, 3 Feb 2025 10:19:58 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf12.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738577998; a=rsa-sha256; cv=none; b=uBKhquXGWX3GYkhAhxlbiY2kfMhjYzGoRuMX0KoEr5V23CjmaCnk0lc2A9ib9EZeUiBaJA 4/VLf8iGuajWZJXm5d09uvfJzWBGuv5qahcpmTH7X7au7nXC2SblbMk4+zN7ec5VaJ1+lZ wb7hfWyCGHb3EseJmwTCAwoegUbF09c= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf12.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738577998; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=95g1WdvJ15ICDChHvnYK8BK/i5y+G7j4b0SSgWLyqq8=; b=Vtf4VEp6JiNh/JH/GxPZO8MOhTwcR7RO7o8kvNoB9Zy/Uc5glFgA7gDZQHo+sM0wdX5At2 qr6zYYaV6rSVKpeGzpPRKFcXmhhBaR11PhbDf8+PwMQppyoFPkTSwY2tt19CUF/5168AJj XKRsp8HqVxJiXEjxywkQdvxqqu44Fjc= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BF4FA11FB; Mon, 3 Feb 2025 02:20:21 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CD9ED3F63F; Mon, 3 Feb 2025 02:19:53 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 04/15] arm64: Introduce por_set_pkey_perms() helper Date: Mon, 3 Feb 2025 10:18:28 +0000 Message-ID: <20250203101839.1223008-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4C3D640009 X-Stat-Signature: npje75fumbd7qejz31czz7g6hzsykoyt X-HE-Tag: 1738577998-353607 X-HE-Meta: U2FsdGVkX19wfUkSSHINaz9/tPwCXmBpzYZKtqZRYPqgy7Wtw/S5NcZ/ofqQr8tnHonGxMby1VxMRI6PPSjGKPEc7XqfJX9G1vJEzbICpKz1tKQMrvjSecklb2KEJ2Bf8Kb4DAaiVlwPnn4TYP5xUTFR8Ap18Ed77sNYyf892iWxWwe9NuXej5084YENQqovH2dthVafnFrqi/+P0iKx1IxU70GKaKTDyBMEDKjjzMXX9T80C4XY9oFGZ751ctpIt/9V0AUkDVLejM2R0O0YH8c/3XKu6NAloor8w9lLHwdkQcWrkQfDJ0dgIXKHewu5XMJvOd560bkD+Vdytf5e1cLMfs+qWnabFwK4Ii4zPv1YBKQbGWvntKW2RT49IBHJ9U2nD3GgXl66PIZ87vn73UZL6OxdOnXMBtYOgk8E5UJLcVqrbdUthMGfOGshSghSK3kfT6vuiQqar7M8WzaAHxNr3YPvhiT6ZvOEi9dHJcTR7J7gxVpRBeT+N+o/w1qPfFKLj+4ysD6xztaZeAidR5OwpFl+XE+s5Y9Zx5wOzgqPnyobdtgJY1jxa82GeRKnml3f/g9cX4e7br++CLWOn1JEkf+Mvojz8tw07TqrnwpeFSrIi6WuPXJvfWcCvy5e00NB5sO3YntezhksVqH9mO++cx18buK72etOEJ+2TqYr/GXFK+K08cC6jOhpnVS1V7NX2cvkmzDcFHGTxXyX+b52Mi7rg+kteLPUpwZEKq1ikv5pMs7YlOX1aez189JpJiGN6zjCYRdBf1Me8Il7RglJD9dLxDwacuoneF+qt2vau7S3kVTmnDAjZvEMLN1T1WwFUzTgXjJD8wEn+6Q5XsaJJwdwGl7onCCq4m6jlz7/J93OGJ4tew5FYJCiskixUIRUWmgh08fcjyI+Jg0m9mx7lTloMf3tJHXNL2/92h7ilY1O9Nzaph2AxOySav2iOr8Y9iKh0eeDIlvyh54 olBH+UPT CQ4pcco/q3TqXNoJFiDuACVcplX0/fnbfYcYHjB7Va0374uapIVrx6oXC3cJNxNg1q4EEIcmOdv35PS32fG+H6tAGrzq8e/Ear7VLVBMCt/FfW3/EyKeol+ZfDMEXfXYoJRYbrJT1b46xC7aNogwWN2tyVwH/dvq6i1aeFroOv95WtIAh1PugFN4XzlTQQXWd9B88+gNBywKJKIQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a helper that sets the permissions of a given pkey (POIndex) in the POR_ELx format, and make use of it in arch_set_user_pkey_access(). Also ensure that is included in asm/por.h to provide the POE_* definitions. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/por.h | 9 +++++++++ arch/arm64/mm/mmu.c | 28 ++++++++++------------------ 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/por.h b/arch/arm64/include/asm/por.h index e06e9f473675..7f0d73980cce 100644 --- a/arch/arm64/include/asm/por.h +++ b/arch/arm64/include/asm/por.h @@ -6,6 +6,8 @@ #ifndef _ASM_ARM64_POR_H #define _ASM_ARM64_POR_H +#include + #define POR_BITS_PER_PKEY 4 #define POR_ELx_IDX(por_elx, idx) (((por_elx) >> ((idx) * POR_BITS_PER_PKEY)) & 0xf) @@ -30,4 +32,11 @@ static inline bool por_elx_allows_exec(u64 por, u8 pkey) return perm & POE_X; } +static inline u64 por_set_pkey_perms(u64 por, u8 pkey, u64 perms) +{ + u64 shift = pkey * POR_BITS_PER_PKEY; + + return (por & ~(POE_MASK << shift)) | (perms << shift); +} + #endif /* _ASM_ARM64_POR_H */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b4df5bc5b1b8..9547183d86cf 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1555,9 +1555,8 @@ void __cpu_replace_ttbr1(pgd_t *pgdp, bool cnp) #ifdef CONFIG_ARCH_HAS_PKEYS int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long init_val) { - u64 new_por = POE_RXW; - u64 old_por; - u64 pkey_shift; + u64 new_perms; + u64 por; if (!system_supports_poe()) return -ENOSPC; @@ -1571,26 +1570,19 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long i return -EINVAL; /* Set the bits we need in POR: */ - new_por = POE_RXW; + new_perms = POE_RXW; if (init_val & PKEY_DISABLE_WRITE) - new_por &= ~POE_W; + new_perms &= ~POE_W; if (init_val & PKEY_DISABLE_ACCESS) - new_por &= ~POE_RW; + new_perms &= ~POE_RW; if (init_val & PKEY_DISABLE_READ) - new_por &= ~POE_R; + new_perms &= ~POE_R; if (init_val & PKEY_DISABLE_EXECUTE) - new_por &= ~POE_X; + new_perms &= ~POE_X; - /* Shift the bits in to the correct place in POR for pkey: */ - pkey_shift = pkey * POR_BITS_PER_PKEY; - new_por <<= pkey_shift; - - /* Get old POR and mask off any old bits in place: */ - old_por = read_sysreg_s(SYS_POR_EL0); - old_por &= ~(POE_MASK << pkey_shift); - - /* Write old part along with new part: */ - write_sysreg_s(old_por | new_por, SYS_POR_EL0); + por = read_sysreg_s(SYS_POR_EL0); + por = por_set_pkey_perms(por, pkey, new_perms); + write_sysreg_s(por, SYS_POR_EL0); return 0; } From patchwork Mon Feb 3 10:18:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA07FC02192 for ; Mon, 3 Feb 2025 10:20:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58285280015; Mon, 3 Feb 2025 05:20:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4943E280002; Mon, 3 Feb 2025 05:20:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35CE8280015; Mon, 3 Feb 2025 05:20:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 10392280002 for ; Mon, 3 Feb 2025 05:20:34 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 617FC142804 for ; Mon, 3 Feb 2025 10:20:03 +0000 (UTC) X-FDA: 83078237928.09.ADAF58E Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf18.hostedemail.com (Postfix) with ESMTP id C9E371C000A for ; Mon, 3 Feb 2025 10:20:01 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf18.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578001; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Snhdnks8KmGT9o0/ZDnykraCyor+6xfRkmoZ7dE4fso=; b=qyjUAp8KoQ4LFvAeHPz8/hfU5zCn3q4lpqvsbKUeIBbAgMfHis7p2FIDwEXjY6wKtYfSYC cn3bVumzI14ILJpDzjBgTymLZWG69TUGmaJSH09bntgRZR/aP+VNtTJ6Ua/RZy/aSa7Er2 DWuOuqb/g3OZAIKTu+ZdaAx90w7mMTE= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf18.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578001; a=rsa-sha256; cv=none; b=bb99Q6Ihfu68/AWaQslfNgPXYo8CR5AusB6h1+Q5rwQiuW6XViv1cvsat8+pr5S/D7vFA7 c0ZmmY95OUERmnmctEzAQhojZuUaQY5Z/2YZgGtIzh6jACV3n2s2Whi7xa2UXvnS6L7cvb 9Oc5DLwAke6qesfohzx2rQbHBGKCpt8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 92D9D1476; Mon, 3 Feb 2025 02:20:25 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A14E23F63F; Mon, 3 Feb 2025 02:19:57 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 05/15] arm64: Implement asm/kpkeys.h using POE Date: Mon, 3 Feb 2025 10:18:29 +0000 Message-ID: <20250203101839.1223008-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C9E371C000A X-Stat-Signature: omqruyswyuabim4ipz9cqm7kn6mk6ik5 X-Rspam-User: X-HE-Tag: 1738578001-650000 X-HE-Meta: U2FsdGVkX1+9cQLdGDTK0WWlYnJWsf5ZXkbb8riEklV1KBGQpSEIZEHfsjErnkZEx3HgZAkdksgqesTnP9gieyqiIifqjdKKPdRA8vip+gC2iDQH1ifiJTEA1LRSV6/xyrpZmOgbS4vmC7y1+NGUXIBlJPdJgIvRbUSM64L2iaIGUn+h7qPDHg9ba0/QOWXmIGtyMhP5djZUqZ3suiZi/BGGSLJuWYk17AUt53lSKDCaYfY+p9liWet8d1pyW0/bnIRi5YlSMBWoFvDAnV+/VY7Ns5/OtfM5ZumSYNXB+XSXk1JLW7wHzUkGRcuGqdaaoUq+QkwxW/sctl0FHQSOVQQAD2qskLY9kaxr8b0Dl71RBDDxc0a3i+Kf/u0aUa8K4dGa6CZ+OFSzm+34mCALY1CtVl+t36A9nTbpvalVgGBIpA2siAPMGJm1fLdwhWqeKueIw/e1asjCuE3zq9XBf4ylxYIQkbcXM/MsyzUF+SEFYlsRNssbmpuYi4XDz/aCeLFr5NMbAPy/qJ96k2u5PLixKmKDvwCjCvSEObLRMd/UMYWbMUeCNOXAw54uoQpMBD8fl+tteD13xehf6VaQKFZk739yg1Iio/WSNCKKBTv4Q96wFKUXNTt8JQXpN3vBFllfm/0oxCDifu1sBccl/7NSuLKgXPKy8JZxt63Ghqm4MgnZX7wvsRysPWtLA44laVxXv78Dkj/pE/S4YfCLQOpesBUiGSNJrUKNp9FGgGMRwI1WTyLL9WwA5LvkoXbY81DqKWKD7XwrdsrvqmTLQ2te33rrUsNrLCDweY22EpUpAJs2iRC6WLtkJgOhIZGESa440s3QEXc4IweFUlywf2BNLHWQh6gkoivxKDqCOYiQMYG2LTtf1T3PoIlLfh9tcgX4qKL7pex+Ev12hFfc/7/TdGuOV+X14FJju+m+dmPbfNjNlCCqrSiWJwTHiLY/XwmEIQfK75xLVVm80R4 jxVuQ/Qc s1c9PXgg0LBt/gGweTOdKlOkEo5uATMUIRBqM5XQBoz6VWgz2Q8djBvdvV4DTgYmkuksFbG18Ytu/ZtwQX7VPQp3OUA5/uSXjTgzEzG6KXkMq9xDIt6TVjr3JBprga927AIfSritrwkZ3etpSQXNuepr/00Apquj9zT/5Svi3M7meknjoYiN3V9yDE4OkfsyUgYQI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Implement the kpkeys interface if CONFIG_ARM64_POE is enabled. The permissions for KPKEYS_PKEY_DEFAULT (pkey 0) are set to RWX as this pkey is also used for code mappings. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 43 +++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 arch/arm64/include/asm/kpkeys.h diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h new file mode 100644 index 000000000000..e17f6df41873 --- /dev/null +++ b/arch/arm64/include/asm/kpkeys.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_KPKEYS_H +#define __ASM_KPKEYS_H + +#include +#include +#include + +#include + +static inline bool arch_kpkeys_enabled(void) +{ + return system_supports_poe(); +} + +#ifdef CONFIG_ARM64_POE + +static inline u64 por_set_kpkeys_level(u64 por, int level) +{ + por = por_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RXW); + + return por; +} + +static inline int arch_kpkeys_set_level(int level) +{ + u64 prev_por = read_sysreg_s(SYS_POR_EL1); + + write_sysreg_s(por_set_kpkeys_level(prev_por, level), SYS_POR_EL1); + isb(); + + return prev_por; +} + +static inline void arch_kpkeys_restore_pkey_reg(u64 pkey_reg) +{ + write_sysreg_s(pkey_reg, SYS_POR_EL1); + isb(); +} + +#endif /* CONFIG_ARM64_POE */ + +#endif /* __ASM_KPKEYS_H */ From patchwork Mon Feb 3 10:18:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 580DCC02192 for ; Mon, 3 Feb 2025 10:22:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E828A280016; Mon, 3 Feb 2025 05:22:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E09AD280015; Mon, 3 Feb 2025 05:22:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C849A280016; Mon, 3 Feb 2025 05:22:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9B5F2280015 for ; Mon, 3 Feb 2025 05:22:12 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 47CFA82572 for ; Mon, 3 Feb 2025 10:20:07 +0000 (UTC) X-FDA: 83078238054.08.E8C1736 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id A54C71A0004 for ; Mon, 3 Feb 2025 10:20:05 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578005; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oHokXrrIxgFozOdPdKGv7pm7boYyoVv2BPENAFW6e/0=; b=JSTNNFOMZYk5krifUTB5mrek8+MfTHWKqiR/CXU6kBSt2QRH7WF3c85UK1qvwQRVTF4SDV t8giKzCuAvrt1JN79TpcrffXdTFYcmUDCHmFRSqwHs8YJonmcSFPTbuDxN8UQgZH4TvUC5 zr7EID7I59ZxOEmbOOLwf0QyuSaM5X0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578005; a=rsa-sha256; cv=none; b=jbocxVqDas5oOVUAZZmmgLtHl6Q5efc4TdyMWK1DsoskoknqdQXY3LXk7WOLN8cneFnwEc 6qCMGAIGSClKuocUmIqE8ruzhhx9XA8LLszEWTbzqQSo4xnLNAFE9onHnV6cjIqKi2OEox vZBt8q6kbHPwhHp5ch9ieUHJyZEI62M= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 66BB711FB; Mon, 3 Feb 2025 02:20:29 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 74BCD3F63F; Mon, 3 Feb 2025 02:20:01 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 06/15] arm64: set_memory: Implement set_memory_pkey() Date: Mon, 3 Feb 2025 10:18:30 +0000 Message-ID: <20250203101839.1223008-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: k4uajsgm5nxzzj84qiwbpzu1c5kttxrj X-Rspamd-Queue-Id: A54C71A0004 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738578005-468914 X-HE-Meta: U2FsdGVkX1+3vzfQp5whxWA63f/Bb2wNJA1qahpzDYdYoJAcAR7NpudL/NLwXzhwdgDSOqLa/4XO3y0RSYx0ugCuBZHPAKTzqnNkEUUvWD7gJQRBm5aJZNyeh4jS2FRset64nD99pFFAbR1wVGXf1OM/YGsXIgIFN6w2SIHRhnz6MZkrq38JVSc0Nx8EY7j0qjPHHg6ldx3YRJ+Qa7wquVMNCmin3xqcnRn0QTD1Xe6R5ejg5dd6rmN5zzoPKWHRt4so5xNVZCkZ9kWIkTkhXkK54Z35Pkkdl3cVhR68dNCs49QyGLx1ObDXmDlDOKTmEkd4rET2XHBc1njfieaAzjSRaAutpsrBujjkdQUZyRM8ZLE34X4i3LOdKrdKt6rFK1lgqyxKTXIJjCattnriAs1SqWFW//FDEg+toWzA5R/O2KEILPKn7dyuFkOXMm0Tm5YCq5VlBqJpsca10PypU6I0vaZAEJH2mIWfe9bbBANWNSBzC5csYFcjdfVJduOILHwv1g7nNFXBeBaOxDl8fznQMMldxMlCiXs3iTrGkbyjD96p0/C90DGomN4hfobYa+qb82VEItYI1dbO9WkxW4xCy6TqgegDSbIc3fNL1ynb1Q01vVa/m/HloGatch8sRaTsliEFVmLNHeEkH/4FJz3PBp5MSHo46c3QjijgCszwBqEKL9Tmd38lFdPxh9SPuSox6dm6Ho2nrnNr7HAQlOQvIVX5Y1u7Vz02ymh0FEO6kHyzser3/svbSLdZoi6yuSX4erD1psTcrRdti2TH0mp09u6kcak7HhuxgC5hH1Z57JD4Enq78jaqoSMllqwiD26IwNvs4ARRgYzw+bMdZgrvqnSa9NZBl2LFReuUIfepe3tt0Ze9kNZln1HH6MZPwIcIYc1cersFoJCfphulcNJ0sJWpidFYKdYgkGBvi5S+FOu/lhubfSiMZQ+LFBvEiV3gc1KrVJL4dPT6Fph phV1ZXD+ gBl71PJsdts+oVOtihTw/ZFyurU2GLQ1QyB08dE4U5Hx0FVgCRHiBZ5JwnZmchD3Md8CBzW+gyHcSmZZXUkSsREJuLBXoa+7iWY9Dz2nnKotdWpXz3gFw9lygYGN6paAmCh8/XaHuymXfCUPhjqes959Osi4Xc4rm00kX2uR/U/VCRLYbBKXo09KYDbItDoVMjTmfqNF16tgjbmlB7SKj7QUcrahhQGhRNsDxPOnNCw47FHA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Implement set_memory_pkey() using POE if supported. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/set_memory.h | 4 ++++ arch/arm64/mm/pageattr.c | 25 +++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 90f61b17275e..b6cd6de34abf 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -19,4 +19,8 @@ bool kernel_page_present(struct page *page); int set_memory_encrypted(unsigned long addr, int numpages); int set_memory_decrypted(unsigned long addr, int numpages); +#ifdef CONFIG_ARCH_HAS_KPKEYS +int set_memory_pkey(unsigned long addr, int numpages, int pkey); +#endif + #endif /* _ASM_ARM64_SET_MEMORY_H */ diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 39fd1f7ff02a..9721a74adbe2 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -292,6 +293,30 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid) return set_memory_valid(addr, nr, valid); } +#ifdef CONFIG_ARCH_HAS_KPKEYS +int set_memory_pkey(unsigned long addr, int numpages, int pkey) +{ + unsigned long set_prot = 0; + + if (!system_supports_poe()) + return 0; + + if (!__is_lm_address(addr)) + return -EINVAL; + + if (pkey >= arch_max_pkey()) + return -EINVAL; + + set_prot |= pkey & BIT(0) ? PTE_PO_IDX_0 : 0; + set_prot |= pkey & BIT(1) ? PTE_PO_IDX_1 : 0; + set_prot |= pkey & BIT(2) ? PTE_PO_IDX_2 : 0; + + return __change_memory_common(addr, PAGE_SIZE * numpages, + __pgprot(set_prot), + __pgprot(PTE_PO_IDX_MASK)); +} +#endif + #ifdef CONFIG_DEBUG_PAGEALLOC /* * This is - apart from the return value - doing the same From patchwork Mon Feb 3 10:18:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5356C02193 for ; Mon, 3 Feb 2025 10:20:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C0EE9280013; Mon, 3 Feb 2025 05:20:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BBEB4280002; Mon, 3 Feb 2025 05:20:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A37D4280013; Mon, 3 Feb 2025 05:20:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 68C01280002 for ; Mon, 3 Feb 2025 05:20:12 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1E2A2436CF for ; Mon, 3 Feb 2025 10:20:11 +0000 (UTC) X-FDA: 83078238222.27.39E0E20 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf25.hostedemail.com (Postfix) with ESMTP id 73FA0A000C for ; Mon, 3 Feb 2025 10:20:09 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578009; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1RJ6r3cxM9fgt2mbPkhm9caNHqt7aSNUg8ljSrwtyWY=; b=tvgyqkv0f2zq60ZQI9qhYjDLZOJN7m46ah8y/TUnCCBMbWyLwoifYwRGFs0k6TuilAZKGJ 3bDygVBJkV23tfVKpNm5FAyxThP93ni5teEVtBsIBOhiCjbAMwq0uTl33OJ9XuPpf6NXMa 0cpt5BDet5DHk993eHlGj8NQG3nbVP4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578009; a=rsa-sha256; cv=none; b=fcgIYt+jHFqiPwHytKFyYzOrvaa/gUaKgsNDH++UuuEkVIjF3zgKKNvs1Erl3TD8fQRM5P WF1OL9Gy7HX6M+FyLmMQTknwuWkWg1J8ZiniWqa5xjKgTkRvwIPskGObx+AlcPR3Kmh+Xd GHOdmthqeiIifhcymNqV+XjullNNL/0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3B2CE1682; Mon, 3 Feb 2025 02:20:33 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 48ED23F63F; Mon, 3 Feb 2025 02:20:05 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 07/15] arm64: Enable kpkeys Date: Mon, 3 Feb 2025 10:18:31 +0000 Message-ID: <20250203101839.1223008-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: 1bnu3njgmprtyh5t9krsbhs87bxnepyo X-Rspamd-Queue-Id: 73FA0A000C X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738578009-688607 X-HE-Meta: U2FsdGVkX19ZnzjoHRYAK+qdGRy7zExRx5BIA45avvIU4o4v2j7zweDuc0nE1Mo8Ehjp/Vi/Pq+mG6BO3LEmtnmTmAPOoOfFO1NAq5i4c1XztPhHxj7mTUEN/deQ7+k+mHKJk4mVdSE0s8cefvea8bSUxUmarc92eIiaGerfYiZTIz6Tg1LRmloY+N4N5dyTaCME+CRml296xVjaiAXRrHyzVupaV1QUQsmS1b43TkqxM9fPp8NxyVpN3jRZsFnlUueH1+/xLwG9CqKdxfnN4pLSvTAZO8aFBtFmjL/YR1BlgXzevu2Ex4xdZ89lzzpt/ZRcJ4yr8qG1VRF2ZqDEePoQhxts4YUh/lQt6w3kERuISZsMFfYm3ilkC0oNKG2Uej6puyJf2X+hBP8gleYc52p93WQjFYL6h2YkjsshdsiKJjzXp1dMGwY7vJZ6SdXn2B34n66vRbJlgHZt+MhwOlN8wulC2IxB/ix3EHop7WID9CpxgwPnQbQKz1wKP9jG4jR4PsJBoql+FcSUtlgY7DZ0rhplTvOg2bdXIqlG8gbgUbIdLV9Zqo08Jx2SXg06i60t4JFpPbWyWarU7e+PONDODokh5AvIluSmBofng1X/kXRqyvhd06CkFPb7jPsdRjwYLE0++Uc/T5Jtdlw8doKA/973vy+cNmXuCBA1t9sGPsOXSWhmbdcRic8ht4fKR8c5Phm4LbT1IX/w5LZM6hRge9qWtG4RAgD2cJCdzRPg8wggZZXkbsudMqoXcc7xYmIM4q/o63QpdQZ9w/OtB2BZ1IL/ddUdtJKE6UVKuP08UQtAgB5vwUPI+4XlvvD8q9K4jqYwxpywvK9yxkdZLqoLlZhft7SjnvV3CZO7jsj5Ji1C5HZdj7iWr/Jd5F1iE2GmWXIJmIj8Kmkmzb7acMjlphJ//oH8ZrvPR6LjpsfF/spBgwDmyuEzgCDzD2tE8gv8tY+Yq0GOki+dDZK Ey7jTaqC vKerx6jLDLHqYjkTKi+ivDPPpYSvLodwwpQdH0r3gkFHfGkL9OzW8v2JgBiZ1pE96Bz9uaEqpE7i2iXSNOY89qQOqNGIwr74/8TnaffUmUaemlO0iD3Dp6hn3HmgMfI4+lNX7EK282LH7MpYfq0qsDFDn4Z/bghDelMxD3XHf1yI2k4qijlyv83lnQ0ozRJSkwhCHcWuGUUSzD7y9HArTh2hXvoN5bGWFYJ8P4aht3Fu+0W4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is the final step to enable kpkeys on arm64. We enable POE at EL1 by setting TCR2_EL1.POE, and initialise POR_EL1 so that it enables access to the default pkey/POIndex (default kpkeys level). An ISB is added so that POE restrictions are enforced immediately. Having done this, we can now select ARCH_HAS_KPKEYS if ARM64_POE is enabled. Signed-off-by: Kevin Brodsky --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/cpufeature.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index fcdd0ed3eca8..34f15348ada8 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2186,6 +2186,7 @@ config ARM64_POE def_bool y select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS + select ARCH_HAS_KPKEYS help The Permission Overlay Extension is used to implement Memory Protection Keys. Memory Protection Keys provides a mechanism for diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 4eb7c6698ae4..28b8c93c60c7 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -76,6 +76,7 @@ #include #include #include +#include #include #include @@ -2385,8 +2386,10 @@ static void cpu_enable_mops(const struct arm64_cpu_capabilities *__unused) #ifdef CONFIG_ARM64_POE static void cpu_enable_poe(const struct arm64_cpu_capabilities *__unused) { - sysreg_clear_set(REG_TCR2_EL1, 0, TCR2_EL1_E0POE); + write_sysreg_s(por_set_kpkeys_level(0, KPKEYS_LVL_DEFAULT), SYS_POR_EL1); + sysreg_clear_set(REG_TCR2_EL1, 0, TCR2_EL1_E0POE | TCR2_EL1_POE); sysreg_clear_set(CPACR_EL1, 0, CPACR_EL1_E0POE); + isb(); } #endif From patchwork Mon Feb 3 10:18:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61570C02196 for ; Mon, 3 Feb 2025 10:20:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CA8B28001A; Mon, 3 Feb 2025 05:20:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 95153280017; Mon, 3 Feb 2025 05:20:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50AE228001A; Mon, 3 Feb 2025 05:20:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1D2CC280018 for ; Mon, 3 Feb 2025 05:20:51 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id F3853825A5 for ; Mon, 3 Feb 2025 10:20:14 +0000 (UTC) X-FDA: 83078238432.12.97C9D4E Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf30.hostedemail.com (Postfix) with ESMTP id 5C75C80002 for ; Mon, 3 Feb 2025 10:20:13 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf30.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578013; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hMHebzscLOUd5of8jcCqK/oKCbj1tqbVmE+gDk9lCiQ=; b=5Lwtd538c5cSeXpG5013fTfo7FwHMgUgFO5cBzwRsenYV6HNhMyEs/yEYfUaFRQYatoDWg /EyJnCuF2cbHdwfFcoOC5cWHuPQxVlI/kNwHWFwCkh/7XZBm8B7kyZ1zDsx7YHdzl5CtQN i9r3Yj7vTg4/oLpQe4p1iU4k9AcTv4A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578013; a=rsa-sha256; cv=none; b=ItrhavKfhfgEm6e0jTjf9QU/Drl7lEa0nUX9CHDWzmX/f310oVWo6IONw1W9dQ2Z0mMPLq DjqdSZkG6GiOwz1OYrxrdkFlD/nAGZZLyztG4uJ8/2Uf5PrbhMNIQiIL58b6TbJ9D5Vp+s H+IP7XqwKd6vdZmGUl+sx0H83Mx2Yqk= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf30.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0F1A111FB; Mon, 3 Feb 2025 02:20:37 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1D2BF3F63F; Mon, 3 Feb 2025 02:20:08 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 08/15] mm: Introduce kernel_pgtables_set_pkey() Date: Mon, 3 Feb 2025 10:18:32 +0000 Message-ID: <20250203101839.1223008-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: nj73oh8tesptedm5edix9q3c9b5p15ux X-Rspam-User: X-Rspamd-Queue-Id: 5C75C80002 X-Rspamd-Server: rspam03 X-HE-Tag: 1738578013-437478 X-HE-Meta: U2FsdGVkX19VB0X6qe5p5XM0aVHjUmk1pbdQ+mf+2hFl5wzklhlCtewk7sWsBtiQu8a8H9ElTbVXi9oowe6SpSR5Nnaswd16yckZy4l6oTmkctZftrovoPMg/eHoz54my5ECD/pUwBCVOnUvVffAf6h01rSIE8URpTBPW2+Vp/8pr83KwR8wFVY+28cSdIJNEpYKc9e6bR0yjGh0nhXRTnzVwcxzFLDORP9W+twv/915vhEVKaDduvv27Md57H5HG8lah9bidXkpWUGlAzVm1FegHcrNP6YoZK8RSSWrMFA15nVNodE0X6XerRhK1ISdLOIkPsxP8MM+S9iZYMYwUBVsCgK40RfV8V77CP0ummEACrvf7RBYDeZR7+Wi+lDL/MR4drYTk/p7BZoXmnWqziEps77yJ8qiMQJtmJ9RqaTK73KM4VChirp4B5SxyOB5YyGWQsL8c/uXkmZCGuI1iFJBRSmvbPGLYcF4AcwWlbP16X6JbuH7cA+62s/XiQFffyfJixZ18dmfITWxGEGiY5jbNITxaJHNniw4/zof/NZBBpmZrInJ8TsfQsQ5jxwQmcLOTqxQdb+pJTmwJh8j9OrDsjTmrD6nrxy4BR922wKIDJbAXuL6hvHLoibO0fyOs2wo7yoRzXSZ9oyX70WjAea80AuqPZqjOPDJ03sbOG+VisV3ufDqJUQp3uchIau9HGqMZp0+sdXs2c217mTPboLatRXjtHjcMtrUthLPhgKgfUFPuYh2KjIJUa83x7KGZ1UEb6sullpNHeigAZQlgc0tjqIhMPTrQvglwFma8vg1w1/SE6ArvGMUSyphg+W2wOPO65sQkuxQ3k1xOizUoTLa7iABimaWKuA4z4RlZW0M2EEZvLVgLa+4mwJS7GFCG0T1N1CvxlsWF64mD/6VckzGPgrGPV4S+Rq6cEuVQG91VF1wndvSysvUZuZ5p/1X0X4h3Pty7hxs8I37Z+p Jh9DkjHT Kl8tG2yzUlTyPLkb3f5wRFQug2g9/BK88aLV8a9Mj7UvlwiNXM7bFa53u8vT5vdlCAYOva2nWtxpDCNTU6eFZ06UcwJoB9Hx1hGzY+ucwhcBli+8QoqkUgAIwvpcgSrWFPWlEI59Nok6fAWF9iVS1xBEmNx7UmY1AYcNanfKObfKuOaLIqebNnNDAll111ju6j5X658Hxoc4L++0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kernel_pgtables_set_pkey() allows setting the pkey of all page table pages in swapper_pg_dir, recursively. This will be needed by kpkeys_hardened_pgtables, as it relies on all PTPs being mapped with a non-default pkey. Those initial kernel page tables cannot practically be assigned a non-default pkey right when they are allocated, so mutating them during (early) boot is required. Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 2 + mm/memory.c | 137 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7b1068ddcbb7..c3998b78f6a5 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4155,4 +4155,6 @@ int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *st int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); +int kernel_pgtables_set_pkey(int pkey); + #endif /* _LINUX_MM_H */ diff --git a/mm/memory.c b/mm/memory.c index 539c0f7c6d54..1386b9cfb459 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -77,6 +77,8 @@ #include #include #include +#include +#include #include @@ -7066,3 +7068,138 @@ void vma_pgtable_walk_end(struct vm_area_struct *vma) if (is_vm_hugetlb_page(vma)) hugetlb_vma_unlock_read(vma); } + +static int set_page_pkey(void *p, int pkey) +{ + unsigned long addr = (unsigned long)p; + + /* + * swapper_pg_dir itself will be made read-only by mark_rodata_ro() + * so there is no point in changing its pkey. + */ + if (p == swapper_pg_dir) + return 0; + + return set_memory_pkey(addr, 1, pkey); +} + +static int set_pkey_pte(pmd_t *pmd, int pkey) +{ + pte_t *pte; + int err; + + pte = pte_offset_kernel(pmd, 0); + err = set_page_pkey(pte, pkey); + + return err; +} + +static int set_pkey_pmd(pud_t *pud, int pkey) +{ + pmd_t *pmd; + int i, err = 0; + + pmd = pmd_offset(pud, 0); + + err = set_page_pkey(pmd, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (pmd_none(pmd[i]) || pmd_bad(pmd[i]) || pmd_leaf(pmd[i])) + continue; + err = set_pkey_pte(&pmd[i], pkey); + if (err) + break; + } + + return err; +} + +static int set_pkey_pud(p4d_t *p4d, int pkey) +{ + pud_t *pud; + int i, err = 0; + + if (mm_pmd_folded(&init_mm)) + return set_pkey_pmd((pud_t *)p4d, pkey); + + pud = pud_offset(p4d, 0); + + err = set_page_pkey(pud, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_PUD; i++) { + if (pud_none(pud[i]) || pud_bad(pud[i]) || pud_leaf(pud[i])) + continue; + err = set_pkey_pmd(&pud[i], pkey); + if (err) + break; + } + + return err; +} + +static int set_pkey_p4d(pgd_t *pgd, int pkey) +{ + p4d_t *p4d; + int i, err = 0; + + if (mm_pud_folded(&init_mm)) + return set_pkey_pud((p4d_t *)pgd, pkey); + + p4d = p4d_offset(pgd, 0); + + err = set_page_pkey(p4d, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_P4D; i++) { + if (p4d_none(p4d[i]) || p4d_bad(p4d[i]) || p4d_leaf(p4d[i])) + continue; + err = set_pkey_pud(&p4d[i], pkey); + if (err) + break; + } + + return err; +} + +/** + * kernel_pgtables_set_pkey - set pkey for all kernel page table pages + * @pkey: pkey to set the page table pages to + * + * Walks swapper_pg_dir setting the protection key of every page table page (at + * all levels) to @pkey. swapper_pg_dir itself is left untouched as it is + * expected to be mapped read-only by mark_rodata_ro(). + * + * No-op if the architecture does not support kpkeys. + */ +int kernel_pgtables_set_pkey(int pkey) +{ + pgd_t *pgd = swapper_pg_dir; + int i, err = 0; + + if (!arch_kpkeys_enabled()) + return 0; + + spin_lock(&init_mm.page_table_lock); + + if (mm_p4d_folded(&init_mm)) { + err = set_pkey_p4d(pgd, pkey); + goto out; + } + + for (i = 0; i < PTRS_PER_PGD; i++) { + if (pgd_none(pgd[i]) || pgd_bad(pgd[i]) || pgd_leaf(pgd[i])) + continue; + err = set_pkey_p4d(&pgd[i], pkey); + if (err) + break; + } + +out: + spin_unlock(&init_mm.page_table_lock); + return err; +} From patchwork Mon Feb 3 10:18:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 024C3C02193 for ; Mon, 3 Feb 2025 10:20:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9013F28000E; Mon, 3 Feb 2025 05:20:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 88ACC280002; Mon, 3 Feb 2025 05:20:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7041A28000E; Mon, 3 Feb 2025 05:20:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4F6D7280002 for ; Mon, 3 Feb 2025 05:20:32 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B9B82142854 for ; Mon, 3 Feb 2025 10:20:18 +0000 (UTC) X-FDA: 83078238558.15.E08D300 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id 2002514000F for ; Mon, 3 Feb 2025 10:20:16 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578017; a=rsa-sha256; cv=none; b=0U3AF5idvDz5cvnaUPZoeLUeGirDjsfGoYYmnU/cv8sJy2BxCF24qPGxGggznro9NxK8RY 6HrT1ydSCgK1MWc+8HHjq3JBibCB+oPG3rjMSGawxCi1ZOxwvJuYs+6eu+LSnCBTZCyI+U 9SCNB3ZOlPUZCtiemyqkqsTq8rpAhAU= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578017; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oM5qEeuq9WxuNGRz1OckJ8gXPrP/PTm/sS7h9Gbd7jY=; b=AO2IHJBjGQf/kBuQKV6DeMpEF6XdmO3XB5RjeKxbbjtttkpP8aE8Sut9RsscLfMQ2frH20 ZPSBicyBD2mZJPhSSooKQURETW7lp0twCRE/13xXSvAfUkLKfduk0Uhw/lTpjH1+VAN59Q 1GkdAEnuQlTccDnuw6ro88Rt+hTRMaQ= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D770F1476; Mon, 3 Feb 2025 02:20:40 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E587F3F63F; Mon, 3 Feb 2025 02:20:12 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 09/15] mm: Introduce kpkeys_hardened_pgtables Date: Mon, 3 Feb 2025 10:18:33 +0000 Message-ID: <20250203101839.1223008-10-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2002514000F X-Stat-Signature: 1aomjjeuowc7mzj7zpqhuprc34a6rw93 X-Rspam-User: X-HE-Tag: 1738578016-566040 X-HE-Meta: U2FsdGVkX18bryrNwxxtTdUppHqQ8HmIirWJouAFBQFg42npG6Ybd0oRElQABvtJjYCSJeqb9LA0+zTwn29ahrYrNCg5tKwlMKROGg/cbIL6mMVNG/+H/eTAi9CKJHuLDR1bi7bMojAfA/yKIKoq5vtmxTLJ7HWwb94JBgzOIQRJQD0Q0MMTewYLQJz+qIwlpHS9rA2ZVsxzmIt+1HKx/iCIVTDS/73jNxKtY1BrkAXxV22f0Px9qa4UErrUnXmev3ATlrHLPSGKubYK7notyqotSkiEXu0WDgp0uh9lK31OZXLPG9QyjYHoMH/sifmiZdczS0k2NHrjwgW3XkW3sA0O/4XSMO+QV3EibQGD6MK6OaMHo1I5WG+X2815NaNCA+1qgXC1Md0R0s9i816C0rXCd1p1PIn87E2I7u3cr6utTHFMTcfAvKAjz4GGjHAQCMiirbGPvE7wCK5xb+VvfjSam0iON7FesvI7HoLGy42RrK+EOfh2wjwyEWfx0MzIbjzetSynMYwZXe4xD2FwhSQA2GgHV+FRIcKfuWYEITSStqUD98JfupR77gbxzhvJY2yyFBC3c/tjNUup+xflme4170SXe3zTL+ULBbsCZaC64WckWhzOs5yh/GzhAkmNla8zYZDkdY8eVxk9zPPU4cxFiYK9Ez0qpCe7wmKXCdJKsA34fLtMWGEQ3b/dYkAJvQhcp8oXzbqTeK9o9hLAdZmUNoYPbCpSbEv9MQK97FnuYo8fGsuKGSRyMwqiYKyPxjGgoCw3Jf9RogtrsEAQgfRkgAQtunN3kBVcM5AmTxHszo7J5ZgY2p27LGCDHP7f3YXbmWjC9rkJiRqnfum2uuR648C4B4Jr2XdEc/U+MToSr+b74QIL87bgAQINCvJPSEXl5Z853xo3pZY7iAzDtMbL/Et6EKOyDQi6MsA4/B68r8nFR60EfJtIjCCx4q3wd9EqpZ+UEOOYdJ4I24b N4YxUkaA fkg1NixJkPxbNFqVY46neUM7S8wxgvE600TnoAVwx1tI40MA1qadR8einrbsLYzEYMKXpMlgNFRE+FqSy92S5xl4eLbNHdKR7Gdg164cChZiDx4/KX3BN0F5ZQWxitm9+8MFaokv4Zz6I74Y3F+SDyqEtvXdTwk2YUlXh8l6DEZB8qWREsSVA5Xq3dazBAewHeUsgfA7pQVXl5ag= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys_hardened_pgtables is a hardening feature based on kpkeys. It aims to prevent the corruption of page tables by: 1. mapping all page table pages, both kernel and user, with a privileged pkey (KPKEYS_PKEY_PGTABLES), and 2. granting write access to that pkey only when running at a higher kpkeys level (KPKEYS_LVL_PGTABLES). The feature is exposed as CONFIG_KPKEYS_HARDENED_PGTABLES; it requires explicit architecture opt-in by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES, since much of the page table handling is arch-specific. This patch introduces an API to modify the PTPs' pkey and switch kpkeys level using a guard object. Because this API is going to be called from low-level pgtable helpers (setters, allocators), it must be inactive on boot and explicitly switched on if and when kpkeys become available. A static key is used for that purpose; it is the responsibility of each architecture supporting kpkeys_hardened_pgtables to call kpkeys_hardened_pgtables_enable() as early as possible to switch on that static key. The initial kernel page tables are also walked to set their pkey, since they have already been allocated at that point. The definition of the kpkeys_hardened_pgtables guard class does not use the static key on the restore path to avoid mismatched set/restore pairs. Indeed, enabling the static key itself involves modifying page tables, and it is thus possible that the guard object is created when the static key appears as false, and destroyed when it appears as true. To avoid this situation, we reserve an invalid value for the pkey register and use it to disable the restore path. Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 4 ++++ include/linux/kpkeys.h | 45 ++++++++++++++++++++++++++++++++++- mm/Kconfig | 3 +++ mm/Makefile | 1 + mm/kpkeys_hardened_pgtables.c | 44 ++++++++++++++++++++++++++++++++++ security/Kconfig.hardening | 12 ++++++++++ 6 files changed, 108 insertions(+), 1 deletion(-) create mode 100644 mm/kpkeys_hardened_pgtables.c diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h index ab819f157d6a..cec92334a9f3 100644 --- a/include/asm-generic/kpkeys.h +++ b/include/asm-generic/kpkeys.h @@ -2,6 +2,10 @@ #ifndef __ASM_GENERIC_KPKEYS_H #define __ASM_GENERIC_KPKEYS_H +#ifndef KPKEYS_PKEY_PGTABLES +#define KPKEYS_PKEY_PGTABLES 1 +#endif + #ifndef KPKEYS_PKEY_DEFAULT #define KPKEYS_PKEY_DEFAULT 0 #endif diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 62f897c65658..645eaf00096c 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -4,11 +4,15 @@ #include #include +#include + +struct folio; #define KPKEYS_LVL_DEFAULT 0 +#define KPKEYS_LVL_PGTABLES 1 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT -#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_PGTABLES #define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ @@ -110,4 +114,43 @@ static inline bool arch_kpkeys_enabled(void) #endif /* CONFIG_ARCH_HAS_KPKEYS */ +#ifdef CONFIG_KPKEYS_HARDENED_PGTABLES + +DECLARE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_enabled); + +/* + * Use guard(kpkeys_hardened_pgtables)() to temporarily grant write access + * to page tables. + */ +KPKEYS_GUARD_COND(kpkeys_hardened_pgtables, KPKEYS_LVL_PGTABLES, + static_branch_unlikely(&kpkeys_hardened_pgtables_enabled)) + +int kpkeys_protect_pgtable_memory(struct folio *folio); +int kpkeys_unprotect_pgtable_memory(struct folio *folio); + +/* + * Enables kpkeys_hardened_pgtables and switches existing kernel page tables to + * a privileged pkey (KPKEYS_PKEY_PGTABLES). + * + * Should be called as early as possible by architecture code, after (k)pkeys + * are initialised and before any user task is spawned. + */ +void kpkeys_hardened_pgtables_enable(void); + +#else /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + +KPKEYS_GUARD_NOOP(kpkeys_hardened_pgtables) + +static inline int kpkeys_protect_pgtable_memory(struct folio *folio) +{ + return 0; +} +static inline int kpkeys_unprotect_pgtable_memory(struct folio *folio) +{ + return 0; +} +static inline void kpkeys_hardened_pgtables_enable(void) {} + +#endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + #endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 71edc478f111..2a8ebe780e64 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1149,6 +1149,9 @@ config ARCH_HAS_PKEYS bool config ARCH_HAS_KPKEYS bool +# ARCH_HAS_KPKEYS must be selected when selecting this option +config ARCH_HAS_KPKEYS_HARDENED_PGTABLES + bool config ARCH_USES_PG_ARCH_2 bool diff --git a/mm/Makefile b/mm/Makefile index 850386a67b3e..130691364172 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -147,3 +147,4 @@ obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o diff --git a/mm/kpkeys_hardened_pgtables.c b/mm/kpkeys_hardened_pgtables.c new file mode 100644 index 000000000000..c6eb7fb6ae56 --- /dev/null +++ b/mm/kpkeys_hardened_pgtables.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +DEFINE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_enabled); + +int kpkeys_protect_pgtable_memory(struct folio *folio) +{ + unsigned long addr = (unsigned long)folio_address(folio); + unsigned int order = folio_order(folio); + int ret = 0; + + if (static_branch_unlikely(&kpkeys_hardened_pgtables_enabled)) + ret = set_memory_pkey(addr, 1 << order, KPKEYS_PKEY_PGTABLES); + + WARN_ON(ret); + return ret; +} + +int kpkeys_unprotect_pgtable_memory(struct folio *folio) +{ + unsigned long addr = (unsigned long)folio_address(folio); + unsigned int order = folio_order(folio); + int ret = 0; + + if (static_branch_unlikely(&kpkeys_hardened_pgtables_enabled)) + ret = set_memory_pkey(addr, 1 << order, KPKEYS_PKEY_DEFAULT); + + WARN_ON(ret); + return ret; +} + +void __init kpkeys_hardened_pgtables_enable(void) +{ + int ret; + + if (!arch_kpkeys_enabled()) + return; + + static_branch_enable(&kpkeys_hardened_pgtables_enabled); + ret = kernel_pgtables_set_pkey(KPKEYS_PKEY_PGTABLES); + WARN_ON(ret); +} diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index b56e001e0c6a..f729488eac56 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -301,6 +301,18 @@ config BUG_ON_DATA_CORRUPTION If unsure, say N. +config KPKEYS_HARDENED_PGTABLES + bool "Harden page tables using kernel pkeys" + depends on ARCH_HAS_KPKEYS_HARDENED_PGTABLES + help + This option makes all page tables mostly read-only by + allocating them with a non-default protection key (pkey) and + only enabling write access to that pkey in routines that are + expected to write to page table entries. + + This option has no effect if the system does not support + kernel pkeys. + endmenu config CC_HAS_RANDSTRUCT From patchwork Mon Feb 3 10:18:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 534C4C02192 for ; Mon, 3 Feb 2025 10:20:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50FEF280018; Mon, 3 Feb 2025 05:20:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A095280017; Mon, 3 Feb 2025 05:20:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2986128001A; Mon, 3 Feb 2025 05:20:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 046E4280017 for ; Mon, 3 Feb 2025 05:20:50 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9AAFFC25E7 for ; Mon, 3 Feb 2025 10:20:22 +0000 (UTC) X-FDA: 83078238684.08.F39BFAF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id F15DB100012 for ; Mon, 3 Feb 2025 10:20:20 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578021; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s7RWiBc91Kcnb6ZXaxAa5m0a+mqmeQb+Ok8r2YCJNYY=; b=3714We2I7sK71iShc1F7ygASrXmBiMKdhrBoyB6j7wcNCNRqtBsxJlOXhk9Y/Q1Zdi4K0c NQ8Mmhw2+7LwohAX/D7nnenkDMYYzxMU6fsvsS4PP/i5cNducb285O7H4BWr0vkOrAvQe+ JoFjRW1vTu1Ia/w+Z0TYNt9eFd75KwU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578021; a=rsa-sha256; cv=none; b=f0yk9vY84Hq2XkVFz4qXpJit75fkSL4eMQuvXs8JxLmqtcwcR6INN0hewZPWlQvjDU3n0D a9weGzhyRfDTH86MLTknQ2Co5ceFFdgRtZxTMrOm4lqJE0qIcORL/mYNfGfct4jjnK0c8p IbFIkYvZtoiPNCyUUsR/YEKYDOs+6hE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AB9A711FB; Mon, 3 Feb 2025 02:20:44 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B9E8C3F63F; Mon, 3 Feb 2025 02:20:16 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 10/15] mm: Allow __pagetable_ctor() to fail Date: Mon, 3 Feb 2025 10:18:34 +0000 Message-ID: <20250203101839.1223008-11-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: F15DB100012 X-Stat-Signature: ywuat7hrdnj6bee96wun16j4u15s53ke X-Rspam-User: X-HE-Tag: 1738578020-879512 X-HE-Meta: U2FsdGVkX1+OH8m7au2awa7Vy6f/pYISySK3YkWe6Osf7rtqw4729uVt9/GplniBlcIG0A87uNnNepS0/Lkml1oxrhHryFVohaD0VuYX9yPHv0GvkCdi/27dkks8hI0HI6a67L0RISzQeUdsRt8wJSKEh9jTdrglYnhxNhiz4ikfdGu/CGz6ga66DdWLzQOl1g8yZs4KOVCx4WN7UJ4Du0t2pZH6nd884/IMVNJfsbStxP9t3egSvf//EFuj3AfDVFomKDf5+R/QUnBDWUT70bad9W5xgStJX/kYk1RiuWGIZfLbtj0yWkH/dC/LpLzc8zxAnKP1DEL+Or4XvcYcj78np1AHsGElvwhq+aF9EGPXVpWl4GVBjg9RLC0RU7ZefYY8AfTdKtXADQ9yUxE1H18Pl581Di1tG4fbm7qw6+WXf02xV7TSHExAKzMsTvjyWN44MFqSsyVtenN1MLsqWZAY/A5nvNYtjT9OqqmUFG4/nCnatMuXwVb3r9rEFc14qesaKft4N8T+rzBfWJqAUlX/MFEvqXG6dhOXVxA8lVJE1Dc1n5frWbO+N7fVHVMQTZmQddIhADXLqJKHsuXgp0cHO1ZeFtVnbTgBrTbZrtx16X4bpqFHNWvgrqsuDpQu9PVr237Bac7U5UQxuSqT8KyVvog/TUMwfAnYfaGjUbI+hsuOtSgT0oUrEy9IpHaLTZIX6anKrWaQ2YuglqNC0Kgd81EkAIfX2BRbEyGO2ifXh3DDjQcVykKdYxYOWx1WvWz1EGKifnRmxLnF/Bur2NYKh+qL5xRFq/mK37ZOuNeujPS4gukm9AUEHxrkjl+kZJf9aaKWXB9MFS1FWdVGitXAKhABfWgQ7n1X5VkfOFTWH4HFtqCnopkiklj+fq1MWLJH74AXHWMcu0myDm52hvDxamT93VzBQULTf3pmhyF97rUFDXP3etda5+S71t/HuV+iZRGdu7nnhxdvBCr 3IPdTmA+ 1p9L5on/+Ev6T8wwDpk/ImAMKcWZmCqrVY+C6dIRX/yTOzGpPvoD7A7PwTcGnSmcQLqUd51uCeBpHmr1a+LZTSEZ5TyHTTL07GUSLBuOCGE0XX1kPJTDog1PR+x0X/6Ctx2V2TGsYQUafKynLkAJY92WFeTqKJlH2s+Bm66fGWZv7xpK/5IM9VXbzrD+4q/sJgw7E3uaBiB5NXd4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation for adding construction hooks (that may fail) to __pagetable_ctor(), make __pagetable_ctor() return a bool, propagate it to pagetable_*_ctor() and handle failure in the generic {pud,p4d,pgd}_alloc. Signed-off-by: Kevin Brodsky --- include/asm-generic/pgalloc.h | 15 ++++++++++++--- include/linux/mm.h | 21 ++++++++++----------- 2 files changed, 22 insertions(+), 14 deletions(-) diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 892ece4558a2..9962f7454d0c 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -173,7 +173,10 @@ static inline pud_t *__pud_alloc_one_noprof(struct mm_struct *mm, unsigned long if (!ptdesc) return NULL; - pagetable_pud_ctor(ptdesc); + if (!pagetable_pud_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } #define __pud_alloc_one(...) alloc_hooks(__pud_alloc_one_noprof(__VA_ARGS__)) @@ -227,7 +230,10 @@ static inline p4d_t *__p4d_alloc_one_noprof(struct mm_struct *mm, unsigned long if (!ptdesc) return NULL; - pagetable_p4d_ctor(ptdesc); + if (!pagetable_p4d_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } #define __p4d_alloc_one(...) alloc_hooks(__p4d_alloc_one_noprof(__VA_ARGS__)) @@ -271,7 +277,10 @@ static inline pgd_t *__pgd_alloc_noprof(struct mm_struct *mm, unsigned int order if (!ptdesc) return NULL; - pagetable_pgd_ctor(ptdesc); + if (!pagetable_pgd_ctor(ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } #define __pgd_alloc(...) alloc_hooks(__pgd_alloc_noprof(__VA_ARGS__)) diff --git a/include/linux/mm.h b/include/linux/mm.h index c3998b78f6a5..721e779647f3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2992,12 +2992,13 @@ static inline bool ptlock_init(struct ptdesc *ptdesc) { return true; } static inline void ptlock_free(struct ptdesc *ptdesc) {} #endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ -static inline void __pagetable_ctor(struct ptdesc *ptdesc) +static inline bool __pagetable_ctor(struct ptdesc *ptdesc) { struct folio *folio = ptdesc_folio(ptdesc); __folio_set_pgtable(folio); lruvec_stat_add_folio(folio, NR_PAGETABLE); + return true; } static inline void pagetable_dtor(struct ptdesc *ptdesc) @@ -3019,8 +3020,7 @@ static inline bool pagetable_pte_ctor(struct ptdesc *ptdesc) { if (!ptlock_init(ptdesc)) return false; - __pagetable_ctor(ptdesc); - return true; + return __pagetable_ctor(ptdesc); } pte_t *___pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp); @@ -3126,8 +3126,7 @@ static inline bool pagetable_pmd_ctor(struct ptdesc *ptdesc) if (!pmd_ptlock_init(ptdesc)) return false; ptdesc_pmd_pts_init(ptdesc); - __pagetable_ctor(ptdesc); - return true; + return __pagetable_ctor(ptdesc); } /* @@ -3149,19 +3148,19 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud) return ptl; } -static inline void pagetable_pud_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pud_ctor(struct ptdesc *ptdesc) { - __pagetable_ctor(ptdesc); + return __pagetable_ctor(ptdesc); } -static inline void pagetable_p4d_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_p4d_ctor(struct ptdesc *ptdesc) { - __pagetable_ctor(ptdesc); + return __pagetable_ctor(ptdesc); } -static inline void pagetable_pgd_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pgd_ctor(struct ptdesc *ptdesc) { - __pagetable_ctor(ptdesc); + return __pagetable_ctor(ptdesc); } extern void __init pagecache_init(void); From patchwork Mon Feb 3 10:18:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9711CC02192 for ; Mon, 3 Feb 2025 10:20:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2245B6B0083; Mon, 3 Feb 2025 05:20:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AEDE6B008A; Mon, 3 Feb 2025 05:20:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF4A2280016; Mon, 3 Feb 2025 05:20:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BDD14280002 for ; Mon, 3 Feb 2025 05:20:36 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 56B671C9E6D for ; Mon, 3 Feb 2025 10:20:26 +0000 (UTC) X-FDA: 83078238852.14.0038C4F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id C42F7C0006 for ; Mon, 3 Feb 2025 10:20:24 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578024; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w3d8h0fhxSI8KBCb1MJSQD5BFSFLbtAEyQ369Pj935A=; b=Cr4VLXXu4AbNgHYMzJDydGOG2hABBqmfeDiug/T+/TL0LZX2i71h39lsAlDmcd3eFwOQ9C nT8Sqwei/fkizeXjhDSd5Q3qv3LVPjechDYA4G8fRt5zWbsSMrhzHaGbMHwbAWOYoBgTcQ ptGVPmX+mkCi51jhdhiz782FuQ7/eq0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578024; a=rsa-sha256; cv=none; b=zj55BrhzwU3mpBGCDmL1YlTqU/5mBz/foFpiY9HcYEROZNW60ZdVkQsk/QgECo5v102mZN likCQZ93Mk6VZwH4VKXZnuLwOXqYxtZJ5FgdrUFTiFVrlI2Yeh4jPdmuAoftiJBGoz35cB QYa59ppXBSdHpeBBfqz48uY3RJcO6uo= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8059E1476; Mon, 3 Feb 2025 02:20:48 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8DE443F63F; Mon, 3 Feb 2025 02:20:20 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 11/15] mm: Map page tables with privileged pkey Date: Mon, 3 Feb 2025 10:18:35 +0000 Message-ID: <20250203101839.1223008-12-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C42F7C0006 X-Stat-Signature: po7c5tti3x1ocaewtgt7ecgdyb3sejxd X-Rspam-User: X-HE-Tag: 1738578024-724353 X-HE-Meta: U2FsdGVkX1+asfeizqJ0s1V5/kHXifHhrpQOLWjuYnMzUBco8xhlOQqn3XY9pRCs3F6NnVy8BtMZFAu3fN8j9JmdUwwtT+NYVKTdAlV36Sl3M7f8A6+aVe7IrxOlMmjSICnL7IlX72t4jmqPDxPFe0THgnI+UzYxEmEVDr2JHpgMARrn27Q49FjFHMP6UQDZ9RDfpLIkledH8nuDMQRyOMXojs7CZ5L8+h4PneuWYmLCg60zaKdIsXgCQRy4EXgOdNp1hYpXzmGMrjtidADQJ8KdVTEAo/FAXWwQ99onLg5qqUc8nU/5mASiJY7L1WJdz2p5vW/0E3bfBdCH3mMVmw0Ae0aRjhcOnVNbbmr/FksdhPKTPNqTIBR2wflel+oXGT6zD22cP7QfUBiasurXFetm8u5fZ1mR88s6Nbrl1hPUSXZ5zCmnINMRYuxQKR28tpIcH5zUTkOjibEXv5MfqrxyVtfdicoVO3SCiK5iFDso3gbaspR3yLlLyb1JSQcxWX7EtoN1/NbmIRmkyz6DAu4kDEElIM9Z5vd6gDjJ7J/3U6nbv6NBp5WHE+R4ReTrsd4sAsN9/8U0/H5s6RR+EuxFVro0RSNBvWy5aH2/3ANe0ENLTmhYwiYUJN5wRQiV2am+E5MNcYM0vBwElcx4j2+5d/MN3SGEizUDRgXAsW9A2leji6jmRkElaV9fzwMCQBTU9GR1nhbbVCMe5sOUiNiKvldQxxKzoD8/YPqr3DeuEdNiTECz0NGWh0WSsdkJkHJsRAMKAjD4wzatF02Toe+rTY27UHDV0ePRZLk0VWksBNWHDwXIjxUSGbaST6byX6pYNtLfBcHuZcXjtHsgCkHGHh4TM40zls692jg7YDKdUvWWkofKk6RXf9fAOe8JeherLxkYwIj7zj9tmlqGK/rF0QcOX3WWWkBAbp2JrIzG4wshEtKp8dp/WI+N0SuBoNY3thC3Ax5S9VAPh9F vUsRf4NS GxAIoM18hQuy2f/18fyKySAeJLpeWyuTITcAn/2Gx6mzrhfPHz8kSQgNMsy8Xopam6wCERBD1iAXEhE8p/mm0KG2FO9ITPtjpURfsCSivWYKBaRf6JM9nzGeujcNNm/LHf4PeeIVSYzPVujTLax9hx95Dux6v2wG8GCFbqMAmLpvtU2tVSnAArbyiA6SIDgCJ/v1h X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, map allocated page table pages using a privileged pkey (KPKEYS_PKEY_PGTABLES), so that page tables can only be written under guard(kpkeys_hardened_pgtables). This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 721e779647f3..aa01f51fdc6f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -32,6 +32,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -2998,6 +2999,8 @@ static inline bool __pagetable_ctor(struct ptdesc *ptdesc) __folio_set_pgtable(folio); lruvec_stat_add_folio(folio, NR_PAGETABLE); + if (kpkeys_protect_pgtable_memory(folio)) + return false; return true; } @@ -3008,6 +3011,7 @@ static inline void pagetable_dtor(struct ptdesc *ptdesc) ptlock_free(ptdesc); __folio_clear_pgtable(folio); lruvec_stat_sub_folio(folio, NR_PAGETABLE); + kpkeys_unprotect_pgtable_memory(folio); } static inline void pagetable_dtor_free(struct ptdesc *ptdesc) From patchwork Mon Feb 3 10:18:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D10AC02192 for ; Mon, 3 Feb 2025 10:20:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F13A280002; Mon, 3 Feb 2025 05:20:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 72946280017; Mon, 3 Feb 2025 05:20:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46DF0280018; Mon, 3 Feb 2025 05:20:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1C0F7280017 for ; Mon, 3 Feb 2025 05:20:46 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 497A1B4FC4 for ; Mon, 3 Feb 2025 10:20:30 +0000 (UTC) X-FDA: 83078239062.13.BB015E1 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id A3294A000E for ; Mon, 3 Feb 2025 10:20:28 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iT19TPacBkHeIN8ActdEF8KEGxhSD8l1/qAasL3IKEI=; b=SwTrXrS9wllt2LrRJMiry/zAgoBWWMpmXItLsPD6waLr8CU4tZRpZXzpb59r/yEbbwSR32 ehbU0+ivJcHtVRhqmopahNuR9AW9r6wv6n0MqOnRP/PGsRSrQ/IGUTCTC6dvy7l4DQHQzK KiWjWFcN6oJPxCeqfB0rLoDIVrExqqA= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578028; a=rsa-sha256; cv=none; b=H2jxyh7OBb2A+F8m2ZJCA5CwjZldPKuDnkTO6u66HcrdNgAkjRo61HukVCIGtog4ZcFmfB daI38XEEx0WGMFLnaIVxk3XJcWQQOLLVnayYi/r3Yhp1s0brP8bos9pyPK60Xb0I4GZOOo NksCQ1VaoA3LSwBwCyY4KM3JJOFey8w= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 52D9E11FB; Mon, 3 Feb 2025 02:20:52 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 617B33F63F; Mon, 3 Feb 2025 02:20:24 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 12/15] arm64: kpkeys: Support KPKEYS_LVL_PGTABLES Date: Mon, 3 Feb 2025 10:18:36 +0000 Message-ID: <20250203101839.1223008-13-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A3294A000E X-Stat-Signature: 5arqrsnwzyczuexm4q5mtte83wqwtizx X-Rspam-User: X-HE-Tag: 1738578028-272770 X-HE-Meta: U2FsdGVkX1+zCpKpWhGuZODkKmsLO2Xh2n5CmtVo2Q0DOoMnG72NCiZw8k5NksQdjQkr7pCJ1kKxAtwtUrY/oPwc5FiZq8G/ZpPmqy0gFZ2KC0TZfiPIKfIveXYjvy1/AgICus2bZALXXpjQLA23vW8DHu2pBI4zOol38UdM+oCm1xg1oJ+qfzvNlAvj66/1T8WwaB7cRdv0RtP2wGjblNUyEiNDdvbs7rJa7Ruh6SQ+eElCz8Jp3kmMICjT6bwZCIoNDzBdX6mKm6G/z7MisRBAXiPBWo6QWB7GRz4+9U9JQmEqPsWCE0+5qe9NgwaLYASCxpYWQ+dxmzuo8hT0dad5dcZwbiQs099qJGmMYh9vM4Jd1Ruge/6qCi+RyOkiq4oQUD9q9c/51UddOIM9RRxQMSajE1NJ5O+VgaZJCsPv4+8GeVeCwWxPedFlocVAdXPPD73NHJONUHcHoa3GbSGJXaVApcc+z6xslX6Yt25zgffKs3aFIPY4qlQ3DmigY9pek4J2W9CPjx9qv3D825y8vizG2v1p6frssp6onPcgxSc5m64BCKni8e9UiHHcszj7wWPVGiY3VupwZ1EDa9cIG7HMpsqx7eKc6JeHPu8sj/mlcKQUtKkAUxhWM/ZVvBoIGCNmNb8yTkaKiBJdlZNhuLDArhF/RCZpPneG7VZ/0zpgOEdfrO4uTgkuQIq+ODKFHyq5LBaJI5b77lxE0SDEEThsoT50vX+8lIT+T7mVR9ko3hr7Q8H0lrUsaQN+1TO0Z+PDh94Hri4T6nGiRqBsXzfri2prTAdDF5S9oHAyaDQqscDmjLsMuZ2iL9mD05dKv6ofhoFNXEEHq77lm7wRMgM0hklVFpBQSwGfU/5CT/aN/15wNKim4DHmDPJeX/fXGh+V43Nc7SVUfKrWPl5SKbADtXBFvlW3XfjXEODybNAfrCQjivpuyptIUG3G2nwDSWg5doWSCBMFpyN 0O6wwZty KAOVLNNgsatXKDmg+rzrim53lRokvu/g7zeWgTgulVZkeldEszYEtU+uhMI8PBBVwyyweWUA7tSIjSDu/voHCZvP98yOhzH2fymX+gaRwMwxjEDlPVPG/o24bQjuGf7Gfy+b3sWRkvautQclx0w7iOu0V2bsn0kq10qO56QxVBLlg782CpzaNSjofOlzMYOGyKlLAIQ/RiOnsZBjcIVi3Si4h9HjV8REVVf7/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Enable RW access to KPKEYS_PKEY_PGTABLES (used to map page table pages) if switching to KPKEYS_LVL_PGTABLES, otherwise only grant RO access. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index e17f6df41873..4854e1f3babd 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -18,6 +18,8 @@ static inline bool arch_kpkeys_enabled(void) static inline u64 por_set_kpkeys_level(u64 por, int level) { por = por_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RXW); + por = por_set_pkey_perms(por, KPKEYS_PKEY_PGTABLES, + level == KPKEYS_LVL_PGTABLES ? POE_RW : POE_R); return por; } From patchwork Mon Feb 3 10:18:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00040C02192 for ; Mon, 3 Feb 2025 10:20:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89033280017; Mon, 3 Feb 2025 05:20:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8166B280012; Mon, 3 Feb 2025 05:20:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 668B7280017; Mon, 3 Feb 2025 05:20:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4199E280012 for ; Mon, 3 Feb 2025 05:20:58 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 18E2D4BEF7 for ; Mon, 3 Feb 2025 10:20:34 +0000 (UTC) X-FDA: 83078239188.06.8B1487B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP id 5A3C04000B for ; Mon, 3 Feb 2025 10:20:32 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578032; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MjcCREu7o3D3vgXRrg7Q6JZZYY2EnTmko3puDev0uB0=; b=6eFZTp/ut1RJUmYNexW+gBuNw/M8ywCD+AgfJaC+sL+4JgCRNLY0ahnJKT8UFdK2bXGhhg rTKo6VuLhU8JT6NTzQ+cn4NqKjqnimjpc1X12p2DJtkYOxFkrFMlz7a5oXLUIsw2+oKHjE wMKVdilfSHZVrsnNzfadSFTlMWf6agU= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578032; a=rsa-sha256; cv=none; b=m1fY1T13ydw92wZhuQlR11u/AVWExhgl4vw08k5Dfntt9nYLU6y3dct1DuvUnJM5H2DJB9 XFwwzMd4VVcfhLxyj2dcSgMsqfxhxZNSmNRy9nL4dFWdMlfyHjUo8yCpTftDzNjgTqC4q8 v4b0WmgF9yotLYLpIsVUawGgqstELFo= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 268A51682; Mon, 3 Feb 2025 02:20:56 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 34BDF3F63F; Mon, 3 Feb 2025 02:20:28 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 13/15] arm64: mm: Guard page table writes with kpkeys Date: Mon, 3 Feb 2025 10:18:37 +0000 Message-ID: <20250203101839.1223008-14-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5A3C04000B X-Stat-Signature: 6jud5pmgpedtbu5n1xkqt165qpxmmatd X-Rspam-User: X-HE-Tag: 1738578032-761787 X-HE-Meta: U2FsdGVkX18c8h6en15KV5Cae1kuf08Zxsv5caa8nCCpERXBvqksUsXrXgjUiNys2rFRuG49yxQ+ddX3TnzErMWMLnkTLEISgSDrU2icpCmXFRasL5X+e57iJ3aeJLwcBxTLQ2ODSQEgza4m9jtElRZZG0lWowSLP3ltbyLEB8XXV6MpLiIPMT1kZj6NaBk811jGOTkrPbznukq9JNJj96AX5eBluW6Z3mEC7AUrZa4bZduq/Cy8s4gvDhuMdssQ8iDeISkfKhwl5x2wmZfRlKoXY0X/yoKtDlqzfNBINvxNtklYkXF4TOB8g2WqK1SnzDfw+cLZ5qWD0GXvaw5hFagnVmdgBU/1oYd70Xnod2U0P00RAsYsxBUcrhml6p1evDpKfg3oYzgaVKxiYXnpIF8BUwNL1vsUlfGwQoThyiW0Sf7Z1OxXgqrZU1fKFgjKk/wuaF4Dn9fhN8EmaKeBla/qOkEovL5G7Q7Rse1dYFctlGxxPo6vnDQznB6uQ4QM/2XwDbMW8rg8m0JkD9zVYe5Ga1QwtdSBF8qpSexLF4lw+sHdecD7XY7ol1/kjgxOxIN5x26O2f3oWq8qy9OEfMvdohRvVXurc523cOHgZ5gEhEjdWgSAqG7pzqjI4LPTeDmeXvrxSTQoMSxv9B5Hho1kyw9ggNV/k1B+jODpm2aWfGahyJukgRlZmxzA40WeHUlwD67tfSU0/SHt8NIdIpAmecVKyBwvcu4a2qPQBwRSo4Os/e1O3v5x+mKy0YKrKrOdQ1ciPgwPaZg94OFZyRGafDw6fUfMw/Pb2Lr2L1Asy6G8srRHkN/ZmxNTPiuTpVNCYXQqbdpkOl3h1DyUWD0J9U3/YkRvjJVQ/o4vVDh/9ABSqNnRxbdTObQ3gv6OsOEpbVTyvGCILNt5UZ2X5A0vABuQFOx/1OtV48wWEGTmjFUTbvRkhsWP0yOJyHROfZOMks9Rg5wkyiULec9 met3BGK8 Kj5QVUSa80xPp8RtKmV+SHnD/6Yu3IFRxP5mkD3demkhb96V9s72xqVFPJ/F5YEf8N/xx+YWJBL+q10aPwT2MRwTqWpoePvFyR6e9cNTB0LW8ycvig4LtH6DNFdHnFv944wfGfpUWakXW1VlgaX6F9YAVEFMnXUL+wdCE7bKeGKplh+/8x3fdVBLHNl7exZA9YCg0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, page tables (both user and kernel) are mapped with a privileged pkey in the linear mapping. As a result, they can only be written under the kpkeys_hardened_pgtables guard, which sets POR_EL1 appropriately to allow such writes. Use this guard wherever page tables genuinely need to be written, keeping its scope as small as possible (so that POR_EL1 is reset as fast as possible). Where atomics are involved, the guard's scope encompasses the whole loop to avoid switching POR_EL1 unnecessarily. This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable.h | 20 ++++++++++++++++++-- arch/arm64/mm/fault.c | 2 ++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 0b2a2ad1b9e8..37ce03a6ab70 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -39,6 +39,7 @@ #include #include #include +#include #ifdef CONFIG_TRANSPARENT_HUGEPAGE #define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE @@ -314,6 +315,7 @@ static inline pte_t pte_clear_uffd_wp(pte_t pte) static inline void __set_pte_nosync(pte_t *ptep, pte_t pte) { + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*ptep, pte); } @@ -758,6 +760,7 @@ static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) } #endif /* __PAGETABLE_PMD_FOLDED */ + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pmdp, pmd); if (pmd_valid(pmd)) { @@ -825,6 +828,7 @@ static inline void set_pud(pud_t *pudp, pud_t pud) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pudp, pud); if (pud_valid(pud)) { @@ -906,6 +910,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*p4dp, p4d); dsb(ishst); isb(); @@ -1033,6 +1038,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pgdp, pgd); dsb(ishst); isb(); @@ -1233,6 +1239,7 @@ static inline int __ptep_test_and_clear_young(struct vm_area_struct *vma, { pte_t old_pte, pte; + guard(kpkeys_hardened_pgtables)(); pte = __ptep_get(ptep); do { old_pte = pte; @@ -1279,7 +1286,10 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma, static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, unsigned long address, pte_t *ptep) { - pte_t pte = __pte(xchg_relaxed(&pte_val(*ptep), 0)); + pte_t pte; + + scoped_guard(kpkeys_hardened_pgtables) + pte = __pte(xchg_relaxed(&pte_val(*ptep), 0)); page_table_check_pte_clear(mm, pte); @@ -1322,7 +1332,10 @@ static inline pte_t __get_and_clear_full_ptes(struct mm_struct *mm, static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long address, pmd_t *pmdp) { - pmd_t pmd = __pmd(xchg_relaxed(&pmd_val(*pmdp), 0)); + pmd_t pmd; + + scoped_guard(kpkeys_hardened_pgtables) + pmd = __pmd(xchg_relaxed(&pmd_val(*pmdp), 0)); page_table_check_pmd_clear(mm, pmd); @@ -1336,6 +1349,7 @@ static inline void ___ptep_set_wrprotect(struct mm_struct *mm, { pte_t old_pte; + guard(kpkeys_hardened_pgtables)(); do { old_pte = pte; pte = pte_wrprotect(pte); @@ -1369,6 +1383,7 @@ static inline void __clear_young_dirty_pte(struct vm_area_struct *vma, { pte_t old_pte; + guard(kpkeys_hardened_pgtables)(); do { old_pte = pte; @@ -1416,6 +1431,7 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp, pmd_t pmd) { page_table_check_pmd_set(vma->vm_mm, pmdp, pmd); + guard(kpkeys_hardened_pgtables)(); return __pmd(xchg_relaxed(&pmd_val(*pmdp), pmd_val(pmd))); } #endif diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ef63651099a9..ab45047155b9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -220,6 +220,8 @@ int __ptep_set_access_flags(struct vm_area_struct *vma, if (pte_same(pte, entry)) return 0; + guard(kpkeys_hardened_pgtables)(); + /* only preserve the access flags and write permission */ pte_val(entry) &= PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; From patchwork Mon Feb 3 10:18:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3B17C02192 for ; Mon, 3 Feb 2025 10:20:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64459280016; Mon, 3 Feb 2025 05:20:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CDBE280002; Mon, 3 Feb 2025 05:20:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F7B6280016; Mon, 3 Feb 2025 05:20:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1AC8A280002 for ; Mon, 3 Feb 2025 05:20:42 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BEC631A265A for ; Mon, 3 Feb 2025 10:20:37 +0000 (UTC) X-FDA: 83078239314.29.F9C7725 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 38CC1120007 for ; Mon, 3 Feb 2025 10:20:36 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578036; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4DSfztf+YFHh7lJoCiiO7DeGAKOV5B4L4xTaMeERC30=; b=vX2uXWS4op5ebt/3UdF7bkJAM1FGhpfmHX+xPLX2n9viXjHYV0lBUihsBnUbUZQdhUO1Ej Cis/uwEsS0vmyC2bXlPheS04YfdxW8PawLBQ2GbBVBobd8Q1oeArXD/C0ubsBJnfrFckow obmeXbKVFkaMQyejT9ZilbJ/Gq49AU4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578036; a=rsa-sha256; cv=none; b=PULE+462ISbOcQ/V4UBpv88PmLVadxHyUFjPhwUM9zfeubAWpFU1vzseHEJXV84Qalx68M c+i9OUZo7b6fyBaToNoUjEgsqx2zDDPVObf9vyRGGPyhHZ7fPJCzMOtC/GJ+UBng3Dyvtk HzsMPwpNnvlX95/u3Ta+Nq6qsNEpnYA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EDDA811FB; Mon, 3 Feb 2025 02:20:59 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 086153F63F; Mon, 3 Feb 2025 02:20:31 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 14/15] arm64: Enable kpkeys_hardened_pgtables support Date: Mon, 3 Feb 2025 10:18:38 +0000 Message-ID: <20250203101839.1223008-15-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: m6f9p7syu4wg9ahwmixx39qt3fhochsj X-Rspam-User: X-Rspamd-Queue-Id: 38CC1120007 X-Rspamd-Server: rspam03 X-HE-Tag: 1738578036-686802 X-HE-Meta: U2FsdGVkX19gE8F8rX4C2IFycJ1Z50vAoauCvpenjfPMLsC9jCbLvGd4YQa2NqPwMjPKcTQhQfuMAJpBdE/eKUu2yULUtK7lS02QccHc+Aq58pbUFIEq5UI0LoXmr+6n5JHJ7hDwTfaa6XHKHkwcH+NtZZdybRYUud9KdOyotckv8Vh6NfeAtUyWESlP9Mwtax9l1FEWclEsb0xjdCK/aPbB2w2NaIPGz/2c/tNBVtRw99ZE9DbwalCRmVzh9t4YkOKRAKfo6keUQChJOAi1RNWKaWMi8LG390d8Ilv6NcQLDEdD4sKBdPJDpOzfzagToD78zbpwyTUFz8fZvZkFzDbOCpUiO5fTQbhdb/0n6bdlThQ3CqkQ1siRKX4tO4D34XrPceO2nK2tGqYQ10omf+hGo6haAcH74ltTExuLBcAzKcDVU/4H/FSiKHPCU3/T6awAVA56NdQCRPtglyyKcCHQKOo/ejnc5bgSVzQI1Z0YgcBeS/Tz7y1xzQFneX7hYoYSShobhuaGmVSr66E4/MNIC/BN+PbZbYFA66H5b3BfUHmQt7AUszQcIPYCSt3qjnZp40/ru1jgWs/Oo91D6DtwBsHxCBUmcgVlAVUSRVR4Pokxtgedocn4iciYjVqpjB9J5OxGyXV0Zk17coaipN2rRrxn7Svfvs7lYvIe70l6MohD0+XdOT8D5OOZiyRLbQO6HvFi4ra+t1OT+eU8UM8Zi/xt+jyxnhX6UnYGLIbtPcFg8E15NCpYFMEruNccfvm+0fghgK7rW0oIWUk6iMmv350LixOnmUfFvOtFGT+tMVIJz1fObD7yDbcww8xILtWmZ03F7OkWkwHdOgkqXyeKPmBkMPLAWzTj4/XUQJW9pqVgVvLyj7X1GG9M9pNYQt82xhcDHmEygmwuHk2jQDUWJ8bQhAB+azmsbu1zajoKyMNRxiRHXr7/Fx5qEanj/2FJ7PByvWFNMPIgNNj D9Nd5Wd2 JppOQF81ZQVYm6wFZ7+pZ594/kEgAXPO//0rmxl643Fnils1CcWSpbYO7D40cFRGfOV2VIsUN+li+A1V9DmDVKRXV890tAiQ4ZQ8lIK+WEkIBc8vwqBt6m65zS5Q5qScghllsCP3W0oOT8YGk4CpC/rTTwZxwuf0pYidNdoRfv7gKsU9zXKcvPgW+HUNhdAv2lWj7+ora0OojB5JJdUWuM9G2rg1l1AwMiWb2QFQum3QhlS2+O8qBMLTREvWrYAQZY/2xO5utuRPvakGEwFrNjqCB4LRc7uG2HqMy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys_hardened_pgtables should be enabled as early as possible (if selected). It does however require kpkeys being available, which means on arm64 POE being detected and enabled. POE is a boot feature, so calling kpkeys_hardened_pgtables_enable() just after setup_boot_cpu_features() in smp_prepare_boot_cpu() is the best we can do. With that done, all the bits are in place and we can advertise support for kpkeys_hardened_pgtables by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES if ARM64_POE is enabled. Signed-off-by: Kevin Brodsky --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/smp.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 34f15348ada8..df26902d385c 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2187,6 +2187,7 @@ config ARM64_POE select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS select ARCH_HAS_KPKEYS + select ARCH_HAS_KPKEYS_HARDENED_PGTABLES help The Permission Overlay Extension is used to implement Memory Protection Keys. Memory Protection Keys provides a mechanism for diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 3b3f6b56e733..074cab55f9db 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -468,6 +469,7 @@ void __init smp_prepare_boot_cpu(void) if (system_uses_irq_prio_masking()) init_gic_priority_masking(); + kpkeys_hardened_pgtables_enable(); kasan_init_hw_tags(); /* Init percpu seeds for random tags after cpus are set up. */ kasan_init_sw_tags(); From patchwork Mon Feb 3 10:18:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFF10C02193 for ; Mon, 3 Feb 2025 10:20:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4734B280019; Mon, 3 Feb 2025 05:20:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A8EC280002; Mon, 3 Feb 2025 05:20:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22564280018; Mon, 3 Feb 2025 05:20:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EF1D2280002 for ; Mon, 3 Feb 2025 05:20:45 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B7ABB120AAF for ; Mon, 3 Feb 2025 10:20:41 +0000 (UTC) X-FDA: 83078239482.02.FEFC364 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 0EE862000B for ; Mon, 3 Feb 2025 10:20:39 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uAEP4bJ4DAnLZnFE79/GJlPC9W5ScHGYEPhyiOeUp20=; b=Ji+RW6OSjXEBiMFagQCMXZg29QBYVOXrwbeZV0SWzfYAWq7D2ADG09bzslrvHzTDbDyAhc f1LI3av6wkum5Hj9BEujAQGsaXN/fK/K6FCwY2zlNWqSfAdhzr4a6DgDZvG+yT6VGKcs6x D/1VUyEKQ3e7/FwPNgMFsuEklyW9UQA= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578040; a=rsa-sha256; cv=none; b=y1pHLqS1+jz8ES6MQtGRqbWqYwQtiW4y8FqQq+08vRn1a3iX8rvNd4fxIKpiWdtNODxbR7 iegBnOdKfS9d2709MfaTT1hy1rTAgH8OIGtAldhWR7XA7jeBQMP3G/8l0nG4HL5KUCayJ2 2MerBIY2gbjQG6L4wESBGhy5AfJ9gy0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C21571476; Mon, 3 Feb 2025 02:21:03 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D01593F63F; Mon, 3 Feb 2025 02:20:35 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v3 15/15] mm: Add basic tests for kpkeys_hardened_pgtables Date: Mon, 3 Feb 2025 10:18:39 +0000 Message-ID: <20250203101839.1223008-16-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203101839.1223008-1-kevin.brodsky@arm.com> References: <20250203101839.1223008-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0EE862000B X-Stat-Signature: qat41jphqkspcgdiks5foeoj4xa57bbk X-Rspam-User: X-HE-Tag: 1738578039-905027 X-HE-Meta: U2FsdGVkX1+7GsN4kuDtDCUzpjUOjcpAxdmsccb2gfp1JVz0KaiMbTnrABfwrQLXqKHYkwkAkjaintyBua4lvJvSj6s9EudTixNW7i32cx1LyEgS1lvGpSu/AuMB25pP1wcRnhJmrgZg2nx/1plo6BRs3E2m6jNmCPYVyu18P6ovCAHlmcoQFZ9iRNqt7oeMbHDQZ8oqpBCCD6syL/xsHHxE55uZrh8oOvQ4hjFjabiAkt309Dkcf/WE0Yfs8nz8CaSb0q4xFJPQT3Te3F0g2XA9VfVqhiReb4waEXQ4xVr27fFAOFbX+wtdQkqOvPOD+hW1bJMMAn01VmJH/hX02s+LXjQo/llJiQQfs4qZceqXehgs18UOXC56sq6Wa9DtyHx1G4MMMgqX1vTpNWFm9iI7xQDzDn7oMprTnydBGY5wkSJUEdPyTxc/fRsRg0pu+KlLxr6dCcfvSzjoxM/kYCDvb2dTU5bXr3w8WwT/KZR2+MJmE2pV7V2wvJVLtjit/HG40RkDtBwJ5gW0XX5pGax0VtbYNwp8fybsWN92KbT3y6Hlcrnj2kNdQGMXdKPM2Di4dlvU/sFHEIhUTI6iJ+CkbZtPwKCbggq9pRCVHXOv7Ky7HSgNpTK+5e+AuHCScjSIkQ6YlBF3YDzE8xclMiu79YbauoPvt0MhElLoJuDkXs8WHiUdeKABzdowVtBLOVt/eRZ5lMTBy8pjwjnMXTr1aPyKPNxoYfa4C9F++bUswTzXYHLT2ZjBpqyM7gjOcBRfBNdEylwcB7Hetf2L8D6v0mOIPsSQa/SEFV6uC7aXLUp7w2DHuTvY5XQFwb/QrgaJjFtVMQxiP258ro42/yJp1So7SwS1ID/IL/Sc90Gf+kLdl9FtYES5dHxIQoSTTEdeWqG9UsSm3gfpE0a+crfGM+llFDCyaKWBV7LP+SszePv1QHG9PyQRWOQNmfJGTFkfhL8qRSC1X/UyXeS 9BWo54vQ 48IG2LUmtsn8kweWRpZZNhK+BLRqFt8N8JwFKA64yY0GNgVM/f10GimqDow1uPhJXeORZzfHpGBoqwY9zy8GHKZdkXRyGrDhOMqRfB1X2JnKa4N8liXH/p/9X87M02Xi9z2mr5JM8sCZKJh6bbMFaec/Zkue11XZycPoRVsLMAN+GdBupAAg7JDVsU0u0AEYpOUDt31EU2Fdilhw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add basic tests for the kpkeys_hardened_pgtables feature: try to perform a direct write to some kernel and user page table entry and ensure it fails. Signed-off-by: Kevin Brodsky --- mm/Makefile | 1 + mm/kpkeys_hardened_pgtables_test.c | 72 ++++++++++++++++++++++++++++++ security/Kconfig.hardening | 12 +++++ 3 files changed, 85 insertions(+) create mode 100644 mm/kpkeys_hardened_pgtables_test.c diff --git a/mm/Makefile b/mm/Makefile index 130691364172..f7263b7f45b8 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -148,3 +148,4 @@ obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES_TEST) += kpkeys_hardened_pgtables_test.o diff --git a/mm/kpkeys_hardened_pgtables_test.c b/mm/kpkeys_hardened_pgtables_test.c new file mode 100644 index 000000000000..86d862d43bea --- /dev/null +++ b/mm/kpkeys_hardened_pgtables_test.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +static void write_kernel_pte(struct kunit *test) +{ + pte_t *ptep; + pte_t pte; + int ret; + + /* + * The choice of address is mostly arbitrary - we just need a page + * that is definitely mapped, such as the current function. + */ + ptep = virt_to_kpte((unsigned long)&write_kernel_pte); + KUNIT_ASSERT_NOT_NULL_MSG(test, ptep, "Failed to get PTE"); + + pte = ptep_get(ptep); + pte = set_pte_bit(pte, __pgprot(PTE_WRITE)); + ret = copy_to_kernel_nofault(ptep, &pte, sizeof(pte)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PTE write wasn't prevented"); +} + +static void write_user_pmd(struct kunit *test) +{ + pmd_t *pmdp; + pmd_t pmd; + unsigned long uaddr; + int ret; + + uaddr = kunit_vm_mmap(test, NULL, 0, PAGE_SIZE, PROT_READ, + MAP_ANONYMOUS | MAP_PRIVATE | MAP_POPULATE, 0); + KUNIT_ASSERT_NE_MSG(test, uaddr, 0, "Could not create userspace mm"); + + /* We passed MAP_POPULATE so a PMD should already be allocated */ + pmdp = pmd_off(current->mm, uaddr); + KUNIT_ASSERT_NOT_NULL_MSG(test, pmdp, "Failed to get PMD"); + + pmd = pmdp_get(pmdp); + pmd = set_pmd_bit(pmd, __pgprot(PROT_SECT_NORMAL)); + ret = copy_to_kernel_nofault(pmdp, &pmd, sizeof(pmd)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PMD write wasn't prevented"); +} + +static int kpkeys_hardened_pgtables_suite_init(struct kunit_suite *suite) +{ + if (!arch_kpkeys_enabled()) { + pr_err("Cannot run kpkeys_hardened_pgtables tests: kpkeys are not supported\n"); + return 1; + } + + return 0; +} + +static struct kunit_case kpkeys_hardened_pgtables_test_cases[] = { + KUNIT_CASE(write_kernel_pte), + KUNIT_CASE(write_user_pmd), + {} +}; + +static struct kunit_suite kpkeys_hardened_pgtables_test_suite = { + .name = "Hardened pgtables using kpkeys", + .test_cases = kpkeys_hardened_pgtables_test_cases, + .suite_init = kpkeys_hardened_pgtables_suite_init, +}; +kunit_test_suite(kpkeys_hardened_pgtables_test_suite); + +MODULE_DESCRIPTION("Tests for the kpkeys_hardened_pgtables feature"); +MODULE_LICENSE("GPL"); diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index f729488eac56..649847535fc3 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -313,6 +313,18 @@ config KPKEYS_HARDENED_PGTABLES This option has no effect if the system does not support kernel pkeys. +config KPKEYS_HARDENED_PGTABLES_TEST + tristate "KUnit tests for kpkeys_hardened_pgtables" if !KUNIT_ALL_TESTS + depends on KPKEYS_HARDENED_PGTABLES + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable this option to check that the kpkeys_hardened_pgtables feature + functions as intended, i.e. prevents arbitrary writes to user and + kernel page tables. + + If unsure, say N. + endmenu config CC_HAS_RANDSTRUCT