From patchwork Fri Dec 6 10:10:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 989E0E77173 for ; Fri, 6 Dec 2024 10:15:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=EbTVnOiNcStaWUw5UawsePvamituuMQLd58uIG221JA=; b=S7Dunl8UblUrZuqCM9ithV5aN1 L7GH3tk9mh1wFtmorBv0yU99gwTwWLNfa/WpBGbK8AmtUXvUc7DbGgDVL1nvhMiacYlgsINE2tdrX eNtJMiLjq3b/L23nLIOk8zwTFGLXSk/vYg0dKoYtCXNKzDHyDUZJDK3dMpg/1TtPY/VTzIo6Nv8CJ 2K3x7JyUIFu02GbyjCBgIbvrztmTto3z535HKSZ+f6WXb/Y29Bm9BMYYNB36KnpJovmGjEXQEjsu4 gxfVcskNgjR2C1E5402ipiwfKBmFQ4yoIIaQi0F8Df7Kwz6smxmVk1adeKktB2YwhzLkIou+1QZPp GVmMi7rg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVMp-00000001Dmm-0SPQ; Fri, 06 Dec 2024 10:15:23 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVKm-00000001D9n-3R9l for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:18 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CFAEC15A1; Fri, 6 Dec 2024 02:13:43 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E643D3F71E; Fri, 6 Dec 2024 02:13:12 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 01/16] mm: Introduce kpkeys Date: Fri, 6 Dec 2024 10:10:55 +0000 Message-ID: <20241206101110.1646108-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021316_943608_BA4D22EE X-CRM114-Status: GOOD ( 18.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kpkeys is a simple framework to enable the use of protection keys (pkeys) to harden the kernel itself. This patch introduces the basic API in : a couple of functions to set and restore the pkey register and a macro to define guard objects. kpkeys introduces a new concept on top of pkeys: the kpkeys level. Each level is associated to a set of permissions for the pkeys managed by the kpkeys framework. kpkeys_set_level(lvl) sets those permissions according to lvl, and returns the original pkey register, to be later restored by kpkeys_restore_pkey_reg(). To start with, only KPKEYS_LVL_DEFAULT is available, which is meant to grant RW access to KPKEYS_PKEY_DEFAULT (i.e. all memory since this is the only available pkey for now). Because each architecture implementing pkeys uses a different representation for the pkey register, and may reserve certain pkeys for specific uses, support for kpkeys must be explicitly indicated by selecting ARCH_HAS_KPKEYS and defining the following functions in , in addition to the macros provided in : - arch_kpkeys_set_level() - arch_kpkeys_restore_pkey_reg() - arch_kpkeys_enabled() Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 9 +++++ include/linux/kpkeys.h | 67 ++++++++++++++++++++++++++++++++++++ mm/Kconfig | 2 ++ 3 files changed, 78 insertions(+) create mode 100644 include/asm-generic/kpkeys.h create mode 100644 include/linux/kpkeys.h diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h new file mode 100644 index 000000000000..3404ce249757 --- /dev/null +++ b/include/asm-generic/kpkeys.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_GENERIC_KPKEYS_H +#define __ASM_GENERIC_KPKEYS_H + +#ifndef KPKEYS_PKEY_DEFAULT +#define KPKEYS_PKEY_DEFAULT 0 +#endif + +#endif /* __ASM_GENERIC_KPKEYS_H */ diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h new file mode 100644 index 000000000000..bcc063425926 --- /dev/null +++ b/include/linux/kpkeys.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _LINUX_KPKEYS_H +#define _LINUX_KPKEYS_H + +#include +#include +#include + +#define KPKEYS_LVL_DEFAULT 0 + +#define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT + +#define KPKEYS_GUARD(_name, set_level, restore_pkey_reg) \ + __DEFINE_CLASS_IS_CONDITIONAL(_name, false); \ + DEFINE_CLASS(_name, u64, \ + restore_pkey_reg, set_level, void); \ + static inline void *class_##_name##_lock_ptr(u64 *_T) \ + { return _T; } + +#ifdef CONFIG_ARCH_HAS_KPKEYS + +#include + +/** + * kpkeys_set_level() - switch kpkeys level + * @level: the level to switch to + * + * Switches the kpkeys level to the specified value. @level must be a + * compile-time constant. The arch-specific pkey register will be updated + * accordingly, and the original value returned. + * + * Return: the original pkey register value. + */ +static inline u64 kpkeys_set_level(int level) +{ + BUILD_BUG_ON_MSG(!__builtin_constant_p(level), + "kpkeys_set_level() only takes constant levels"); + BUILD_BUG_ON_MSG(level < KPKEYS_LVL_MIN || level > KPKEYS_LVL_MAX, + "Invalid level passed to kpkeys_set_level()"); + + return arch_kpkeys_set_level(level); +} + +/** + * kpkeys_restore_pkey_reg() - restores a pkey register value + * @pkey_reg: the pkey register value to restore + * + * This function is meant to be passed the value returned by kpkeys_set_level(), + * in order to restore the pkey register to its original value (thus restoring + * the original kpkeys level). + */ +static inline void kpkeys_restore_pkey_reg(u64 pkey_reg) +{ + arch_kpkeys_restore_pkey_reg(pkey_reg); +} + +#else /* CONFIG_ARCH_HAS_KPKEYS */ + +static inline bool arch_kpkeys_enabled(void) +{ + return false; +} + +#endif /* CONFIG_ARCH_HAS_KPKEYS */ + +#endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 84000b016808..f51dffca9d4e 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1104,6 +1104,8 @@ config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS bool +config ARCH_HAS_KPKEYS + bool config ARCH_USES_PG_ARCH_2 bool From patchwork Fri Dec 6 10:10:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D739E77173 for ; Fri, 6 Dec 2024 10:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=DAvN/ymhOC9RH1764Pqdo1P8xtkdW4aJF6g43VRuQIg=; b=MCluhuXODMb87bNxnTF+v2xZKQ US8wLuXYHbyEEHoFxBYqGBaQBeK3bfqUf9Lp2YAAsJ66D2JCx6hXWJYT1+bApwo53c6y19YYG3KuS UxRHKVdLZ9Aw38lJMXJr23uf7aHqPIFscWi6mKfDs4rfnRMwtAJqAYX/4tx0k6/o+4uUtkQ/bNb72 pow7JxHZUy8JaP3ICIXP37pfBqGli+Ybcw6M4HAH2JwRoSYeWiqAv4Q/aTb1FAQHsugCTud9eyNEP r8loRu+9M6qZaHYaqZEl27OkbR9WyuNC59ezzngIOtpsae9Tg7V+l9HMVp8nXkz67WKanFfFnfHkH Z7hlK2AQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVNp-00000001E3x-2z6n; Fri, 06 Dec 2024 10:16:25 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVKr-00000001DBJ-3DFF for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:22 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 12CBC12FC; Fri, 6 Dec 2024 02:13:47 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 29AAC3F71E; Fri, 6 Dec 2024 02:13:16 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 02/16] set_memory: Introduce set_memory_pkey() stub Date: Fri, 6 Dec 2024 10:10:56 +0000 Message-ID: <20241206101110.1646108-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021321_845439_99E8259C X-CRM114-Status: UNSURE ( 7.65 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a new function, set_memory_pkey(), which sets the protection key (pkey) of pages in the specified linear mapping range. Architectures implementing kernel pkeys (kpkeys) must provide a suitable implementation; an empty stub is added as fallback. Signed-off-by: Kevin Brodsky --- include/linux/set_memory.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 3030d9245f5a..7b3a8bfde3c6 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -84,4 +84,11 @@ static inline int set_memory_decrypted(unsigned long addr, int numpages) } #endif /* CONFIG_ARCH_HAS_MEM_ENCRYPT */ +#ifndef CONFIG_ARCH_HAS_KPKEYS +static inline int set_memory_pkey(unsigned long addr, int numpages, int pkey) +{ + return 0; +} +#endif + #endif /* _LINUX_SET_MEMORY_H_ */ From patchwork Fri Dec 6 10:10:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75BC4E77173 for ; Fri, 6 Dec 2024 10:17:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SIEGBD80S/lUoDDrPSEcayu5EzxVe8qGIrwAYqVA0Yg=; b=b/WWsCm+k13lVNbCbM10XUHgtM fjLUbsV1ycTgMN8zmsNL/aU2f22oqJWEzCe3FtaYJ3oWlMPJe16y1YPl0Dn6SWZMBffaMxXL/byht zs1qesLoaMulF8CdVZPwOBw8b9s1AvrdvlbGRtLzgw0qUGulYi7Xqj8QRbdQH26w8k2YfqIyTWZyf TV0TIEKViLvd9nQ17ZaX4zUFkG7h2kuUdEIUXJ2Va64uLiiYQHJtMgVr/yexxydN3hnVkLrnDzxEh mlXC38lmGEGW8X0n6QAtGRgyjY71zbpt2Byhxy48vxSoISQJqEiHZbBKQlwziu2XjvJfpt9V7F+Tx PXkv+Qeg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVOq-00000001EeR-1hWt; Fri, 06 Dec 2024 10:17:28 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVKt-00000001DCR-0waX for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:24 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4A6A515A1; Fri, 6 Dec 2024 02:13:50 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 611A33F71E; Fri, 6 Dec 2024 02:13:19 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 03/16] arm64: mm: Enable overlays for all EL1 indirect permissions Date: Fri, 6 Dec 2024 10:10:57 +0000 Message-ID: <20241206101110.1646108-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021323_301653_4B85C044 X-CRM114-Status: UNSURE ( 8.69 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation of using POE inside the kernel, enable "Overlay applied" for all stage 1 base permissions in PIR_EL1. This ensures that the permissions set in POR_EL1 affect all kernel mappings. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable-prot.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 9f9cf13bbd95..a1c4f3837ea9 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -174,13 +174,13 @@ static inline bool __pure lpa2_is_enabled(void) PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_GCS_RO), PIE_NONE_O) | \ PIRx_ELx_PERM(pte_pi_index(_PAGE_EXECONLY), PIE_NONE_O) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY), PIE_R) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_ROX), PIE_RX) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_EXEC), PIE_RWX) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_RO), PIE_R) | \ - PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL), PIE_RW)) + PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY_EXEC), PIE_R_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED_EXEC), PIE_RW_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_READONLY), PIE_R_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_SHARED), PIE_RW_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_ROX), PIE_RX_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_EXEC), PIE_RWX_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL_RO), PIE_R_O) | \ + PIRx_ELx_PERM(pte_pi_index(_PAGE_KERNEL), PIE_RW_O)) #endif /* __ASM_PGTABLE_PROT_H */ From patchwork Fri Dec 6 10:10:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5383BE77173 for ; Fri, 6 Dec 2024 10:18:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=z0mOxBustPxojlu+qPM7wRKUHEpzeQd+SLH4Hfq3xaM=; b=WpO+SUsyQlxDqRsceOwpH5ESPs 006v2VaqV684rxYJ9MpOaM7XMu8E5fQ3lhNkK5aAXlzlUpu2qHaLeS5sqYheZLZgd7TdEPm02oQY8 gFG/m9PKBRtsLTVtmRmHlhdYEgS7f/KVZf/8tyimBksP6Xo7Dp102nwj2MndF7jkqmNpImQJ3R29J M7o4AOWCDQJvL48i54rDxwamvjzKY9k5Wfx/NiDbwbC7gSvK1qrMIcknmttQd0gUaPVRWOGZJGD1v +t+lzEtqJyNFrc0E9bcqL6Ftd1TQk6RrehYXtrgDPXJ5o0cQgePLWW/nSIbnJu7EqKq9faeJb8SiU m3WVxTcA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVPq-00000001F6K-2hCV; Fri, 06 Dec 2024 10:18:30 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVKw-00000001DDz-1HJM for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:27 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8224512FC; Fri, 6 Dec 2024 02:13:53 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 98E183F71E; Fri, 6 Dec 2024 02:13:22 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 04/16] arm64: Introduce por_set_pkey_perms() helper Date: Fri, 6 Dec 2024 10:10:58 +0000 Message-ID: <20241206101110.1646108-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021326_381102_1E999A2C X-CRM114-Status: GOOD ( 11.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a helper that sets the permissions of a given pkey (POIndex) in the POR_ELx format, and make use of it in arch_set_user_pkey_access(). Also ensure that is included in asm/por.h to provide the POE_* definitions. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/por.h | 9 +++++++++ arch/arm64/mm/mmu.c | 28 ++++++++++------------------ 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/por.h b/arch/arm64/include/asm/por.h index e06e9f473675..7f0d73980cce 100644 --- a/arch/arm64/include/asm/por.h +++ b/arch/arm64/include/asm/por.h @@ -6,6 +6,8 @@ #ifndef _ASM_ARM64_POR_H #define _ASM_ARM64_POR_H +#include + #define POR_BITS_PER_PKEY 4 #define POR_ELx_IDX(por_elx, idx) (((por_elx) >> ((idx) * POR_BITS_PER_PKEY)) & 0xf) @@ -30,4 +32,11 @@ static inline bool por_elx_allows_exec(u64 por, u8 pkey) return perm & POE_X; } +static inline u64 por_set_pkey_perms(u64 por, u8 pkey, u64 perms) +{ + u64 shift = pkey * POR_BITS_PER_PKEY; + + return (por & ~(POE_MASK << shift)) | (perms << shift); +} + #endif /* _ASM_ARM64_POR_H */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index e2739b69e11b..20e0390ee382 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1554,9 +1554,8 @@ void __cpu_replace_ttbr1(pgd_t *pgdp, bool cnp) #ifdef CONFIG_ARCH_HAS_PKEYS int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long init_val) { - u64 new_por = POE_RXW; - u64 old_por; - u64 pkey_shift; + u64 new_perms; + u64 por; if (!system_supports_poe()) return -ENOSPC; @@ -1570,26 +1569,19 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long i return -EINVAL; /* Set the bits we need in POR: */ - new_por = POE_RXW; + new_perms = POE_RXW; if (init_val & PKEY_DISABLE_WRITE) - new_por &= ~POE_W; + new_perms &= ~POE_W; if (init_val & PKEY_DISABLE_ACCESS) - new_por &= ~POE_RW; + new_perms &= ~POE_RW; if (init_val & PKEY_DISABLE_READ) - new_por &= ~POE_R; + new_perms &= ~POE_R; if (init_val & PKEY_DISABLE_EXECUTE) - new_por &= ~POE_X; + new_perms &= ~POE_X; - /* Shift the bits in to the correct place in POR for pkey: */ - pkey_shift = pkey * POR_BITS_PER_PKEY; - new_por <<= pkey_shift; - - /* Get old POR and mask off any old bits in place: */ - old_por = read_sysreg_s(SYS_POR_EL0); - old_por &= ~(POE_MASK << pkey_shift); - - /* Write old part along with new part: */ - write_sysreg_s(old_por | new_por, SYS_POR_EL0); + por = read_sysreg_s(SYS_POR_EL0); + por = por_set_pkey_perms(por, pkey, new_perms); + write_sysreg_s(por, SYS_POR_EL0); return 0; } From patchwork Fri Dec 6 10:10:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B854EE77173 for ; Fri, 6 Dec 2024 10:19:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Snhdnks8KmGT9o0/ZDnykraCyor+6xfRkmoZ7dE4fso=; b=aLPleZtymRh2kFsO93C2wkCzmU ci3iRy2uqVh2H+Y3fSPaKUw7EtQy1wy92BmnPxaqxyif8Ha7Pom9bu22Mvv32TmjJ20q0j8UdlDzC HgZZWu+Z5rV6I6gtjFYU4hGPPPv4YXQwWaDFoLRgBFrh8XmzpNA66dyo2/gRC1rFXIoaaPY6x5kuQ rtF8aTFZItiAzvjoL3g1ToktAzEqFf4YGQ/z4Wy69PlLItkQBHuJ/7TbFxWsaJXSVGtwNyGX28JCl kzdfMtglrk7dw23IO/u2G23N7iorDyFNIVTFDv1yiaarjSNm/eTPRdEL57a/bZXXRSyJ23FthBMxq 5IfJBizg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVQr-00000001FTr-1n3a; Fri, 06 Dec 2024 10:19:33 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVKz-00000001DFT-1ddm for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BA05612FC; Fri, 6 Dec 2024 02:13:56 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D091A3F71E; Fri, 6 Dec 2024 02:13:25 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 05/16] arm64: Implement asm/kpkeys.h using POE Date: Fri, 6 Dec 2024 10:10:59 +0000 Message-ID: <20241206101110.1646108-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021329_482945_D0D29B85 X-CRM114-Status: GOOD ( 11.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Implement the kpkeys interface if CONFIG_ARM64_POE is enabled. The permissions for KPKEYS_PKEY_DEFAULT (pkey 0) are set to RWX as this pkey is also used for code mappings. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 43 +++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 arch/arm64/include/asm/kpkeys.h diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h new file mode 100644 index 000000000000..e17f6df41873 --- /dev/null +++ b/arch/arm64/include/asm/kpkeys.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_KPKEYS_H +#define __ASM_KPKEYS_H + +#include +#include +#include + +#include + +static inline bool arch_kpkeys_enabled(void) +{ + return system_supports_poe(); +} + +#ifdef CONFIG_ARM64_POE + +static inline u64 por_set_kpkeys_level(u64 por, int level) +{ + por = por_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RXW); + + return por; +} + +static inline int arch_kpkeys_set_level(int level) +{ + u64 prev_por = read_sysreg_s(SYS_POR_EL1); + + write_sysreg_s(por_set_kpkeys_level(prev_por, level), SYS_POR_EL1); + isb(); + + return prev_por; +} + +static inline void arch_kpkeys_restore_pkey_reg(u64 pkey_reg) +{ + write_sysreg_s(pkey_reg, SYS_POR_EL1); + isb(); +} + +#endif /* CONFIG_ARM64_POE */ + +#endif /* __ASM_KPKEYS_H */ From patchwork Fri Dec 6 10:11:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5CC61E7717B for ; Fri, 6 Dec 2024 10:20:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dVSpYM+QE00njreDMoRWvdvnnjf642S716a8thAJyH4=; b=pThi6lZQ8qtk9snGkNdiDs/ZZi HqGcqD7rtuZU8nX+7RWtQdlRgMmTYJkeMxTN7zwTbjfz+LrNWi/fYuGgCSUM0JYiBDpEKQbNlGalj wUIEH/NnHUKgE9qo2UzopWEic1L6pvAmoPZkCynf7mbFV9v0VC9TC3U/sol6iCF9tci2p24pv2Afx n8p0X7e/wQCl/mDg3zP3Cc4hzUaFwe+52hctg2ItY1UYovz9Hfi45B6vNwSYt0XWrd+H3YVal8ccd vOQbjBD4GmU4VSQXHw5dce1PchgiH42CxE5/c+qfCoAUVOihrEQwdlI4Nog2CXdyPF7vDayENelnD q4l0rhkg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVRr-00000001Fjy-0UCx; Fri, 06 Dec 2024 10:20:35 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVL2-00000001DH4-3MuN for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:33 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F1AD712FC; Fri, 6 Dec 2024 02:13:59 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 142D83F71E; Fri, 6 Dec 2024 02:13:28 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 06/16] arm64: set_memory: Implement set_memory_pkey() Date: Fri, 6 Dec 2024 10:11:00 +0000 Message-ID: <20241206101110.1646108-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021332_882389_2BFB8F92 X-CRM114-Status: UNSURE ( 9.85 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Implement set_memory_pkey() using POE if supported. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/set_memory.h | 4 ++++ arch/arm64/mm/pageattr.c | 21 +++++++++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 90f61b17275e..b6cd6de34abf 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -19,4 +19,8 @@ bool kernel_page_present(struct page *page); int set_memory_encrypted(unsigned long addr, int numpages); int set_memory_decrypted(unsigned long addr, int numpages); +#ifdef CONFIG_ARCH_HAS_KPKEYS +int set_memory_pkey(unsigned long addr, int numpages, int pkey); +#endif + #endif /* _ASM_ARM64_SET_MEMORY_H */ diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 39fd1f7ff02a..3b8fec532b18 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -292,6 +292,27 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid) return set_memory_valid(addr, nr, valid); } +#ifdef CONFIG_ARCH_HAS_KPKEYS +int set_memory_pkey(unsigned long addr, int numpages, int pkey) +{ + unsigned long set_prot = 0; + + if (!system_supports_poe()) + return 0; + + if (!__is_lm_address(addr)) + return -EINVAL; + + set_prot |= pkey & BIT(0) ? PTE_PO_IDX_0 : 0; + set_prot |= pkey & BIT(1) ? PTE_PO_IDX_1 : 0; + set_prot |= pkey & BIT(2) ? PTE_PO_IDX_2 : 0; + + return __change_memory_common(addr, PAGE_SIZE * numpages, + __pgprot(set_prot), + __pgprot(PTE_PO_IDX_MASK)); +} +#endif + #ifdef CONFIG_DEBUG_PAGEALLOC /* * This is - apart from the return value - doing the same From patchwork Fri Dec 6 10:11:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7933E77173 for ; Fri, 6 Dec 2024 10:21:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zWUViwKb9YyQ8v4y65dgLraNpNyCQDT2uHU/J7LxPdg=; b=4VZxk9oJ69+zEZeV/JTfmn8ukh tRdXhE3F8jDVhtqx6ezDOD36oyEuZdX6ELLazHp10p7NcGWO+Z+oyiluCfelrIn1caoAKLL6H5ck2 9b6YBOSUPsTHH45/28uV7bTO0oQXMm2RkojfNKrTN54K1oik6idGhyccVtlTg7Ye8Y07ZRVin/IKa 7kQajm684mv779aLTPkix0YZCu9tPX2Akd1g5BNjur++D0pNSJL+xHdJyYabTMhckjHk7Rf9+4ndl CTb4Vi8qW3T07b1wCIsgtIjwRsGe72EAZI0zj85Va2Nl7bIDtr9fHnEotL9Q6J2DiJQ2lSjPqYU8W a9nO8R3w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVSq-00000001G2U-3AT5; Fri, 06 Dec 2024 10:21:36 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVL5-00000001DIJ-28nc for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:37 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 358B312FC; Fri, 6 Dec 2024 02:14:03 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4BFBA3F71E; Fri, 6 Dec 2024 02:13:32 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 07/16] arm64: Enable kpkeys Date: Fri, 6 Dec 2024 10:11:01 +0000 Message-ID: <20241206101110.1646108-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021335_588997_7D48F13E X-CRM114-Status: GOOD ( 11.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This is the final step to enable kpkeys on arm64. We enable POE at EL1 by setting TCR2_EL1.POE, and initialise POR_EL1 so that it enables access to the default pkey/POIndex (default kpkeys level). An ISB is added so that POE restrictions are enforced immediately. Having done this, we can now select ARCH_HAS_KPKEYS if ARM64_POE is enabled. Signed-off-by: Kevin Brodsky --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/cpufeature.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 100570a048c5..f35964641c1a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2183,6 +2183,7 @@ config ARM64_POE def_bool y select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS + select ARCH_HAS_KPKEYS help The Permission Overlay Extension is used to implement Memory Protection Keys. Memory Protection Keys provides a mechanism for diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 6ce71f444ed8..3925bf04fb2f 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -75,6 +75,7 @@ #include #include #include +#include #include #include @@ -2376,8 +2377,10 @@ static void cpu_enable_mops(const struct arm64_cpu_capabilities *__unused) #ifdef CONFIG_ARM64_POE static void cpu_enable_poe(const struct arm64_cpu_capabilities *__unused) { - sysreg_clear_set(REG_TCR2_EL1, 0, TCR2_EL1x_E0POE); + write_sysreg_s(por_set_kpkeys_level(0, KPKEYS_LVL_DEFAULT), SYS_POR_EL1); + sysreg_clear_set(REG_TCR2_EL1, 0, TCR2_EL1x_E0POE | TCR2_EL1x_POE); sysreg_clear_set(CPACR_EL1, 0, CPACR_ELx_E0POE); + isb(); } #endif From patchwork Fri Dec 6 10:11:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5AC4E77173 for ; Fri, 6 Dec 2024 10:22:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=59fciq8B+Agc81G/BRtnxHEnCmv7pc5HwbA3OuaCGvE=; b=YKpjwLIRSLVO2cBadQ6wR0eNQM bCc0CxpaNtsYpyes5prAwE8/qsmTy8J2SG2n9Ypl9F74SShMV9bC00n9x4XfCoPoi0+KULSTy3HZN viUkx3zDsuaRa91y9cVeaQlRdaqXYXQiZW4iNCXYNUqCn9C7YLQtyeaYTYg9yQnIDuBAZk7fVQhk/ ELlvNWXEHfTB3kDUc+/bLD4gtseUanUOmM86Sb123dNgDfiISfJFv/bZCxpt3D6chmC0AUpWsKTtL qmHrKCiTRF2ydenufn7whehp3aCNP0mLvnSqAhVHEq4CsnzkL3irXORKpJoS96j4oEt1AeyguBVnY YG9PZPuw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVTt-00000001GIY-1oTz; Fri, 06 Dec 2024 10:22:41 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVL8-00000001DJL-3OR8 for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:39 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6D4CC15A1; Fri, 6 Dec 2024 02:14:06 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8400A3F71E; Fri, 6 Dec 2024 02:13:35 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 08/16] mm: Introduce kernel_pgtables_set_pkey() Date: Fri, 6 Dec 2024 10:11:02 +0000 Message-ID: <20241206101110.1646108-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021338_936991_20E86CBD X-CRM114-Status: GOOD ( 16.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kernel_pgtables_set_pkey() allows setting the pkey of all page table pages in swapper_pg_dir, recursively. This will be needed by kpkeys_hardened_pgtables, as it relies on all PTPs being mapped with a non-default pkey. Those initial kernel page tables cannot practically be assigned a non-default pkey right when they are allocated, so mutating them during (early) boot is required. Signed-off-by: Kevin Brodsky --- It feels that some sort of locking is called for in kernel_pgtables_set_pkey(), but I couldn't figure out what would be appropriate. --- include/linux/mm.h | 2 + mm/memory.c | 130 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index c39c4945946c..683e883dae77 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4179,4 +4179,6 @@ int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *st int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); +int kernel_pgtables_set_pkey(int pkey); + #endif /* _LINUX_MM_H */ diff --git a/mm/memory.c b/mm/memory.c index 75c2dfd04f72..278ddf9f6249 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -76,6 +76,7 @@ #include #include #include +#include #include @@ -6974,3 +6975,132 @@ void vma_pgtable_walk_end(struct vm_area_struct *vma) if (is_vm_hugetlb_page(vma)) hugetlb_vma_unlock_read(vma); } + +static int set_page_pkey(void *p, int pkey) +{ + unsigned long addr = (unsigned long)p; + + /* + * swapper_pg_dir itself will be made read-only by mark_rodata_ro() + * so there is no point in changing its pkey. + */ + if (p == swapper_pg_dir) + return 0; + + return set_memory_pkey(addr, 1, pkey); +} + +static int set_pkey_pte(pmd_t *pmd, int pkey) +{ + pte_t *pte; + int err; + + pte = pte_offset_kernel(pmd, 0); + err = set_page_pkey(pte, pkey); + + return err; +} + +static int set_pkey_pmd(pud_t *pud, int pkey) +{ + pmd_t *pmd; + int i, err = 0; + + pmd = pmd_offset(pud, 0); + + err = set_page_pkey(pmd, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (pmd_none(pmd[i]) || pmd_bad(pmd[i]) || pmd_leaf(pmd[i])) + continue; + err = set_pkey_pte(&pmd[i], pkey); + if (err) + break; + } + + return err; +} + +static int set_pkey_pud(p4d_t *p4d, int pkey) +{ + pud_t *pud; + int i, err = 0; + + if (mm_pmd_folded(&init_mm)) + return set_pkey_pmd((pud_t *)p4d, pkey); + + pud = pud_offset(p4d, 0); + + err = set_page_pkey(pud, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_PUD; i++) { + if (pud_none(pud[i]) || pud_bad(pud[i]) || pud_leaf(pud[i])) + continue; + err = set_pkey_pmd(&pud[i], pkey); + if (err) + break; + } + + return err; +} + +static int set_pkey_p4d(pgd_t *pgd, int pkey) +{ + p4d_t *p4d; + int i, err = 0; + + if (mm_pud_folded(&init_mm)) + return set_pkey_pud((p4d_t *)pgd, pkey); + + p4d = p4d_offset(pgd, 0); + + err = set_page_pkey(p4d, pkey); + if (err) + return err; + + for (i = 0; i < PTRS_PER_P4D; i++) { + if (p4d_none(p4d[i]) || p4d_bad(p4d[i]) || p4d_leaf(p4d[i])) + continue; + err = set_pkey_pud(&p4d[i], pkey); + if (err) + break; + } + + return err; +} + +/** + * kernel_pgtables_set_pkey - set pkey for all kernel page table pages + * @pkey: pkey to set the page table pages to + * + * Walks swapper_pg_dir setting the protection key of every page table page (at + * all levels) to @pkey. swapper_pg_dir itself is left untouched as it is + * expected to be mapped read-only by mark_rodata_ro(). + * + * No-op if the architecture does not support kpkeys. + */ +int kernel_pgtables_set_pkey(int pkey) +{ + pgd_t *pgd = swapper_pg_dir; + int i, err = 0; + + if (!arch_kpkeys_enabled()) + return 0; + + if (mm_p4d_folded(&init_mm)) + return set_pkey_p4d(pgd, pkey); + + for (i = 0; i < PTRS_PER_PGD; i++) { + if (pgd_none(pgd[i]) || pgd_bad(pgd[i]) || pgd_leaf(pgd[i])) + continue; + err = set_pkey_p4d(&pgd[i], pkey); + if (err) + break; + } + + return err; +} From patchwork Fri Dec 6 10:11:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 49F60E77173 for ; Fri, 6 Dec 2024 10:23:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=DseRajfS8W1T/f3Rhtm2iIvVD4CreMkqE69JYThDAM0=; b=ryPOJEMjcTPkAk4Nx4M4SUVim2 YODAAWwKqAFOM62mGxsPVgrqC/hlIZwqdw5ls3hKUibnDNAjKU50Qal9AxjwZtKVvGOjdFFiWn/nA gJ35UheDcMt7o3u9QpJf9sMrS+Kz42dmyrhBK1B0PBQK239NwjVZmp0U/D+nMhGL27kDkiGiJZxg3 AYiNA/aCcDTzxa2nHzqSkq//8xblsSNd6yP3dyCsK53cTSgFPTrqsQ15T2brQUutA2OdhGdGRXI/k qPCVMjwkAj37BfocZynC3bIZCi7VEV6KmAr3Deeavr4DCV3YiQJUJEtIK3YDZ2mmPFFbsNg97YOIk TlsgLnUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVUw-00000001Gbw-0bf6; Fri, 06 Dec 2024 10:23:46 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLC-00000001DKz-1aMW for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:43 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A511B12FC; Fri, 6 Dec 2024 02:14:09 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BBBCB3F71E; Fri, 6 Dec 2024 02:13:38 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 09/16] mm: Introduce kpkeys_hardened_pgtables Date: Fri, 6 Dec 2024 10:11:03 +0000 Message-ID: <20241206101110.1646108-10-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021342_504715_F3DA24BF X-CRM114-Status: GOOD ( 25.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kpkeys_hardened_pgtables is a hardening feature based on kpkeys. It aims to prevent the corruption of page tables by: 1. mapping all page table pages, both kernel and user, with a privileged pkey (KPKEYS_PKEY_PGTABLES), and 2. granting write access to that pkey only when running at a higher kpkeys level (KPKEYS_LVL_PGTABLES). The feature is exposed as CONFIG_KPKEYS_HARDENED_PGTABLES; it requires explicit architecture opt-in by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES, since much of the page table handling is arch-specific. This patch introduces an API to modify the PTPs' pkey and switch kpkeys level using a guard object. Because this API is going to be called from low-level pgtable helpers (setters, allocators), it must be inactive on boot and explicitly switched on if and when kpkeys become available. A static key is used for that purpose; it is the responsibility of each architecture supporting kpkeys_hardened_pgtables to call kpkeys_hardened_pgtables_enable() as early as possible to switch on that static key. The initial kernel page tables are also walked to set their pkey, since they have already been allocated at that point. The definition of the kpkeys_hardened_pgtables guard class does not use the static key on the restore path to avoid mismatched set/restore pairs. Indeed, enabling the static key itself involves modifying page tables, and it is thus possible that the guard object is created when the static key appears as false, and destroyed when it appears as true. To avoid this situation, we reserve an invalid value for the pkey register and use it to disable the restore path. Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 12 +++++++ include/linux/kpkeys.h | 67 ++++++++++++++++++++++++++++++++++- mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/kpkeys_hardened_pgtables.c | 17 +++++++++ security/Kconfig.hardening | 12 +++++++ 6 files changed, 111 insertions(+), 1 deletion(-) create mode 100644 mm/kpkeys_hardened_pgtables.c diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h index 3404ce249757..cec92334a9f3 100644 --- a/include/asm-generic/kpkeys.h +++ b/include/asm-generic/kpkeys.h @@ -2,8 +2,20 @@ #ifndef __ASM_GENERIC_KPKEYS_H #define __ASM_GENERIC_KPKEYS_H +#ifndef KPKEYS_PKEY_PGTABLES +#define KPKEYS_PKEY_PGTABLES 1 +#endif + #ifndef KPKEYS_PKEY_DEFAULT #define KPKEYS_PKEY_DEFAULT 0 #endif +/* + * Represents a pkey register value that cannot be used, typically disabling + * access to all keys. + */ +#ifndef KPKEYS_PKEY_REG_INVAL +#define KPKEYS_PKEY_REG_INVAL 0 +#endif + #endif /* __ASM_GENERIC_KPKEYS_H */ diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index bcc063425926..bd3e0f36d2d6 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -4,12 +4,14 @@ #include #include +#include #include #define KPKEYS_LVL_DEFAULT 0 +#define KPKEYS_LVL_PGTABLES 1 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT -#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_PGTABLES #define KPKEYS_GUARD(_name, set_level, restore_pkey_reg) \ __DEFINE_CLASS_IS_CONDITIONAL(_name, false); \ @@ -64,4 +66,67 @@ static inline bool arch_kpkeys_enabled(void) #endif /* CONFIG_ARCH_HAS_KPKEYS */ +#ifdef CONFIG_KPKEYS_HARDENED_PGTABLES + +DECLARE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_enabled); + +/* + * Use guard(kpkeys_hardened_pgtables)() to temporarily grant write access + * to page tables. + */ +KPKEYS_GUARD(kpkeys_hardened_pgtables, + static_branch_unlikely(&kpkeys_hardened_pgtables_enabled) ? + kpkeys_set_level(KPKEYS_LVL_PGTABLES) : + KPKEYS_PKEY_REG_INVAL, + _T != KPKEYS_PKEY_REG_INVAL ? + kpkeys_restore_pkey_reg(_T) : + (void)0) + +static inline int kpkeys_protect_pgtable_memory(unsigned long addr, int numpages) +{ + int ret = 0; + + if (static_branch_unlikely(&kpkeys_hardened_pgtables_enabled)) + ret = set_memory_pkey(addr, numpages, KPKEYS_PKEY_PGTABLES); + + WARN_ON(ret); + return ret; +} + +static inline int kpkeys_unprotect_pgtable_memory(unsigned long addr, int numpages) +{ + int ret = 0; + + if (static_branch_unlikely(&kpkeys_hardened_pgtables_enabled)) + ret = set_memory_pkey(addr, numpages, KPKEYS_PKEY_DEFAULT); + + WARN_ON(ret); + return ret; +} + +/* + * Enables kpkeys_hardened_pgtables and switches existing kernel page tables to + * a privileged pkey (KPKEYS_PKEY_PGTABLES). + * + * Should be called as early as possible by architecture code, after (k)pkeys + * are initialised and before any user task is spawned. + */ +void kpkeys_hardened_pgtables_enable(void); + +#else /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + +KPKEYS_GUARD(kpkeys_hardened_pgtables, 0, (void)_T) + +static inline int kpkeys_protect_pgtable_memory(unsigned long addr, int numpages) +{ + return 0; +} +static inline int kpkeys_unprotect_pgtable_memory(unsigned long addr, int numpages) +{ + return 0; +} +static inline void kpkeys_hardened_pgtables_enable(void) {} + +#endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + #endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index f51dffca9d4e..07ae45a1395f 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1106,6 +1106,9 @@ config ARCH_HAS_PKEYS bool config ARCH_HAS_KPKEYS bool +# ARCH_HAS_KPKEYS must be selected when selecting this option +config ARCH_HAS_KPKEYS_HARDENED_PGTABLES + bool config ARCH_USES_PG_ARCH_2 bool diff --git a/mm/Makefile b/mm/Makefile index dba52bb0da8a..ffe799c1c897 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -146,3 +146,4 @@ obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o diff --git a/mm/kpkeys_hardened_pgtables.c b/mm/kpkeys_hardened_pgtables.c new file mode 100644 index 000000000000..e26fc20bdafe --- /dev/null +++ b/mm/kpkeys_hardened_pgtables.c @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include + +DEFINE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_enabled); + +void __init kpkeys_hardened_pgtables_enable(void) +{ + int ret; + + if (!arch_kpkeys_enabled()) + return; + + static_branch_enable(&kpkeys_hardened_pgtables_enabled); + ret = kernel_pgtables_set_pkey(KPKEYS_PKEY_PGTABLES); + WARN_ON(ret); +} diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index c9d5ca3d8d08..95f93f1d4055 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -300,6 +300,18 @@ config BUG_ON_DATA_CORRUPTION If unsure, say N. +config KPKEYS_HARDENED_PGTABLES + bool "Harden page tables using kernel pkeys" + depends on ARCH_HAS_KPKEYS_HARDENED_PGTABLES + help + This option makes all page tables mostly read-only by + allocating them with a non-default protection key (pkey) and + only enabling write access to that pkey in routines that are + expected to write to page table entries. + + This option has no effect if the system does not support + kernel pkeys. + endmenu config CC_HAS_RANDSTRUCT From patchwork Fri Dec 6 10:11:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DFB16E77173 for ; Fri, 6 Dec 2024 10:25:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0kY/h5GnEYP+1jrlPtC6VCHp7In/U5CKU4bQyxtRMrA=; b=biV+xADINZ+vsLG49DC3TwajhT Wb5nmTRQrgcWl67GsUKCNxGG6Q6yRd0FR7Nrd8TknUe5eNMjLuQYAfDthI7CnCTTNqrPqnCiM6g88 UiZ0aLdS1Dpz2RfjSzKPdAQBHNpOnZEfdM7spP5hlgOLr3hyxuQMeFFY275NHAl0SLyo4nRlTIPp4 rTDZ27csxStmS761BanuPYVvF6HIeR0y7NSxWEr3WSuyrKKoce2n75XB3Bd+0Prr+FzZASTIs9WHC qSP8T/fFIWY/K2jqfi6yJ4JN/71ro6RLvuwAPNEEvZ2BsGoHnRrbW/r29XkIzZWUTJEinXqeryHWZ gjXnJ5Qg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVVy-00000001Grv-3Yk0; Fri, 06 Dec 2024 10:24:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLF-00000001DMJ-19us for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:46 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DD1011691; Fri, 6 Dec 2024 02:14:12 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F3AD33F71E; Fri, 6 Dec 2024 02:13:41 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 10/16] mm: Map page tables with privileged pkey Date: Fri, 6 Dec 2024 10:11:04 +0000 Message-ID: <20241206101110.1646108-11-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021345_351971_4FFBACEC X-CRM114-Status: GOOD ( 11.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, map allocated page table pages using a privileged pkey (KPKEYS_PKEY_PGTABLES), so that page tables can only be written under guard(kpkeys_hardened_pgtables). This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 683e883dae77..4fb25454ba85 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -2895,7 +2896,19 @@ static inline bool pagetable_is_reserved(struct ptdesc *pt) */ static inline struct ptdesc *pagetable_alloc_noprof(gfp_t gfp, unsigned int order) { - struct page *page = alloc_pages_noprof(gfp | __GFP_COMP, order); + struct page *page; + int ret; + + page = alloc_pages_noprof(gfp | __GFP_COMP, order); + if (!page) + return NULL; + + ret = kpkeys_protect_pgtable_memory((unsigned long)page_address(page), + 1 << order); + if (ret) { + __free_pages(page, order); + return NULL; + } return page_ptdesc(page); } @@ -2911,8 +2924,11 @@ static inline struct ptdesc *pagetable_alloc_noprof(gfp_t gfp, unsigned int orde static inline void pagetable_free(struct ptdesc *pt) { struct page *page = ptdesc_page(pt); + unsigned int order = compound_order(page); - __free_pages(page, compound_order(page)); + kpkeys_unprotect_pgtable_memory((unsigned long)page_address(page), + 1 << order); + __free_pages(page, order); } #if defined(CONFIG_SPLIT_PTE_PTLOCKS) From patchwork Fri Dec 6 10:11:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E5C8E77179 for ; Fri, 6 Dec 2024 10:26:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iT19TPacBkHeIN8ActdEF8KEGxhSD8l1/qAasL3IKEI=; b=etAx+4k0OIKJLJIi6F5/HJdGo7 M7XYOh0+RXyRnXaK6G/7TqoQgxMhYB5V5/JzhD5jqrI/TqPXttYTS6qGccZHhOuaHevVGZp7JGuha ux8uQYWRO6LlblWd35oxtppE12kVjNY5NJMCdoeuoAo8fJiPmiYSWxC6CCHcYHWyvpcV3Sb4L9JGz BVFG9Bnir5izadqd0+qMfiosAGfQTVHr+tzciLVxuQZ1BmJx/kJp/KuuVqqKl+nxfh0cCJxkA1APD Ik/QZlLBxjIBxzjcLJc/RTilBjL3S8Csmzhb3GP0s+LmTaw9m7MTTCzjhStjXIQgGC8bRgr9a3pio FKTYPo/Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVX1-00000001HCO-22ar; Fri, 06 Dec 2024 10:25:55 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLI-00000001DNf-1u66 for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:49 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 209FA169E; Fri, 6 Dec 2024 02:14:16 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 371C13F71E; Fri, 6 Dec 2024 02:13:45 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 11/16] arm64: kpkeys: Support KPKEYS_LVL_PGTABLES Date: Fri, 6 Dec 2024 10:11:05 +0000 Message-ID: <20241206101110.1646108-12-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021348_531709_B43FD180 X-CRM114-Status: UNSURE ( 7.58 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Enable RW access to KPKEYS_PKEY_PGTABLES (used to map page table pages) if switching to KPKEYS_LVL_PGTABLES, otherwise only grant RO access. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index e17f6df41873..4854e1f3babd 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -18,6 +18,8 @@ static inline bool arch_kpkeys_enabled(void) static inline u64 por_set_kpkeys_level(u64 por, int level) { por = por_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RXW); + por = por_set_pkey_perms(por, KPKEYS_PKEY_PGTABLES, + level == KPKEYS_LVL_PGTABLES ? POE_RW : POE_R); return por; } From patchwork Fri Dec 6 10:11:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D55FE77179 for ; Fri, 6 Dec 2024 10:27:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wnhguXoH5f/gzWJpTgSeKyxUKNVSbH3M4baDMDbBb1o=; b=vDl4LAj3+Bul46JxDx6fnA1D45 SiLZBOa8IbLVcfOlBHCb58Wnh8KoY/HuxDebuAJxGgHhsgMuqqE+v555+Z7LcTO20FAfPgqT013mg ajNVXe/4y3YnJBQz5ad1R6EFhyWm6Bn6ap50aI4LCM2iTH42657zK2/NNe/QtrDyop43kmLf869eA VzC7aH6tjjYize0PHuwFEefUg7TdzDrWVo9IQgAs2foFbgkMnybznBR1z8aaUtBv1mmNkdfrZlb3Z d7VWl7CRGNSHRDAJt4ey6ObuprRcXNTO9hse6baBhUbPMatrhcB4Oe0Q4Z/P5yP2tF3OsKVq8eT3+ 0olYgzHw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVY4-00000001HWs-0kKq; Fri, 06 Dec 2024 10:27:00 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLL-00000001DPD-3f54 for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:53 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 58708169E; Fri, 6 Dec 2024 02:14:19 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6ED083F7D8; Fri, 6 Dec 2024 02:13:48 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 12/16] arm64: mm: Map p4d/pgd with privileged pkey Date: Fri, 6 Dec 2024 10:11:06 +0000 Message-ID: <20241206101110.1646108-13-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021352_004776_0A865429 X-CRM114-Status: GOOD ( 11.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, map p4d/pgd pages using a privileged pkey (KPKEYS_PKEY_PGTABLES), so that they can only be written under guard(kpkeys_hardened_pgtables). The case where pgd is not page-sized is not currently handled - this is pending support for pkeys in kmem_cache. This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgalloc.h | 21 ++++++++++++++++++--- arch/arm64/mm/pgd.c | 30 ++++++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index e75422864d1b..c006aecd6ba5 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -88,18 +88,33 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgdp, p4d_t *p4dp) static inline p4d_t *p4d_alloc_one(struct mm_struct *mm, unsigned long addr) { gfp_t gfp = GFP_PGTABLE_USER; + int ret; if (mm == &init_mm) gfp = GFP_PGTABLE_KERNEL; - return (p4d_t *)get_zeroed_page(gfp); + + addr = get_zeroed_page(gfp); + if (!addr) + return NULL; + + ret = kpkeys_protect_pgtable_memory(addr, 1); + if (ret) { + free_page(addr); + return NULL; + } + + return (p4d_t *)addr; } static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) { + unsigned long addr = (unsigned long)p4d; + if (!pgtable_l5_enabled()) return; - BUG_ON((unsigned long)p4d & (PAGE_SIZE-1)); - free_page((unsigned long)p4d); + BUG_ON(addr & (PAGE_SIZE-1)); + kpkeys_unprotect_pgtable_memory(addr, 1); + free_page(addr); } #define __p4d_free_tlb(tlb, p4d, addr) p4d_free((tlb)->mm, p4d) diff --git a/arch/arm64/mm/pgd.c b/arch/arm64/mm/pgd.c index 0c501cabc238..3577cc1821af 100644 --- a/arch/arm64/mm/pgd.c +++ b/arch/arm64/mm/pgd.c @@ -28,12 +28,38 @@ static bool pgdir_is_page_size(void) return false; } +static pgd_t *pgd_page_alloc(gfp_t gfp) +{ + unsigned long addr; + int ret; + + addr = __get_free_page(gfp); + if (!addr) + return NULL; + + ret = kpkeys_protect_pgtable_memory(addr, 1); + if (ret) { + free_page(addr); + return NULL; + } + + return (pgd_t *)addr; +} + +static void pgd_page_free(pgd_t *pgd) +{ + unsigned long addr = (unsigned long)pgd; + + kpkeys_unprotect_pgtable_memory(addr, 1); + free_page(addr); +} + pgd_t *pgd_alloc(struct mm_struct *mm) { gfp_t gfp = GFP_PGTABLE_USER; if (pgdir_is_page_size()) - return (pgd_t *)__get_free_page(gfp); + return pgd_page_alloc(gfp); else return kmem_cache_alloc(pgd_cache, gfp); } @@ -41,7 +67,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) void pgd_free(struct mm_struct *mm, pgd_t *pgd) { if (pgdir_is_page_size()) - free_page((unsigned long)pgd); + pgd_page_free(pgd); else kmem_cache_free(pgd_cache, pgd); } From patchwork Fri Dec 6 10:11:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C31F6E77173 for ; Fri, 6 Dec 2024 10:28:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XgcmeDZOa8E4pKh8qkoCSM0EnIW5k+WcCNukwL1J2N4=; b=xMzKCEz85K3r7HOyflCu5sQL1E x1lhNLKTMS1l3swqiJkzMuMAALoX+BuGud1OH5fWE4Td4AZzykV9XBeXRj6OkoqgQhhdbR/BeN+mO SaOpdt46bacBka8j58zjfkbmCMbaUWenX1Vo1CqHfIrLlvPnPw1rLVXw1bWzUmB0jev2MbPzIntik 8QjxwhMbKu3mfkwEvlCUmdlUd66iWg9U/5TXATDPS90Byuzf9+N2juAvK/ZnujXh8+BHBZlxMzgmA JNtQuNUFOysC2oMI2grvG9s3w2lIso6JxegjhOUtR6943UmEnalHJZ9o9SKtioziy/3byfJGdY2HC VA2Ugevg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVZ5-00000001HnZ-3yLG; Fri, 06 Dec 2024 10:28:03 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLP-00000001DQO-04AY for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:56 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 901F1176C; Fri, 6 Dec 2024 02:14:22 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A69F93F71E; Fri, 6 Dec 2024 02:13:51 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 13/16] arm64: mm: Reset pkey in __tlb_remove_table() Date: Fri, 6 Dec 2024 10:11:07 +0000 Message-ID: <20241206101110.1646108-14-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021355_093911_08AF4123 X-CRM114-Status: GOOD ( 10.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Page table pages are typically freed via tlb_remove_table() and friends. Ensure that the linear mapping for those pages is reset to the default pkey when CONFIG_KPKEYS_HARDENED_PGTABLES is enabled. This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/tlb.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index a947c6e784ed..d1611ffa6d91 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -10,10 +10,14 @@ #include #include +#include static inline void __tlb_remove_table(void *_table) { - free_page_and_swap_cache((struct page *)_table); + struct page *page = (struct page *)_table; + + kpkeys_unprotect_pgtable_memory((unsigned long)page_address(page), 1); + free_page_and_swap_cache(page); } #define tlb_flush tlb_flush From patchwork Fri Dec 6 10:11:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20E00E77173 for ; Fri, 6 Dec 2024 10:29:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YNr3Mvp78Rfuf46cqOpUA3bgNmv3XpNz0Lb8lC56yyQ=; b=gjlacTrbF5Gu6xKHlDzorNCXTg nDW6qZaupBw+kH0oQMSM0uww+XKIbj/XHOnEnPyDZwChKsDBhkzQn0Yuqpp3Tkd07dbn3dopZeWvx otQwThRhjlVNALPxvKlF0F00Noc1WPbYGARV4QllwbG+QzCrFiqQdiZsmnNespsF/fgfZvK+/VoWq bvPNiwrBxP38HiEZy7y9gSjRC1A7N9/q7hjYdMVdtlihVHTMlw0NPqgJtBHs86s7NLjWW7qwN7Tky R6FprAmSPx4Cv575x+5xGd5x8fwc6yTw4Iq+k8BJFqSSs1FLFRAhu7RqdUhzg4WYgfYwXTu28KUKl 7P2KJ/oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVa8-00000001I66-2MrR; Fri, 06 Dec 2024 10:29:08 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLS-00000001DR7-2YJC for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:13:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C7FAF1A25; Fri, 6 Dec 2024 02:14:25 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DE71F3F71E; Fri, 6 Dec 2024 02:13:54 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 14/16] arm64: mm: Guard page table writes with kpkeys Date: Fri, 6 Dec 2024 10:11:08 +0000 Message-ID: <20241206101110.1646108-15-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021358_730738_77ED9999 X-CRM114-Status: GOOD ( 12.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, page tables (both user and kernel) are mapped with a privileged pkey in the linear mapping. As a result, they can only be written under the kpkeys_hardened_pgtables guard, which sets POR_EL1 appropriately to allow such writes. Use this guard wherever page tables genuinely need to be written, keeping its scope as small as possible (so that POR_EL1 is reset as fast as possible). Where atomics are involved, the guard's scope encompasses the whole loop to avoid switching POR_EL1 unnecessarily. This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable.h | 19 +++++++++++++++++-- arch/arm64/mm/fault.c | 2 ++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 6986345b537a..5f9d748f08ee 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -39,6 +39,7 @@ #include #include #include +#include #ifdef CONFIG_TRANSPARENT_HUGEPAGE #define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE @@ -314,6 +315,7 @@ static inline pte_t pte_clear_uffd_wp(pte_t pte) static inline void __set_pte_nosync(pte_t *ptep, pte_t pte) { + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*ptep, pte); } @@ -758,6 +760,7 @@ static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) } #endif /* __PAGETABLE_PMD_FOLDED */ + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pmdp, pmd); if (pmd_valid(pmd)) { @@ -825,6 +828,7 @@ static inline void set_pud(pud_t *pudp, pud_t pud) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pudp, pud); if (pud_valid(pud)) { @@ -906,6 +910,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*p4dp, p4d); dsb(ishst); isb(); @@ -1033,6 +1038,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) return; } + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pgdp, pgd); dsb(ishst); isb(); @@ -1233,6 +1239,7 @@ static inline int __ptep_test_and_clear_young(struct vm_area_struct *vma, { pte_t old_pte, pte; + guard(kpkeys_hardened_pgtables)(); pte = __ptep_get(ptep); do { old_pte = pte; @@ -1279,7 +1286,10 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma, static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, unsigned long address, pte_t *ptep) { - pte_t pte = __pte(xchg_relaxed(&pte_val(*ptep), 0)); + pte_t pte; + + scoped_guard(kpkeys_hardened_pgtables) + pte = __pte(xchg_relaxed(&pte_val(*ptep), 0)); page_table_check_pte_clear(mm, pte); @@ -1322,7 +1332,10 @@ static inline pte_t __get_and_clear_full_ptes(struct mm_struct *mm, static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long address, pmd_t *pmdp) { - pmd_t pmd = __pmd(xchg_relaxed(&pmd_val(*pmdp), 0)); + pmd_t pmd; + + scoped_guard(kpkeys_hardened_pgtables) + pmd = __pmd(xchg_relaxed(&pmd_val(*pmdp), 0)); page_table_check_pmd_clear(mm, pmd); @@ -1336,6 +1349,7 @@ static inline void ___ptep_set_wrprotect(struct mm_struct *mm, { pte_t old_pte; + guard(kpkeys_hardened_pgtables)(); do { old_pte = pte; pte = pte_wrprotect(pte); @@ -1416,6 +1430,7 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp, pmd_t pmd) { page_table_check_pmd_set(vma->vm_mm, pmdp, pmd); + guard(kpkeys_hardened_pgtables)(); return __pmd(xchg_relaxed(&pmd_val(*pmdp), pmd_val(pmd))); } #endif diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ef63651099a9..ab45047155b9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -220,6 +220,8 @@ int __ptep_set_access_flags(struct vm_area_struct *vma, if (pte_same(pte, entry)) return 0; + guard(kpkeys_hardened_pgtables)(); + /* only preserve the access flags and write permission */ pte_val(entry) &= PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; From patchwork Fri Dec 6 10:11:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896940 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6CBC2E77173 for ; Fri, 6 Dec 2024 10:30:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=biEkaHTbzHosmskt6hjwupIYqCkEYLd73BiTY0Z2nr0=; b=QmBds9XbkQizNfA6KEBhdMQLvx rOCBgOMFPThEUh5mR7UGXfx2JKl2MiLjtkONvr35+pSEeNaNoBSGNP5YwmSBOsR+dwcX64sbVBVSp 061wnVcgccutueykNC/Nb5zfVQrLAlRnrLsIQxXrWwhJ/NU+AeqmZPMu7wrfHyemH1Dmrqw5Yjegi x8TA1Cbdxi+pZPtN/4gP3yRkJZ/ZzgyIySvftHxhebvTRQoWJ/b6cqw09I6cPrlAPAepO2KS/CAG/ DxiNfwmi1TOKjnxsPV65Hiki08fyvgQ+E+rTlgY+05hXP7OawuDvYEGjdl0iHcFSf02jvoLooeEpO IbEQQq5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVbB-00000001IP3-134B; Fri, 06 Dec 2024 10:30:13 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLV-00000001DRo-2Dr3 for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:14:02 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0B95E1BA8; Fri, 6 Dec 2024 02:14:29 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 220533F71E; Fri, 6 Dec 2024 02:13:57 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 15/16] arm64: Enable kpkeys_hardened_pgtables support Date: Fri, 6 Dec 2024 10:11:09 +0000 Message-ID: <20241206101110.1646108-16-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021401_636572_B3F1FEE2 X-CRM114-Status: UNSURE ( 9.70 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kpkeys_hardened_pgtables should be enabled as early as possible (if selected). It does however require kpkeys being available, which means on arm64 POE being detected and enabled. POE is a boot feature, so calling kpkeys_hardened_pgtables_enable() just after setup_boot_cpu_features() in smp_prepare_boot_cpu() is the best we can do. With that done, all the bits are in place and we can advertise support for kpkeys_hardened_pgtables by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES if ARM64_POE is enabled. Signed-off-by: Kevin Brodsky --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/smp.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index f35964641c1a..dac2f9a64826 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2184,6 +2184,7 @@ config ARM64_POE select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS select ARCH_HAS_KPKEYS + select ARCH_HAS_KPKEYS_HARDENED_PGTABLES help The Permission Overlay Extension is used to implement Memory Protection Keys. Memory Protection Keys provides a mechanism for diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 3b3f6b56e733..074cab55f9db 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -468,6 +469,7 @@ void __init smp_prepare_boot_cpu(void) if (system_uses_irq_prio_masking()) init_gic_priority_masking(); + kpkeys_hardened_pgtables_enable(); kasan_init_hw_tags(); /* Init percpu seeds for random tags after cpus are set up. */ kasan_init_sw_tags(); From patchwork Fri Dec 6 10:11:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13896941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 140A4E77173 for ; Fri, 6 Dec 2024 10:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CSEQJK+ksYiuBwJLNZIu14y1HbYeyt5RPpbKUWbkgOM=; b=wqVFZ3QUKQzs3mZJIwyhaOFKky fWlestjyKq/lFPG44oQE+6wYS4hphR4bsZ00L6Q7QDfjE8BtR35RsabItWGc252Edw9FDTZdFl2Gl 0Ob0jl9ZElbFkjbAKYMz2KdrFc0J0rutUfK1R4i6UwpY2Mr3oQzIYcmAvgnULNOtvsWXNlRk7TzWi OdF6QZpmkxZVgaMqU4jhQA+gZZ4JuyHKZIuIElrck1lhex1HUUMzkTezAfmY7BMYlImLasuquAhpm iT979JHjY3xRGm1EsNX7IlK/l1/GrBHD0xfDkL/ytjUg+dtTGrdInjKtBccgkIQULGUelDXgtdF9+ Gq9z9N9Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVcC-00000001Idq-3dvm; Fri, 06 Dec 2024 10:31:16 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJVLY-00000001DTt-47Tn for linux-arm-kernel@lists.infradead.org; Fri, 06 Dec 2024 10:14:06 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 436591C01; Fri, 6 Dec 2024 02:14:32 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 59F153F71E; Fri, 6 Dec 2024 02:14:01 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , aruna.ramakrishna@oracle.com, broonie@kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, jannh@google.com, jeffxu@chromium.org, joey.gouly@arm.com, kees@kernel.org, maz@kernel.org, pierre.langlois@arm.com, qperret@google.com, ryan.roberts@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH 16/16] mm: Add basic tests for kpkeys_hardened_pgtables Date: Fri, 6 Dec 2024 10:11:10 +0000 Message-ID: <20241206101110.1646108-17-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206101110.1646108-1-kevin.brodsky@arm.com> References: <20241206101110.1646108-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241206_021405_101083_2116DF2E X-CRM114-Status: GOOD ( 19.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add basic tests for the kpkeys_hardened_pgtables feature: try to perform a direct write to some kernel and user page table entry and ensure it fails. Signed-off-by: Kevin Brodsky --- mm/Makefile | 1 + mm/kpkeys_hardened_pgtables_test.c | 71 ++++++++++++++++++++++++++++++ security/Kconfig.hardening | 12 +++++ 3 files changed, 84 insertions(+) create mode 100644 mm/kpkeys_hardened_pgtables_test.c diff --git a/mm/Makefile b/mm/Makefile index ffe799c1c897..49ac16ae6875 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -147,3 +147,4 @@ obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES_TEST) += kpkeys_hardened_pgtables_test.o diff --git a/mm/kpkeys_hardened_pgtables_test.c b/mm/kpkeys_hardened_pgtables_test.c new file mode 100644 index 000000000000..37b6ffaa55e6 --- /dev/null +++ b/mm/kpkeys_hardened_pgtables_test.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +static void write_kernel_pte(struct kunit *test) +{ + pte_t *ptep; + pte_t pte; + int ret; + + /* + * The choice of address is mostly arbitrary - we just need a page + * that is definitely mapped, such as the current function. + */ + ptep = virt_to_kpte((unsigned long)&write_kernel_pte); + KUNIT_ASSERT_NOT_NULL_MSG(test, ptep, "Failed to get PTE"); + + pte = ptep_get(ptep); + pte = set_pte_bit(pte, __pgprot(PTE_WRITE)); + ret = copy_to_kernel_nofault(ptep, &pte, sizeof(pte)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PTE write wasn't prevented"); +} + +static void write_user_pmd(struct kunit *test) +{ + pmd_t *pmdp; + pmd_t pmd; + unsigned long uaddr; + int ret; + + uaddr = kunit_vm_mmap(test, NULL, 0, PAGE_SIZE, PROT_READ, + MAP_ANONYMOUS | MAP_PRIVATE | MAP_POPULATE, 0); + KUNIT_ASSERT_NE_MSG(test, uaddr, 0, "Could not create userspace mm"); + + /* We passed MAP_POPULATE so a PMD should already be allocated */ + pmdp = pmd_off(current->mm, uaddr); + KUNIT_ASSERT_NOT_NULL_MSG(test, pmdp, "Failed to get PMD"); + + pmd = pmdp_get(pmdp); + pmd = set_pmd_bit(pmd, __pgprot(PROT_SECT_NORMAL)); + ret = copy_to_kernel_nofault(pmdp, &pmd, sizeof(pmd)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Direct PMD write wasn't prevented"); +} + +static int kpkeys_hardened_pgtables_suite_init(struct kunit_suite *suite) +{ + if (!arch_kpkeys_enabled()) { + pr_err("Cannot run kpkeys_hardened_pgtables tests: kpkeys are not supported\n"); + return 1; + } + + return 0; +} + +static struct kunit_case kpkeys_hardened_pgtables_test_cases[] = { + KUNIT_CASE(write_kernel_pte), + KUNIT_CASE(write_user_pmd), + {} +}; + +static struct kunit_suite kpkeys_hardened_pgtables_test_suite = { + .name = "Hardened pgtables using kpkeys", + .test_cases = kpkeys_hardened_pgtables_test_cases, + .suite_init = kpkeys_hardened_pgtables_suite_init, +}; +kunit_test_suite(kpkeys_hardened_pgtables_test_suite); + +MODULE_LICENSE("GPL"); diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index 95f93f1d4055..8bc5d7235f6d 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -312,6 +312,18 @@ config KPKEYS_HARDENED_PGTABLES This option has no effect if the system does not support kernel pkeys. +config KPKEYS_HARDENED_PGTABLES_TEST + tristate "KUnit tests for kpkeys_hardened_pgtables" if !KUNIT_ALL_TESTS + depends on KPKEYS_HARDENED_PGTABLES + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable this option to check that the kpkeys_hardened_pgtables feature + functions as intended, i.e. prevents arbitrary writes to user and + kernel page tables. + + If unsure, say N. + endmenu config CC_HAS_RANDSTRUCT