From patchwork Mon Feb 3 10:28:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 247E0C02193 for ; Mon, 3 Feb 2025 11:45:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vkmJBE2xe1rD7fvTP6vWNu0WpILKMHVEtvlvMXZaL4Q=; b=bELm1/8Z+dgV063QwAFATFaFPm wJCl5wK//TFPQMZFJdWgdk3WyWMP2QTjZKPktGFXR2BG5949z0No9/RE8eLV631FANOOAP28Of/0f 1NmnP34PHICveUHYIljibhwe7fYUYGrU0Vfmy9mcjgQXgFr/l9lw7pjfKm2AJIDMNq15TszsNUXMW a5VspOHFiqJIm8Pf4RScJBiHNCmyUxFahcoN9xHUvscL4gC10DETWjOYWUnxjf/q+o4RbufTB7xvM kFDRB0FhmGxk9wshJpZe04iFz9ly+PppNJHXs0wv5jXYR01FrXhGDjJqRXD13MZja21C/EOpQM6QX /ZDhdGjA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1teutg-0000000FGoz-2lKO; Mon, 03 Feb 2025 11:45:48 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tetgx-0000000F7PJ-06Oh for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:28:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vkmJBE2xe1rD7fvTP6vWNu0WpILKMHVEtvlvMXZaL4Q=; b=alxfeKXxJ7aSdmRuqBDuL2s3gs Kust79Ay2ovL48tin0cjA6JRnWwuvUUL6v+aM0+o9vH8KPjJVxjWwTOjnkbpVXs79zcx/kPqkUVCw AKEYR0laNn1iHQr8bVNYyMVcfVuzLkHZEQOSB7ar3GK6YBoVxN0B3+cNN8ZhzEmsV0NKqS6yrMXxz Nl9wcxGIHhSw9ZAyCIdpstNfRizSncvm1Y3Jb/DS99psMUwmks3iLPm54aYTKOUeoc3cB/J+DUbAg LW5jDXyVTaitBGf+Zad0Dej3xUsvo3sxiGlZaNZU8Ux55yF8LbrO+Tm7QmhumR2PBGWR8DM1EUkga AL/7qAKw==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tetgt-0000000G3pM-3zQh for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:33 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AA49E1476; Mon, 3 Feb 2025 02:28:55 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7B5A13F63F; Mon, 3 Feb 2025 02:28:27 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 1/8] arm64: kpkeys: Avoid unnecessary writes to POR_EL1 Date: Mon, 3 Feb 2025 10:28:02 +0000 Message-ID: <20250203102809.1223255-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102832_390265_748D37F4 X-CRM114-Status: GOOD ( 12.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Nested uses of kpkeys guards are about to be introduced, which means that kpkeys_set_level() may not actually need to change the value of POR_EL1. Since updating POR_EL1 requires an expensive ISB, let's skip the write if the value is unchanged, by returning KPKEYS_PKEY_REG_INVAL. This will cause the matching kpkeys_restore_pkey_reg() call to bail out without calling arch_kpkeys_restore_pkey_reg(). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index 4854e1f3babd..3f16584d495a 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -27,8 +27,12 @@ static inline u64 por_set_kpkeys_level(u64 por, int level) static inline int arch_kpkeys_set_level(int level) { u64 prev_por = read_sysreg_s(SYS_POR_EL1); + u64 new_por = por_set_kpkeys_level(prev_por, level); - write_sysreg_s(por_set_kpkeys_level(prev_por, level), SYS_POR_EL1); + if (new_por == prev_por) + return KPKEYS_PKEY_REG_INVAL; + + write_sysreg_s(new_por, SYS_POR_EL1); isb(); return prev_por; From patchwork Mon Feb 3 10:28:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5A5CAC02194 for ; Mon, 3 Feb 2025 12:00:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=oHzh0GIvxYZInKVNXjFtmOiyhq3s9+3Bn3ETwjH4vJU=; b=hkE5EwFtmzDP0ez47TpEwrKAfO /OZ5ljTk8nzfsxdq7jyhqbcOWVQxCAjfL2JhAYhu+03k5HSyJsogPl0LukLEGdmkF8yENKrhUWncH q+9Jcfq2wCaTZVk59LHVKABQCSK43kGtazjWEKtXrgXOKs1nDLAijKrDI3f3hH1nXWntoYo1fPckO VDkUS/XVwMSRqlDM4Br7k0XsUxcB8NMCu1NT3/azmtQ33kIMAWnOSSbgBbCnQb2jOJqNEI27m3gp2 uqmXjsg1cMC0u2+cEFM7EKO5ITUpnufw6r+gBzys/+DWIspGtkErmVRaHVorweVakSAupVg98Pyhn ChJtn7sg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tev8C-0000000FJBt-3Ufx; Mon, 03 Feb 2025 12:00:48 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1teth1-0000000F7Qh-0Chv for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:28:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=oHzh0GIvxYZInKVNXjFtmOiyhq3s9+3Bn3ETwjH4vJU=; b=WIbzXna9IrYRkOjIiL7Tx2eSF3 HbPBFlugTLTDo1+k1JJxPDOqp0H5P88fB3pwgJyIH0uHYUeXwxUxfaw2raNNCcmBmlY8mjMHrauD/ PCwpHn59BizXeVH6p96lUc7irGmh/aBF1CHSLsl5Zm++5RJGDzJbWy/LhkeMDiqkEcoaBEpCK9q2F 9iBRvgw5kiZDBmNq8wk+kgZMfB1N/8+NVWl1QgCVxzkDHgAedDjEf6rZOqpHHBMUTWWSnuhdt/Aun OGPwDe6u7+yvGv2nNEv7BxEwVDcokso4vV8sFbqSaEtMke5Ui6cGstT77uFuj/Jw94/0o6yD1CMTR ZMmTm7vg==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tetgy-0000000G3r0-06Ep for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:37 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BBF761682; Mon, 3 Feb 2025 02:28:59 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8CF223F63F; Mon, 3 Feb 2025 02:28:31 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 2/8] mm: kpkeys: Introduce unrestricted level Date: Mon, 3 Feb 2025 10:28:03 +0000 Message-ID: <20250203102809.1223255-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102836_379308_D29E68C8 X-CRM114-Status: GOOD ( 11.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Highly privileged components, such as allocators, may require write access to arbitrary data. To that end, introduce a kpkeys level that grants write access to all kpkeys. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 4 +++- include/linux/kpkeys.h | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index 3f16584d495a..ab2305ca24b7 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -19,7 +19,9 @@ static inline u64 por_set_kpkeys_level(u64 por, int level) { por = por_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RXW); por = por_set_pkey_perms(por, KPKEYS_PKEY_PGTABLES, - level == KPKEYS_LVL_PGTABLES ? POE_RW : POE_R); + level == KPKEYS_LVL_PGTABLES || + level == KPKEYS_LVL_UNRESTRICTED + ? POE_RW : POE_R); return por; } diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 645eaf00096c..9d9feec83ccf 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -10,9 +10,10 @@ struct folio; #define KPKEYS_LVL_DEFAULT 0 #define KPKEYS_LVL_PGTABLES 1 +#define KPKEYS_LVL_UNRESTRICTED 2 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT -#define KPKEYS_LVL_MAX KPKEYS_LVL_PGTABLES +#define KPKEYS_LVL_MAX KPKEYS_LVL_UNRESTRICTED #define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ From patchwork Mon Feb 3 10:28:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E466EC02194 for ; Mon, 3 Feb 2025 12:00:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wydQ6t7sf5CwaeBJ0WjER4INB2IsNUN8+Dj6vCyx8B4=; b=HWrSgnRAjpFHbQoOTYzRTIqMGt DKO47h8c0cko7OxZe6xj6t/p1jWF4eGW7QKxJvGP06CdZzTfB/Uhp/C6zZfKPp6hK78xkojxY++ui z2zXX//R9laf/Qr77ZLJRmoRrvjh7kuc3+vOZG3ggZiZJ5X7EIL1FbTlOE4QOuyscJhqYrKF+4Gh+ uMsTNyRVCXKji6lBxUvN6GdtTeVbxr38oPGS2tLvfaPbT2+TgizBrOcoZr64J+39JI5yTu8zK/yhi HVW5hqyfNnmXLybzoTO7NErWgscXMRIKzxrRI0Lu5xGmRkv8autcHGJYqOKJpiyluK5jgEL6DqeVn S5FDkXoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tev8D-0000000FJCH-1a2y; Mon, 03 Feb 2025 12:00:49 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1teth5-0000000F7RT-21s5 for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:28:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=wydQ6t7sf5CwaeBJ0WjER4INB2IsNUN8+Dj6vCyx8B4=; b=iIpvLbRG8ZgvrmdcMo/uFoti2y +hcOmhZeLfOlafvVYMVQhWoPjqaxhwKs7nuUoz/2Q40mJ1e9ozbPdYPZAbDuhMG7FEU8ciYNG23qT qpq6vfrks577DDAlPSZK1sIeigKXAyFLtIcqSgSxVupKorrGycC9RtU/ptHXGkr5bWE1iYX7y5Bqq 5DzYJa52IWs7NqpzESDdsF5PtSk0fjCO1kETBDsQUWME1PW9wfa1Jl1wsKWIlsJG98A/IBVT+cW35 ezQweiWDZH0Aidesbi6d7aMvUv4XUjCpu44GUIvla6OyhxJPieC+2QOd54vDu6vPQw/OBEJ7kNBTB OoTkceAA==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1teth2-0000000G3s2-1Bj9 for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:42 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0159F1A32; Mon, 3 Feb 2025 02:29:04 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9F1B53F63F; Mon, 3 Feb 2025 02:28:35 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 3/8] slab: Introduce SLAB_SET_PKEY Date: Mon, 3 Feb 2025 10:28:04 +0000 Message-ID: <20250203102809.1223255-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102840_759784_2E49A0F7 X-CRM114-Status: GOOD ( 27.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce the SLAB_SET_PKEY flag to request a kmem_cache whose slabs are mapped with a non-default pkey, if kernel pkeys (kpkeys) are supported. The pkey to be used is specified via a new pkey field in struct kmem_cache_args. The setting/resetting of the pkey is done directly at the slab level (allocate_slab/__free_slab) to avoid having to propagate the pkey value down to the page level. Memory mapped with a non-default pkey cannot be written to at the default kpkeys level. This is handled by switching to the unrestricted kpkeys level (granting write access to all pkeys) when writing to a slab with SLAB_SET_PKEY. The merging of slabs with SLAB_SET_PKEY is conservatively prevented, though it should be possible to merge slabs with the same configured pkey. Signed-off-by: Kevin Brodsky --- include/linux/slab.h | 21 ++++++++++++++++ mm/slab.h | 7 +++++- mm/slab_common.c | 2 +- mm/slub.c | 58 +++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 85 insertions(+), 3 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 09eedaecf120..cc2e757b16ec 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -58,6 +58,9 @@ enum _slab_flag_bits { _SLAB_CMPXCHG_DOUBLE, #ifdef CONFIG_SLAB_OBJ_EXT _SLAB_NO_OBJ_EXT, +#endif +#ifdef CONFIG_ARCH_HAS_KPKEYS + _SLAB_SET_PKEY, #endif _SLAB_FLAGS_LAST_BIT }; @@ -234,6 +237,12 @@ enum _slab_flag_bits { #define SLAB_NO_OBJ_EXT __SLAB_FLAG_UNUSED #endif +#ifdef CONFIG_ARCH_HAS_KPKEYS +#define SLAB_SET_PKEY __SLAB_FLAG_BIT(_SLAB_SET_PKEY) +#else +#define SLAB_SET_PKEY __SLAB_FLAG_UNUSED +#endif + /* * freeptr_t represents a SLUB freelist pointer, which might be encoded * and not dereferenceable if CONFIG_SLAB_FREELIST_HARDENED is enabled. @@ -331,6 +340,18 @@ struct kmem_cache_args { * %NULL means no constructor. */ void (*ctor)(void *); + /** + * @pkey: The pkey to map the allocated pages with. + * + * If the SLAB flags include SLAB_SET_PKEY, and if kernel pkeys are + * supported, objects are allocated in pages mapped with the protection + * key specified by @pkey. Otherwise, this field is ignored. + * + * Note that if @pkey is a non-default pkey, some overhead is incurred + * when internal slab functions switch the pkey register to write to the + * slab (e.g. setting a free pointer). + */ + int pkey; }; struct kmem_cache *__kmem_cache_create_args(const char *name, diff --git a/mm/slab.h b/mm/slab.h index 1a081f50f947..d5cf5927634a 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -311,6 +311,10 @@ struct kmem_cache { unsigned int usersize; /* Usercopy region size */ #endif +#ifdef CONFIG_ARCH_HAS_KPKEYS + int pkey; +#endif + struct kmem_cache_node *node[MAX_NUMNODES]; }; @@ -462,7 +466,8 @@ static inline bool is_kmalloc_normal(struct kmem_cache *s) SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS | \ SLAB_NOLEAKTRACE | SLAB_RECLAIM_ACCOUNT | \ SLAB_TEMPORARY | SLAB_ACCOUNT | \ - SLAB_NO_USER_FLAGS | SLAB_KMALLOC | SLAB_NO_MERGE) + SLAB_NO_USER_FLAGS | SLAB_KMALLOC | SLAB_NO_MERGE | \ + SLAB_SET_PKEY) #define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_CONSISTENCY_CHECKS) diff --git a/mm/slab_common.c b/mm/slab_common.c index 69f9afd85f9f..21323d2a108e 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -47,7 +47,7 @@ struct kmem_cache *kmem_cache; */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_NO_MERGE) + SLAB_FAILSLAB | SLAB_NO_MERGE | SLAB_SET_PKEY) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) diff --git a/mm/slub.c b/mm/slub.c index 1f50129dcfb3..75b543e255d9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include @@ -459,6 +460,15 @@ static nodemask_t slab_nodes; static struct workqueue_struct *flushwq; #endif +#ifdef CONFIG_ARCH_HAS_KPKEYS +KPKEYS_GUARD_COND(kpkeys_slab_write, + KPKEYS_LVL_UNRESTRICTED, + unlikely(s->flags & SLAB_SET_PKEY), + struct kmem_cache *s) +#else +KPKEYS_GUARD_NOOP(kpkeys_slab_write, struct kmem_cache *s) +#endif + /******************************************************************** * Core slab cache functions *******************************************************************/ @@ -545,6 +555,8 @@ static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) BUG_ON(object == fp); /* naive detection of double free or corruption */ #endif + guard(kpkeys_slab_write)(s); + freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr); *(freeptr_t *)freeptr_addr = freelist_ptr_encode(s, fp, freeptr_addr); } @@ -765,6 +777,8 @@ static inline void set_orig_size(struct kmem_cache *s, p += get_info_end(s); p += sizeof(struct track) * 2; + guard(kpkeys_slab_write)(s); + *(unsigned int *)p = orig_size; } @@ -949,6 +963,8 @@ static void set_track_update(struct kmem_cache *s, void *object, { struct track *p = get_track(s, object, alloc); + guard(kpkeys_slab_write)(s); + #ifdef CONFIG_STACKDEPOT p->handle = handle; #endif @@ -973,6 +989,8 @@ static void init_tracking(struct kmem_cache *s, void *object) if (!(s->flags & SLAB_STORE_USER)) return; + guard(kpkeys_slab_write)(s); + p = get_track(s, object, TRACK_ALLOC); memset(p, 0, 2*sizeof(struct track)); } @@ -1137,6 +1155,8 @@ static void init_object(struct kmem_cache *s, void *object, u8 val) u8 *p = kasan_reset_tag(object); unsigned int poison_size = s->object_size; + guard(kpkeys_slab_write)(s); + if (s->flags & SLAB_RED_ZONE) { /* * Here and below, avoid overwriting the KMSAN shadow. Keeping @@ -2335,6 +2355,8 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bool init, int rsize; unsigned int inuse, orig_size; + guard(kpkeys_slab_write)(s); + inuse = get_info_end(s); orig_size = get_orig_size(s, x); if (!kasan_has_integrated_init()) @@ -2563,6 +2585,8 @@ static __always_inline void unaccount_slab(struct slab *slab, int order, -(PAGE_SIZE << order)); } +static void __free_slab(struct kmem_cache *s, struct slab *slab); + static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) { struct slab *slab; @@ -2612,6 +2636,18 @@ static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) setup_slab_debug(s, slab, start); +#ifdef CONFIG_ARCH_HAS_KPKEYS + if (unlikely(s->flags & SLAB_SET_PKEY)) { + int ret = set_memory_pkey((unsigned long)start, + 1 << oo_order(oo), s->pkey); + + if (WARN_ON(ret)) { + __free_slab(s, slab); + return NULL; + } + } +#endif + shuffle = shuffle_freelist(s, slab); if (!shuffle) { @@ -2652,6 +2688,11 @@ static void __free_slab(struct kmem_cache *s, struct slab *slab) __folio_clear_slab(folio); mm_account_reclaimed_pages(pages); unaccount_slab(slab, order, s); +#ifdef CONFIG_ARCH_HAS_KPKEYS + if (unlikely(s->flags & SLAB_SET_PKEY)) + WARN_ON(set_memory_pkey((unsigned long)folio_address(folio), + pages, 0)); +#endif free_frozen_pages(&folio->page, order); } @@ -4053,9 +4094,11 @@ static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) { if (unlikely(slab_want_init_on_free(s)) && obj && - !freeptr_outside_object(s)) + !freeptr_outside_object(s)) { + guard(kpkeys_slab_write)(s); memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); + } } static __fastpath_inline @@ -4798,6 +4841,7 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) /* Zero out spare memory. */ if (want_init_on_alloc(flags)) { kasan_disable_current(); + guard(kpkeys_slab_write)(s); if (orig_size && orig_size < new_size) memset(kasan_reset_tag(p) + orig_size, 0, new_size - orig_size); else @@ -4807,6 +4851,7 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) /* Setup kmalloc redzone when needed */ if (s && slub_debug_orig_size(s)) { + guard(kpkeys_slab_write)(s); set_orig_size(s, (void *)p, new_size); if (s->flags & SLAB_RED_ZONE && new_size < ks) memset_no_sanitize_memory(kasan_reset_tag(p) + new_size, @@ -6162,6 +6207,17 @@ int do_kmem_cache_create(struct kmem_cache *s, const char *name, s->useroffset = args->useroffset; s->usersize = args->usersize; #endif +#ifdef CONFIG_ARCH_HAS_KPKEYS + s->pkey = args->pkey; + + if (s->flags & SLAB_SET_PKEY) { + if (s->pkey >= arch_max_pkey()) + goto out; + + if (!arch_kpkeys_enabled() || s->pkey == KPKEYS_PKEY_DEFAULT) + s->flags &= ~SLAB_SET_PKEY; + } +#endif if (!calculate_sizes(args, s)) goto out; From patchwork Mon Feb 3 10:28:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8418C02196 for ; Mon, 3 Feb 2025 12:00:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gXEcaQ/nat4AynYmLE14uvl2r7aUGOOMKf84Ghy3AwI=; b=GVaeRJHdOxJNESfjSSz9Opsd82 L/k4pcINcs6nGkfBfFS2RqS/zIjFfBvpYw8IquQcF8EKMbNNGGvY5c3vsX3P5kI8GIhxIRVaJc+st Zft8osB+gERcg2w8fUXH2Xfr2D68NZqTLcvHKYTSvYOGzgejxTEvlA3O7KxTaDAl+eU5HZC7ZAPZV yLfcvi4nj00k82I2+3GdntWtz5K0B/dFaa70NOwsJbE5mIdU1Z2LpcmgHOv+p7OECWBbj4m3A1eg8 fE/4d7dbMNT39FHYEosI2Db9yQfLHNSa3uho5boml1Q1W2ExYHWeBXn5hrkSPm5lRTE5fdvVMjlqb Tz3Ip60Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tev8D-0000000FJCr-3ux7; Mon, 03 Feb 2025 12:00:49 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tethA-0000000F7S6-0Ihx for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:28:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=gXEcaQ/nat4AynYmLE14uvl2r7aUGOOMKf84Ghy3AwI=; b=pq1p14Uaw+XsugF0yoySi62CEV aHjMZ9qb+EiY2wO+Z6v1MA4Xla/BkkgvHbFCRsArl4xIaNCakLarsqFLdxo3C4pXQQgU43Se3RQp3 WfuHRCOI8Uo10WfQKHeZy07wmGcVmjVxj53n3d9ZqsS06uHsYEaI5yTAcAcQ4/scD8Bqp7NUIFjIc IIo3LvQvvXl1QDvA1lPHzlvOXjuJ1XBs/Dm8nStbVYFS1KRSN99Jropn7/djw8RHtf21Hnd8xavWv UbJzhIH8GWdzUCIJyNvzuHwHFQNYODdKfGxx31Ln7FmzgyBDHhK9fzrOnWK9OViNWZmBRC1neriiu 7DHyN4bg==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1teth6-0000000G3tD-1Nj6 for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:46 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 128781BA8; Mon, 3 Feb 2025 02:29:08 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D8CF83F63F; Mon, 3 Feb 2025 02:28:39 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 4/8] rcu: Allow processing kpkeys-protected data Date: Mon, 3 Feb 2025 10:28:05 +0000 Message-ID: <20250203102809.1223255-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102845_099403_15A7B261 X-CRM114-Status: GOOD ( 16.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Data assigned a non-default pkey is not writable at the default kpkeys level. If such data is managed via RCU, some mechanism is required to temporarily grant write access to the data's struct rcu_head, for instance when zeroing the callback pointer. There is unfortunately no straightforward way for RCU to know whether the managed data is mapped with a non-default pkey. This patch takes the easy route and switches to the unrestricted kpkeys level whenever struct rcu_head is written; this should work reliably but it is clearly suboptimal. That behaviour is enabled by selecting CONFIG_KPKEYS_UNRESTRICTED_RCU. This patch isn't comprehensive, in particular it does not take care of Tiny RCU. Signed-off-by: Kevin Brodsky --- include/linux/kpkeys.h | 6 ++++++ kernel/rcu/rcu_segcblist.c | 10 +++++++--- kernel/rcu/tree.c | 4 +++- mm/Kconfig | 2 ++ 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 9d9feec83ccf..c5d804c1ab7b 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -154,4 +154,10 @@ static inline void kpkeys_hardened_pgtables_enable(void) {} #endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ +#ifdef CONFIG_KPKEYS_UNRESTRICTED_RCU +KPKEYS_GUARD(kpkeys_rcu, KPKEYS_LVL_UNRESTRICTED) +#else +KPKEYS_GUARD_NOOP(kpkeys_rcu) +#endif + #endif /* _LINUX_KPKEYS_H */ diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c index 298a2c573f02..a9b5552b53a5 100644 --- a/kernel/rcu/rcu_segcblist.c +++ b/kernel/rcu/rcu_segcblist.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "rcu_segcblist.h" @@ -332,7 +333,8 @@ void rcu_segcblist_enqueue(struct rcu_segcblist *rsclp, rcu_segcblist_inc_len(rsclp); rcu_segcblist_inc_seglen(rsclp, RCU_NEXT_TAIL); rhp->next = NULL; - WRITE_ONCE(*rsclp->tails[RCU_NEXT_TAIL], rhp); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[RCU_NEXT_TAIL], rhp); WRITE_ONCE(rsclp->tails[RCU_NEXT_TAIL], &rhp->next); } @@ -381,7 +383,8 @@ void rcu_segcblist_extract_done_cbs(struct rcu_segcblist *rsclp, rclp->len = rcu_segcblist_get_seglen(rsclp, RCU_DONE_TAIL); *rclp->tail = rsclp->head; WRITE_ONCE(rsclp->head, *rsclp->tails[RCU_DONE_TAIL]); - WRITE_ONCE(*rsclp->tails[RCU_DONE_TAIL], NULL); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[RCU_DONE_TAIL], NULL); rclp->tail = rsclp->tails[RCU_DONE_TAIL]; for (i = RCU_CBLIST_NSEGS - 1; i >= RCU_DONE_TAIL; i--) if (rsclp->tails[i] == rsclp->tails[RCU_DONE_TAIL]) @@ -436,7 +439,8 @@ void rcu_segcblist_insert_done_cbs(struct rcu_segcblist *rsclp, if (!rclp->head) return; /* No callbacks to move. */ rcu_segcblist_add_seglen(rsclp, RCU_DONE_TAIL, rclp->len); - *rclp->tail = rsclp->head; + scoped_guard(kpkeys_rcu) + *rclp->tail = rsclp->head; WRITE_ONCE(rsclp->head, rclp->head); for (i = RCU_DONE_TAIL; i < RCU_CBLIST_NSEGS; i++) if (&rsclp->head == rsclp->tails[i]) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 475f31deed14..48d9d14a2af6 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -64,6 +64,7 @@ #include #include #include +#include #include "../time/tick-internal.h" #include "tree.h" @@ -2542,7 +2543,8 @@ static void rcu_do_batch(struct rcu_data *rdp) f = rhp->func; debug_rcu_head_callback(rhp); - WRITE_ONCE(rhp->func, (rcu_callback_t)0L); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(rhp->func, (rcu_callback_t)0L); f(rhp); rcu_lock_release(&rcu_callback_map); diff --git a/mm/Kconfig b/mm/Kconfig index 2a8ebe780e64..e2671c57e047 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1152,6 +1152,8 @@ config ARCH_HAS_KPKEYS # ARCH_HAS_KPKEYS must be selected when selecting this option config ARCH_HAS_KPKEYS_HARDENED_PGTABLES bool +config KPKEYS_UNRESTRICTED_RCU + bool config ARCH_USES_PG_ARCH_2 bool From patchwork Mon Feb 3 10:28:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 78C28C02192 for ; Mon, 3 Feb 2025 10:53:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tqpNjfnw7iMi+gWoBv0avsygAJN2y4ykdULl/ADYxb8=; b=tGH2yADwiEZ9DXvMyNGKdJ7wyJ hD/YY0cse2Mbmy3WnewRDWyNDcPI519BOWyjzl40Q2/TK9uKNswm2MevixfqwTaT0JarjqAau6rHy srFiai1s3aEwO0V/V7kj/0OwWyb7dtJVsuO/YXM0JgVKg2RaCszoAmPiAVnJm3qemD7SJao6vw/Fp /3LqRteZIErSz0qmiHHKadrN64KVGNlu5mqs/LQHvGAosbpQgktRiB3gu5w33P6VEuOh+FLus7TaW 0/mLIL5SAE4BwJ2H5z+HHuYVi7quIlAIJ+shezRpCiBq8qmHmlzqtXNkThO4icKCG5wGWcr74kg7i LGDvmkYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1teu4u-0000000FAAl-0FPp; Mon, 03 Feb 2025 10:53:20 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tethD-0000000F7T3-1GVL for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:28:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=tqpNjfnw7iMi+gWoBv0avsygAJN2y4ykdULl/ADYxb8=; b=dIW2Kww47pVIM+wdmlgTiLv2fv Lfa3lnfVJqCH/T9s+MEJF8jXqxGGbAMEseUkEjKz0oVA6KN3wWZ4AgK276pXLJiMiXlP87lmcaEoa L7LP6ZLpqGK5PMN86LY4DKgmfIGSwUYxDzX3PzlisOCuXkrb2NZlDgz6VWEWtVvFKNruLIKhaQMep SoXRQ8tPfQuBCJAdiTswVtB08aNc09UoD/YlxoQB5FGix0ctsnapcY277a6kZE/i8jD0pHaujB6L+ 6K+xEdzVarNiMEt1PTRTuXeqMEJGqMhE26JjW/6QZARoTmVrSaC2WrimBWF3vysYzfLQaMXauqLra CnvX0VMw==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tethA-0000000G3ut-1fG9 for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:50 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 23BF31BB0; Mon, 3 Feb 2025 02:29:12 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E99793F63F; Mon, 3 Feb 2025 02:28:43 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 5/8] mm: kpkeys: Introduce cred pkey/level Date: Mon, 3 Feb 2025 10:28:06 +0000 Message-ID: <20250203102809.1223255-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102848_710029_19BB3D4C X-CRM114-Status: GOOD ( 12.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will need a separate pkey to protect struct cred. Allocate one as well as a new kpkeys level that grants write access to that pkey, and add a guard that switches to that level. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 4 ++++ include/asm-generic/kpkeys.h | 4 ++++ include/linux/kpkeys.h | 9 ++++++++- 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index ab2305ca24b7..f5797e579fb9 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -22,6 +22,10 @@ static inline u64 por_set_kpkeys_level(u64 por, int level) level == KPKEYS_LVL_PGTABLES || level == KPKEYS_LVL_UNRESTRICTED ? POE_RW : POE_R); + por = por_set_pkey_perms(por, KPKEYS_PKEY_CRED, + level == KPKEYS_LVL_CRED || + level == KPKEYS_LVL_UNRESTRICTED + ? POE_RW : POE_R); return por; } diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h index cec92334a9f3..56a2fc9fe4a6 100644 --- a/include/asm-generic/kpkeys.h +++ b/include/asm-generic/kpkeys.h @@ -2,6 +2,10 @@ #ifndef __ASM_GENERIC_KPKEYS_H #define __ASM_GENERIC_KPKEYS_H +#ifndef KPKEYS_PKEY_CRED +#define KPKEYS_PKEY_CRED 2 +#endif + #ifndef KPKEYS_PKEY_PGTABLES #define KPKEYS_PKEY_PGTABLES 1 #endif diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index c5d804c1ab7b..a478eaf2e14f 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -10,7 +10,8 @@ struct folio; #define KPKEYS_LVL_DEFAULT 0 #define KPKEYS_LVL_PGTABLES 1 -#define KPKEYS_LVL_UNRESTRICTED 2 +#define KPKEYS_LVL_CRED 2 +#define KPKEYS_LVL_UNRESTRICTED 3 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT #define KPKEYS_LVL_MAX KPKEYS_LVL_UNRESTRICTED @@ -160,4 +161,10 @@ KPKEYS_GUARD(kpkeys_rcu, KPKEYS_LVL_UNRESTRICTED) KPKEYS_GUARD_NOOP(kpkeys_rcu) #endif +#ifdef CONFIG_KPKEYS_HARDENED_CRED +KPKEYS_GUARD(kpkeys_hardened_cred, KPKEYS_LVL_CRED) +#else +KPKEYS_GUARD_NOOP(kpkeys_hardened_cred) +#endif + #endif /* _LINUX_KPKEYS_H */ From patchwork Mon Feb 3 10:28:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957344 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DAE10C02194 for ; Mon, 3 Feb 2025 12:00:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=H8lNEuBGD5WSMCCeKCge9CS2olSopCyu/v81KbZt5Mo=; b=2crDG1MFWx9kDxfvP47yTotmzL rl0HE2nPM7kB6UGvR07mXaEnjFNzQj3S2+lAz7rXxiTDyGwhGVf38cdGL6zBebErad9SO/YCCWLn7 Vy7cRqbbTmQYx3tQB8c9ox1S984g3PbwcKgzppMiyYwWCS2r0dc+Aj7OX1nd4XygtrPNzSC4bYem2 tSgIE+/4dlMxkIuKqIYoiLb+2q1ZAyrJgLWJnCrrzod/gnjqKEE7TkgcRP2L2W/iP35ckogIewHOh dEt1cfrygti0E7336DrVagDhTCyFbZbHDHp4Qq96YqfvCaaxs5aAQOjb8DYZ2dzJjzcQzXya25Npm 2lNJwpGw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tev8E-0000000FJDr-3u4C; Mon, 03 Feb 2025 12:00:50 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tethH-0000000F7TU-1rTh for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:28:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=H8lNEuBGD5WSMCCeKCge9CS2olSopCyu/v81KbZt5Mo=; b=ag8VCmSMQxuQLlbSnTPSjxJJfB t9OoPev/RBmcMlTqo1W04XVHvdBvywL9GhKlj93/rDV45Gjpmp1vYq2ptLMT6StdkVs1FEvegCdHB Q1pgFUTG8eYNRYOASyrPjRQ7oVn5nTDeRqHlYtYT52IZWj4kXHlvLVI2odH3t8hwytcCe0Ud/Jena XRM/s1hqwMajg27WbtY7i8OayIgs3q1E+2EKOASVtvNzh2LvQpL9sPt0J4tD7gNYFY8JzNTFFfjN9 RbMaSHqaAQCQcslUxF0yBMqVrHSSgHa36r5U4nJeAR+IDlts8N9OhUmxEv94gcICNPYYzschSvFS5 8qJc4R6A==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tethE-0000000G3vu-24VU for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:54 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3737B1BB2; Mon, 3 Feb 2025 02:29:16 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 074ED3F63F; Mon, 3 Feb 2025 02:28:47 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 6/8] cred: Protect live struct cred with kpkeys Date: Mon, 3 Feb 2025 10:28:07 +0000 Message-ID: <20250203102809.1223255-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102852_883633_9C35FA35 X-CRM114-Status: GOOD ( 35.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch introduces a feature to prevent unintended modifications of live credentials, by moving them to protected memory when they are installed via commit_creds(). The protection mechanism is kernel pkeys (kpkeys): protected memory is mapped with a non-default pkey and write access is disabled by default. As a result, task->{cred,real_cred} can only be written to by switching to a higher kpkeys level. The kpkeys_hardened_cred feature is enabled by choosing CONFIG_KPKEYS_HARDENED_CRED=y and running on a system supporting kpkeys. Credentials are not directly allocated in protected memory, as that would force all code preparing new credentials to switch kpkeys level. To avoid such disruption, prepare_creds() and variants still allocate standard memory. When commit_creds() is called, the credentials are copied to protected memory, and the temporary object (in a standard kmalloc slab) is freed. This approach does not work so transparently when it comes to override_creds(), because it does not consume the reference: the object it gets passed cannot be moved. Callers of override_creds() will need to explicitly call a new protect_creds() helper to move the credentials to protected memory once they are done preparing them. Some of these callers use the unmodified output of prepare_creds(); prepare_protected_creds() is introduced to avoid an unnecessary copy in such cases. This patch does not handle these situations, but it does not break them either (credentials installed by override_creds() will simply be unprotected). Various helpers need to modify live credentials. To that end, guard(kpkeys_hardened_cred) is introduced to switch to the kpkeys level that enables write access to KPKEYS_PKEY_CRED. Signed-off-by: Kevin Brodsky --- include/linux/cred.h | 6 ++ kernel/cred.c | 178 +++++++++++++++++++++++++++++++------ security/Kconfig.hardening | 13 +++ 3 files changed, 170 insertions(+), 27 deletions(-) diff --git a/include/linux/cred.h b/include/linux/cred.h index 0c3c4b16b469..b854adce7462 100644 --- a/include/linux/cred.h +++ b/include/linux/cred.h @@ -16,6 +16,7 @@ #include #include #include +#include struct cred; struct inode; @@ -162,6 +163,8 @@ extern int set_create_files_as(struct cred *, struct inode *); extern int cred_fscmp(const struct cred *, const struct cred *); extern void __init cred_init(void); extern int set_cred_ucounts(struct cred *); +extern struct cred *prepare_protected_creds(void); +extern struct cred *protect_creds(struct cred *); static inline bool cap_ambient_invariant_ok(const struct cred *cred) { @@ -205,6 +208,7 @@ static inline const struct cred *get_cred_many(const struct cred *cred, int nr) struct cred *nonconst_cred = (struct cred *) cred; if (!cred) return cred; + guard(kpkeys_hardened_cred)(); nonconst_cred->non_rcu = 0; atomic_long_add(nr, &nonconst_cred->usage); return cred; @@ -229,6 +233,7 @@ static inline const struct cred *get_cred_rcu(const struct cred *cred) struct cred *nonconst_cred = (struct cred *) cred; if (!cred) return NULL; + guard(kpkeys_hardened_cred)(); if (!atomic_long_inc_not_zero(&nonconst_cred->usage)) return NULL; nonconst_cred->non_rcu = 0; @@ -252,6 +257,7 @@ static inline void put_cred_many(const struct cred *_cred, int nr) struct cred *cred = (struct cred *) _cred; if (cred) { + guard(kpkeys_hardened_cred)(); if (atomic_long_sub_and_test(nr, &cred->usage)) __put_cred(cred); } diff --git a/kernel/cred.c b/kernel/cred.c index 9676965c0981..81664ffef8f7 100644 --- a/kernel/cred.c +++ b/kernel/cred.c @@ -20,6 +20,8 @@ #include #include +#include "../mm/slab.h" + #if 0 #define kdebug(FMT, ...) \ printk("[%-5.5s%5u] " FMT "\n", \ @@ -62,6 +64,48 @@ struct cred init_cred = { .ucounts = &init_ucounts, }; +static bool hardened_cred_enabled(void) +{ + return IS_ENABLED(CONFIG_KPKEYS_HARDENED_CRED) && arch_kpkeys_enabled(); +} + +static bool cred_is_protected(const struct cred *cred) +{ + struct slab *slab; + + slab = virt_to_slab(cred); + if (!slab) + return false; + + return slab->slab_cache->flags & SLAB_SET_PKEY; +} + +static struct cred *alloc_unprotected_creds(gfp_t flags) +{ + if (hardened_cred_enabled()) + return kmalloc(sizeof(struct cred), flags); + else + return kmem_cache_alloc(cred_jar, flags); +} + +static struct cred *alloc_protected_creds(gfp_t flags) +{ + return kmem_cache_alloc(cred_jar, flags); +} + +static void free_creds(struct cred *cred) +{ + bool cred_in_jar = true; + + if (hardened_cred_enabled()) + cred_in_jar = cred_is_protected(cred); + + if (cred_in_jar) + kmem_cache_free(cred_jar, cred); + else + kfree(cred); +} + /* * The RCU callback to actually dispose of a set of credentials */ @@ -75,7 +119,8 @@ static void put_cred_rcu(struct rcu_head *rcu) panic("CRED: put_cred_rcu() sees %p with usage %ld\n", cred, atomic_long_read(&cred->usage)); - security_cred_free(cred); + scoped_guard(kpkeys_hardened_cred) + security_cred_free(cred); key_put(cred->session_keyring); key_put(cred->process_keyring); key_put(cred->thread_keyring); @@ -86,7 +131,7 @@ static void put_cred_rcu(struct rcu_head *rcu) if (cred->ucounts) put_ucounts(cred->ucounts); put_user_ns(cred->user_ns); - kmem_cache_free(cred_jar, cred); + free_creds(cred); } /** @@ -174,7 +219,7 @@ struct cred *cred_alloc_blank(void) { struct cred *new; - new = kmem_cache_zalloc(cred_jar, GFP_KERNEL); + new = alloc_unprotected_creds(GFP_KERNEL | __GFP_ZERO); if (!new) return NULL; @@ -189,29 +234,10 @@ struct cred *cred_alloc_blank(void) return NULL; } -/** - * prepare_creds - Prepare a new set of credentials for modification - * - * Prepare a new set of task credentials for modification. A task's creds - * shouldn't generally be modified directly, therefore this function is used to - * prepare a new copy, which the caller then modifies and then commits by - * calling commit_creds(). - * - * Preparation involves making a copy of the objective creds for modification. - * - * Returns a pointer to the new creds-to-be if successful, NULL otherwise. - * - * Call commit_creds() or abort_creds() to clean up. - */ -struct cred *prepare_creds(void) +static struct cred *__prepare_creds(struct cred *new) { struct task_struct *task = current; const struct cred *old; - struct cred *new; - - new = kmem_cache_alloc(cred_jar, GFP_KERNEL); - if (!new) - return NULL; kdebug("prepare_creds() alloc %p", new); @@ -248,8 +274,57 @@ struct cred *prepare_creds(void) abort_creds(new); return NULL; } + +/** + * prepare_creds - Prepare a new set of credentials for modification + * + * Prepare a new set of task credentials for modification. A task's creds + * shouldn't generally be modified directly, therefore this function is used to + * prepare a new copy, which the caller then modifies and then commits by + * calling commit_creds(). + * + * Preparation involves making a copy of the objective creds for modification. + * + * Returns a pointer to the new creds-to-be if successful, NULL otherwise. + * + * Call commit_creds() or abort_creds() to clean up. + */ +struct cred *prepare_creds(void) +{ + struct cred *new; + + new = alloc_unprotected_creds(GFP_KERNEL); + if (!new) + return NULL; + + return __prepare_creds(new); +} EXPORT_SYMBOL(prepare_creds); + +/** + * prepare_protected_creds - Prepare a new set of credentials in protected + * memory + * + * This function is equivalent to protect_creds(prepare_creds()), but avoids + * the copy in prepare_creds() by directly allocating the credentials in + * protected memory. The returned object may only be modified by switching to + * a higher kpkeys level, if kpkeys_hardened_cred is enabled. + */ +struct cred *prepare_protected_creds(void) +{ + struct cred *new; + + new = alloc_protected_creds(GFP_KERNEL); + if (!new) + return NULL; + + guard(kpkeys_hardened_cred)(); + + return __prepare_creds(new); +} +EXPORT_SYMBOL(prepare_protected_creds); + /* * Prepare credentials for current to perform an execve() * - The caller must hold ->cred_guard_mutex @@ -309,7 +384,9 @@ int copy_creds(struct task_struct *p, unsigned long clone_flags) return 0; } - new = prepare_creds(); + guard(kpkeys_hardened_cred)(); + + new = prepare_protected_creds(); if (!new) return -ENOMEM; @@ -400,6 +477,10 @@ int commit_creds(struct cred *new) BUG_ON(task->cred != old); BUG_ON(atomic_long_read(&new->usage) < 1); + guard(kpkeys_hardened_cred)(); + + new = protect_creds(new); + get_cred(new); /* we will require a ref for the subj creds too */ /* dumpability changes */ @@ -555,9 +636,16 @@ int set_cred_ucounts(struct cred *new) */ void __init cred_init(void) { + slab_flags_t flags = SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT; + struct kmem_cache_args args = {}; + + if (hardened_cred_enabled()) { + flags |= SLAB_SET_PKEY; + args.pkey = KPKEYS_PKEY_CRED; + } + /* allocate a slab in which we can store credentials */ - cred_jar = KMEM_CACHE(cred, - SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT); + cred_jar = kmem_cache_create("cred", sizeof(struct cred), &args, flags); } /** @@ -584,7 +672,7 @@ struct cred *prepare_kernel_cred(struct task_struct *daemon) if (WARN_ON_ONCE(!daemon)) return NULL; - new = kmem_cache_alloc(cred_jar, GFP_KERNEL); + new = alloc_unprotected_creds(GFP_KERNEL); if (!new) return NULL; @@ -627,6 +715,42 @@ struct cred *prepare_kernel_cred(struct task_struct *daemon) } EXPORT_SYMBOL(prepare_kernel_cred); +/** + * protect_creds - Move a set of credentials to protected memory + * @cred: The credentials to protect + * + * If kpkeys_hardened_cred is enabled, this function transfers @cred to + * protected memory. The returned object may only be modified by switching to a + * higher kpkeys level, for instance by using guard(kpkeys_hardened_cred). + * + * Because the credentials are copied to a new location and the old location is + * freed, any exising reference to @cred becomes invalid after this function is + * called. For this reason only the caller should have a reference to @cred. + * + * If any failure occurs, or if kpkeys_hardened_cred is disabled, @cred is + * returned unmodified. + */ +struct cred *protect_creds(struct cred *cred) +{ + struct cred *protected_cred; + + if (!hardened_cred_enabled()) + return cred; + + if (WARN_ON(atomic_long_read(&cred->usage) != 1)) + return cred; + + protected_cred = alloc_protected_creds(GFP_KERNEL); + if (WARN_ON(!protected_cred)) + return cred; + + guard(kpkeys_hardened_cred)(); + + *protected_cred = *cred; + kfree(cred); + return protected_cred; +} + /** * set_security_override - Set the security ID in a set of credentials * @new: The credentials to alter diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index 649847535fc3..1af3a9dae645 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -325,6 +325,19 @@ config KPKEYS_HARDENED_PGTABLES_TEST If unsure, say N. +config KPKEYS_HARDENED_CRED + bool "Harden task credentials using kernel pkeys" + depends on ARCH_HAS_KPKEYS + select KPKEYS_UNRESTRICTED_RCU + help + This option enforces the immutability of tasks credentials + (struct cred) by allocating them with a non-default protection (pkey) + and only enabling write access to that pkey in a limited set of cred + helpers. + + This option has no effect if the system does not support + kernel pkeys. + endmenu config CC_HAS_RANDSTRUCT From patchwork Mon Feb 3 10:28:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 76B31C02192 for ; Mon, 3 Feb 2025 10:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VjrjF09VQnxDeNYh4e1Ha75V6UufRCaFP71EexFz7fY=; b=KqtbKC4JcNQY+dH0QQ0cFOEyQq ibI+cn4HhXYRzN/Wb9XqcXpaoBH4cpcwITaTt0D8EG/RIusnGAKRdp0O1Akx1+ZpUQsPdN7Fxw4qU keRPABoAgwzFaUr57voG5lT3mndHdbaDSNVoMDrDlZYu6S0XRwLtpdxZ3/3vQe3wHMZRj4HcslvZE D3ZQbme/5di/DtJ27xVA132pGCilJ8AN6xhf65qPrTZvA2fafFiyqld/T0UHqvhhhWlpUu8twxkug BHUqFUkfDau1CCjzFPaHifIF01eMCFk+SPtcm4U+t+s/8hnNTKieHx8FQ+XYjFMR6OivPkzrU/9of 548orzVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1teu7V-0000000FAbk-1JSJ; Mon, 03 Feb 2025 10:56:01 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tethI-0000000F7Ts-11KZ for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:28:58 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 480031BC0; Mon, 3 Feb 2025 02:29:20 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 196943F63F; Mon, 3 Feb 2025 02:28:51 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 7/8] fs: Protect creds installed by override_creds() Date: Mon, 3 Feb 2025 10:28:08 +0000 Message-ID: <20250203102809.1223255-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_022856_368952_519639DC X-CRM114-Status: GOOD ( 18.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kpkeys_hardened_cred feature, when enabled, automatically protects credentials installed by commit_creds(). However, because override_creds() does not consume its argument, it is up to its callers to protect the credentials before calling override_creds(). This is done by calling protect_creds(), moving the credentials to a protected memory location. In some cases, the credentials returned by prepare_creds() are passed to override_creds() as-is. In such situation where write access to the credentials is not needed, prepare_protected_creds() is used to avoid the copy incurred by a separate call to protect_creds(). This patch covers the main users of override_creds(), but it is not comprehensive. This patch is a no-op if kpkeys_hardened_cred isn't enabled. Signed-off-by: Kevin Brodsky --- fs/aio.c | 2 +- fs/fuse/passthrough.c | 2 +- fs/nfs/nfs4idmap.c | 2 +- fs/nfsd/auth.c | 2 +- fs/nfsd/nfs4recover.c | 2 +- fs/nfsd/nfsfh.c | 2 +- fs/open.c | 2 +- fs/overlayfs/dir.c | 2 +- fs/overlayfs/super.c | 2 +- 9 files changed, 9 insertions(+), 9 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 7b976b564cfc..ab9f4c8d778a 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -1657,7 +1657,7 @@ static int aio_fsync(struct fsync_iocb *req, const struct iocb *iocb, if (unlikely(!req->file->f_op->fsync)) return -EINVAL; - req->creds = prepare_creds(); + req->creds = prepare_protected_creds(); if (!req->creds) return -ENOMEM; diff --git a/fs/fuse/passthrough.c b/fs/fuse/passthrough.c index 607ef735ad4a..4451651b1e51 100644 --- a/fs/fuse/passthrough.c +++ b/fs/fuse/passthrough.c @@ -248,7 +248,7 @@ int fuse_backing_open(struct fuse_conn *fc, struct fuse_backing_map *map) goto out_fput; fb->file = file; - fb->cred = prepare_creds(); + fb->cred = prepare_protected_creds(); refcount_set(&fb->count, 1); res = fuse_backing_id_alloc(fc, fb); diff --git a/fs/nfs/nfs4idmap.c b/fs/nfs/nfs4idmap.c index 25a7c771cfd8..6ff25dd5c2fb 100644 --- a/fs/nfs/nfs4idmap.c +++ b/fs/nfs/nfs4idmap.c @@ -228,7 +228,7 @@ int nfs_idmap_init(void) set_bit(KEY_FLAG_ROOT_CAN_CLEAR, &keyring->flags); cred->thread_keyring = keyring; cred->jit_keyring = KEY_REQKEY_DEFL_THREAD_KEYRING; - id_resolver_cache = cred; + id_resolver_cache = protect_creds(cred); return 0; failed_reg_legacy: diff --git a/fs/nfsd/auth.c b/fs/nfsd/auth.c index 4dc327e02456..09b377a97147 100644 --- a/fs/nfsd/auth.c +++ b/fs/nfsd/auth.c @@ -79,7 +79,7 @@ int nfsd_setuser(struct svc_cred *cred, struct svc_export *exp) else new->cap_effective = cap_raise_nfsd_set(new->cap_effective, new->cap_permitted); - put_cred(override_creds(new)); + put_cred(override_creds(protect_creds(new))); return 0; oom: diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c index 28f4d5311c40..095664648103 100644 --- a/fs/nfsd/nfs4recover.c +++ b/fs/nfsd/nfs4recover.c @@ -81,7 +81,7 @@ nfs4_save_creds(const struct cred **original_creds) new->fsuid = GLOBAL_ROOT_UID; new->fsgid = GLOBAL_ROOT_GID; - *original_creds = override_creds(new); + *original_creds = override_creds(protect_creds(new)); return 0; } diff --git a/fs/nfsd/nfsfh.c b/fs/nfsd/nfsfh.c index 32019751a41e..d64d23e9357e 100644 --- a/fs/nfsd/nfsfh.c +++ b/fs/nfsd/nfsfh.c @@ -221,7 +221,7 @@ static __be32 nfsd_set_fh_dentry(struct svc_rqst *rqstp, struct net *net, new->cap_effective = cap_raise_nfsd_set(new->cap_effective, new->cap_permitted); - put_cred(override_creds(new)); + put_cred(override_creds(protect_creds(new))); } else { error = nfsd_setuser_and_check_port(rqstp, cred, exp); if (error) diff --git a/fs/open.c b/fs/open.c index 932e5a6de63b..3b5331b7c0f0 100644 --- a/fs/open.c +++ b/fs/open.c @@ -457,7 +457,7 @@ static const struct cred *access_override_creds(void) * freeing. */ override_cred->non_rcu = 1; - return override_creds(override_cred); + return override_creds(protect_creds(override_cred)); } static long do_faccessat(int dfd, const char __user *filename, int mode, int flags) diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c index c9993ff66fc2..943ec4300ddb 100644 --- a/fs/overlayfs/dir.c +++ b/fs/overlayfs/dir.c @@ -580,7 +580,7 @@ static const struct cred *ovl_setup_cred_for_create(struct dentry *dentry, * We must be called with creator creds already, otherwise we risk * leaking creds. */ - old_cred = override_creds(override_cred); + old_cred = override_creds(protect_creds(override_cred)); WARN_ON_ONCE(old_cred != ovl_creds(dentry->d_sb)); return override_cred; diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index 86ae6f6da36b..3489a62c5d8a 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -1318,7 +1318,7 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc) sb->s_d_op = &ovl_dentry_operations; err = -ENOMEM; - ofs->creator_cred = cred = prepare_creds(); + ofs->creator_cred = cred = prepare_protected_creds(); if (!cred) goto out_err; From patchwork Mon Feb 3 10:28:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 55EE0C02196 for ; Mon, 3 Feb 2025 10:57:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zoyyTWy8+Zu7JkfSAKSCzAU5xw+8zxp6lIz7/LeWBsU=; b=Jngtw8WXnMgQWyVPVaFX5m++cu F26RaWAtU9/6VhWCUMm1+15VQ8+5w0TgCHycaNTNy/zYbASjEf90qG4nbFltvQMOIMD9pvA1lUhEZ WaH7T3V6J2ILf+k4Dh/q+33bM8MgF/1uLGA0m4zl75l9DsmBXLPaj7tXzkvedv3FbY1VTaSOV3AB5 DZf5u6HonkCC3sbGPbfJUf6Rt0366BHpWattA4381YsW5VIO/B9lJ2InQ3BLk8c+nlKJ22v4IJZcA A43oBjEb4Kygyc6JUu+XAIcY5mGDRnKERtnSDQ2GFjChGyKBFh7ffnOR9N8zdvMd0qzXRc2X1/kCQ lNSWYM+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1teu8m-0000000FAlZ-3uao; Mon, 03 Feb 2025 10:57:20 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tethP-0000000F7WE-3TTe for linux-arm-kernel@bombadil.infradead.org; Mon, 03 Feb 2025 10:29:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=zoyyTWy8+Zu7JkfSAKSCzAU5xw+8zxp6lIz7/LeWBsU=; b=NqhUCjVzzNH9IYAYME2YURfEs+ 10+BBOC2Lv61RFNIHywHyCixVHcLyCnxaTqT7VpIBS/hFYt3UjUwJ44ocnvbavu2mjK0in8J39chd tdGegS9+NhqwlZShrDnu0GG4ZHpADkKpfvc4CbH18ZquEWiU/t1roilPZY6V3Llr9iQ/PJYZCpacJ UBK30uH3HfuNTFKfmFlkOfzKulSIK1J5wF7K0rXmbNQ8UYMqj6bJu4vnaUBJRvDEwuPsjSItaruvt 0d0KeFMJ0NUJqyPG/qlSQOVBffYdXhNvvECrWEFfIxIIGPX3gFdck+2gDwgEB6MVNnUZ9j3sUMeE0 Z7V6sCgg==; Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tethM-0000000G3y1-2q4F for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 10:29:02 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5909D1BCA; Mon, 3 Feb 2025 02:29:24 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2AE363F7BD; Mon, 3 Feb 2025 02:28:56 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 8/8] mm: Add basic tests for kpkeys_hardened_cred Date: Mon, 3 Feb 2025 10:28:09 +0000 Message-ID: <20250203102809.1223255-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_102901_044221_CB0E2635 X-CRM114-Status: GOOD ( 18.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add basic tests for the kpkeys_hardened_pgtables feature: try to perform a direct write to current->{cred,real_cred} and ensure it fails. Signed-off-by: Kevin Brodsky --- mm/Makefile | 1 + mm/kpkeys_hardened_cred_test.c | 42 ++++++++++++++++++++++++++++++++++ security/Kconfig.hardening | 11 +++++++++ 3 files changed, 54 insertions(+) create mode 100644 mm/kpkeys_hardened_cred_test.c diff --git a/mm/Makefile b/mm/Makefile index f7263b7f45b8..2024226902d4 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -149,3 +149,4 @@ obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES_TEST) += kpkeys_hardened_pgtables_test.o +obj-$(CONFIG_KPKEYS_HARDENED_CRED_TEST) += kpkeys_hardened_cred_test.o diff --git a/mm/kpkeys_hardened_cred_test.c b/mm/kpkeys_hardened_cred_test.c new file mode 100644 index 000000000000..46048098f99d --- /dev/null +++ b/mm/kpkeys_hardened_cred_test.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include + +static void write_cred(struct kunit *test) +{ + long zero = 0; + int ret; + + ret = copy_to_kernel_nofault((unsigned long *)current->cred, &zero, sizeof(zero)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Write to current->cred wasn't prevented"); + + ret = copy_to_kernel_nofault((unsigned long *)current->real_cred, &zero, sizeof(zero)); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Write to current->real_cred wasn't prevented"); +} + +static int kpkeys_hardened_cred_suite_init(struct kunit_suite *suite) +{ + if (!arch_kpkeys_enabled()) { + pr_err("Cannot run kpkeys_hardened_cred tests: kpkeys are not supported\n"); + return 1; + } + + return 0; +} + +static struct kunit_case kpkeys_hardened_cred_test_cases[] = { + KUNIT_CASE(write_cred), + {} +}; + +static struct kunit_suite kpkeys_hardened_cred_test_suite = { + .name = "Hardened credentials using kpkeys", + .test_cases = kpkeys_hardened_cred_test_cases, + .suite_init = kpkeys_hardened_cred_suite_init, +}; +kunit_test_suite(kpkeys_hardened_cred_test_suite); + +MODULE_DESCRIPTION("Tests for the kpkeys_hardened_cred feature"); +MODULE_LICENSE("GPL"); diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index 1af3a9dae645..9b0563a03ab4 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -338,6 +338,17 @@ config KPKEYS_HARDENED_CRED This option has no effect if the system does not support kernel pkeys. +config KPKEYS_HARDENED_CRED_TEST + tristate "KUnit tests for kpkeys_hardened_cred" if !KUNIT_ALL_TESTS + depends on KPKEYS_HARDENED_CRED + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable this option to check that the kpkeys_hardened_cred feature + functions as intended, i.e. prevents arbitrary writes to live credentials. + + If unsure, say N. + endmenu config CC_HAS_RANDSTRUCT