From patchwork Mon Feb 3 10:28:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 13957221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDE9DC02193 for ; Mon, 3 Feb 2025 10:29:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6701E280011; Mon, 3 Feb 2025 05:29:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 61FF528000E; Mon, 3 Feb 2025 05:29:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C0BC280011; Mon, 3 Feb 2025 05:29:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2946028000E for ; Mon, 3 Feb 2025 05:29:01 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D985B1C9D8F for ; Mon, 3 Feb 2025 10:28:45 +0000 (UTC) X-FDA: 83078259810.13.0DF4AFC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id 4713C1A0008 for ; Mon, 3 Feb 2025 10:28:44 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738578524; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gXEcaQ/nat4AynYmLE14uvl2r7aUGOOMKf84Ghy3AwI=; b=F+Jd/lEPd1GsFTzmGdV7DzhZhv7KE7RhnRSF6Xdp6n8ACWGSP6sh7MIYXEKNSe/P+pT2E1 +PKp+rolQa6XwGriqrfAbhuzx+9Sg9IbiDM1mPGDC2+vIqriOEAxIqEU50UW61OHLxGeyQ sDX6Qr5LXnfR9M3SJ6wwHyyvN2DPZ5M= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738578524; a=rsa-sha256; cv=none; b=F0nPnhCTaTY14N++No//UUDyxzaLm4biRX8HePQ2Uhg3rdVSmgfchCbAIoEFs2DTp6e7ml yGACaK1HMFhz14D3OF3Tpwdot8spbG6BYLn9KFAfAecCnvsuButADMfZLAvMPKRQtF184c A62bfcschTupCmnYN6emz5kHSG8UV64= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 128781BA8; Mon, 3 Feb 2025 02:29:08 -0800 (PST) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D8CF83F63F; Mon, 3 Feb 2025 02:28:39 -0800 (PST) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH 4/8] rcu: Allow processing kpkeys-protected data Date: Mon, 3 Feb 2025 10:28:05 +0000 Message-ID: <20250203102809.1223255-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250203102809.1223255-1-kevin.brodsky@arm.com> References: <20250203102809.1223255-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4713C1A0008 X-Stat-Signature: zoxo9cn1eqroxx9j1p4w1993e9f8qrri X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1738578524-612693 X-HE-Meta: U2FsdGVkX1/NSxoSuRfhTw+dJFLWX2xZDfdr9rCby3Ii4vjGBkON/WUk8QwKXOBzT/8nx5mxs11Mp3kEY+bwzIo5Bg35t5RRTaR7JMmOopppVwDvDoq4iw3fDo62tfZq6wTWD/BZ8hiiImn1R8I2xPVdqyP2F1tr3XBQoQHnFxxsgUlieFF+htdiFdDr0vXpkpcXOVsKKq66e3UiperDf/BRP23XQ1hkYk64CyJX0DGTH99VpI4tsAdEt7sOT76dm4N3Z6ONMDFTni0/ZkAT4YdktYdDJBshHkYyui6+5nG31fsqUAPkLSgMjATfPfwIY9G1Bz+Ac3oPaH26obJIazhluy2/dghEke5+j/MxGJXrEY5KyL0mpCsj6iJg3pde1DbfZlVt3tvukyCa59dPFv9K8X9UG8MhyfbuUrqz3Kkazx68LO4Nv4QHbieKbsSWTthZhzQWgXwiZ8aMEyHKxkra5p/WhMmACLFP+LLKVPIvmx5ukR+r0PDr8spuA3g+LwIpoIgC1SqH/J2F39Gp3eebxl1giH75JdH0eeK+sSEwH1e2JmcgV00fEMr81SrLorsP0q2gpIj/9uN+XSOvI14DF+Fo5W76JtIvYLSsoTgf+iR9xtpgycYWWx39Ne+TAaMrOKBvzmNX2AaFT+O0bwcIaczV9mnGUZ611AvDqiBLO5LHS+osQHuwlxMzquXTWMg9ECh4c+IiQPWhPcr1eand/SmJVZ6LEkw3f+lf4DAqiIMVe9NWj4m9nEIJP9adKp3jOgo/dUsdc5CA+mGOLNGJeG2h2orUQZ/+E+wckSUxVy4EQqL8edXWhNQyH86NdghrKp+kNWCmP/BaPLvFRWseJydUKB/U2z2AkKrJ6gBK7mdpDD/JOA2mEKS0h49ufGmDZnFNDcgNa039cKkNcsY8pe2Tma4Iuh3yuWFmUnJP+QiVBW1SVrDux2ZsiNiLPabtY/5YKpDyxfiH7up DQEwab+Y OCId9MxRbY9M29Q7KjbgEKHXi0cf+35GGM78ailWsCaZpSgmLUqKOGumMAP1IUcqN5wcNo7s6IyY8BvYXteO8uJeXyHoCnn1/0lhdEdHhTf8xY0b95LwzE41sfYLMjPtwiL1UBgxhKgOmiX0GXyEq5xxvDbWNANJrS5CqoOyccZ9veKREAIh8SflVvs7FRInXkNEE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Data assigned a non-default pkey is not writable at the default kpkeys level. If such data is managed via RCU, some mechanism is required to temporarily grant write access to the data's struct rcu_head, for instance when zeroing the callback pointer. There is unfortunately no straightforward way for RCU to know whether the managed data is mapped with a non-default pkey. This patch takes the easy route and switches to the unrestricted kpkeys level whenever struct rcu_head is written; this should work reliably but it is clearly suboptimal. That behaviour is enabled by selecting CONFIG_KPKEYS_UNRESTRICTED_RCU. This patch isn't comprehensive, in particular it does not take care of Tiny RCU. Signed-off-by: Kevin Brodsky --- include/linux/kpkeys.h | 6 ++++++ kernel/rcu/rcu_segcblist.c | 10 +++++++--- kernel/rcu/tree.c | 4 +++- mm/Kconfig | 2 ++ 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 9d9feec83ccf..c5d804c1ab7b 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -154,4 +154,10 @@ static inline void kpkeys_hardened_pgtables_enable(void) {} #endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ +#ifdef CONFIG_KPKEYS_UNRESTRICTED_RCU +KPKEYS_GUARD(kpkeys_rcu, KPKEYS_LVL_UNRESTRICTED) +#else +KPKEYS_GUARD_NOOP(kpkeys_rcu) +#endif + #endif /* _LINUX_KPKEYS_H */ diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c index 298a2c573f02..a9b5552b53a5 100644 --- a/kernel/rcu/rcu_segcblist.c +++ b/kernel/rcu/rcu_segcblist.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "rcu_segcblist.h" @@ -332,7 +333,8 @@ void rcu_segcblist_enqueue(struct rcu_segcblist *rsclp, rcu_segcblist_inc_len(rsclp); rcu_segcblist_inc_seglen(rsclp, RCU_NEXT_TAIL); rhp->next = NULL; - WRITE_ONCE(*rsclp->tails[RCU_NEXT_TAIL], rhp); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[RCU_NEXT_TAIL], rhp); WRITE_ONCE(rsclp->tails[RCU_NEXT_TAIL], &rhp->next); } @@ -381,7 +383,8 @@ void rcu_segcblist_extract_done_cbs(struct rcu_segcblist *rsclp, rclp->len = rcu_segcblist_get_seglen(rsclp, RCU_DONE_TAIL); *rclp->tail = rsclp->head; WRITE_ONCE(rsclp->head, *rsclp->tails[RCU_DONE_TAIL]); - WRITE_ONCE(*rsclp->tails[RCU_DONE_TAIL], NULL); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[RCU_DONE_TAIL], NULL); rclp->tail = rsclp->tails[RCU_DONE_TAIL]; for (i = RCU_CBLIST_NSEGS - 1; i >= RCU_DONE_TAIL; i--) if (rsclp->tails[i] == rsclp->tails[RCU_DONE_TAIL]) @@ -436,7 +439,8 @@ void rcu_segcblist_insert_done_cbs(struct rcu_segcblist *rsclp, if (!rclp->head) return; /* No callbacks to move. */ rcu_segcblist_add_seglen(rsclp, RCU_DONE_TAIL, rclp->len); - *rclp->tail = rsclp->head; + scoped_guard(kpkeys_rcu) + *rclp->tail = rsclp->head; WRITE_ONCE(rsclp->head, rclp->head); for (i = RCU_DONE_TAIL; i < RCU_CBLIST_NSEGS; i++) if (&rsclp->head == rsclp->tails[i]) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 475f31deed14..48d9d14a2af6 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -64,6 +64,7 @@ #include #include #include +#include #include "../time/tick-internal.h" #include "tree.h" @@ -2542,7 +2543,8 @@ static void rcu_do_batch(struct rcu_data *rdp) f = rhp->func; debug_rcu_head_callback(rhp); - WRITE_ONCE(rhp->func, (rcu_callback_t)0L); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(rhp->func, (rcu_callback_t)0L); f(rhp); rcu_lock_release(&rcu_callback_map); diff --git a/mm/Kconfig b/mm/Kconfig index 2a8ebe780e64..e2671c57e047 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1152,6 +1152,8 @@ config ARCH_HAS_KPKEYS # ARCH_HAS_KPKEYS must be selected when selecting this option config ARCH_HAS_KPKEYS_HARDENED_PGTABLES bool +config KPKEYS_UNRESTRICTED_RCU + bool config ARCH_USES_PG_ARCH_2 bool