From patchwork Mon Jul 17 18:03:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13316155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9952C0015E for ; Mon, 17 Jul 2023 18:03:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229917AbjGQSDV (ORCPT ); Mon, 17 Jul 2023 14:03:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229868AbjGQSDU (ORCPT ); Mon, 17 Jul 2023 14:03:20 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86C8BB3; Mon, 17 Jul 2023 11:03:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1B95C61063; Mon, 17 Jul 2023 18:03:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E373C433C8; Mon, 17 Jul 2023 18:03:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689616998; bh=Ll8sW1RjttvaLOWSpYGmiHDsZNfDqJfD+od6jxaPQrA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S0RMFCjkONoOuMxS9wbNLk3KMgyDCBwGCX+WPfnD+5EYKbt6/ucnLgi5R8siUgpll W/6Y3RIP1LGm04ZLXoCHRtcLXM4ICJ9Nwr8100cyoH8CAnlUZzsTyx929Gj1WGg42r 4IJUwSiBKfnw+X8IebGMKsl0zMKY8h4to7O1yWmVbwJevFbhel1AuM2yVi9BXWQoQ3 pB3/YRFMK0wjQj3VEL0v2o9LuvpQ9SBrielyTypx3CpivMgW97EJeI5DNm1GLokb1v I0ond5XpzKMLCkuOYM0aghvvuqvmT0Udt/q33jKPHxmkxRXTV0riv1U2ji6c7OTC66 A/LpxD4wRu66A== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 363F1CE03F1; Mon, 17 Jul 2023 11:03:18 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 1/6] rcu: Update synchronize_rcu_mult() comment for call_rcu_hurry() Date: Mon, 17 Jul 2023 11:03:12 -0700 Message-Id: <20230717180317.1097590-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> References: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org Those who have worked with RCU for some time will naturally think in terms of the long-standing call_rcu() API rather than the much newer call_rcu_hurry() API. But it is call_rcu_hurry() that you should normally pass to synchronize_rcu_mult(). This commit therefore updates the header comment to point this out. Signed-off-by: Paul E. McKenney Reviewed-by: Joel Fernandes (Google) --- include/linux/rcupdate_wait.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/rcupdate_wait.h b/include/linux/rcupdate_wait.h index 699b938358bf..5e0f74f2f8ca 100644 --- a/include/linux/rcupdate_wait.h +++ b/include/linux/rcupdate_wait.h @@ -42,6 +42,11 @@ do { \ * call_srcu() function, with this wrapper supplying the pointer to the * corresponding srcu_struct. * + * Note that call_rcu_hurry() should be used instead of call_rcu() + * because in kernels built with CONFIG_RCU_LAZY=y the delay between the + * invocation of call_rcu() and that of the corresponding RCU callback + * can be multiple seconds. + * * The first argument tells Tiny RCU's _wait_rcu_gp() not to * bother waiting for RCU. The reason for this is because anywhere * synchronize_rcu_mult() can be called is automatically already a full From patchwork Mon Jul 17 18:03:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13316156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80906EB64DC for ; Mon, 17 Jul 2023 18:03:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229583AbjGQSDX (ORCPT ); Mon, 17 Jul 2023 14:03:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229562AbjGQSDV (ORCPT ); Mon, 17 Jul 2023 14:03:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD4C7E4C; Mon, 17 Jul 2023 11:03:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2D9C16119D; Mon, 17 Jul 2023 18:03:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E093C433C9; Mon, 17 Jul 2023 18:03:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689616998; bh=v7j+4/RDCywUwH0JVY5YSjpILugIRR8+kpY9IyawFFE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RA/y3TFUBbF+jj4qXI5cAWvGTAEUWF8bE2wU5hgIC3hnyJaC1899+YYO0wMnVFP96 dG2kZN61Uzgx8U4QpAByTA4JhEdt9K9ZEfBGar8/r6gc5n9jDFCSmwoU3dILafVC6X IStR1cWC30coigyp2eW1S9B62ajApXZ7zT+pvpxbCR/NOcKQ23Qvfmq3lNbrh2R+U3 Nh3JGGIbwnkdChC5uNZU+oggLbR3SkhwX7lJdpe0pvBjHc/qO3607gZ5VGaEbsX2jR 19bTlHkQ6XrOKvo6N+HXoRGzl1bj4QKpbO0rS4tjotG/UlU0c1lWooxQkCmnwz1Azh 1A1+N8q6CchvQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 3A439CE04CD; Mon, 17 Jul 2023 11:03:18 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Masami Hiramatsu Subject: [PATCH rcu 2/6] rcu: Clarify rcu_is_watching() kernel-doc comment Date: Mon, 17 Jul 2023 11:03:13 -0700 Message-Id: <20230717180317.1097590-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> References: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org Make it clear that this function always returns either true or false without other planned failure modes. Reported-by: Masami Hiramatsu Signed-off-by: Paul E. McKenney Reviewed-by: Joel Fernandes (Google) --- kernel/rcu/tree.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 1449cb69a0e0..fae9b4e29c93 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -679,10 +679,14 @@ static void rcu_disable_urgency_upon_qs(struct rcu_data *rdp) /** * rcu_is_watching - see if RCU thinks that the current CPU is not idle * - * Return true if RCU is watching the running CPU, which means that this - * CPU can safely enter RCU read-side critical sections. In other words, - * if the current CPU is not in its idle loop or is in an interrupt or - * NMI handler, return true. + * Return @true if RCU is watching the running CPU and @false otherwise. + * An @true return means that this CPU can safely enter RCU read-side + * critical sections. + * + * More specifically, if the current CPU is not deep within its idle + * loop, return @true. Note that rcu_is_watching() will return @true if + * invoked from an interrupt or NMI handler, even if that interrupt or + * NMI interrupted the CPU while it was deep within its idle loop. * * Make notrace because it can be called by the internal functions of * ftrace, and making this notrace removes unnecessary recursion calls. From patchwork Mon Jul 17 18:03:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13316157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21911C001DC for ; Mon, 17 Jul 2023 18:03:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229458AbjGQSDX (ORCPT ); Mon, 17 Jul 2023 14:03:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229819AbjGQSDV (ORCPT ); Mon, 17 Jul 2023 14:03:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCE4A1BE; Mon, 17 Jul 2023 11:03:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 44FF7611C6; Mon, 17 Jul 2023 18:03:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9653EC433CA; Mon, 17 Jul 2023 18:03:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689616998; bh=w65onYuj7ePMCkLvjZLbWr2mrrWxblP9v1QgU7VLyv0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K+3yOAwjLMvwjPlg9dCrC2H0t0nf8sR/shzrFrp+rKAiRP7ALllkz6jIR1V76IbVm 07qhF6IJSI3QXZnULiX9lH0lhN4N0yqIHNoJH+G+DH3FlfeJZOKNdmKdE3lpuSNB0o yjkaPXFPEUPSVz4lqOXjvoLzPKQXLLdMfbddElVJ0d3/7mMCpRiQhzUA4FLJfzzEZZ CDkkg27uaPSs6Hz8lq8mVC5z4EFxGrPrDEYn4SeafPT/7CFABT08KbjhI8lPqQwKld MK2hhI/EzgwfSBJYM9EYVSq1uWc+lom+YwZR20YicE62ZBOZRhkperItRpQdWU79IA KpQpqsZ6/D63g== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 3CF50CE0806; Mon, 17 Jul 2023 11:03:18 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Linus Torvalds , atthias Brugger , "Rafael J. Wysocki" , =?utf-8?b?TWljaGHFgiBN?= =?utf-8?b?aXJvc8WCYXc=?= , Dmitry Osipenko , Sachin Sant , Joel Fernandes Subject: [PATCH rcu 3/6] srcu,notifier: Remove #ifdefs in favor of SRCU Tiny srcu_usage Date: Mon, 17 Jul 2023 11:03:14 -0700 Message-Id: <20230717180317.1097590-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> References: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org This commit removes two #ifdef directives from include/linux/notifier.h by causing SRCU Tiny to provide a dummy srcu_usage structure and a dummy __SRCU_USAGE_INIT() macro. Suggested-by: Linus Torvalds Cc: atthias Brugger Cc: "Rafael J. Wysocki" Cc: "Michał Mirosław" Cc: Dmitry Osipenko Cc: Sachin Sant Cc: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney --- include/linux/notifier.h | 11 ----------- include/linux/srcutiny.h | 7 +++++++ 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/notifier.h b/include/linux/notifier.h index 86544707236a..45702bdcbceb 100644 --- a/include/linux/notifier.h +++ b/include/linux/notifier.h @@ -73,9 +73,7 @@ struct raw_notifier_head { struct srcu_notifier_head { struct mutex mutex; -#ifdef CONFIG_TREE_SRCU struct srcu_usage srcuu; -#endif struct srcu_struct srcu; struct notifier_block __rcu *head; }; @@ -106,7 +104,6 @@ extern void srcu_init_notifier_head(struct srcu_notifier_head *nh); #define RAW_NOTIFIER_INIT(name) { \ .head = NULL } -#ifdef CONFIG_TREE_SRCU #define SRCU_NOTIFIER_INIT(name, pcpu) \ { \ .mutex = __MUTEX_INITIALIZER(name.mutex), \ @@ -114,14 +111,6 @@ extern void srcu_init_notifier_head(struct srcu_notifier_head *nh); .srcuu = __SRCU_USAGE_INIT(name.srcuu), \ .srcu = __SRCU_STRUCT_INIT(name.srcu, name.srcuu, pcpu), \ } -#else -#define SRCU_NOTIFIER_INIT(name, pcpu) \ - { \ - .mutex = __MUTEX_INITIALIZER(name.mutex), \ - .head = NULL, \ - .srcu = __SRCU_STRUCT_INIT(name.srcu, name.srcuu, pcpu), \ - } -#endif #define ATOMIC_NOTIFIER_HEAD(name) \ struct atomic_notifier_head name = \ diff --git a/include/linux/srcutiny.h b/include/linux/srcutiny.h index ebd72491af99..06ce65518974 100644 --- a/include/linux/srcutiny.h +++ b/include/linux/srcutiny.h @@ -48,6 +48,13 @@ void srcu_drive_gp(struct work_struct *wp); #define DEFINE_STATIC_SRCU(name) \ static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name, name) +// Dummy structure for srcu_notifier_head. +struct srcu_usage { + char srcuu_dummy; +}; + +#define __SRCU_USAGE_INIT(name) { .srcuu_dummy = 0, } + void synchronize_srcu(struct srcu_struct *ssp); /* From patchwork Mon Jul 17 18:03:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13316158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A140FC0015E for ; Mon, 17 Jul 2023 18:03:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230206AbjGQSDZ (ORCPT ); Mon, 17 Jul 2023 14:03:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229938AbjGQSDV (ORCPT ); Mon, 17 Jul 2023 14:03:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7521E55; Mon, 17 Jul 2023 11:03:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4CF5A611CE; Mon, 17 Jul 2023 18:03:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6588C433CB; Mon, 17 Jul 2023 18:03:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689616998; bh=qrmkRJUcCb33D57HkzURt5PuKRrWNnsRo/tBMS92MhE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UdwHjLqXUhPxQ2N5R1zmC27110AaNXYiFtwwZ2a8+8xHPpWEmvCrbhUkry60YJLIM b2Pczt9fyDGyuVibGeaSgkeS2izPNY+H7icYNdr9eNUOovesJykc4EcyDMFnWuAS+S kCngrQmTBoYtZwBj6K4eqy8DQqSPsVLoVWZPolmCscxt9R7ihK7lIz3NExXJxFsf6r UndJA48RETusgsPrsvYbuZ33QpIHlpyRWNrL793sSCTGZcRg0UtGfUPzmYIWnXeHH7 fjPMd18adrwHyXXDQeS4qcKtNMvo/2yMAUC7Whq/EI9VspYLBpIIv3JbBOJOGIpvrU XZcXo6W0YVc5Q== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 3F9C0CE0836; Mon, 17 Jul 2023 11:03:18 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 4/6] rcu: Mark __rcu_irq_enter_check_tick() ->rcu_urgent_qs load Date: Mon, 17 Jul 2023 11:03:15 -0700 Message-Id: <20230717180317.1097590-4-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> References: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org The rcu_request_urgent_qs_task() function does a cross-CPU store to ->rcu_urgent_qs, so this commit therefore marks the load in __rcu_irq_enter_check_tick() with READ_ONCE(). Signed-off-by: Paul E. McKenney Reviewed-by: Joel Fernandes (Google) --- kernel/rcu/tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index fae9b4e29c93..aec07f2ec638 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -632,7 +632,7 @@ void __rcu_irq_enter_check_tick(void) // prevents self-deadlock. So we can safely recheck under the lock. // Note that the nohz_full state currently cannot change. raw_spin_lock_rcu_node(rdp->mynode); - if (rdp->rcu_urgent_qs && !rdp->rcu_forced_tick) { + if (READ_ONCE(rdp->rcu_urgent_qs) && !rdp->rcu_forced_tick) { // A nohz_full CPU is in the kernel and RCU needs a // quiescent state. Turn on the tick! WRITE_ONCE(rdp->rcu_forced_tick, true); From patchwork Mon Jul 17 18:03:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13316159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1037CC001DF for ; Mon, 17 Jul 2023 18:03:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230084AbjGQSDY (ORCPT ); Mon, 17 Jul 2023 14:03:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229937AbjGQSDV (ORCPT ); Mon, 17 Jul 2023 14:03:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14D90E60; Mon, 17 Jul 2023 11:03:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6A458611CF; Mon, 17 Jul 2023 18:03:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B8EEBC433CD; Mon, 17 Jul 2023 18:03:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689616998; bh=faENVi6cC50zYs1JsPUdeoJna1PA0xy0tmxx8qvTGUs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gwsgXiGGyLxGtlNnGaH4qkO5xQ+pXbWqFQn67T/rhn/Kk9ilaql1M1NQwI5VTfXtz favsZTYWhw/fpKAmN13CT5s+9eRGKUzzI4WrJDUtw7HjFEGdD7kJa4BiIue7QiMA6i lWZXDSmmOIXMHOrJg9slg6rosvhwhRiQWiKKSu62ipxsDMyegCMPKp7OHMeovFcbQT +F2h1r8TaOcpEfRwQqmWq/bp4x9Z+IGUt9si8blRDYzOwi+vYU9f5pUx5Q1S8+XQI8 FOukS7Y9EIb569hDIMCtl38Tm63De0QF2Kfk059vU3q0YYfdaWiwuSg6Ep3ODSoZx8 Q8UStCGfAqAlw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 41D52CE0902; Mon, 17 Jul 2023 11:03:18 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 5/6] rcu: Make the rcu_nocb_poll boot parameter usable via boot config Date: Mon, 17 Jul 2023 11:03:16 -0700 Message-Id: <20230717180317.1097590-5-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> References: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org The rcu_nocb_poll kernel boot parameter is defined via early_param(), whose parsing functions are invoked from parse_early_param() which is in turn invoked by setup_arch(), which is very early indeed.  It is invoked so early that the console output timestamps read 0.000000, in other words, before time begins. This use of early_param() means that the rcu_nocb_poll kernel boot parameter cannot usefully be embedded into the kernel image. Yes, you can embed it, but setup_boot_config() is invoked from start_kernel() too late for it to be parsed. But it makes no sense to parse this parameter so early. After all, it cannot do anything until the rcuog kthreads are created, which is long after rcu_init() time, let alone setup_boot_config() time. This commit therefore switches the rcu_nocb_poll kernel boot parameter from early_param() to __setup(), which allows boot-config parsing of this parameter, in turn allowing it to be embedded into the kernel image. Signed-off-by: Paul E. McKenney Reviewed-by: Joel Fernandes (Google) --- kernel/rcu/tree_nocb.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 43229d2b0c44..5598212d1f27 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -77,9 +77,9 @@ __setup("rcu_nocbs", rcu_nocb_setup); static int __init parse_rcu_nocb_poll(char *arg) { rcu_nocb_poll = true; - return 0; + return 1; } -early_param("rcu_nocb_poll", parse_rcu_nocb_poll); +__setup("rcu_nocb_poll", parse_rcu_nocb_poll); /* * Don't bother bypassing ->cblist if the call_rcu() rate is low. From patchwork Mon Jul 17 18:03:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13316160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 310EDC001E0 for ; Mon, 17 Jul 2023 18:03:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229999AbjGQSD0 (ORCPT ); Mon, 17 Jul 2023 14:03:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229970AbjGQSDV (ORCPT ); Mon, 17 Jul 2023 14:03:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D49B499; Mon, 17 Jul 2023 11:03:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AA8A9611C5; Mon, 17 Jul 2023 18:03:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E07CAC4339A; Mon, 17 Jul 2023 18:03:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689616998; bh=vEUUOY6M/3b2MdG2PW8TiiGilUJqe947oj/T9vFTNTA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dwsvUF/M6y1htH9x4F8FD9RJ7dMM3iqjK0L9G/KO/bHFbfAbvKW9LQqgdffTCkrJ8 Q3VAWs2tJvzYG01qCc+I0khfxeLsX5IXuA1qB7E7B6bmN6KzPvg0NDwchVgnJZIkRt l/BgTtbHEtZ8VzJoDXONq4GP1/9sdbeAh16Ap3IUrKuqUoEdH5IIJkkIX1TXbu5hZI ZMocGgOmJXnGa8S6m/1xMih1P4I3vb6g6tBoxrHzFJD1HGwLxm8KgklhyGYt0jh8x3 PoBhOvFNITxVPY0ovU5TqzlWXzTrJ9FCy1Vy3qu9PymxoGgX6Z0t1iYAQmHbcobG8s ZyVrbtuztfxPw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 441F6CE092F; Mon, 17 Jul 2023 11:03:18 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, Alan Huang , "Paul E . McKenney" Subject: [PATCH rcu 6/6] rcu: Use WRITE_ONCE() for assignments to ->next for rculist_nulls Date: Mon, 17 Jul 2023 11:03:17 -0700 Message-Id: <20230717180317.1097590-6-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> References: <6127192c-da9b-4599-9738-6e8f92e6c75c@paulmck-laptop> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Alan Huang When the objects managed by rculist_nulls are allocated with SLAB_TYPESAFE_BY_RCU, old readers may still hold references to an object even though it is just now being added, which means the modification of ->next is visible to readers. This patch therefore uses WRITE_ONCE() for assignments to ->next. Signed-off-by: Alan Huang Signed-off-by: Paul E. McKenney Reviewed-by: Joel Fernandes (Google) --- include/linux/rculist_nulls.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/rculist_nulls.h b/include/linux/rculist_nulls.h index ba4c00dd8005..89186c499dd4 100644 --- a/include/linux/rculist_nulls.h +++ b/include/linux/rculist_nulls.h @@ -101,7 +101,7 @@ static inline void hlist_nulls_add_head_rcu(struct hlist_nulls_node *n, { struct hlist_nulls_node *first = h->first; - n->next = first; + WRITE_ONCE(n->next, first); WRITE_ONCE(n->pprev, &h->first); rcu_assign_pointer(hlist_nulls_first_rcu(h), n); if (!is_a_nulls(first)) @@ -137,7 +137,7 @@ static inline void hlist_nulls_add_tail_rcu(struct hlist_nulls_node *n, last = i; if (last) { - n->next = last->next; + WRITE_ONCE(n->next, last->next); n->pprev = &last->next; rcu_assign_pointer(hlist_nulls_next_rcu(last), n); } else {