From patchwork Wed Oct 19 22:46:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DAA1C433FE for ; Wed, 19 Oct 2022 22:47:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229644AbiJSWrE (ORCPT ); Wed, 19 Oct 2022 18:47:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230235AbiJSWrD (ORCPT ); Wed, 19 Oct 2022 18:47:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CE62188107; Wed, 19 Oct 2022 15:47:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 88595619E5; Wed, 19 Oct 2022 22:47:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E601EC433C1; Wed, 19 Oct 2022 22:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=Ygtgs0PHtrfe91imqGDUAOtSTrIZREFcGSd4BCKgdo0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MDQYW34tUukSZiX8K55GD3wJ5gibLdND5HFZUo8qjQ3KQ7iMdSFIbxKJopZ6jw13B Fe9D657uTPDoZEbvaEEiK1aNR2w5t4ORXEOpjyPcautM9bICAH7PVP7nr9GZy3zmQV SpbqN56G5zFTHsB/mAaZUZAbadveQ+Q9TPMlpH1AVliutFzGqxAhvMHOLNnh32uruG 0tcL1XtCEojAV5R3jWM6dGfk5Ucjbcb+gOlsf+31yVcECbcc2wlyzpgodfvNR2FDmD yHt8Ru3lc593r6juGUX71bZt4y2Wo53LyHCqSOmaeslA266I7PWMG2RNeCnxC9MhCl CfK73ddbVu4tw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id A393B5C06B4; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Zqiang , "Paul E . McKenney" Subject: [PATCH rcu 1/8] rcu: Remove duplicate RCU exp QS report from rcu_report_dead() Date: Wed, 19 Oct 2022 15:46:52 -0700 Message-Id: <20221019224659.2499511-1-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Zqiang The rcu_report_dead() function invokes rcu_report_exp_rdp() in order to force an immediate expedited quiescent state on the outgoing CPU, and then it invokes rcu_preempt_deferred_qs() to provide any required deferred quiescent state of either sort. Because the call to rcu_preempt_deferred_qs() provides the expedited RCU quiescent state if requested, the call to rcu_report_exp_rdp() is potentially redundant. One possible issue is a concurrent start of a new expedited RCU grace period, but this situation is already handled correctly by __sync_rcu_exp_select_node_cpus(). This function will detect that the CPU is going offline via the error return from its call to smp_call_function_single(). In that case, it will retry, and eventually stop retrying due to rcu_report_exp_rdp() clearing the ->qsmaskinitnext bit corresponding to the target CPU. As a result, __sync_rcu_exp_select_node_cpus() will report the necessary quiescent state after dealing with any remaining CPU. This change assumes that control does not enter rcu_report_dead() within an RCU read-side critical section, but then again, the surviving call to rcu_preempt_deferred_qs() has always made this assumption. This commit therefore removes the call to rcu_report_exp_rdp(), thus relying on rcu_preempt_deferred_qs() to handle both normal and expedited quiescent states. Signed-off-by: Zqiang Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 6bb8e72bc8151..0ca21ac0f0648 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4276,8 +4276,6 @@ void rcu_report_dead(unsigned int cpu) // Do any dangling deferred wakeups. do_nocb_deferred_wakeup(rdp); - /* QS for any half-done expedited grace period. */ - rcu_report_exp_rdp(rdp); rcu_preempt_deferred_qs(current); /* Remove outgoing CPU from mask in the leaf rcu_node structure. */ From patchwork Wed Oct 19 22:46:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6375FC4167D for ; Wed, 19 Oct 2022 22:47:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230452AbiJSWrI (ORCPT ); Wed, 19 Oct 2022 18:47:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230285AbiJSWrE (ORCPT ); Wed, 19 Oct 2022 18:47:04 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 966D01849BE; Wed, 19 Oct 2022 15:47:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4CD7BB82613; Wed, 19 Oct 2022 22:47:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4999C433D6; Wed, 19 Oct 2022 22:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=zhI+nSG0LEQR2PAAAvwCyyYQdtfxOPlUmkdh0E4iu6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ie/0CtZv4oEdoViQqjdmdwAOkuUbEniXSDznUSiM+fM8btdjJ+wZZM2Nh2vOHGJBK kuQUWupa2Mqkf1yLJFOutIuIHllryTzfrPxQyLNBM4qdYDtaHkftNSP3NVz68HSGB0 IIUB+ijLG9OUCiSW8lCz5etbYQY3MHEHNmg0wwb/8d5S9ximif6nNJQpgMmBPNNZZP tS8EnPBF1QFqoIJD96pTmSVl2edtkzBI6abWSzoYOroWhUZhPPIYAcKUWvpNyH43vD 9iuN2l8GfB7H79el4Pi63Mw0C+d6nAORSLbaPu0MJXOakr8ZDhujDXh/SPrsKFgKBH YlnijjYTyPRSw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id A6FAF5C0879; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Pingfan Liu , David Woodhouse , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , "Jason A. Donenfeld" , "Paul E . McKenney" Subject: [PATCH rcu 2/8] rcu: Synchronize ->qsmaskinitnext in rcu_boost_kthread_setaffinity() Date: Wed, 19 Oct 2022 15:46:53 -0700 Message-Id: <20221019224659.2499511-2-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Pingfan Liu Once either rcutree_online_cpu() or rcutree_dead_cpu() is invoked concurrently, the following rcu_boost_kthread_setaffinity() race can occur: CPU 1 CPU2 mask = rcu_rnp_online_cpus(rnp); ... mask = rcu_rnp_online_cpus(rnp); ... set_cpus_allowed_ptr(t, cm); set_cpus_allowed_ptr(t, cm); This results in CPU2's update being overwritten by that of CPU1, and thus the possibility of ->boost_kthread_task continuing to run on a to-be-offlined CPU. This commit therefore eliminates this race by relying on the pre-existing acquisition of ->boost_kthread_mutex to serialize the full process of changing the affinity of ->boost_kthread_task. Signed-off-by: Pingfan Liu Cc: David Woodhouse Cc: Frederic Weisbecker Cc: Neeraj Upadhyay Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers Cc: Lai Jiangshan Cc: Joel Fernandes Cc: "Jason A. Donenfeld" Signed-off-by: Paul E. McKenney --- kernel/rcu/tree_plugin.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index e3142ee35fc6a..7b0fe741a0886 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1221,11 +1221,13 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp) * We don't include outgoingcpu in the affinity set, use -1 if there is * no outgoing CPU. If there are no CPUs left in the affinity set, * this function allows the kthread to execute on any CPU. + * + * Any future concurrent calls are serialized via ->boost_kthread_mutex. */ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu) { struct task_struct *t = rnp->boost_kthread_task; - unsigned long mask = rcu_rnp_online_cpus(rnp); + unsigned long mask; cpumask_var_t cm; int cpu; @@ -1234,6 +1236,7 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu) if (!zalloc_cpumask_var(&cm, GFP_KERNEL)) return; mutex_lock(&rnp->boost_kthread_mutex); + mask = rcu_rnp_online_cpus(rnp); for_each_leaf_node_possible_cpu(rnp, cpu) if ((mask & leaf_node_cpu_bit(rnp, cpu)) && cpu != outgoingcpu) From patchwork Wed Oct 19 22:46:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 314BAC433FE for ; Wed, 19 Oct 2022 22:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230398AbiJSWrH (ORCPT ); Wed, 19 Oct 2022 18:47:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230282AbiJSWrE (ORCPT ); Wed, 19 Oct 2022 18:47:04 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91E0317C167; Wed, 19 Oct 2022 15:47:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 514B7B8261A; Wed, 19 Oct 2022 22:47:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EB837C433B5; Wed, 19 Oct 2022 22:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=XdRd/D7YtnjF/VbtVQdPblQxfOL6jWA9yUi4elYsgMo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CATtEuYG11NQ97X+dUT5iqaZitwvMN1GjLxxebL7NPDD4dxFA0yPhlkM/csd1ceU0 5YpYWBZTlDcIDBhwd6d0OdN3MkPVI2tLW93lrNsg8w9aKtopPk/i42xNiXpnSQrikF 9opL8UGvgpS2dC64Wl4SQKHNG9djEdMGQzGo5+nqC2ZAE0qEe7BEwQ9bVoEDpM1P1I zqxyAoxz6PzHlS9NsFx7aMOnhGD5niqY5M1NnljreFhSr0HeZEDtpzzYi5U1llCaEK 2fsg1/4UBIGvOSBX7KbTKIxuiO9jPari1IRNCcENz39AozIrPniJi6mGm6EzuGUpMZ pDSXKJwdKXczw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id A93345C0890; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Zeng Heng , Mukesh Ojha , "Paul E . McKenney" Subject: [PATCH rcu 3/8] rcu: Remove unused 'cpu' in rcu_virt_note_context_switch() Date: Wed, 19 Oct 2022 15:46:54 -0700 Message-Id: <20221019224659.2499511-3-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Zeng Heng This commit removes the unused function argument 'cpu'. This does not change functionality, but might save a cycle or two. Signed-off-by: Zeng Heng Acked-by: Mukesh Ojha Signed-off-by: Paul E. McKenney --- include/linux/kvm_host.h | 2 +- include/linux/rcutiny.h | 2 +- include/linux/rcutree.h | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 32f259fa58013..381b92d146c71 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -416,7 +416,7 @@ static __always_inline void guest_context_enter_irqoff(void) */ if (!context_tracking_guest_enter()) { instrumentation_begin(); - rcu_virt_note_context_switch(smp_processor_id()); + rcu_virt_note_context_switch(); instrumentation_end(); } } diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 768196a5f39d6..9bc025aa79a30 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -142,7 +142,7 @@ static inline int rcu_needs_cpu(void) * Take advantage of the fact that there is only one CPU, which * allows us to ignore virtualization-based context switches. */ -static inline void rcu_virt_note_context_switch(int cpu) { } +static inline void rcu_virt_note_context_switch(void) { } static inline void rcu_cpu_stall_reset(void) { } static inline int rcu_jiffies_till_stall_check(void) { return 21 * HZ; } static inline void rcu_irq_exit_check_preempt(void) { } diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 5efb51486e8af..70795386b9ffa 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -27,7 +27,7 @@ void rcu_cpu_stall_reset(void); * wrapper around rcu_note_context_switch(), which allows TINY_RCU * to save a few bytes. The caller must have disabled interrupts. */ -static inline void rcu_virt_note_context_switch(int cpu) +static inline void rcu_virt_note_context_switch(void) { rcu_note_context_switch(false); } From patchwork Wed Oct 19 22:46:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5740C4332F for ; Wed, 19 Oct 2022 22:47:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230270AbiJSWrD (ORCPT ); Wed, 19 Oct 2022 18:47:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230192AbiJSWrC (ORCPT ); Wed, 19 Oct 2022 18:47:02 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CBEE17C167; Wed, 19 Oct 2022 15:47:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 928ED619E9; Wed, 19 Oct 2022 22:47:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC272C43470; Wed, 19 Oct 2022 22:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=+mIw1jy8wj3gxm+o9ZlbmvXenuiwqx+JElmYZGbXaeg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jZ9Z0O4bxNI1YlzPyEqaNEYwOgorClEeEvpkHvp2pmO6GzXevqCV3HxCsEQqCysOu bmpwnxQkAHY3unTeNRtyIKVQfx5peSYFKdS3cri2lzyajtFdxKd1Sy8izG2XH0tEfT DxQ/ZPj+Q0TRLxJCPp2RHrMk//Q8/2JZTfYObUG4EhM1yL+w0aGD5BMW07/pLaSWu+ ihoL++h+iI9JvfAYPU4ifHt13KidQ6HWdyj0+w9H4FrRSepBvE3fDIUCa9EHYZ5pbj sTOvJlTYpyQeR8xJSK16uUQV3qWt6w3SVDmE3Pz2U3wMDeiYgBraK7p+EGRFse76Ab PHR0j5Jf0qN5A== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id AB23C5C0920; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Joel Fernandes (Google)" , Frederic Weisbecker , "Paul E . McKenney" Subject: [PATCH rcu 4/8] rcu: Use READ_ONCE() for lockless read of rnp->qsmask Date: Wed, 19 Oct 2022 15:46:55 -0700 Message-Id: <20221019224659.2499511-4-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: "Joel Fernandes (Google)" The rnp->qsmask is locklessly accessed from rcutree_dying_cpu(). This may help avoid load tearing due to concurrent access, KCSAN issues, and preserve sanity of people reading the mask in tracing. Reviewed-by: Frederic Weisbecker Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 0ca21ac0f0648..5ec97e3f7468f 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2106,7 +2106,7 @@ int rcutree_dying_cpu(unsigned int cpu) if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) return 0; - blkd = !!(rnp->qsmask & rdp->grpmask); + blkd = !!(READ_ONCE(rnp->qsmask) & rdp->grpmask); trace_rcu_grace_period(rcu_state.name, READ_ONCE(rnp->gp_seq), blkd ? TPS("cpuofl-bgp") : TPS("cpuofl")); return 0; From patchwork Wed Oct 19 22:46:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2852C4332F for ; Wed, 19 Oct 2022 22:47:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230393AbiJSWrJ (ORCPT ); Wed, 19 Oct 2022 18:47:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230299AbiJSWrE (ORCPT ); Wed, 19 Oct 2022 18:47:04 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B463E186D70; Wed, 19 Oct 2022 15:47:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 64496B825E1; Wed, 19 Oct 2022 22:47:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0562DC4347C; Wed, 19 Oct 2022 22:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=U+1luf3mbgyTSKOMl8vtX4dFmH8rl+GKNY2TDjN1IyY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XMSKFGEu/uzR3N52gimUSRvlUY77ksX6bwSP5MEEbrhoYubbKoJVO455NMN69R4xj x9GRwiRrinWyIWvqcLaI+JhV5ZHw0Ow+vjol3spPlLV+EDFsRspfhySJYB1hNYEyDD aSGLKC8qpWBT4nbxwHNQOo7RmG1Z0Y+5PDZjmaBTcnkG3j3YRGb2Ycr+/iXr0GVAM8 f3R8P+eO4JCLRSgp7mHQtfS9keccHmF/hGloP0UzUPKujvCNZbkgV3dyp0o5SzcWdJ GLdFjAAokYxUviKsgseded/9rmE6dJI2yUYwAvyjELLvaU6fXnXMYEQiD9ItgNieST uSvkQeYinB1kw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id AD5D05C0A04; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org Subject: [PATCH rcu 5/8] slab: Explain why SLAB_DESTROY_BY_RCU reference before locking Date: Wed, 19 Oct 2022 15:46:56 -0700 Message-Id: <20221019224659.2499511-5-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org It is not obvious to the casual user why it is absolutely necessary to acquire a reference to a SLAB_DESTROY_BY_RCU structure before acquiring a lock in that structure. Therefore, add a comment explaining this point. Signed-off-by: Paul E. McKenney Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Andrew Morton Cc: Vlastimil Babka Cc: Roman Gushchin Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Acked-by: Vlastimil Babka --- include/linux/slab.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/linux/slab.h b/include/linux/slab.h index 90877fcde70bd..446303e385265 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -76,6 +76,12 @@ * rcu_read_lock before reading the address, then rcu_read_unlock after * taking the spinlock within the structure expected at that address. * + * Note that it is not possible to acquire a lock within a structure + * allocated with SLAB_DESTROY_BY_RCU without first acquiring a reference + * as described above. The reason is that SLAB_DESTROY_BY_RCU pages are + * not zeroed before being given to the slab, which means that any locks + * must be initialized after each and every kmem_struct_alloc(). + * * Note that SLAB_TYPESAFE_BY_RCU was originally named SLAB_DESTROY_BY_RCU. */ /* Defer freeing slabs to RCU */ From patchwork Wed Oct 19 22:46:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E447C43217 for ; Wed, 19 Oct 2022 22:47:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231128AbiJSWrL (ORCPT ); Wed, 19 Oct 2022 18:47:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230371AbiJSWrF (ORCPT ); Wed, 19 Oct 2022 18:47:05 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B056188AAD; Wed, 19 Oct 2022 15:47:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DA989B8260A; Wed, 19 Oct 2022 22:47:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 506FBC4314A; Wed, 19 Oct 2022 22:47:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=fFc3hA0ZPr//nb0oSgszXsNmNDNbjld6HNvW7I07HJ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Oy/1fcvMSiFHp/xKwJ9CqwfRJHoqRkZcrv5jCD4aB/NXbeEvG+CAwUn+1laykl7t5 MMmXgTMbAhBmVfvWRKyYlC5jUxmRPZ6dEJEz0m9ZwTUj4tfcDKHGlv9zWZk+A+WIg0 EqUOn0Qr+yCnEZl+/MBE/objul/fDbsW25LyJ9tkZ7Pq6SRXK/2/7uPjOk/JmrDBtJ mVGDhL9L+jbomtlachTuFWKndhB7zvpLpYgWGxho7hvOURaDR6/s/GPVo+BLtxO4uV B56yRqi+MUbeTK0SrMpyjd1/6lJenX52g0gLYrHty/L61W6B5paQBeYXfdxCgTivgS 7jlhUKv59foUw== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id AF2425C0A40; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Yipeng Zou , Frederic Weisbecker , "Paul E . McKenney" Subject: [PATCH rcu 6/8] rcu: Remove rcu_is_idle_cpu() Date: Wed, 19 Oct 2022 15:46:57 -0700 Message-Id: <20221019224659.2499511-6-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Yipeng Zou The commit 3fcd6a230fa7 ("x86/cpu: Avoid cpuinfo-induced IPIing of idle CPUs") introduced rcu_is_idle_cpu() in order to identify the current CPU idle state. But commit f3eca381bd49 ("x86/aperfmperf: Replace arch_freq_get_on_cpu()") switched to using MAX_SAMPLE_AGE, so rcu_is_idle_cpu() is no longer used. This commit therefore removes it. Fixes: f3eca381bd49 ("x86/aperfmperf: Replace arch_freq_get_on_cpu()") Signed-off-by: Yipeng Zou Reviewed-by: Frederic Weisbecker Signed-off-by: Paul E. McKenney --- include/linux/rcutiny.h | 2 -- include/linux/rcutree.h | 2 -- kernel/rcu/tree.c | 6 ------ 3 files changed, 10 deletions(-) diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 9bc025aa79a30..5c271bf3a1e7e 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -146,8 +146,6 @@ static inline void rcu_virt_note_context_switch(void) { } static inline void rcu_cpu_stall_reset(void) { } static inline int rcu_jiffies_till_stall_check(void) { return 21 * HZ; } static inline void rcu_irq_exit_check_preempt(void) { } -#define rcu_is_idle_cpu(cpu) \ - (is_idle_task(current) && !in_nmi() && !in_hardirq() && !in_serving_softirq()) static inline void exit_rcu(void) { } static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t) { diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 70795386b9ffa..4003bf6cfa1c2 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -87,8 +87,6 @@ bool poll_state_synchronize_rcu_full(struct rcu_gp_oldstate *rgosp); void cond_synchronize_rcu(unsigned long oldstate); void cond_synchronize_rcu_full(struct rcu_gp_oldstate *rgosp); -bool rcu_is_idle_cpu(int cpu); - #ifdef CONFIG_PROVE_RCU void rcu_irq_exit_check_preempt(void); #else diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 5ec97e3f7468f..f6561aa401c04 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -301,12 +301,6 @@ static bool rcu_dynticks_in_eqs(int snap) return !(snap & RCU_DYNTICKS_IDX); } -/* Return true if the specified CPU is currently idle from an RCU viewpoint. */ -bool rcu_is_idle_cpu(int cpu) -{ - return rcu_dynticks_in_eqs(rcu_dynticks_snap(cpu)); -} - /* * Return true if the CPU corresponding to the specified rcu_data * structure has spent some time in an extended quiescent state since From patchwork Wed Oct 19 22:46:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73D1AC433FE for ; Wed, 19 Oct 2022 22:47:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230481AbiJSWrK (ORCPT ); Wed, 19 Oct 2022 18:47:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230368AbiJSWrF (ORCPT ); Wed, 19 Oct 2022 18:47:05 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B1E6188107; Wed, 19 Oct 2022 15:47:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B8D70B8261B; Wed, 19 Oct 2022 22:47:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 505A7C43148; Wed, 19 Oct 2022 22:47:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=IeaAKnXCrTJxEtb/hM2Q5kn1ssJ75eQ9FsOhFMTira4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=s2eRhzIvlTeDSe1B8OI9k7XhSR+HrCnEujCmMR1fA0ltBZ3oeJzvsdkc2x8DaEYo6 vip4yyk3uBScrCJ2kpln3iH9pg39A/8gA4spvtnSP5RsmBB6WdCboK04Lo+doYVmeF sfAVQs7GcyBG7/mXI324vwx0aN7ztx59oUdwFoYECI7und9aTWpjb+OvBu6aLg3jsE kbe+8LCQ6WxrFfgnMEdnfL4tH3furhSuBLAWGjrb50IEm/wOYENRcCYoJkwr+qaFYl 0VWesc45R5xRXk47xsq/OWFN/sqiKo2xtO0cwWgpzeW77txeSnyUIr0DJZexx8qx4z jss+1AKGZ7jvg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id B11CC5C0AC5; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, "Paul E. McKenney" , Alexei Starovoitov Subject: [PATCH rcu 7/8] rcu-tasks: Make grace-period-age message human-readable Date: Wed, 19 Oct 2022 15:46:58 -0700 Message-Id: <20221019224659.2499511-7-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org This commit adds a few words to the informative message that appears every ten seconds in RCU Tasks and RCU Tasks Trace grace periods. This message currently reads as follows: rcu_tasks_wait_gp: rcu_tasks grace period 1046 is 10088 jiffies old. After this change, it provides additional context, instead reading as follows: rcu_tasks_wait_gp: rcu_tasks grace period number 1046 (since boot) is 10088 jiffies old. Reported-by: Alexei Starovoitov Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index f5bf6fb430dab..b0b885e071fa8 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -728,7 +728,7 @@ static void rcu_tasks_wait_gp(struct rcu_tasks *rtp) if (rtsi > 0 && !reported && time_after(j, lastinfo + rtsi)) { lastinfo = j; rtsi = rtsi * rcu_task_stall_info_mult; - pr_info("%s: %s grace period %lu is %lu jiffies old.\n", + pr_info("%s: %s grace period number %lu (since boot) is %lu jiffies old.\n", __func__, rtp->kname, rtp->tasks_gp_seq, j - rtp->gp_start); } } From patchwork Wed Oct 19 22:46:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13012447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C669C4321E for ; Wed, 19 Oct 2022 22:47:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230387AbiJSWrG (ORCPT ); Wed, 19 Oct 2022 18:47:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230234AbiJSWrD (ORCPT ); Wed, 19 Oct 2022 18:47:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8820717FD5C; Wed, 19 Oct 2022 15:47:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 22755619D6; Wed, 19 Oct 2022 22:47:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5060CC43149; Wed, 19 Oct 2022 22:47:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666219621; bh=CDl8H2MGOjWvC458fdEn7dT11YxGntP9ARrgviEeMhY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tuC9VoJUsxYY/4c+0Pc8LrQ1NGvAKs1dMjfH9r+iKyPsUll5AXr+M8mzGLDsSgWsc wNZkFN8/urxwPLqOoviVRke4/rJGfytC7VoI/4ExKL6ikIH8zSDpmSs7VGFH1rf/bT AE9DZl2rrmOoaHRLKi9uSR2a6dK851gFDFHp3a04uSuhov9GeK36VtvC5mCmne1dpU PAn+ouBxcTUFL9Civ4mINIPu8vnvSSs6P5heivUd6ApQ9BEkJmoKY9dvA6huIWeH/+ Ew+TTrGXjp4GD/qunPYACNRTIUTSMSBEAwcN98aCDMgs1y03E41YnNyklrkA2rCEXJ dZAatlp2GWijA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id B2E8E5C0B8F; Wed, 19 Oct 2022 15:47:00 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Zqiang , Joel Fernandes , "Paul E . McKenney" Subject: [PATCH rcu 8/8] rcu: Fix __this_cpu_read() lockdep warning in rcu_force_quiescent_state() Date: Wed, 19 Oct 2022 15:46:59 -0700 Message-Id: <20221019224659.2499511-8-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> References: <20221019224652.GA2499358@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Zqiang Running rcutorture with non-zero fqs_duration module parameter in a kernel built with CONFIG_PREEMPTION=y results in the following splat: BUG: using __this_cpu_read() in preemptible [00000000] code: rcu_torture_fqs/398 caller is __this_cpu_preempt_check+0x13/0x20 CPU: 3 PID: 398 Comm: rcu_torture_fqs Not tainted 6.0.0-rc1-yoctodev-standard+ Call Trace: dump_stack_lvl+0x5b/0x86 dump_stack+0x10/0x16 check_preemption_disabled+0xe5/0xf0 __this_cpu_preempt_check+0x13/0x20 rcu_force_quiescent_state.part.0+0x1c/0x170 rcu_force_quiescent_state+0x1e/0x30 rcu_torture_fqs+0xca/0x160 ? rcu_torture_boost+0x430/0x430 kthread+0x192/0x1d0 ? kthread_complete_and_exit+0x30/0x30 ret_from_fork+0x22/0x30 The problem is that rcu_force_quiescent_state() uses __this_cpu_read() in preemptible code instead of the proper raw_cpu_read(). This commit therefore changes __this_cpu_read() to raw_cpu_read(). Signed-off-by: Zqiang Reviewed-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index f6561aa401c04..1e1d333d07ffe 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2410,7 +2410,7 @@ void rcu_force_quiescent_state(void) struct rcu_node *rnp_old = NULL; /* Funnel through hierarchy to reduce memory contention. */ - rnp = __this_cpu_read(rcu_data.mynode); + rnp = raw_cpu_read(rcu_data.mynode); for (; rnp != NULL; rnp = rnp->parent) { ret = (READ_ONCE(rcu_state.gp_flags) & RCU_GP_FLAG_FQS) || !raw_spin_trylock(&rnp->fqslock);