From patchwork Fri Aug 2 00:34:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13750964 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8BDD10A24; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; cv=none; b=sJHOzR1ODLVVd6vYLtUAAgkcx1k1Cfu/HbnNy0istKKBUPd+jFMpWBLycxJg5yL8xVqrLWrveQ8TiX3ZbqedxV2v89Mzzbqao7ma8NwoQQxHW8NZWlWzgEaAexfCJ0cTw8L9c7Jvp8tPuM7CP1rm1F7HTIxiZj4f8PfdYTphRRY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; c=relaxed/simple; bh=Bs6t1S51lhnIfRf7LZhnT/4oX7/PDkMznq5WhI41PoQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cLz3M11COXkT4Oej3J2E06cmm9nns5i6M8nCk81/OZyaC7/013st3esm0+wQZjkGAYHKiBVGMUhc3y2WbBGMADIBjuOBTSmDpF12Qb3E6Q2zAAXcpGnjvqhYPY5t5OPWm8nN80+WXim7kXRzAfXxQ3cxfjbZcn45i5z9FBVjBIs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dY2TyUYS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dY2TyUYS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35B41C32786; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722558868; bh=Bs6t1S51lhnIfRf7LZhnT/4oX7/PDkMznq5WhI41PoQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dY2TyUYSoX3gzVYa2usg91lrH/mRiL+T9is4/2zCw7rke6ahHg9rjhCCcYPv0n6lE QobqxTAsGzyXhBKXZOXq8lq0EnKOwG9uicJzohd0seNr1tMLwV3zlctKRdD5e/PpNb uBNRY1JT1jO1aOoW789XWI0g0VUisAFhObTXULB9EoUt1A9IcKmyRfHI1VTsSZssP4 uK7FhbTuPI4OkYbqLtoO/WEkcoL1hwf+U6oUa1GCjmuYnXFLqh5jur3nIhpvV9rVXo IriTwfwVvwVpNKqAkeSYTCsXX6AT+d5zohYrtXNz6prMy2ZiNjGfMpLv6qiOzTdEiW Mly/w0bDOfFJA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E01F9CE09F8; Thu, 1 Aug 2024 17:34:27 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 1/6] rcu/tasks: Check processor-ID assumptions Date: Thu, 1 Aug 2024 17:34:21 -0700 Message-Id: <20240802003426.4134196-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> References: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The current mapping of smp_processor_id() to a CPU processing Tasks-RCU callbacks makes some assumptions about layout. This commit therefore adds a WARN_ON() to check these assumptions. Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index b6fcf744af75d..d473846a572aa 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -355,6 +355,7 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, rcu_read_lock(); ideal_cpu = smp_processor_id() >> READ_ONCE(rtp->percpu_enqueue_shift); chosen_cpu = cpumask_next(ideal_cpu - 1, cpu_possible_mask); + WARN_ON_ONCE(chosen_cpu >= nr_cpu_ids); rtpcp = per_cpu_ptr(rtp->rtpcpu, chosen_cpu); if (!raw_spin_trylock_rcu_node(rtpcp)) { // irqs already disabled. raw_spin_lock_rcu_node(rtpcp); // irqs already disabled. From patchwork Fri Aug 2 00:34:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13750960 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60484EAE7; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; cv=none; b=Z59PlDOJ22A00U7lz2uaTgUJMtxkO/oMnIHlN1wN563S523aWB7D6kgbeqGHk7nGdPzxqfY8rBk5qyca89xd1+xSrFkVpg6DTrW95GN6QU2U96IVE38sci+lPCTaZOdWS75sMNEU/SoXxP3u3w/OTAalIznKjNpjqOYvqQ1EZlQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; c=relaxed/simple; bh=WbIoVnZfKmvIceCjC2LaFnl97Erf9A6epdp+H3kywvw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=n2iCfO0JUHRw8n2XX60BblgknQUUP3QwLCC18VEZCnCg5nrhiNCfvclfg32c1LgxbrxtNdps7kaPw9e/eVHJK4sfXAUS0NmMUdzgXOXzeFjxYGQ/jjhzWCMyNQbSMDSMaWN84i5i+rMO6lIblpff3HIf18tZCv6FWFTKS/6oHr4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nVEKpVvv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nVEKpVvv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 40591C4AF0C; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722558868; bh=WbIoVnZfKmvIceCjC2LaFnl97Erf9A6epdp+H3kywvw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nVEKpVvvg1cQbbjIiMOXTZ9+2q8cAN1JFDoSL6ukgX3omie260BGMTRNksBplWz4F WyiPcaURRaQsPMRFJnyXjhHxpz1RMyQU4FpODAK57QC2cyZTiQ8NVTysmx/YR9h8+R iFq+j+YW0pyZAPMZymFOrI1Ss0kM92epB52ONy+t/W/v1PLG0ELMhh8yd9irX2GZK1 RrZK4LsN/IISztHARodvjjKCgeDLjndOI+jPyP025eBsDfhQFMbvwRqxisO+dlAMt5 tvkkCdkcqOu927Vcyqg/voc2FNejqHMW/Jrm24D5SPmbhWtB6lP4DdLM4lkbQ8zegO rEx99LgA/8aLA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E478BCE0A01; Thu, 1 Aug 2024 17:34:27 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 2/6] rcu/tasks: Update rtp->tasks_gp_seq comment Date: Thu, 1 Aug 2024 17:34:22 -0700 Message-Id: <20240802003426.4134196-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> References: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The rtp->tasks_gp_seq grace-period sequence number is not a strict count, but rather the usual RCU sequence number with the lower few bits tracking per-grace-period state and the upper bits the count of grace periods since boot, give or take the initial value. This commit therefore adjusts this comment. Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index d473846a572aa..cf273c554f3b0 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -64,7 +64,7 @@ struct rcu_tasks_percpu { * @init_fract: Initial backoff sleep interval. * @gp_jiffies: Time of last @gp_state transition. * @gp_start: Most recent grace-period start in jiffies. - * @tasks_gp_seq: Number of grace periods completed since boot. + * @tasks_gp_seq: Number of grace periods completed since boot in upper bits. * @n_ipis: Number of IPIs sent to encourage grace periods to end. * @n_ipis_fails: Number of IPI-send failures. * @kthread_ptr: This flavor's grace-period/callback-invocation kthread. From patchwork Fri Aug 2 00:34:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13750963 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8B7DF9EC; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; cv=none; b=bzcUObm2zv3PnOiV6ibToOPdghoa48ca/c+W41CzUY7UlDCPYAh7cCp3dlB9EkifvU7mVNIDcrHdeWfp2FWN8nYmY668mN7CN7/kZJL2bq0dsvXiGeoMAmROGqSC78NeIwoJF4UsUp/Rjvni2B6nx3p64uIPP0lO0q5ZpI7cuLw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; c=relaxed/simple; bh=X9aHmvddPuSA+i4410NgEvfsaBk41rG80LyCQw3wKdg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XayUtcAVu+uj5tUQc9B7UqMGKDFMscTXdwBysljclPZghYlsCT8c3BH1zm1BZ48/EkhruuG4cJVIZbjFYKuU+w8Qk2Cqx/RDfgvWwyhdk7kDTQcORzvnYn1hy65DAPUjZN3BRmaoHsGfHD5muDkkZYfwOAFxGsBvEm2v5lfx+w4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=vMnq8Etj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="vMnq8Etj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 520C1C4AF0D; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722558868; bh=X9aHmvddPuSA+i4410NgEvfsaBk41rG80LyCQw3wKdg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vMnq8Etjy6vct9F/udWnJEKAOwphoo0Ri5n/MWEWd2HYszyCcdf3Xb+ldFzA2J3Dc UAkf8QzVvaz1YInuoCvoJfCk4VQG0b0nfCS/kXO5bmFYXtk/Kx4KpwUWuqdWS+p/1D 64Xl8j9UqFeaNypZPzQ5T7JrPyy5LXj2Sb5pTFxKwCDqKJF+AC3N/FmFiQh6mW3YFZ tq7L4ulKEUy4NvpkBlJWtkwhyyOoIxWVwrK1/L/bwOP4j6Vs76fZjoxZZylTYEgzPj avFslSZyMAlM8k6Y3ZVKpduMAIGYF59zPRmMh4KG5gZQRVGPhanERUdwo9BZQvN01b l17UNRYNMBjmA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id E7438CE0BC3; Thu, 1 Aug 2024 17:34:27 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 3/6] rcu: Provide rcu_barrier_cb_is_done() to check rcu_barrier() CBs Date: Thu, 1 Aug 2024 17:34:23 -0700 Message-Id: <20240802003426.4134196-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> References: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit provides a rcu_barrier_cb_is_done() function that returns true if the *rcu_barrier*() callback passed in is done. This will be used when printing grace-period debugging information. Signed-off-by: Paul E. McKenney --- kernel/rcu/rcu.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index f035fd4e579b4..feb3ac1dc5d59 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -252,6 +252,11 @@ static inline void debug_rcu_head_callback(struct rcu_head *rhp) kmem_dump_obj(rhp); } +static inline bool rcu_barrier_cb_is_done(struct rcu_head *rhp) +{ + return rhp->next == rhp; +} + extern int rcu_cpu_stall_suppress_at_boot; static inline bool rcu_stall_is_suppressed_at_boot(void) From patchwork Fri Aug 2 00:34:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13750962 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8BB5FC11; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; cv=none; b=k9XiCysAbMTyvY1kqTZPwFwfurqmE+a20whpaW2EZi/hxGob3N3eZmUHckXTRbGPrmUXW4GPGLIhkVYon/+GS3gc1ZkllOnkNYfgwXLJ4Pygb6bZgFN9kvC3rh64JNOoU45+fFZw5dTFde8cFqJN5Z+AsSuS6Ku5Xt9D0+xMkcA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; c=relaxed/simple; bh=WJLXF1GgJdCg9l8dn7wE1jff0VI3lwIKjJqHLcAH/5w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pXPdzBqhjAJim+e7coIjSPcjyiSNAScqlLq85h16mq0Km8krczPZBFfl0LhMyxE7NvulvKKNq3QWCXU+3FRIzdXUW33xvJJ7Y1LeXtmoNLA5rbqMf4RTvUbjrLUK0RmIEp2/Y+eWdTsD0J2sTwr4wjb/M0OXMiN1AQAimJSpfy4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CMPh5fDZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CMPh5fDZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5BCE5C4AF11; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722558868; bh=WJLXF1GgJdCg9l8dn7wE1jff0VI3lwIKjJqHLcAH/5w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CMPh5fDZMdEgSp4Yus7Z0MxD/aGB9QEPISJmjUPfxtrODPdU/8XfLA3udQGtqqxQQ 1MW6X8JsEd1aL83lHvuhXOw4bANrQEz5bkICNUgejronVf33Np2bIMMU3ZXQJ8cjaj ebFlT4311PBCeOqpJuSj19epAvb+h8eiqGzuWtR/sfD0LnL+MxG2rDMpJLirE0QBJl Xz4L7dY8wNWz9y9oDeS/g52qyvOMF0LKztYA7bD+V8jYJkj9Vgk9KV4+HuXTd/qWzb W0KQXtowjPspa8TsWs3qi2/G/7KrgoYz8v0VRlAoV7GyPgQw8FlmVBBJ987OqijFXw 3mkVXJu84naBg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id EA26FCE0E0B; Thu, 1 Aug 2024 17:34:27 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 4/6] rcu/tasks: Mark callbacks not currently participating in barrier operation Date: Thu, 1 Aug 2024 17:34:24 -0700 Message-Id: <20240802003426.4134196-4-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> References: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Each Tasks RCU flavor keeps a count of the number of callbacks that the current rcu_barrier_tasks*() is waiting on, but there is currently no easy way to work out which callback is stuck. One way to do this is to mark idle RCU-barrier callbacks by making the ->next pointer point to the callback itself, and this commit does just that. Later commits will use this for debug output. Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index cf273c554f3b0..6f1e82e548bbc 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -281,6 +281,7 @@ static void cblist_init_generic(struct rcu_tasks *rtp) INIT_LIST_HEAD(&rtpcp->rtp_blkd_tasks); if (!rtpcp->rtp_exit_list.next) INIT_LIST_HEAD(&rtpcp->rtp_exit_list); + rtpcp->barrier_q_head.next = &rtpcp->barrier_q_head; maxcpu = cpu; } @@ -405,6 +406,7 @@ static void rcu_barrier_tasks_generic_cb(struct rcu_head *rhp) struct rcu_tasks *rtp; struct rcu_tasks_percpu *rtpcp; + rhp->next = rhp; // Mark the callback as having been invoked. rtpcp = container_of(rhp, struct rcu_tasks_percpu, barrier_q_head); rtp = rtpcp->rtpp; if (atomic_dec_and_test(&rtp->barrier_q_count)) From patchwork Fri Aug 2 00:34:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13750961 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 804E8EAE9; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; cv=none; b=dkXQgm20L/gqSXaboZavZerl6H4Xrjk/eyYedM4tMI5GwRzQykrSY0EdbMc5EJ6ChpdLA/emYV24K4EQt2XQXA6GhrZZPI7XWJX+2Y27RWwClY8o+02e9rX3pM9L6EWWBpDRyskv4SJqJ1Giy4mXDGroENDlUxOQMm84cwzkBwQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558868; c=relaxed/simple; bh=S/VfjVcS8Be7DWVforJ3o2DzecpeB4W6woz2WULZdnY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=daaJh66q2TkTKlgZCRX/mbbh9+GhtTxs3kfv6+fWtrKqNoBc11g5yXUQaBv+lXI0kTfy2X6e3S1B8L/PXnAtJWDWkScf473ePbTpGDPA/JWVBf5cl3W8Ne7EGILwMw95mgSkp6vxB3kgOaaKnlwVLJWjTldlJsgqPhjuDM/8fGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PiZjm0+N; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PiZjm0+N" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B17EC4AF0F; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722558868; bh=S/VfjVcS8Be7DWVforJ3o2DzecpeB4W6woz2WULZdnY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PiZjm0+Nn936+7RteanWAYOc+FAWLt5WdjGPvXFL23bqd8ilawnnad6hCMAVUL+oZ UEbLTRXIf/BbdM+/Uhf2z+Oo0aGb50mcn+2Ub6DiD5mu3Buq0tV5iZq4HAXRd1bH2Q 5rlM7ycyTC4TS7XEL+1JSa8s45Nr4BqXGVgxemcf87U/9Oz0+Ojf289aZ5rsqZiHJv Y/w0SKskEbtIeD+8rDSoIo2DD+cysxPoxd3gIH/IyfvTIbrGGL8IBAyxxeR2YoenjP PGp+ewpatxofqipKySdwGrLG1vidB2VczT8k7EQuT0lKnxeI99Nx6o2ixocnLKzJnN ds+aZ6kHmHOxg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id ECB4BCE0F69; Thu, 1 Aug 2024 17:34:27 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 5/6] rcu/tasks: Add detailed grace-period and barrier diagnostics Date: Thu, 1 Aug 2024 17:34:25 -0700 Message-Id: <20240802003426.4134196-5-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> References: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit adds rcu_tasks_torture_stats_print(), rcu_tasks_trace_torture_stats_print(), and rcu_tasks_rude_torture_stats_print() functions that provide detailed diagnostics on grace-period, callback, and barrier state. Signed-off-by: Paul E. McKenney --- include/linux/rcupdate.h | 3 ++ kernel/rcu/tasks.h | 63 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index e40a726fe1a03..58d84c59f3dda 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -174,6 +174,7 @@ static inline void rcu_nocb_flush_deferred_wakeup(void) { } } while (0) void call_rcu_tasks(struct rcu_head *head, rcu_callback_t func); void synchronize_rcu_tasks(void); +void rcu_tasks_torture_stats_print(char *tt, char *tf); # else # define rcu_tasks_classic_qs(t, preempt) do { } while (0) # define call_rcu_tasks call_rcu @@ -200,6 +201,7 @@ void rcu_tasks_trace_qs_blkd(struct task_struct *t); rcu_tasks_trace_qs_blkd(t); \ } \ } while (0) +void rcu_tasks_trace_torture_stats_print(char *tt, char *tf); # else # define rcu_tasks_trace_qs(t) do { } while (0) # endif @@ -212,6 +214,7 @@ do { \ # ifdef CONFIG_TASKS_RUDE_RCU void synchronize_rcu_tasks_rude(void); +void rcu_tasks_rude_torture_stats_print(char *tt, char *tf); # endif #define rcu_note_voluntary_context_switch(t) rcu_tasks_qs(t, false) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 6f1e82e548bbc..37597f7c581ca 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -712,9 +712,7 @@ static void __init rcu_tasks_bootup_oddness(void) #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */ } -#endif /* #ifndef CONFIG_TINY_RCU */ -#ifndef CONFIG_TINY_RCU /* Dump out rcutorture-relevant state common to all RCU-tasks flavors. */ static void show_rcu_tasks_generic_gp_kthread(struct rcu_tasks *rtp, char *s) { @@ -748,6 +746,52 @@ static void show_rcu_tasks_generic_gp_kthread(struct rcu_tasks *rtp, char *s) rtp->lazy_jiffies, s); } + +/* Dump out more rcutorture-relevant state common to all RCU-tasks flavors. */ +static void rcu_tasks_torture_stats_print_generic(struct rcu_tasks *rtp, char *tt, + char *tf, char *tst) +{ + cpumask_var_t cm; + int cpu; + bool gotcb = false; + unsigned long j = jiffies; + + pr_alert("%s%s Tasks%s RCU g%ld gp_start %lu gp_jiffies %lu gp_state %d (%s).\n", + tt, tf, tst, data_race(rtp->tasks_gp_seq), + j - data_race(rtp->gp_start), j - data_race(rtp->gp_jiffies), + data_race(rtp->gp_state), tasks_gp_state_getname(rtp)); + pr_alert("\tEnqueue shift %d limit %d Dequeue limit %d gpseq %lu.\n", + data_race(rtp->percpu_enqueue_shift), + data_race(rtp->percpu_enqueue_lim), + data_race(rtp->percpu_dequeue_lim), + data_race(rtp->percpu_dequeue_gpseq)); + (void)zalloc_cpumask_var(&cm, GFP_KERNEL); + pr_alert("\tCallback counts:"); + for_each_possible_cpu(cpu) { + long n; + struct rcu_tasks_percpu *rtpcp = per_cpu_ptr(rtp->rtpcpu, cpu); + + if (cpumask_available(cm) && !rcu_barrier_cb_is_done(&rtpcp->barrier_q_head)) + cpumask_set_cpu(cpu, cm); + n = rcu_segcblist_n_cbs(&rtpcp->cblist); + if (!n) + continue; + pr_cont(" %d:%ld", cpu, n); + gotcb = true; + } + if (gotcb) + pr_cont(".\n"); + else + pr_cont(" (none).\n"); + pr_alert("\tBarrier seq %lu count %d holdout CPUs ", + data_race(rtp->barrier_q_seq), atomic_read(&rtp->barrier_q_count)); + if (cpumask_available(cm) && !cpumask_empty(cm)) + pr_cont(" %*pbl.\n", cpumask_pr_args(cm)); + else + pr_cont("(none).\n"); + free_cpumask_var(cm); +} + #endif // #ifndef CONFIG_TINY_RCU static void exit_tasks_rcu_finish_trace(struct task_struct *t); @@ -1199,6 +1243,11 @@ void show_rcu_tasks_classic_gp_kthread(void) show_rcu_tasks_generic_gp_kthread(&rcu_tasks, ""); } EXPORT_SYMBOL_GPL(show_rcu_tasks_classic_gp_kthread); +void rcu_tasks_torture_stats_print(char *tt, char *tf) +{ + rcu_tasks_torture_stats_print_generic(&rcu_tasks, tt, tf, ""); +} +EXPORT_SYMBOL_GPL(rcu_tasks_torture_stats_print); #endif // !defined(CONFIG_TINY_RCU) struct task_struct *get_rcu_tasks_gp_kthread(void) @@ -1360,6 +1409,11 @@ void show_rcu_tasks_rude_gp_kthread(void) show_rcu_tasks_generic_gp_kthread(&rcu_tasks_rude, ""); } EXPORT_SYMBOL_GPL(show_rcu_tasks_rude_gp_kthread); +void rcu_tasks_rude_torture_stats_print(char *tt, char *tf) +{ + rcu_tasks_torture_stats_print_generic(&rcu_tasks_rude, tt, tf, ""); +} +EXPORT_SYMBOL_GPL(rcu_tasks_rude_torture_stats_print); #endif // !defined(CONFIG_TINY_RCU) struct task_struct *get_rcu_tasks_rude_gp_kthread(void) @@ -2037,6 +2091,11 @@ void show_rcu_tasks_trace_gp_kthread(void) show_rcu_tasks_generic_gp_kthread(&rcu_tasks_trace, buf); } EXPORT_SYMBOL_GPL(show_rcu_tasks_trace_gp_kthread); +void rcu_tasks_trace_torture_stats_print(char *tt, char *tf) +{ + rcu_tasks_torture_stats_print_generic(&rcu_tasks_trace, tt, tf, ""); +} +EXPORT_SYMBOL_GPL(rcu_tasks_trace_torture_stats_print); #endif // !defined(CONFIG_TINY_RCU) struct task_struct *get_rcu_tasks_trace_gp_kthread(void) From patchwork Fri Aug 2 00:34:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13750965 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CF91168DA; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558869; cv=none; b=g5tTesnLxmNwuZ2ZKwmtlYHQutUKCRp3ysJozvP97M+dYhaCr+vXRRpl7ReAhtMbshQ6pK7anD7cQOSW9WktfETquETYpdVlhhjGS4o7VUvn1ev3hX0BWievQZDUKHINzNXMjPmscUT5lj4huM15gmsPQFzOXi7nzfz1PWLPRGk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722558869; c=relaxed/simple; bh=9C6fIiB/R+CnzPnUcGPw5yPIwJNQLck0Q+T4uibP3zg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FR4NNSiRbGFMtnlUEUyicKH7RVslS59NvUD01QMP5MdFgAmhl4Ax9/bzkqBXrKA7swwr5mw6vP5c8y9PT8aEsFCM+iNdRC7aC9xOYP5h5EXy0okgV++vjuJUAkX9mDbBFMyO5Fih6VhKWRCMRFmm6lNpiH98vuYAzDS3QkoYt60= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nJeZdika; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nJeZdika" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9903AC4AF15; Fri, 2 Aug 2024 00:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722558868; bh=9C6fIiB/R+CnzPnUcGPw5yPIwJNQLck0Q+T4uibP3zg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nJeZdikadeB8zMehwiJaMByVnqDWfOB3bNNwugWjvE2xxBNfwVH1DVEsGCcBSeClk 3+2gy5m6qBgoO/uvVtqXx7wDnibQISE9gYfIoA6FLMV9rxYqMAomKlY1BPr00OHLG1 PpOk2SILhUlN//PCThh5Mm25O+9YaWmgYa6CE/iz8yEQwcAJzl6Y5cFj/6DN0hnzuT CQBUbDw6Z8Hd5SCoTEawSavdT6BU9I5xCrVdV1eRTNs4gbC6gdKzNuolO0JUTeuKit bJkemCSDdjGh6wkaDprZXWVS7r6ZgDBxk3tKJ6T3a9LvaD3ikZlNAwSZJCrXzYOvs6 q3WQntdSCc8Tg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id EF11CCE0FA1; Thu, 1 Aug 2024 17:34:27 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 6/6] rcu/tasks: Add rcu_barrier_tasks*() start time to diagnostics Date: Thu, 1 Aug 2024 17:34:26 -0700 Message-Id: <20240802003426.4134196-6-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> References: <41c68c6b-2c55-4c2d-ab70-f2a5b38eb374@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit adds the start time, in jiffies, of the most recently started rcu_barrier_tasks*() operation to the diagnostic output used by rcuscale. This information can be helpful in distinguishing a hung barrier operation from a long series of barrier operations. Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 37597f7c581ca..2f8d6c8e3c4ce 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -85,6 +85,7 @@ struct rcu_tasks_percpu { * @barrier_q_count: Number of queues being waited on. * @barrier_q_completion: Barrier wait/wakeup mechanism. * @barrier_q_seq: Sequence number for barrier operations. + * @barrier_q_start: Most recent barrier start in jiffies. * @name: This flavor's textual name. * @kname: This flavor's kthread name. */ @@ -120,6 +121,7 @@ struct rcu_tasks { atomic_t barrier_q_count; struct completion barrier_q_completion; unsigned long barrier_q_seq; + unsigned long barrier_q_start; char *name; char *kname; }; @@ -428,6 +430,7 @@ static void __maybe_unused rcu_barrier_tasks_generic(struct rcu_tasks *rtp) mutex_unlock(&rtp->barrier_q_mutex); return; } + rtp->barrier_q_start = jiffies; rcu_seq_start(&rtp->barrier_q_seq); init_completion(&rtp->barrier_q_completion); atomic_set(&rtp->barrier_q_count, 2); @@ -783,8 +786,9 @@ static void rcu_tasks_torture_stats_print_generic(struct rcu_tasks *rtp, char *t pr_cont(".\n"); else pr_cont(" (none).\n"); - pr_alert("\tBarrier seq %lu count %d holdout CPUs ", - data_race(rtp->barrier_q_seq), atomic_read(&rtp->barrier_q_count)); + pr_alert("\tBarrier seq %lu start %lu count %d holdout CPUs ", + data_race(rtp->barrier_q_seq), j - data_race(rtp->barrier_q_start), + atomic_read(&rtp->barrier_q_count)); if (cpumask_available(cm) && !cpumask_empty(cm)) pr_cont(" %*pbl.\n", cpumask_pr_args(cm)); else