From patchwork Fri Aug 16 06:49:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765512 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE3A980BFF; Fri, 16 Aug 2024 06:50:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791028; cv=none; b=ge+liJvu9nqgn6YzuDLX0srK6yiwB4+WtOBoe8ox2Fim/iyqGm6O9GMFiH7QoZal4VfSZbe6YR7izfvP34e0R++VjvzJqXaCAiFCDTrIc4xMXSgBjnrkpu+7l9DZQ1v9i7wKPiRtEv1F2Gujf1kGVUbzsjrYF/EVQT2QBq7mqXY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791028; c=relaxed/simple; bh=yM39PubOSo+Kk7oYwByRSf3L9oA2uYH1PGJ91ZPWzFo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VF7DJKdpCzozPzDtK3nSlmhgMMtTT/sRShLeTMogWt5zm93Fjm5QbX3w8nJ0KD6eMcP7Ufrd76oBiy43W5slENeyYOT7k1xIbJS/UCJFAH90cR8n1o43Zh5C43fQlV3DwA0h+ku/nxrfepS1ymO1GZXQ78RDXcYg9C2DiFbYyc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=d86oQ4jU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="d86oQ4jU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6DE54C4AF10; Fri, 16 Aug 2024 06:50:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791028; bh=yM39PubOSo+Kk7oYwByRSf3L9oA2uYH1PGJ91ZPWzFo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d86oQ4jU4nQqMrpRibi267xjgrPxr5ufty8EfuUXe68kDzrjMFIi3g6f5K6r2OcNd H8rI8qjJZGNOQVEslSsZ+I2UQmxm1/nTGpsNhownVaQIp8bTFuBtdxfzKAkWiIXUX4 6SskE8wi0V+8/hqhXlAfzCaWH19dT5SnrjTr76ttZeTvo5EMzr8C4AQntsxiLiclfP PQjyQ3uV3RdL/k3hu98XxWgpwxAvG0kjd73GeBTVah/FQ79yJG1SCzHAy88ME5a9t6 cG5ZIC+zqxGdsd4cXbGrjvZkdu91/ujojJMK0HmuCo1Jf1mN0FiG04vb4pdbi5jXdY LHNiqYDDt5jdQ== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 01/12] tools/rcu: Remove RCU Tasks Rude asynchronous APIs from rcu-updaters.sh Date: Fri, 16 Aug 2024 12:19:10 +0530 Message-Id: <20240816064921.57645-1-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The call_rcu_tasks_rude() and rcu_barrier_tasks_rude() APIs are no longer. This commit therefore removes them from the rcu-updaters.sh script. Signed-off-by: Paul E. McKenney Signed-off-by: Neeraj Upadhyay --- tools/rcu/rcu-updaters.sh | 2 -- 1 file changed, 2 deletions(-) diff --git a/tools/rcu/rcu-updaters.sh b/tools/rcu/rcu-updaters.sh index 4ef1397927bb..8a5df3f22550 100755 --- a/tools/rcu/rcu-updaters.sh +++ b/tools/rcu/rcu-updaters.sh @@ -21,12 +21,10 @@ fi bpftrace -e 'kprobe:kvfree_call_rcu, kprobe:call_rcu, kprobe:call_rcu_tasks, - kprobe:call_rcu_tasks_rude, kprobe:call_rcu_tasks_trace, kprobe:call_srcu, kprobe:rcu_barrier, kprobe:rcu_barrier_tasks, - kprobe:rcu_barrier_tasks_rude, kprobe:rcu_barrier_tasks_trace, kprobe:srcu_barrier, kprobe:synchronize_rcu, From patchwork Fri Aug 16 06:49:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765513 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 138108289E; Fri, 16 Aug 2024 06:50:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791042; cv=none; b=ncTeDfpIhAOhPoJPR8TQHz6CpYzI1aua0MXQqDKzzya425VFExidkXutQnVtpOk9JRxGyeOKjxLG8abnlF/3H/YD+f8UxnAzKhAvAXfloGIFBxDlnihKY2YV38Yuh4SQJpLO7oryFtbXeF8PZwChm2LUQVA0z/wCh1zaMSYHsOI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791042; c=relaxed/simple; bh=+XHlKyHnlRafoh2XYAWbtt0V7ZsBf8HadwtpSouqSuY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Y/Z7EGI67WMkEPawE0gG5dmobpueQRg2Eeg0AuYbcrDd1lMmaFyA6oTmDVw3zzgzYAyZDPZX10C89sZMiEtgi8rrYQeFklFGQUe1aa2vvgwZrsAs2p+BdMeju+JjWDwEAzY9achVZuB+xFfRUYH4fbcUzFNfvMcBlS3DS75pxWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FdRTBEDt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FdRTBEDt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2BC6C4AF09; Fri, 16 Aug 2024 06:50:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791041; bh=+XHlKyHnlRafoh2XYAWbtt0V7ZsBf8HadwtpSouqSuY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FdRTBEDtYPLLb5+i/GLGA0IaqjXFLjV5LQ9v2isJNiZpvRbS9WFMuGKcG9RMUlTYV Uq4wiFPW3Pxyjj2NC/HvjbzKhFc4wJtiNpkDH2SZLq9KKY5KZqPJlfbEcFGbCXauX7 9H2/0cSRfkTfy0Owz4S1DECNiI3Q0okvgfYZUalXx12hH6uQNlezRNacJ2MU652iDE /M+h3cwu/29ltgYEaNHlkqB2OCOyPqYpNGYH83hPBwAFkuSgsuTVMizMC8gPg7LJzZ ZLI9I0brLX1rHdhzgy6OFPXB/bLcdx0lRvrjclv3ppbcXpCwDjBi9M7AJ01Pjoq8jn BDcnJCf1n6SQg== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 02/12] doc: Remove RCU Tasks Rude asynchronous APIs Date: Fri, 16 Aug 2024 12:19:11 +0530 Message-Id: <20240816064921.57645-2-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The call_rcu_tasks_rude() and rcu_barrier_tasks_rude() APIs are no longer. This commit therefore removes them from the documentation. Signed-off-by: Paul E. McKenney Signed-off-by: Neeraj Upadhyay --- .../RCU/Design/Requirements/Requirements.rst | 3 +- Documentation/RCU/checklist.rst | 61 +++++++++---------- Documentation/RCU/whatisRCU.rst | 2 +- .../admin-guide/kernel-parameters.txt | 8 --- 4 files changed, 30 insertions(+), 44 deletions(-) diff --git a/Documentation/RCU/Design/Requirements/Requirements.rst b/Documentation/RCU/Design/Requirements/Requirements.rst index f511476b4550..6125e7068d2c 100644 --- a/Documentation/RCU/Design/Requirements/Requirements.rst +++ b/Documentation/RCU/Design/Requirements/Requirements.rst @@ -2649,8 +2649,7 @@ those that are idle from RCU's perspective) and then Tasks Rude RCU can be removed from the kernel. The tasks-rude-RCU API is also reader-marking-free and thus quite compact, -consisting of call_rcu_tasks_rude(), synchronize_rcu_tasks_rude(), -and rcu_barrier_tasks_rude(). +consisting solely of synchronize_rcu_tasks_rude(). Tasks Trace RCU ~~~~~~~~~~~~~~~ diff --git a/Documentation/RCU/checklist.rst b/Documentation/RCU/checklist.rst index 3e6407de231c..7de3e308f330 100644 --- a/Documentation/RCU/checklist.rst +++ b/Documentation/RCU/checklist.rst @@ -194,14 +194,13 @@ over a rather long period of time, but improvements are always welcome! when publicizing a pointer to a structure that can be traversed by an RCU read-side critical section. -5. If any of call_rcu(), call_srcu(), call_rcu_tasks(), - call_rcu_tasks_rude(), or call_rcu_tasks_trace() is used, - the callback function may be invoked from softirq context, - and in any case with bottom halves disabled. In particular, - this callback function cannot block. If you need the callback - to block, run that code in a workqueue handler scheduled from - the callback. The queue_rcu_work() function does this for you - in the case of call_rcu(). +5. If any of call_rcu(), call_srcu(), call_rcu_tasks(), or + call_rcu_tasks_trace() is used, the callback function may be + invoked from softirq context, and in any case with bottom halves + disabled. In particular, this callback function cannot block. + If you need the callback to block, run that code in a workqueue + handler scheduled from the callback. The queue_rcu_work() + function does this for you in the case of call_rcu(). 6. Since synchronize_rcu() can block, it cannot be called from any sort of irq context. The same rule applies @@ -254,10 +253,10 @@ over a rather long period of time, but improvements are always welcome! corresponding readers must use rcu_read_lock_trace() and rcu_read_unlock_trace(). - c. If an updater uses call_rcu_tasks_rude() or - synchronize_rcu_tasks_rude(), then the corresponding - readers must use anything that disables preemption, - for example, preempt_disable() and preempt_enable(). + c. If an updater uses synchronize_rcu_tasks_rude(), + then the corresponding readers must use anything that + disables preemption, for example, preempt_disable() + and preempt_enable(). Mixing things up will result in confusion and broken kernels, and has even resulted in an exploitable security issue. Therefore, @@ -326,11 +325,9 @@ over a rather long period of time, but improvements are always welcome! d. Periodically invoke rcu_barrier(), permitting a limited number of updates per grace period. - The same cautions apply to call_srcu(), call_rcu_tasks(), - call_rcu_tasks_rude(), and call_rcu_tasks_trace(). This is - why there is an srcu_barrier(), rcu_barrier_tasks(), - rcu_barrier_tasks_rude(), and rcu_barrier_tasks_rude(), - respectively. + The same cautions apply to call_srcu(), call_rcu_tasks(), and + call_rcu_tasks_trace(). This is why there is an srcu_barrier(), + rcu_barrier_tasks(), and rcu_barrier_tasks_trace(), respectively. Note that although these primitives do take action to avoid memory exhaustion when any given CPU has too many callbacks, @@ -383,17 +380,17 @@ over a rather long period of time, but improvements are always welcome! must use whatever locking or other synchronization is required to safely access and/or modify that data structure. - Do not assume that RCU callbacks will be executed on - the same CPU that executed the corresponding call_rcu(), - call_srcu(), call_rcu_tasks(), call_rcu_tasks_rude(), or - call_rcu_tasks_trace(). For example, if a given CPU goes offline - while having an RCU callback pending, then that RCU callback - will execute on some surviving CPU. (If this was not the case, - a self-spawning RCU callback would prevent the victim CPU from - ever going offline.) Furthermore, CPUs designated by rcu_nocbs= - might well *always* have their RCU callbacks executed on some - other CPUs, in fact, for some real-time workloads, this is the - whole point of using the rcu_nocbs= kernel boot parameter. + Do not assume that RCU callbacks will be executed on the same + CPU that executed the corresponding call_rcu(), call_srcu(), + call_rcu_tasks(), or call_rcu_tasks_trace(). For example, if + a given CPU goes offline while having an RCU callback pending, + then that RCU callback will execute on some surviving CPU. + (If this was not the case, a self-spawning RCU callback would + prevent the victim CPU from ever going offline.) Furthermore, + CPUs designated by rcu_nocbs= might well *always* have their + RCU callbacks executed on some other CPUs, in fact, for some + real-time workloads, this is the whole point of using the + rcu_nocbs= kernel boot parameter. In addition, do not assume that callbacks queued in a given order will be invoked in that order, even if they all are queued on the @@ -507,9 +504,9 @@ over a rather long period of time, but improvements are always welcome! These debugging aids can help you find problems that are otherwise extremely difficult to spot. -17. If you pass a callback function defined within a module to one of - call_rcu(), call_srcu(), call_rcu_tasks(), call_rcu_tasks_rude(), - or call_rcu_tasks_trace(), then it is necessary to wait for all +17. If you pass a callback function defined within a module + to one of call_rcu(), call_srcu(), call_rcu_tasks(), or + call_rcu_tasks_trace(), then it is necessary to wait for all pending callbacks to be invoked before unloading that module. Note that it is absolutely *not* sufficient to wait for a grace period! For example, synchronize_rcu() implementation is *not* @@ -522,7 +519,6 @@ over a rather long period of time, but improvements are always welcome! - call_rcu() -> rcu_barrier() - call_srcu() -> srcu_barrier() - call_rcu_tasks() -> rcu_barrier_tasks() - - call_rcu_tasks_rude() -> rcu_barrier_tasks_rude() - call_rcu_tasks_trace() -> rcu_barrier_tasks_trace() However, these barrier functions are absolutely *not* guaranteed @@ -539,7 +535,6 @@ over a rather long period of time, but improvements are always welcome! - Either synchronize_srcu() or synchronize_srcu_expedited(), together with and srcu_barrier() - synchronize_rcu_tasks() and rcu_barrier_tasks() - - synchronize_tasks_rude() and rcu_barrier_tasks_rude() - synchronize_tasks_trace() and rcu_barrier_tasks_trace() If necessary, you can use something like workqueues to execute diff --git a/Documentation/RCU/whatisRCU.rst b/Documentation/RCU/whatisRCU.rst index d585a5490aee..1ef5784c1b84 100644 --- a/Documentation/RCU/whatisRCU.rst +++ b/Documentation/RCU/whatisRCU.rst @@ -1103,7 +1103,7 @@ RCU-Tasks-Rude:: Critical sections Grace period Barrier - N/A call_rcu_tasks_rude rcu_barrier_tasks_rude + N/A N/A synchronize_rcu_tasks_rude diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index f1384c7b59c9..a6107b745076 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5572,14 +5572,6 @@ of zero will disable batching. Batching is always disabled for synchronize_rcu_tasks(). - rcupdate.rcu_tasks_rude_lazy_ms= [KNL] - Set timeout in milliseconds RCU Tasks - Rude asynchronous callback batching for - call_rcu_tasks_rude(). A negative value - will take the default. A value of zero will - disable batching. Batching is always disabled - for synchronize_rcu_tasks_rude(). - rcupdate.rcu_tasks_trace_lazy_ms= [KNL] Set timeout in milliseconds RCU Tasks Trace asynchronous callback batching for From patchwork Fri Aug 16 06:49:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765514 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E15CA12CDBA; Fri, 16 Aug 2024 06:50:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791048; cv=none; b=cLzzA6n7u7dfAatONp00CW/yd/iGarNydDgVtN8Jpaa+UgmURV6BrZVA+H+6ha0tD3SuYz6QoNLVs732NcnKJ6gGFuOCqB/1hWYpA72qDGF/ozpUOZh0xerYS+8avrF6nzrpVJZYnIIm9tAynLU8dZ9jh1e2GgaY/8BKOK+hqYE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791048; c=relaxed/simple; bh=UeCiYHs1vez36Tx9MRfvIOwMz1Q1YjwQeJtSjmbREkM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ydkbkqu85oLCcAyaSjrBKpEO+7Vf9Spbdv8TSlS5rRFo6uddimi5+YoHi4qehPWGhF1Z59fS5VaJ5nFNCiQxQZ/UAXqu1KYouujhaf5q3rkuJGLacqwVRCwuAXMoP3fIAEfzvb3PrrGwsyPnMBNGTvA1i2XAZxBfPdTkX2J5UoM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JMEGnr9/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JMEGnr9/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F95DC4AF0F; Fri, 16 Aug 2024 06:50:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791047; bh=UeCiYHs1vez36Tx9MRfvIOwMz1Q1YjwQeJtSjmbREkM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JMEGnr9/7eypT7eVq6nFKxLLMekYYnuKxYo6gxbAUW1YOzqcuTgoUz4QC+pWzDD42 Rqj7r/XDIn6rLR+6rxAnbm+6EZxqzfS0gNg8F5vRh8kMMMCAheAf3LMYr0bOwkjlDc WZSMx74Y+1PdIqIQYuHkIHkwJCVt+gnd++UBxhTbGjJCF7Eaub/TnT6Vj/1tSkTA5b ltQ8Fx7h7igek8cxPEI9S+OaIju5jmRMcUgAFdQ87SAEbMJRYQ4L6v1yofPuy/D+U/ +3p2bx/wY0NR5vlSR2MzjaqtRBpSEqvR8xkcGpGtuau/T9xxaWTwZLaGSiZvBsy1WH IEkbOaylaPw7Q== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org, Peter Zijlstra Subject: [PATCH rcu 03/12] rcutorture: Stop testing RCU Tasks Rude asynchronous APIs Date: Fri, 16 Aug 2024 12:19:12 +0530 Message-Id: <20240816064921.57645-3-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The call_rcu_tasks_rude() and rcu_barrier_tasks_rude() APIs are currently unused. Furthermore, the idea is to get rid of RCU Tasks Rude entirely once all architectures have their deep-idle and entry/exit code correctly marked as inline or noinstr. As a first step towards this goal, this commit therefore removes these two functions from rcutorture testing. Signed-off-by: Paul E. McKenney Cc: Peter Zijlstra Signed-off-by: Neeraj Upadhyay --- kernel/rcu/rcutorture.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c index 08bf7c669dd3..cd8752b2a960 100644 --- a/kernel/rcu/rcutorture.c +++ b/kernel/rcu/rcutorture.c @@ -915,11 +915,6 @@ static struct rcu_torture_ops tasks_ops = { * Definitions for rude RCU-tasks torture testing. */ -static void rcu_tasks_rude_torture_deferred_free(struct rcu_torture *p) -{ - call_rcu_tasks_rude(&p->rtort_rcu, rcu_torture_cb); -} - static struct rcu_torture_ops tasks_rude_ops = { .ttype = RCU_TASKS_RUDE_FLAVOR, .init = rcu_sync_torture_init, @@ -927,11 +922,8 @@ static struct rcu_torture_ops tasks_rude_ops = { .read_delay = rcu_read_delay, /* just reuse rcu's version. */ .readunlock = rcu_torture_read_unlock_trivial, .get_gp_seq = rcu_no_completed, - .deferred_free = rcu_tasks_rude_torture_deferred_free, .sync = synchronize_rcu_tasks_rude, .exp_sync = synchronize_rcu_tasks_rude, - .call = call_rcu_tasks_rude, - .cb_barrier = rcu_barrier_tasks_rude, .gp_kthread_dbg = show_rcu_tasks_rude_gp_kthread, .get_gp_data = rcu_tasks_rude_get_gp_data, .cbflood_max = 50000, From patchwork Fri Aug 16 06:49:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765515 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6C44137747; Fri, 16 Aug 2024 06:50:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791053; cv=none; b=VXHcmYju4rwehhHENeFrsbdLREGsUwZKS17lPRGPCFS7noOTs/KqM/6KyivbJV71WcIYTVorWe0axRjiKPgFDipJkjoNgLjM+QH/fgDIgSEgFY3IU9fRTU9l2qSXHP5xwWgGu2ogQJGT77CvNwF6uEmk5ZFRlk+g9cfszufGccs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791053; c=relaxed/simple; bh=Qo8iVEXOB5vQe3YYkQwY0kyj0PXdCKtErMcG9BVvOic=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZB//MJCLYyxv7nbcX7ypAd8NNTKsA4XqGeSdXkncUMazrcX4ejqJyqggIds5SoVpQhyquj4cRUq0sbMkymkeX9gqYOjJBiXv41CvsqAwK6d2N6j6jFFu42KfBC0BEE02ZguKd4KSVtWiyKWeXlGYUOiyLTvEpcDgG2HWVSwoYqk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=feyIAjIs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="feyIAjIs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A798C4AF09; Fri, 16 Aug 2024 06:50:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791053; bh=Qo8iVEXOB5vQe3YYkQwY0kyj0PXdCKtErMcG9BVvOic=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=feyIAjIsF/8CpfxfrRKmpUs3o4VwSDpGKvURV5yqsxxeHLk337oP7ZFb1c2L0JQKE xCg5kDr8EYYBZtugVQ+xiQRVIRCY4hmhiCcTH3hb0XoxkxKWFE9v2pio+02PYOPW1+ ZMHKhX5q/0+ZYQryJyrtvq3rFEkKoCPq+nWiPij+14vufggAfSWlXb0dmIbqcUAcbC jiCaLxuWPKJETr+vflfug9RELdAh9+Ai8VtAwry6DgPiXSHVSls2TMY6p5VVWrBgya ZeqJykDYOdgQwBauR/RYoGabsf4bXzOVuX3am7k2zVegoSag45LVxnPJPOyP4tdEgQ OE2wZEGhOaVZg== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org, Peter Zijlstra Subject: [PATCH rcu 04/12] rcuscale: Stop testing RCU Tasks Rude asynchronous APIs Date: Fri, 16 Aug 2024 12:19:13 +0530 Message-Id: <20240816064921.57645-4-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The call_rcu_tasks_rude() and rcu_barrier_tasks_rude() APIs are currently unused. Furthermore, the idea is to get rid of RCU Tasks Rude entirely once all architectures have their deep-idle and entry/exit code correctly marked as inline or noinstr. As a step towards this goal, this commit therefore removes these two functions from rcuscale testing. Signed-off-by: Paul E. McKenney Cc: Peter Zijlstra Signed-off-by: Neeraj Upadhyay --- kernel/rcu/rcuscale.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c index b53a9e8f5904..d534d4ec2314 100644 --- a/kernel/rcu/rcuscale.c +++ b/kernel/rcu/rcuscale.c @@ -333,8 +333,6 @@ static struct rcu_scale_ops tasks_rude_ops = { .readunlock = tasks_rude_scale_read_unlock, .get_gp_seq = rcu_no_completed, .gp_diff = rcu_seq_diff, - .async = call_rcu_tasks_rude, - .gp_barrier = rcu_barrier_tasks_rude, .sync = synchronize_rcu_tasks_rude, .exp_sync = synchronize_rcu_tasks_rude, .rso_gp_kthread = get_rcu_tasks_rude_gp_kthread, From patchwork Fri Aug 16 06:49:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765516 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 185038289E; Fri, 16 Aug 2024 06:50:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791059; cv=none; b=XMVuOiUPm5Q5GTE5SjfX0AuVDgMmkYy23kIRFwsexy+YXgVpAS8IRYxWE3JDtvyUE2dgPlw+rjxAlygBBGPzv15mUsLBMkQjd5aXNE1r8JXdpldB3Hckxr57PUX0tUXFmfmbA/03pKqDqsA4oa3JjvgwXANUL0n3Ez1TFpQkQj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791059; c=relaxed/simple; bh=xAwbzpZYpYKzUiAmmYCU4rpEXtgjT8ZJynKD/3ssZUI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EILgde0v2b5NVyqcPFAztLjkF/eet8MmB2PPG78oyrvUPZMA6AgHbifJKFfeVP69qPYOitxjj+CKesnO9C4z3Lg7DeYGcYDPwLIr3jj5L62p7PdX1Ooj00Bl8gKRShWqwtrxNsu550RbHnSr1XSS+Eg4ewQbDuAT37+GoCu33co= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fBhcCB1D; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fBhcCB1D" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 193CDC4AF0E; Fri, 16 Aug 2024 06:50:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791059; bh=xAwbzpZYpYKzUiAmmYCU4rpEXtgjT8ZJynKD/3ssZUI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fBhcCB1DFVgb5yH8k6D/q3EkEbZXj8iFGp7pkb1NeL/Q8+Z9XymHjGyu0HZZOkFDJ V/a432xNrlacJ66dp9VeHOgqvLTd/J/Nksz/+UhRsx+3D8on6JUlcs7Vd4nYlP0EMp EwkkhVHwR+8PaJW/CbGGSc9enfgnY2+4gB9oeAcdnXC0AAXAsqfPbkCJHDT1eYtUiL hxUr73/sX0Pxr2OfHalUIvzygzirFlYGOtXzFA5/ZHa1wrEin5rjwPwBIw2pb7gSxm UPi8Yq5RUyQxlSsZyMsRS/GyMj95wJTL7KbkpaA+tJWZX9AHVCvqSwcnIp4xLOsOaT lqRBGMK5ssJ/Q== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org, Peter Zijlstra Subject: [PATCH rcu 05/12] rcu-tasks: Remove RCU Tasks Rude asynchronous APIs Date: Fri, 16 Aug 2024 12:19:14 +0530 Message-Id: <20240816064921.57645-5-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The call_rcu_tasks_rude() and rcu_barrier_tasks_rude() APIs are currently unused. This commit therefore removes their definitions and boot-time self-tests. Signed-off-by: Paul E. McKenney Cc: Peter Zijlstra Signed-off-by: Neeraj Upadhyay --- include/linux/rcupdate.h | 2 -- kernel/rcu/tasks.h | 55 +++++++++++++--------------------------- 2 files changed, 17 insertions(+), 40 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 13f6f00aecf9..31e679c7110e 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -37,7 +37,6 @@ /* Exported common interfaces */ void call_rcu(struct rcu_head *head, rcu_callback_t func); void rcu_barrier_tasks(void); -void rcu_barrier_tasks_rude(void); void synchronize_rcu(void); struct rcu_gp_oldstate; @@ -202,7 +201,6 @@ do { \ } while (0) # ifdef CONFIG_TASKS_RUDE_RCU -void call_rcu_tasks_rude(struct rcu_head *head, rcu_callback_t func); void synchronize_rcu_tasks_rude(void); # endif diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index ba3440a45b6d..4bc038bcc016 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -396,7 +396,7 @@ static void rcu_barrier_tasks_generic_cb(struct rcu_head *rhp) // Wait for all in-flight callbacks for the specified RCU Tasks flavor. // Operates in a manner similar to rcu_barrier(). -static void rcu_barrier_tasks_generic(struct rcu_tasks *rtp) +static void __maybe_unused rcu_barrier_tasks_generic(struct rcu_tasks *rtp) { int cpu; unsigned long flags; @@ -1244,13 +1244,12 @@ void exit_tasks_rcu_finish(void) { exit_tasks_rcu_finish_trace(current); } //////////////////////////////////////////////////////////////////////// // -// "Rude" variant of Tasks RCU, inspired by Steve Rostedt's trick of -// passing an empty function to schedule_on_each_cpu(). This approach -// provides an asynchronous call_rcu_tasks_rude() API and batching of -// concurrent calls to the synchronous synchronize_rcu_tasks_rude() API. -// This invokes schedule_on_each_cpu() in order to send IPIs far and wide -// and induces otherwise unnecessary context switches on all online CPUs, -// whether idle or not. +// "Rude" variant of Tasks RCU, inspired by Steve Rostedt's +// trick of passing an empty function to schedule_on_each_cpu(). +// This approach provides batching of concurrent calls to the synchronous +// synchronize_rcu_tasks_rude() API. This invokes schedule_on_each_cpu() +// in order to send IPIs far and wide and induces otherwise unnecessary +// context switches on all online CPUs, whether idle or not. // // Callback handling is provided by the rcu_tasks_kthread() function. // @@ -1268,11 +1267,11 @@ static void rcu_tasks_rude_wait_gp(struct rcu_tasks *rtp) schedule_on_each_cpu(rcu_tasks_be_rude); } -void call_rcu_tasks_rude(struct rcu_head *rhp, rcu_callback_t func); +static void call_rcu_tasks_rude(struct rcu_head *rhp, rcu_callback_t func); DEFINE_RCU_TASKS(rcu_tasks_rude, rcu_tasks_rude_wait_gp, call_rcu_tasks_rude, "RCU Tasks Rude"); -/** +/* * call_rcu_tasks_rude() - Queue a callback rude task-based grace period * @rhp: structure to be used for queueing the RCU updates. * @func: actual callback function to be invoked after the grace period @@ -1289,12 +1288,14 @@ DEFINE_RCU_TASKS(rcu_tasks_rude, rcu_tasks_rude_wait_gp, call_rcu_tasks_rude, * * See the description of call_rcu() for more detailed information on * memory ordering guarantees. + * + * This is no longer exported, and is instead reserved for use by + * synchronize_rcu_tasks_rude(). */ -void call_rcu_tasks_rude(struct rcu_head *rhp, rcu_callback_t func) +static void call_rcu_tasks_rude(struct rcu_head *rhp, rcu_callback_t func) { call_rcu_tasks_generic(rhp, func, &rcu_tasks_rude); } -EXPORT_SYMBOL_GPL(call_rcu_tasks_rude); /** * synchronize_rcu_tasks_rude - wait for a rude rcu-tasks grace period @@ -1320,26 +1321,9 @@ void synchronize_rcu_tasks_rude(void) } EXPORT_SYMBOL_GPL(synchronize_rcu_tasks_rude); -/** - * rcu_barrier_tasks_rude - Wait for in-flight call_rcu_tasks_rude() callbacks. - * - * Although the current implementation is guaranteed to wait, it is not - * obligated to, for example, if there are no pending callbacks. - */ -void rcu_barrier_tasks_rude(void) -{ - rcu_barrier_tasks_generic(&rcu_tasks_rude); -} -EXPORT_SYMBOL_GPL(rcu_barrier_tasks_rude); - -int rcu_tasks_rude_lazy_ms = -1; -module_param(rcu_tasks_rude_lazy_ms, int, 0444); - static int __init rcu_spawn_tasks_rude_kthread(void) { rcu_tasks_rude.gp_sleep = HZ / 10; - if (rcu_tasks_rude_lazy_ms >= 0) - rcu_tasks_rude.lazy_jiffies = msecs_to_jiffies(rcu_tasks_rude_lazy_ms); rcu_spawn_tasks_kthread_generic(&rcu_tasks_rude); return 0; } @@ -2069,11 +2053,6 @@ static struct rcu_tasks_test_desc tests[] = { /* If not defined, the test is skipped. */ .notrun = IS_ENABLED(CONFIG_TASKS_RCU), }, - { - .name = "call_rcu_tasks_rude()", - /* If not defined, the test is skipped. */ - .notrun = IS_ENABLED(CONFIG_TASKS_RUDE_RCU), - }, { .name = "call_rcu_tasks_trace()", /* If not defined, the test is skipped. */ @@ -2081,6 +2060,7 @@ static struct rcu_tasks_test_desc tests[] = { } }; +#if defined(CONFIG_TASKS_RCU) || defined(CONFIG_TASKS_TRACE_RCU) static void test_rcu_tasks_callback(struct rcu_head *rhp) { struct rcu_tasks_test_desc *rttd = @@ -2090,6 +2070,7 @@ static void test_rcu_tasks_callback(struct rcu_head *rhp) rttd->notrun = false; } +#endif // #if defined(CONFIG_TASKS_RCU) || defined(CONFIG_TASKS_TRACE_RCU) static void rcu_tasks_initiate_self_tests(void) { @@ -2102,16 +2083,14 @@ static void rcu_tasks_initiate_self_tests(void) #ifdef CONFIG_TASKS_RUDE_RCU pr_info("Running RCU Tasks Rude wait API self tests\n"); - tests[1].runstart = jiffies; synchronize_rcu_tasks_rude(); - call_rcu_tasks_rude(&tests[1].rh, test_rcu_tasks_callback); #endif #ifdef CONFIG_TASKS_TRACE_RCU pr_info("Running RCU Tasks Trace wait API self tests\n"); - tests[2].runstart = jiffies; + tests[1].runstart = jiffies; synchronize_rcu_tasks_trace(); - call_rcu_tasks_trace(&tests[2].rh, test_rcu_tasks_callback); + call_rcu_tasks_trace(&tests[1].rh, test_rcu_tasks_callback); #endif } From patchwork Fri Aug 16 06:49:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765517 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62C0584FA0; Fri, 16 Aug 2024 06:51:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791065; cv=none; b=eDoU4EGeyJn3oKiT3WrzQuzg9LVuicvgnTG9WzA+WZVw4p8z+LGTmXxA92wGeWuQmSRzwkUznZ+ATLJMw0FScJtHbrY0RiaxkVoqeLwTsXYNIwuhPiEKHymSVnXJqMKID31kgJ8KlIdfan1pguNfu5jvpT0/7b1U9SBsqZZyZj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791065; c=relaxed/simple; bh=fXZscSYT/8Xl7gzQoqlHNdL62LVt3zUNCLl4GoJexQY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EmX0JuFIIQtrH3h4nWSrCK2UoErQBHbFJBdIcknawffhbNlv0B1ti5NQJ4Xu6R9J7NZ6292s3WNZ+i/jR8yvG+G2uMzfSKSPC4wf0XXY8meNaV/pr40XL16gzIg+Mv7zdS35vA93/dUNEYzPZchAU6g3jwLAO1yQBiMrhmMo6PE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bxWp/u/6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bxWp/u/6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AEF9DC4AF0E; Fri, 16 Aug 2024 06:50:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791064; bh=fXZscSYT/8Xl7gzQoqlHNdL62LVt3zUNCLl4GoJexQY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bxWp/u/6ZCYy0ZJ8ykdwujQy67vkLQFAZ+JowQIXCP1glpZf9xYfen6M/OPhqUYdG ItANZucH1bY9UkvTGjdFoo1sXh2ddrT56NKHtFBPnfGhhFx64+yW7iBxcRyh+F3Orw a3pNIu+MBxHzZQ31+9wtatyBbRWKAGOAEnSPIYpQpNU2LLZb/rJ47awSZOP0QvfasP dNGqLGkce9eLObiqVtgH/Tq4LPCwXdLc6PAsvQuGrExoZvnI78xh/sLE91QZ7KoWCI JLQgQtnkC6V5VXq+XluF8K9r+KXXlvp93JqjDTcrbiiIjs6h4wneky5qM8x8I31XhW L5po3unD1rdzw== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org, Zqiang , Zhixu Liu Subject: [PATCH rcu 06/12] rcu-tasks: Fix access non-existent percpu rtpcp variable in rcu_tasks_need_gpcb() Date: Fri, 16 Aug 2024 12:19:15 +0530 Message-Id: <20240816064921.57645-6-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Zqiang For kernels built with CONFIG_FORCE_NR_CPUS=y, the nr_cpu_ids is defined as NR_CPUS instead of the number of possible cpus, this will cause the following system panic: smpboot: Allowing 4 CPUs, 0 hotplug CPUs ... setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:512 nr_node_ids:1 ... BUG: unable to handle page fault for address: ffffffff9911c8c8 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 0 PID: 15 Comm: rcu_tasks_trace Tainted: G W 6.6.21 #1 5dc7acf91a5e8e9ac9dcfc35bee0245691283ea6 RIP: 0010:rcu_tasks_need_gpcb+0x25d/0x2c0 RSP: 0018:ffffa371c00a3e60 EFLAGS: 00010082 CR2: ffffffff9911c8c8 CR3: 000000040fa20005 CR4: 00000000001706f0 Call Trace: ? __die+0x23/0x80 ? page_fault_oops+0xa4/0x180 ? exc_page_fault+0x152/0x180 ? asm_exc_page_fault+0x26/0x40 ? rcu_tasks_need_gpcb+0x25d/0x2c0 ? __pfx_rcu_tasks_kthread+0x40/0x40 rcu_tasks_one_gp+0x69/0x180 rcu_tasks_kthread+0x94/0xc0 kthread+0xe8/0x140 ? __pfx_kthread+0x40/0x40 ret_from_fork+0x34/0x80 ? __pfx_kthread+0x40/0x40 ret_from_fork_asm+0x1b/0x80 Considering that there may be holes in the CPU numbers, use the maximum possible cpu number, instead of nr_cpu_ids, for configuring enqueue and dequeue limits. [ neeraj.upadhyay: Fix htmldocs build error reported by Stephen Rothwell ] Closes: https://lore.kernel.org/linux-input/CALMA0xaTSMN+p4xUXkzrtR5r6k7hgoswcaXx7baR_z9r5jjskw@mail.gmail.com/T/#u Reported-by: Zhixu Liu Signed-off-by: Zqiang Signed-off-by: Neeraj Upadhyay --- kernel/rcu/tasks.h | 82 ++++++++++++++++++++++++++++++---------------- 1 file changed, 53 insertions(+), 29 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 4bc038bcc016..72d564c84499 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -34,6 +34,7 @@ typedef void (*postgp_func_t)(struct rcu_tasks *rtp); * @rtp_blkd_tasks: List of tasks blocked as readers. * @rtp_exit_list: List of tasks in the latter portion of do_exit(). * @cpu: CPU number corresponding to this entry. + * @index: Index of this CPU in rtpcp_array of the rcu_tasks structure. * @rtpp: Pointer to the rcu_tasks structure. */ struct rcu_tasks_percpu { @@ -49,6 +50,7 @@ struct rcu_tasks_percpu { struct list_head rtp_blkd_tasks; struct list_head rtp_exit_list; int cpu; + int index; struct rcu_tasks *rtpp; }; @@ -76,6 +78,7 @@ struct rcu_tasks_percpu { * @call_func: This flavor's call_rcu()-equivalent function. * @wait_state: Task state for synchronous grace-period waits (default TASK_UNINTERRUPTIBLE). * @rtpcpu: This flavor's rcu_tasks_percpu structure. + * @rtpcp_array: Array of pointers to rcu_tasks_percpu structure of CPUs in cpu_possible_mask. * @percpu_enqueue_shift: Shift down CPU ID this much when enqueuing callbacks. * @percpu_enqueue_lim: Number of per-CPU callback queues in use for enqueuing. * @percpu_dequeue_lim: Number of per-CPU callback queues in use for dequeuing. @@ -110,6 +113,7 @@ struct rcu_tasks { call_rcu_func_t call_func; unsigned int wait_state; struct rcu_tasks_percpu __percpu *rtpcpu; + struct rcu_tasks_percpu **rtpcp_array; int percpu_enqueue_shift; int percpu_enqueue_lim; int percpu_dequeue_lim; @@ -182,6 +186,8 @@ module_param(rcu_task_collapse_lim, int, 0444); static int rcu_task_lazy_lim __read_mostly = 32; module_param(rcu_task_lazy_lim, int, 0444); +static int rcu_task_cpu_ids; + /* RCU tasks grace-period state for debugging. */ #define RTGS_INIT 0 #define RTGS_WAIT_WAIT_CBS 1 @@ -245,6 +251,8 @@ static void cblist_init_generic(struct rcu_tasks *rtp) int cpu; int lim; int shift; + int maxcpu; + int index = 0; if (rcu_task_enqueue_lim < 0) { rcu_task_enqueue_lim = 1; @@ -254,14 +262,9 @@ static void cblist_init_generic(struct rcu_tasks *rtp) } lim = rcu_task_enqueue_lim; - if (lim > nr_cpu_ids) - lim = nr_cpu_ids; - shift = ilog2(nr_cpu_ids / lim); - if (((nr_cpu_ids - 1) >> shift) >= lim) - shift++; - WRITE_ONCE(rtp->percpu_enqueue_shift, shift); - WRITE_ONCE(rtp->percpu_dequeue_lim, lim); - smp_store_release(&rtp->percpu_enqueue_lim, lim); + rtp->rtpcp_array = kcalloc(num_possible_cpus(), sizeof(struct rcu_tasks_percpu *), GFP_KERNEL); + BUG_ON(!rtp->rtpcp_array); + for_each_possible_cpu(cpu) { struct rcu_tasks_percpu *rtpcp = per_cpu_ptr(rtp->rtpcpu, cpu); @@ -273,14 +276,29 @@ static void cblist_init_generic(struct rcu_tasks *rtp) INIT_WORK(&rtpcp->rtp_work, rcu_tasks_invoke_cbs_wq); rtpcp->cpu = cpu; rtpcp->rtpp = rtp; + rtpcp->index = index; + rtp->rtpcp_array[index] = rtpcp; + index++; if (!rtpcp->rtp_blkd_tasks.next) INIT_LIST_HEAD(&rtpcp->rtp_blkd_tasks); if (!rtpcp->rtp_exit_list.next) INIT_LIST_HEAD(&rtpcp->rtp_exit_list); + maxcpu = cpu; } - pr_info("%s: Setting shift to %d and lim to %d rcu_task_cb_adjust=%d.\n", rtp->name, - data_race(rtp->percpu_enqueue_shift), data_race(rtp->percpu_enqueue_lim), rcu_task_cb_adjust); + rcu_task_cpu_ids = maxcpu + 1; + if (lim > rcu_task_cpu_ids) + lim = rcu_task_cpu_ids; + shift = ilog2(rcu_task_cpu_ids / lim); + if (((rcu_task_cpu_ids - 1) >> shift) >= lim) + shift++; + WRITE_ONCE(rtp->percpu_enqueue_shift, shift); + WRITE_ONCE(rtp->percpu_dequeue_lim, lim); + smp_store_release(&rtp->percpu_enqueue_lim, lim); + + pr_info("%s: Setting shift to %d and lim to %d rcu_task_cb_adjust=%d rcu_task_cpu_ids=%d.\n", + rtp->name, data_race(rtp->percpu_enqueue_shift), data_race(rtp->percpu_enqueue_lim), + rcu_task_cb_adjust, rcu_task_cpu_ids); } // Compute wakeup time for lazy callback timer. @@ -348,7 +366,7 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, rtpcp->rtp_n_lock_retries = 0; } if (rcu_task_cb_adjust && ++rtpcp->rtp_n_lock_retries > rcu_task_contend_lim && - READ_ONCE(rtp->percpu_enqueue_lim) != nr_cpu_ids) + READ_ONCE(rtp->percpu_enqueue_lim) != rcu_task_cpu_ids) needadjust = true; // Defer adjustment to avoid deadlock. } // Queuing callbacks before initialization not yet supported. @@ -368,10 +386,10 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, raw_spin_unlock_irqrestore_rcu_node(rtpcp, flags); if (unlikely(needadjust)) { raw_spin_lock_irqsave(&rtp->cbs_gbl_lock, flags); - if (rtp->percpu_enqueue_lim != nr_cpu_ids) { + if (rtp->percpu_enqueue_lim != rcu_task_cpu_ids) { WRITE_ONCE(rtp->percpu_enqueue_shift, 0); - WRITE_ONCE(rtp->percpu_dequeue_lim, nr_cpu_ids); - smp_store_release(&rtp->percpu_enqueue_lim, nr_cpu_ids); + WRITE_ONCE(rtp->percpu_dequeue_lim, rcu_task_cpu_ids); + smp_store_release(&rtp->percpu_enqueue_lim, rcu_task_cpu_ids); pr_info("Switching %s to per-CPU callback queuing.\n", rtp->name); } raw_spin_unlock_irqrestore(&rtp->cbs_gbl_lock, flags); @@ -444,6 +462,8 @@ static int rcu_tasks_need_gpcb(struct rcu_tasks *rtp) dequeue_limit = smp_load_acquire(&rtp->percpu_dequeue_lim); for (cpu = 0; cpu < dequeue_limit; cpu++) { + if (!cpu_possible(cpu)) + continue; struct rcu_tasks_percpu *rtpcp = per_cpu_ptr(rtp->rtpcpu, cpu); /* Advance and accelerate any new callbacks. */ @@ -481,7 +501,7 @@ static int rcu_tasks_need_gpcb(struct rcu_tasks *rtp) if (rcu_task_cb_adjust && ncbs <= rcu_task_collapse_lim) { raw_spin_lock_irqsave(&rtp->cbs_gbl_lock, flags); if (rtp->percpu_enqueue_lim > 1) { - WRITE_ONCE(rtp->percpu_enqueue_shift, order_base_2(nr_cpu_ids)); + WRITE_ONCE(rtp->percpu_enqueue_shift, order_base_2(rcu_task_cpu_ids)); smp_store_release(&rtp->percpu_enqueue_lim, 1); rtp->percpu_dequeue_gpseq = get_state_synchronize_rcu(); gpdone = false; @@ -496,7 +516,9 @@ static int rcu_tasks_need_gpcb(struct rcu_tasks *rtp) pr_info("Completing switch %s to CPU-0 callback queuing.\n", rtp->name); } if (rtp->percpu_dequeue_lim == 1) { - for (cpu = rtp->percpu_dequeue_lim; cpu < nr_cpu_ids; cpu++) { + for (cpu = rtp->percpu_dequeue_lim; cpu < rcu_task_cpu_ids; cpu++) { + if (!cpu_possible(cpu)) + continue; struct rcu_tasks_percpu *rtpcp = per_cpu_ptr(rtp->rtpcpu, cpu); WARN_ON_ONCE(rcu_segcblist_n_cbs(&rtpcp->cblist)); @@ -511,30 +533,32 @@ static int rcu_tasks_need_gpcb(struct rcu_tasks *rtp) // Advance callbacks and invoke any that are ready. static void rcu_tasks_invoke_cbs(struct rcu_tasks *rtp, struct rcu_tasks_percpu *rtpcp) { - int cpu; - int cpunext; int cpuwq; unsigned long flags; int len; + int index; struct rcu_head *rhp; struct rcu_cblist rcl = RCU_CBLIST_INITIALIZER(rcl); struct rcu_tasks_percpu *rtpcp_next; - cpu = rtpcp->cpu; - cpunext = cpu * 2 + 1; - if (cpunext < smp_load_acquire(&rtp->percpu_dequeue_lim)) { - rtpcp_next = per_cpu_ptr(rtp->rtpcpu, cpunext); - cpuwq = rcu_cpu_beenfullyonline(cpunext) ? cpunext : WORK_CPU_UNBOUND; - queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work); - cpunext++; - if (cpunext < smp_load_acquire(&rtp->percpu_dequeue_lim)) { - rtpcp_next = per_cpu_ptr(rtp->rtpcpu, cpunext); - cpuwq = rcu_cpu_beenfullyonline(cpunext) ? cpunext : WORK_CPU_UNBOUND; + index = rtpcp->index * 2 + 1; + if (index < num_possible_cpus()) { + rtpcp_next = rtp->rtpcp_array[index]; + if (rtpcp_next->cpu < smp_load_acquire(&rtp->percpu_dequeue_lim)) { + cpuwq = rcu_cpu_beenfullyonline(rtpcp_next->cpu) ? rtpcp_next->cpu : WORK_CPU_UNBOUND; queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work); + index++; + if (index < num_possible_cpus()) { + rtpcp_next = rtp->rtpcp_array[index]; + if (rtpcp_next->cpu < smp_load_acquire(&rtp->percpu_dequeue_lim)) { + cpuwq = rcu_cpu_beenfullyonline(rtpcp_next->cpu) ? rtpcp_next->cpu : WORK_CPU_UNBOUND; + queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work); + } + } } } - if (rcu_segcblist_empty(&rtpcp->cblist) || !cpu_possible(cpu)) + if (rcu_segcblist_empty(&rtpcp->cblist)) return; raw_spin_lock_irqsave_rcu_node(rtpcp, flags); rcu_segcblist_advance(&rtpcp->cblist, rcu_seq_current(&rtp->tasks_gp_seq)); From patchwork Fri Aug 16 06:49:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765518 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CAF0136E23; Fri, 16 Aug 2024 06:51:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791071; cv=none; b=RSaAtwAqRmkPs5aDB0l5l2k4dXs/8+g/yqy0nFnDcZ9DI7OR9jK1Zo2wit27BaPtn+oDYOGXTH3JXXMopy5XmXOLXLcXC/os7LShD08tLgCdNbHfhfr/bc6OmgsjPwXwT42JA2vSXv6VAEMxsOiv7rhtCOhkzGt1wS5/eWSzYIY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791071; c=relaxed/simple; bh=ezgPeV6bSSDC6r/9nIfy0Dm82DBCIOzBZILfHQLQEio=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gp2hu2WmmIrhUJzpFr8+g9uR7N79ToZkGBz5ACZxDGMsHP8Az1YAfJ8e6vzbsnxeOUSy1dnwmGGCF9YO4cQzJSz3gQmgKee4DLSM2ZN3ehsUXfXg5jUJN9OhBK9DxsxdIaOeo1UBOFfh/YPD7ShX4oSZ7GMjC0SMSp+LfnGKdBY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QO9aWaGQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QO9aWaGQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99CEAC4AF0F; Fri, 16 Aug 2024 06:51:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791070; bh=ezgPeV6bSSDC6r/9nIfy0Dm82DBCIOzBZILfHQLQEio=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QO9aWaGQRgAbQ1FnNFvTCCRy0s+beqUmEF9lS/AHxa7ivqirceEQdNCxZaDpPeZzy 5XyU4VguWnC0RBFApZ/eoPhjd9NBf6ATMPVcmSjAK0L2PsaxeE3cWZBFN0dumYBdhu a8eIJQn0ZQ0hS3yFuB8rwRhPnn6W0AoBG/sfanoNFjTbmgNIoRK2CEPGmBhKjZ5DAe bCIUqCjJt7jK8v/U204GBmBXJEOJ07R+L0NWpCMyObgZnYSnn5UCFqwOTXGmBzVypF nuyUZtdkO/oZWpWUq1yYP2xl1siLZGWnlLNFeXYo5NQsI1+MAG077G8aBzT7EX5XCS NTsqZBqRC5iaQ== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 07/12] rcu/tasks: Check processor-ID assumptions Date: Fri, 16 Aug 2024 12:19:16 +0530 Message-Id: <20240816064921.57645-7-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The current mapping of smp_processor_id() to a CPU processing Tasks-RCU callbacks makes some assumptions about layout. This commit therefore adds a WARN_ON() to check these assumptions. [ neeraj.upadhyay: Replace nr_cpu_ids with rcu_task_cpu_ids. ] Signed-off-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay --- kernel/rcu/tasks.h | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 72d564c84499..9fb8bb0afff7 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -357,6 +357,7 @@ static void call_rcu_tasks_generic(struct rcu_head *rhp, rcu_callback_t func, rcu_read_lock(); ideal_cpu = smp_processor_id() >> READ_ONCE(rtp->percpu_enqueue_shift); chosen_cpu = cpumask_next(ideal_cpu - 1, cpu_possible_mask); + WARN_ON_ONCE(chosen_cpu >= rcu_task_cpu_ids); rtpcp = per_cpu_ptr(rtp->rtpcpu, chosen_cpu); if (!raw_spin_trylock_rcu_node(rtpcp)) { // irqs already disabled. raw_spin_lock_rcu_node(rtpcp); // irqs already disabled. From patchwork Fri Aug 16 06:49:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765519 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6129113B783; Fri, 16 Aug 2024 06:51:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791082; cv=none; b=MOUJvSM33Z4hErcu6UE100JlnZySCM6tT8KQMjvVzO4uxDI+4HTk8/FOuNUM6cR/1rdj+63NkjXgfckVoEsnBJ8QaoKlfbG+BoTioEHNpNBR4p84YgHPQxugk+y4CzNZbgxt18hlLEn/TYKMODQ5HKxGQZ/105PoGEYBfi02nDY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791082; c=relaxed/simple; bh=7Xy0bxjgRFCmWJzteGxUb/ixG8Cfyekq5uFhtPDLC6Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eTdsC9SmBfKzIT4HX8dVukjGf/7srULWIy6HkNNMqp6MrKwifCvvUwLldphEl5qUbDMhwmUtf22M1T4fcU6E+I5IC0CcVqYvGy31Gxm9oGj3jGmZSsFaFOOTuZbQAFeujpPJdOf8/tJ+Q6woLnM8PUq/ikknVNct6xQjg9RiyWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XbSmrq1R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XbSmrq1R" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 04AF9C32782; Fri, 16 Aug 2024 06:51:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791082; bh=7Xy0bxjgRFCmWJzteGxUb/ixG8Cfyekq5uFhtPDLC6Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XbSmrq1RDIAuTG17t/CA5X3mHTHDEb95mUeeZwTSa4l4jxRjvdCi4EDuXQn2oRxeL PQLX1vtBPko4iTsQLBErWZbFWI6SboD+utowuaKvMJbKaDA2/uYG1eldkYilwsDGq0 dXYOPkOVANsCsmIeHksgCVlFlr1qGIJVWbMXtSOPsXeP5S2jzTVD65aTOHWraqLb8T Giire4YMzybzR9EbybDOq65j49qULhiZLqoCNiWrfTWyf+hhzOi8fAvK9a3CgP84hw uIZARzRpxeUoY3pconbn0OIxudoyOaz2DBhjT1HZEetwzJ5QJsSn60qV4f9WUjFU6G aQ505Xut1UvWA== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 08/12] rcu/tasks: Update rtp->tasks_gp_seq comment Date: Fri, 16 Aug 2024 12:19:17 +0530 Message-Id: <20240816064921.57645-8-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The rtp->tasks_gp_seq grace-period sequence number is not a strict count, but rather the usual RCU sequence number with the lower few bits tracking per-grace-period state and the upper bits the count of grace periods since boot, give or take the initial value. This commit therefore adjusts this comment. Signed-off-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay --- kernel/rcu/tasks.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 9fb8bb0afff7..127018280618 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -65,7 +65,7 @@ struct rcu_tasks_percpu { * @init_fract: Initial backoff sleep interval. * @gp_jiffies: Time of last @gp_state transition. * @gp_start: Most recent grace-period start in jiffies. - * @tasks_gp_seq: Number of grace periods completed since boot. + * @tasks_gp_seq: Number of grace periods completed since boot in upper bits. * @n_ipis: Number of IPIs sent to encourage grace periods to end. * @n_ipis_fails: Number of IPI-send failures. * @kthread_ptr: This flavor's grace-period/callback-invocation kthread. From patchwork Fri Aug 16 06:49:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765520 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7520313D262; Fri, 16 Aug 2024 06:51:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791088; cv=none; b=njT51DcpWMDnIFfjhm0uUZapJqKgCzYfy0+qmGCndWyXDiaXvQwA4MMKJzEJ39VqPf1GTOl9TSatDSsJ+hC8lGDLSzO39xie4w44LS2Mbi2kIjvJEihmBziDI2w11Uf98xJJ/7N+TWnJ/W9bWLQCTf/26DsM2xU0k6oQBmcLXHU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791088; c=relaxed/simple; bh=b29osK1yWSOJhUzqZLlk591XC5cm6F2NYvSY/YEC+vk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WwvMYO4zo/uuS/Aek2nidzieqSKDWXHr6eMrXPjx1nPCjKHm78KDgIb3y43oiU5VlryVZAsDIbnW98rGNoaQELwCGidROCCRL8cd+Mnlh2nWLFrZK+MaLZbNjwf2F4spTlOnLsxrk+SKPlQFmaBw0B8wWAV9EUTEGoNh9Wx2saE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mrFemkGJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mrFemkGJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2CF1EC4AF0B; Fri, 16 Aug 2024 06:51:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791088; bh=b29osK1yWSOJhUzqZLlk591XC5cm6F2NYvSY/YEC+vk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mrFemkGJ/ewJ9jeE5VyKL5WOTd7hAfd7SW2GgI7m6spjBlVJ8+ah8suIhD9mR3mGS fpr9AimFacbiwcIQIW3mcSSxinG7FvloAdM06UinFOlcUbKMQS8HAxhhmkrJtO7urQ CKp6FXra9B9gXyHKBYZW5U+W2lPEBy+7OxXy0G8+uS/ixi3epTbCVBH78uyjPdasGq dcdiLUjNLFqngyDyxw7XJ2v+mU9ytGwhyC6nGWrfARP6WPFD+6RgEjPG0ML8Guy3HV Fxd9j62QoKJl22nIanpdpl+FAYgFdeYMcwyzj1MO/mTBIJUL9LccgdgGUKqmvgIMek MK2aj841qDdiA== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 09/12] rcu: Provide rcu_barrier_cb_is_done() to check rcu_barrier() CBs Date: Fri, 16 Aug 2024 12:19:18 +0530 Message-Id: <20240816064921.57645-9-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" This commit provides a rcu_barrier_cb_is_done() function that returns true if the *rcu_barrier*() callback passed in is done. This will be used when printing grace-period debugging information. Signed-off-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay --- kernel/rcu/rcu.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index 38238e595a61..caaed27e476b 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -255,6 +255,11 @@ static inline void debug_rcu_head_callback(struct rcu_head *rhp) kmem_dump_obj(rhp); } +static inline bool rcu_barrier_cb_is_done(struct rcu_head *rhp) +{ + return rhp->next == rhp; +} + extern int rcu_cpu_stall_suppress_at_boot; static inline bool rcu_stall_is_suppressed_at_boot(void) From patchwork Fri Aug 16 06:49:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765521 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 681E1839E3; Fri, 16 Aug 2024 06:51:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791095; cv=none; b=A07C/QZh6yYIzqAKSfcdZvUd70+bIK6QM2CrNtQcEcZc+qYHtK8d8HI/XdVg1jfDcf4jUzZgoV8EYVIfezFuEZCYsXTNSbxqdx18iwFcm4FAnA8NNqnUIb9vTAoEEvI3laqoTzU0CdXrlg7QHB5Ry1xiXbg/3XzC+oUkYR7LwmM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791095; c=relaxed/simple; bh=wi+lEcfaLQ4kKBI2cOT3M3q6JGLW3FpsAu/Mo0gCJ4g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T4gtUHCCWs9iJ+Sr/aC/NX9T+lY6FLlcSVaCQJwe+8Xjtthho4Hg2zqIEkIgs2ZXFGl+OI4O8SVWOPMZ3BfFT2BB+9HHTDF/wrtQMr0dlDm6rxYiOVunxFeEWui0JhYC+bLpBk13UxRoIt4qn6lS11FvU/gKCVrcgjuXabnIm1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FEwAj0gw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FEwAj0gw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DB4DC4AF09; Fri, 16 Aug 2024 06:51:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791094; bh=wi+lEcfaLQ4kKBI2cOT3M3q6JGLW3FpsAu/Mo0gCJ4g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FEwAj0gwhsqb82LY0RTh1HZPLw3jSNKzErVcUE5waOBiPq7qlvXzysY1lwtrEGJFp uM2QkZk7nZu6JmIE4tR+ijPTCi9VYdsSBvB7w7/BlPBnz2/GkMYv4+eTePfnH/ohPe M0aHrEJP+vqgb2qqAdkRH1gbYJjoWrxVQQJwcUJGOfa5nHxompszjEHqfKyW9I6xzf 2A/dFS3IViVg6FKT6e4OTOo4beaGSFBvqUiooLojqGZodM329V3j+eN4qDEmvL5L15 uXUGMtHixw3Qr0k2km72cH8IQ//apyV+T41J5W7v0pMO4+zqi6VLUK5EEczGI32hMW 3BnLfZQzx+33A== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 10/12] rcu/tasks: Mark callbacks not currently participating in barrier operation Date: Fri, 16 Aug 2024 12:19:19 +0530 Message-Id: <20240816064921.57645-10-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" Each Tasks RCU flavor keeps a count of the number of callbacks that the current rcu_barrier_tasks*() is waiting on, but there is currently no easy way to work out which callback is stuck. One way to do this is to mark idle RCU-barrier callbacks by making the ->next pointer point to the callback itself, and this commit does just that. Later commits will use this for debug output. Signed-off-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay --- kernel/rcu/tasks.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 127018280618..d44abcd656d6 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -283,6 +283,7 @@ static void cblist_init_generic(struct rcu_tasks *rtp) INIT_LIST_HEAD(&rtpcp->rtp_blkd_tasks); if (!rtpcp->rtp_exit_list.next) INIT_LIST_HEAD(&rtpcp->rtp_exit_list); + rtpcp->barrier_q_head.next = &rtpcp->barrier_q_head; maxcpu = cpu; } @@ -407,6 +408,7 @@ static void rcu_barrier_tasks_generic_cb(struct rcu_head *rhp) struct rcu_tasks *rtp; struct rcu_tasks_percpu *rtpcp; + rhp->next = rhp; // Mark the callback as having been invoked. rtpcp = container_of(rhp, struct rcu_tasks_percpu, barrier_q_head); rtp = rtpcp->rtpp; if (atomic_dec_and_test(&rtp->barrier_q_count)) From patchwork Fri Aug 16 06:49:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765522 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C19612C54B; Fri, 16 Aug 2024 06:51:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791102; cv=none; b=Fye3uOfY1ZkDakjcCjHbwOF344zj49aoUR8nLtcj2ZxKIySV5pCRnFHr0Fx+zRTbl2A1LCfXUYc4qB8c/wBcez3pUd6p6n5fPdMgg7UHCGYY/3l0Te/fYO6H36qZyER4xfPk/kz8QSutXLEKsExltszGyMVj9KuSnvYE6F1nqCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791102; c=relaxed/simple; bh=KkDJYH3rH2rJUD9sYZd1IWM6+Gk+sX8bQc9VkZJuQM0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CrkJbJO9bhFl4vSHALITv22pTas7nj+/bH6/xXsC++kDJnJuo8su8TGkYmHEW0QdTmmkxGrNPEjkHJcFYEeFGSqPw5O93VVp1K1mvKLzlj9ZmMKioGwZqa/0045N1JC3i9XmlY45J2gLGXNFkAzMf1mbm5Cr/+ggggOVDEYUikg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UY15I7/q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UY15I7/q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE63DC4AF0F; Fri, 16 Aug 2024 06:51:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791102; bh=KkDJYH3rH2rJUD9sYZd1IWM6+Gk+sX8bQc9VkZJuQM0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UY15I7/qzUnEx4mqauQWDzX5wnzBYK20U4WEBn6jjHxx6LVMlzw76recGrSTlyHiR EQXyBIdmZYWLNEyLJfBPmcaV6FuiBz589RibyaAxjhy1PtytuCQkXspYVhJTY1Je5C pEQjUgh5n2ub4JOHDjvXrIS9bIPUEAMbMAakuORpcjxTahJjftY1O6ShzzCrbSz6zS e3XseIU/9K2/WICiYCmOrmh1ox+ouA6lhCjuSuMVm754LtmqFPmL7bwC4dduKoKlOp Gktwf3ipGwo6DRY2A7a1b9fpK5LlHCn7llfQ/b4saQOhfleWYimDXHtlS1XojknCI0 Wgn+fzEDTXfXA== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 11/12] rcu/tasks: Add detailed grace-period and barrier diagnostics Date: Fri, 16 Aug 2024 12:19:20 +0530 Message-Id: <20240816064921.57645-11-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" This commit adds rcu_tasks_torture_stats_print(), rcu_tasks_trace_torture_stats_print(), and rcu_tasks_rude_torture_stats_print() functions that provide detailed diagnostics on grace-period, callback, and barrier state. Signed-off-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay --- include/linux/rcupdate.h | 3 ++ kernel/rcu/tasks.h | 66 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 67 insertions(+), 2 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 31e679c7110e..17463e95b6ef 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -164,6 +164,7 @@ static inline void rcu_nocb_flush_deferred_wakeup(void) { } } while (0) void call_rcu_tasks(struct rcu_head *head, rcu_callback_t func); void synchronize_rcu_tasks(void); +void rcu_tasks_torture_stats_print(char *tt, char *tf); # else # define rcu_tasks_classic_qs(t, preempt) do { } while (0) # define call_rcu_tasks call_rcu @@ -190,6 +191,7 @@ void rcu_tasks_trace_qs_blkd(struct task_struct *t); rcu_tasks_trace_qs_blkd(t); \ } \ } while (0) +void rcu_tasks_trace_torture_stats_print(char *tt, char *tf); # else # define rcu_tasks_trace_qs(t) do { } while (0) # endif @@ -202,6 +204,7 @@ do { \ # ifdef CONFIG_TASKS_RUDE_RCU void synchronize_rcu_tasks_rude(void); +void rcu_tasks_rude_torture_stats_print(char *tt, char *tf); # endif #define rcu_note_voluntary_context_switch(t) rcu_tasks_qs(t, false) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index d44abcd656d6..5f6d80ce1e47 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -714,9 +714,7 @@ static void __init rcu_tasks_bootup_oddness(void) #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */ } -#endif /* #ifndef CONFIG_TINY_RCU */ -#ifndef CONFIG_TINY_RCU /* Dump out rcutorture-relevant state common to all RCU-tasks flavors. */ static void show_rcu_tasks_generic_gp_kthread(struct rcu_tasks *rtp, char *s) { @@ -750,6 +748,52 @@ static void show_rcu_tasks_generic_gp_kthread(struct rcu_tasks *rtp, char *s) rtp->lazy_jiffies, s); } + +/* Dump out more rcutorture-relevant state common to all RCU-tasks flavors. */ +static void rcu_tasks_torture_stats_print_generic(struct rcu_tasks *rtp, char *tt, + char *tf, char *tst) +{ + cpumask_var_t cm; + int cpu; + bool gotcb = false; + unsigned long j = jiffies; + + pr_alert("%s%s Tasks%s RCU g%ld gp_start %lu gp_jiffies %lu gp_state %d (%s).\n", + tt, tf, tst, data_race(rtp->tasks_gp_seq), + j - data_race(rtp->gp_start), j - data_race(rtp->gp_jiffies), + data_race(rtp->gp_state), tasks_gp_state_getname(rtp)); + pr_alert("\tEnqueue shift %d limit %d Dequeue limit %d gpseq %lu.\n", + data_race(rtp->percpu_enqueue_shift), + data_race(rtp->percpu_enqueue_lim), + data_race(rtp->percpu_dequeue_lim), + data_race(rtp->percpu_dequeue_gpseq)); + (void)zalloc_cpumask_var(&cm, GFP_KERNEL); + pr_alert("\tCallback counts:"); + for_each_possible_cpu(cpu) { + long n; + struct rcu_tasks_percpu *rtpcp = per_cpu_ptr(rtp->rtpcpu, cpu); + + if (cpumask_available(cm) && !rcu_barrier_cb_is_done(&rtpcp->barrier_q_head)) + cpumask_set_cpu(cpu, cm); + n = rcu_segcblist_n_cbs(&rtpcp->cblist); + if (!n) + continue; + pr_cont(" %d:%ld", cpu, n); + gotcb = true; + } + if (gotcb) + pr_cont(".\n"); + else + pr_cont(" (none).\n"); + pr_alert("\tBarrier seq %lu count %d holdout CPUs ", + data_race(rtp->barrier_q_seq), atomic_read(&rtp->barrier_q_count)); + if (cpumask_available(cm) && !cpumask_empty(cm)) + pr_cont(" %*pbl.\n", cpumask_pr_args(cm)); + else + pr_cont("(none).\n"); + free_cpumask_var(cm); +} + #endif // #ifndef CONFIG_TINY_RCU static void exit_tasks_rcu_finish_trace(struct task_struct *t); @@ -1201,6 +1245,12 @@ void show_rcu_tasks_classic_gp_kthread(void) show_rcu_tasks_generic_gp_kthread(&rcu_tasks, ""); } EXPORT_SYMBOL_GPL(show_rcu_tasks_classic_gp_kthread); + +void rcu_tasks_torture_stats_print(char *tt, char *tf) +{ + rcu_tasks_torture_stats_print_generic(&rcu_tasks, tt, tf, ""); +} +EXPORT_SYMBOL_GPL(rcu_tasks_torture_stats_print); #endif // !defined(CONFIG_TINY_RCU) struct task_struct *get_rcu_tasks_gp_kthread(void) @@ -1361,6 +1411,12 @@ void show_rcu_tasks_rude_gp_kthread(void) show_rcu_tasks_generic_gp_kthread(&rcu_tasks_rude, ""); } EXPORT_SYMBOL_GPL(show_rcu_tasks_rude_gp_kthread); + +void rcu_tasks_rude_torture_stats_print(char *tt, char *tf) +{ + rcu_tasks_torture_stats_print_generic(&rcu_tasks_rude, tt, tf, ""); +} +EXPORT_SYMBOL_GPL(rcu_tasks_rude_torture_stats_print); #endif // !defined(CONFIG_TINY_RCU) struct task_struct *get_rcu_tasks_rude_gp_kthread(void) @@ -2038,6 +2094,12 @@ void show_rcu_tasks_trace_gp_kthread(void) show_rcu_tasks_generic_gp_kthread(&rcu_tasks_trace, buf); } EXPORT_SYMBOL_GPL(show_rcu_tasks_trace_gp_kthread); + +void rcu_tasks_trace_torture_stats_print(char *tt, char *tf) +{ + rcu_tasks_torture_stats_print_generic(&rcu_tasks_trace, tt, tf, ""); +} +EXPORT_SYMBOL_GPL(rcu_tasks_trace_torture_stats_print); #endif // !defined(CONFIG_TINY_RCU) struct task_struct *get_rcu_tasks_trace_gp_kthread(void) From patchwork Fri Aug 16 06:49:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neeraj Upadhyay X-Patchwork-Id: 13765523 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8FBE13777F; Fri, 16 Aug 2024 06:51:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791109; cv=none; b=jPiSODnJX6E7r4A0x2HCybWld/AbsmyhqaLpMh3vhKWMk/GVslFoSavIbVfSubAb7IhTwyYtqvV21WISOavzGn0ZbkfOe0vTZuM/xT1JXew94QHXDUO42Pahc36bU7jDYgGu+TuaQ5k1LYSau+ek8UZMLN1dp6VJl2z+qliQS+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723791109; c=relaxed/simple; bh=D8s5bN3jgVM8JPbRgvLUs8ZVTdCwr1h49SMsmogsp08=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YumHu4TzzXclsGM1wsthdrKeF70kQ2ISWMs9jnR2lUptpT+6V893m/WHroa9n33XNomgJGgGFfw4LlvwnklY6Va2C4LwNcVTYYYNCmGifi65rdYKpEbt7S9LgALDXh3WpSWJ8IIX4BKRwzewiVaui8OlXoyDal/fourcSlcM+QY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jbGtZ28N; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jbGtZ28N" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 09E1CC32782; Fri, 16 Aug 2024 06:51:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1723791109; bh=D8s5bN3jgVM8JPbRgvLUs8ZVTdCwr1h49SMsmogsp08=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jbGtZ28NtVnsK3IecxIH0GNuFtvvK8noARrlYH8vqERTjYRg/J9DTVcG6a+i8PeMK WbVjnCdRcKwu4GlmCjjNzX1UP2GQcubhd0A5vx40WxEwMlRT2Ra6FcUmLiTFnbXxlb 3EJr5jRtwB3ev8/ZI15CoSSyKCkf92j70vJ21b6eYqRrC6RepL06H7twsMsYXpr5SE 2/+l7XsEjKD+q3GePzoH8AqMHHoVsDjwwF773rQEQ3F1TJ31uPi7JjslId1Wx1Blf0 VcKcVe8zaAxn5gGeQ9R9SdA5EIlVBz8q7DeXDRy+MtyDZBNiXzcwkXwa4Sdz9sVJmP KFUgU8bEauLLw== From: neeraj.upadhyay@kernel.org To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, paulmck@kernel.org, neeraj.upadhyay@kernel.org, neeraj.upadhyay@amd.com, boqun.feng@gmail.com, joel@joelfernandes.org, urezki@gmail.com, frederic@kernel.org Subject: [PATCH rcu 12/12] rcu/tasks: Add rcu_barrier_tasks*() start time to diagnostics Date: Fri, 16 Aug 2024 12:19:21 +0530 Message-Id: <20240816064921.57645-12-neeraj.upadhyay@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240816064730.GA56949@neeraj.linux> References: <20240816064730.GA56949@neeraj.linux> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" This commit adds the start time, in jiffies, of the most recently started rcu_barrier_tasks*() operation to the diagnostic output used by rcuscale. This information can be helpful in distinguishing a hung barrier operation from a long series of barrier operations. Signed-off-by: "Paul E. McKenney" Signed-off-by: Neeraj Upadhyay --- kernel/rcu/tasks.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 5f6d80ce1e47..36be92efb0da 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -87,6 +87,7 @@ struct rcu_tasks_percpu { * @barrier_q_count: Number of queues being waited on. * @barrier_q_completion: Barrier wait/wakeup mechanism. * @barrier_q_seq: Sequence number for barrier operations. + * @barrier_q_start: Most recent barrier start in jiffies. * @name: This flavor's textual name. * @kname: This flavor's kthread name. */ @@ -122,6 +123,7 @@ struct rcu_tasks { atomic_t barrier_q_count; struct completion barrier_q_completion; unsigned long barrier_q_seq; + unsigned long barrier_q_start; char *name; char *kname; }; @@ -430,6 +432,7 @@ static void __maybe_unused rcu_barrier_tasks_generic(struct rcu_tasks *rtp) mutex_unlock(&rtp->barrier_q_mutex); return; } + rtp->barrier_q_start = jiffies; rcu_seq_start(&rtp->barrier_q_seq); init_completion(&rtp->barrier_q_completion); atomic_set(&rtp->barrier_q_count, 2); @@ -785,8 +788,9 @@ static void rcu_tasks_torture_stats_print_generic(struct rcu_tasks *rtp, char *t pr_cont(".\n"); else pr_cont(" (none).\n"); - pr_alert("\tBarrier seq %lu count %d holdout CPUs ", - data_race(rtp->barrier_q_seq), atomic_read(&rtp->barrier_q_count)); + pr_alert("\tBarrier seq %lu start %lu count %d holdout CPUs ", + data_race(rtp->barrier_q_seq), j - data_race(rtp->barrier_q_start), + atomic_read(&rtp->barrier_q_count)); if (cpumask_available(cm) && !cpumask_empty(cm)) pr_cont(" %*pbl.\n", cpumask_pr_args(cm)); else