From patchwork Wed Oct 9 18:00:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828937 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 140B81A76D5; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; cv=none; b=QhpoVUi5hwecunHmO1lhP+DJBVwvlI+GAGepFNOklODPxCw5RMRlt4/QIB+SrIMpvwWYOsmO1zvxX0q/8PXE/NjisFGOy5ftKE/8K5c0dir3KYFjspM+2TNv708I+lect5u2otwUNfRudkWqKyR+utVh9VbHPMH+iJWNRkJZCKo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; c=relaxed/simple; bh=ChclQjdk1wWpYrsOxGeef20vXWr7m8b4FD5ACzrOUE0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WhEYlHBMkqgzH9hYygxu05uWJhHyv+KG+Dn3vTcYJH14Ug7v9+KQ8+ZBAKZk64zVnGOji0W7T4heL8QrG1WJlN+KcXlCS4Ut6qJzHvLqv4yZBJWFjkqVr2ohcsEyYHhyz3aIdzt9ueV5pvhX05HK6f+IafPFwAwh7GVMU+sOS+E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UloGKvEj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UloGKvEj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB5F7C4CECC; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496846; bh=ChclQjdk1wWpYrsOxGeef20vXWr7m8b4FD5ACzrOUE0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UloGKvEjq5TAcI62jhgjmcZ0N9cY+N11CprmxYp48Lp9TbqchMs5ZPJjdTgLbuEZ4 VrjqgeK1hYHexzCx62kSUX/mhiE5OJzH99/N+DTLKYv4aYm9/5dwshq/QjSKgcPtqu +DQGkHSb9/YqyNW7iai6ZUDCN0BAXPVvvAoemfd2784Ys3d0XO0LXX0uf1KutzH8WE F+vvl0AMKQlPap7w71QoVcU7lIB+jCORGB94VHToTUhcKMX1+tjHBN52ASJxhSQu1w cQAwxJ1hla7QZy7hNJfP64yNgVKtS4hMGU0xJUi4Vb4nc2GiOlsJbjIVKlUOS0oGi+ zKQQmKsvdIF2w== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 693FECE08E5; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Olivier Langlois Subject: [PATCH rcu 1/7] rcu: Add rcuog kthreads to RCU_NOCB_CPU help text Date: Wed, 9 Oct 2024 11:00:39 -0700 Message-Id: <20241009180045.777721-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The RCU_NOCB_CPU help text currently fails to mention rcuog kthreads, so this commit adds this information. Reported-by: Olivier Langlois Signed-off-by: Paul E. McKenney --- kernel/rcu/Kconfig | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig index 3e079de0f5b43..b9b6bc55185db 100644 --- a/kernel/rcu/Kconfig +++ b/kernel/rcu/Kconfig @@ -249,16 +249,24 @@ config RCU_NOCB_CPU workloads will incur significant increases in context-switch rates. - This option offloads callback invocation from the set of CPUs - specified at boot time by the rcu_nocbs parameter. For each - such CPU, a kthread ("rcuox/N") will be created to invoke - callbacks, where the "N" is the CPU being offloaded, and where - the "x" is "p" for RCU-preempt (PREEMPTION kernels) and "s" for - RCU-sched (!PREEMPTION kernels). Nothing prevents this kthread - from running on the specified CPUs, but (1) the kthreads may be - preempted between each callback, and (2) affinity or cgroups can - be used to force the kthreads to run on whatever set of CPUs is - desired. + This option offloads callback invocation from the set of + CPUs specified at boot time by the rcu_nocbs parameter. + For each such CPU, a kthread ("rcuox/N") will be created to + invoke callbacks, where the "N" is the CPU being offloaded, + and where the "x" is "p" for RCU-preempt (PREEMPTION kernels) + and "s" for RCU-sched (!PREEMPTION kernels). This option + also creates another kthread for each sqrt(nr_cpu_ids) CPUs + ("rcuog/N", where N is the first CPU in that group to come + online), which handles grace periods for its group. Nothing + prevents these kthreads from running on the specified CPUs, + but (1) the kthreads may be preempted between each callback, + and (2) affinity or cgroups can be used to force the kthreads + to run on whatever set of CPUs is desired. + + The sqrt(nr_cpu_ids) grouping may be overridden using the + rcutree.rcu_nocb_gp_stride kernel boot parameter. This can + be especially helpful for smaller numbers of CPUs, where + sqrt(nr_cpu_ids) can be a bit of a blunt instrument. Say Y here if you need reduced OS jitter, despite added overhead. Say N here if you are unsure. From patchwork Wed Oct 9 18:00:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828938 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 141441DFE24; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; cv=none; b=keFXYM6CEyKhQsmSMKRzL2wrlcMJ1i9mbIIopOZm1MCDJZfyU07e6WzMRrXci1V6vwLhz+wezrtJ4D8UlcEuJXHDzhMK+bFeQv3jRG1XhQDRa+1WFvjeBWBhr1C5KmGWcXPlZ3kCVc4uR2YAY6EMTXYrmD/dvvYhOnc0WbFJUaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; c=relaxed/simple; bh=Dj95o31GQqsBMKISFvlZUrwyZKzM8Z7Ckp1HHM8MZl8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RH2JmaofZCXg7Q2e6BJ91bNli5+WPvNBgJsuyusaWbpLkDzVT/7McjlsHkEGMLFRl/MIGiAWm05+7Yskkl+Nuc3mb44GMNPPE+fNY6Vej8qLuth13LNz7KmEvhokKvmg0WXsQrTKfjwjoR97tzwYbvMQJANwf+eDTAGgcW+x5fQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZUT9cVog; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZUT9cVog" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BF7A9C4CECE; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496846; bh=Dj95o31GQqsBMKISFvlZUrwyZKzM8Z7Ckp1HHM8MZl8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZUT9cVoguQMfcO4rnNpP/A0W+gScXmM3VBfvTC/+c5SezaK8kaq/I5lyq0TeaujxK CkxqUqQ+sDXzUuRngpx2Et4iTn0Lp6bwU2diUGhKi5U46CwEp4MoBniatz9dWTqdDE L01X4uExHij8kZaNlSKRrAqbYVDDTNOvov6T88do3qF6gr3T5svixkLZZgjcDxofMK 19CRBfk3Iz+cI8dXUOBFzfP6/HoZVbS96D7j0EyBwfjZx17NGSiYGz2CFFmEPvuInE PSCMytq76LcIqWCGYD9RlZAw8X62Rmzw2XAZsNqlU2ekGrLadFRU80AkQ/q10ddxbO wo4dNu/movwkA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 6BB0ACE0B68; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 2/7] doc: Add rcuog kthreads to kernel-per-CPU-kthreads.rst Date: Wed, 9 Oct 2024 11:00:40 -0700 Message-Id: <20241009180045.777721-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit adds the rcuog kthreads to the list of callback-offloading kthreads that can be affinitied away from worker CPUs. Signed-off-by: Paul E. McKenney --- Documentation/admin-guide/kernel-per-CPU-kthreads.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-per-CPU-kthreads.rst b/Documentation/admin-guide/kernel-per-CPU-kthreads.rst index b6aeae3327ceb..ea7fa2a8bbf0b 100644 --- a/Documentation/admin-guide/kernel-per-CPU-kthreads.rst +++ b/Documentation/admin-guide/kernel-per-CPU-kthreads.rst @@ -315,7 +315,7 @@ To reduce its OS jitter, do at least one of the following: to do. Name: - rcuop/%d and rcuos/%d + rcuop/%d, rcuos/%d, and rcuog/%d Purpose: Offload RCU callbacks from the corresponding CPU. From patchwork Wed Oct 9 18:00:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828940 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A9671E0DC5; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; cv=none; b=PX5jnZHLxWHWPOyAZH5Xrh9HQiCXlFi5xAAWecKZG895u/PnQPO/lxtHKQWfOvo8zoEP2235znLs6GfYHWVvLbPLQ9kvAACvwNG3uDvEgm+X2vtHHLdXJgstv+TTIKBuhyBzGu0UVDlIOx5k18hcitRX9gBbd2WKm1Ant7Mc2Ak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; c=relaxed/simple; bh=vpuIgx5kDOrF1tvN1wFdkRmFqkJZGZwsKVqT4sAsn8E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pXx1OyqytY0LolwZDHuxL5t/a4q8Ar8hmMTKEGrDk5JMxNFvfk8dofv460W23Ntdt2rpbB7wNFJK4fEiu/6+FWgXlj4jQhEhPgC8t7c6dMDu5AIGD1s8XLfa+pHs9+lWOhqCMQ3hbczQhLvPJS2soNKM7qEIrLR4hKsmLGqgK04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=vGcpGRHG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="vGcpGRHG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C9ACCC4CECF; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496846; bh=vpuIgx5kDOrF1tvN1wFdkRmFqkJZGZwsKVqT4sAsn8E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vGcpGRHGEXMF9dLRiTMFtMsfpal65nz3iH4dwnTOVDbZNXp/l/8IUf8yG85J/dJhb 7yWJY/K1pFKDQP97xf56Q36xSVy8JoHnCkO0bVKr0OAkePB9PsqzjXHZ5WAL1mZIRm 9SM5fTxx1D1P6aI4F0Bd0KHYOHmlgQGBnhNKd9sY0RG/REnBmW9sBY+3ifs6bCpkrb IEkBXd2wuq0d5dD5ZpI7CSq2OZYk8tq6F0sZQDvdJhC3RoXi1X05DbUBxoRJmPcz3U ivpP8uopy1/dpgXilpKAnjqMTE1wsXCUBVCBG7G8TT2tj7dmAaH3YKzz0En8KrkiJD ThTBKgT2m1efA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 6E7C8CE0BB1; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Peter Zijlstra , Neeraj Upadhyay Subject: [PATCH rcu 3/7] rcu: Allow short-circuiting of synchronize_rcu_tasks_rude() Date: Wed, 9 Oct 2024 11:00:41 -0700 Message-Id: <20241009180045.777721-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 There are now architectures for which all deep-idle and entry-exit functions are properly inlined or marked noinstr. Such architectures do not need synchronize_rcu_tasks_rude(), or will not once RCU Tasks has been modified to pay attention to idle tasks. This commit therefore allows a CONFIG_ARCH_HAS_NOINSTR_MARKINGS Kconfig option to turn synchronize_rcu_tasks_rude() into a no-op. To facilitate testing, kernels built by rcutorture scripting will enable RCU Tasks Trace even on systems that do not need it. [ paulmck: Apply Peter Zijlstra feedback. ] Signed-off-by: Paul E. McKenney Cc: Peter Zijlstra Signed-off-by: Neeraj Upadhyay --- kernel/rcu/tasks.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 6333f4ccf024b..dd9730fd44fbe 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -1398,7 +1398,8 @@ static void call_rcu_tasks_rude(struct rcu_head *rhp, rcu_callback_t func) */ void synchronize_rcu_tasks_rude(void) { - synchronize_rcu_tasks_generic(&rcu_tasks_rude); + if (!IS_ENABLED(CONFIG_ARCH_WANTS_NO_INSTR) || IS_ENABLED(CONFIG_FORCE_TASKS_RUDE_RCU)) + synchronize_rcu_tasks_generic(&rcu_tasks_rude); } EXPORT_SYMBOL_GPL(synchronize_rcu_tasks_rude); From patchwork Wed Oct 9 18:00:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828939 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A90F1957F8; Wed, 9 Oct 2024 18:00:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; cv=none; b=ZyK2lj0FA1zjrt7OPM+FPNLCRar6e0yLu/6ST1RBimtHOBsXa3DpXjQpGeBzxIzbtsiLK4CLWJozkADBFttKrD1heWlEcomTPCIAW1YnKvl87YRwr/JmHN50VGygbFNWd+ujl09HhWdkhlguaTy12NaZcidvNYi75Z700aUn+E0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; c=relaxed/simple; bh=OySrHGIYGBCnLfy5/68TxcrLmWNaXjEnRyRhtLieCyE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FIHotcgipeXAvktiLZrlCv0R3FVzaERoOhAx5j7E72SUdjV/ixr1EeRmQHrQtTAdE8xO/SZVQM9mOIHGoc1aXh6vSqS6inMUDY3DL97NATP3kH9InG9Sj4iZTZF57Zn766iU545/uGyeQf6r6HhB8S73WQdS+qFeGtArrrBWtPc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AmpiWNxj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AmpiWNxj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBF26C4CED1; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496846; bh=OySrHGIYGBCnLfy5/68TxcrLmWNaXjEnRyRhtLieCyE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AmpiWNxjj9ykcbMYQabAxAfCZDufS2YtyYEKPaXTrucrMCtyxx2TB5O7ffz0tVj6+ ISHEN6H1n+3shbX7hs+jFBv9Xrll3bRmuscBf/lJW9o3wIQ42STuf0ggSA6W/KHE9X TDPf751b9mjY4AGGw9kgF+g+xBx4hUTFHkbOle7gOrRulC0o7ygiqCoyV4a3WMMDS2 2tmLBABO8wXW++u/lkPAy+KvG36ileuhJSyflWtRk1dnTWNK4e5OJCgShiEnn0Yd1m UZISqF7jJWIVgsUIFMyyZmqB2UfegHPX9tN4qcvXGVcyhZzVL0IxNS9Vy0BIv5nyew znkaBQ6JQukOg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 7118FCE0DD1; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Kent Overstreet Subject: [PATCH rcu 4/7] rcu: Permit start_poll_synchronize_rcu*() with interrupts disabled Date: Wed, 9 Oct 2024 11:00:42 -0700 Message-Id: <20241009180045.777721-4-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The header comment for both start_poll_synchronize_rcu() and start_poll_synchronize_rcu_full() state that interrupts must be enabled when calling these two functions, and there is a lockdep assertion in start_poll_synchronize_rcu_common() enforcing this restriction. However, there is no need for this restrictions, as can be seen in call_rcu(), which does wakeups when interrupts are disabled. This commit therefore removes the lockdep assertion and the comments. Reported-by: Kent Overstreet Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index a60616e69b663..3d6507d0b8d33 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4193,7 +4193,6 @@ static void start_poll_synchronize_rcu_common(void) struct rcu_data *rdp; struct rcu_node *rnp; - lockdep_assert_irqs_enabled(); local_irq_save(flags); rdp = this_cpu_ptr(&rcu_data); rnp = rdp->mynode; @@ -4218,9 +4217,6 @@ static void start_poll_synchronize_rcu_common(void) * grace period has elapsed in the meantime. If the needed grace period * is not already slated to start, notifies RCU core of the need for that * grace period. - * - * Interrupts must be enabled for the case where it is necessary to awaken - * the grace-period kthread. */ unsigned long start_poll_synchronize_rcu(void) { @@ -4241,9 +4237,6 @@ EXPORT_SYMBOL_GPL(start_poll_synchronize_rcu); * grace period (whether normal or expedited) has elapsed in the meantime. * If the needed grace period is not already slated to start, notifies * RCU core of the need for that grace period. - * - * Interrupts must be enabled for the case where it is necessary to awaken - * the grace-period kthread. */ void start_poll_synchronize_rcu_full(struct rcu_gp_oldstate *rgosp) { From patchwork Wed Oct 9 18:00:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828943 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D1DC1E102C; Wed, 9 Oct 2024 18:00:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496848; cv=none; b=jzQrcsB+/ZdCDOjTX+yDIi2Fbh0POt8TixGZ4/Nb0MMvx9uNydoUCR9sF4rSGPgm2x/8g/pGqIOPQrv6f0YXRSSLOgtKFaFzPjwn+nYTo3vEw9g4f7nbVqDbfetoB6Rw/fk/o9hDcyLDuPSH1ByJrPrBNyT0EQm41ttnCc7RlJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496848; c=relaxed/simple; bh=eJja4NpVituNDp/ZVRfcYxyH6Blp+5M5FRFnEJOlmjc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sa+mmQNZ1NkygrnkmvUMH9iD6sSuHsIP7DKTXpS4ebY9uvVSqhoUzaLaTMyF+Z1TQAuswQFQEXzKkBequ1WluXqK3/x+TFtL+VfiDXkBTpDw3zWwBitR5tfqB7GEzSuc17hvIj1qZw/GgRrNwXueHxEn9w/PgysEWzr/ubm8QEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=C1WfJE70; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="C1WfJE70" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DD19BC4CED3; Wed, 9 Oct 2024 18:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496847; bh=eJja4NpVituNDp/ZVRfcYxyH6Blp+5M5FRFnEJOlmjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C1WfJE70RVlPaVlV1CFLPIvs5Zzg+sYaPi6qPaOT8QFEos1+hhdyZt98jERFCK9PW bjCl1GsE3PFRfj48lcdPOzEsYKolsKs0jjpMn21S7siY+QAgspgVkwPxKt5WVR3Fwc FwBVIWuxvZJMWwM4cEv2pZfk9EhwUr6SOguez4hSypfA8U+RyvaDhrGPr+ahJK1zg5 hADEL/XPwaZo+3KJGTJ4Q/mEns1sQwr0VEOG/giFKG/bQhcwwai7iuDlyYeer/4tgC JnSq9pD0y8yqw+g5STk+as5wattygG1+uiM3jvISPcgD44afZjfZAYRvRKNo7VJhFw 6yrd2zPh8t2ZQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 73B16CE0E61; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 5/7] rcutorture: Test start-poll primitives with interrupts disabled Date: Wed, 9 Oct 2024 11:00:43 -0700 Message-Id: <20241009180045.777721-5-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit tests the ->start_poll() and ->start_poll_full() functions with interrupts disabled, but only for RCU variants setting the ->start_poll_irqsoff flag. Signed-off-by: Paul E. McKenney --- kernel/rcu/rcutorture.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c index bb75dbf5c800c..b4cb7623a8bfc 100644 --- a/kernel/rcu/rcutorture.c +++ b/kernel/rcu/rcutorture.c @@ -393,6 +393,7 @@ struct rcu_torture_ops { int slow_gps; int no_pi_lock; int debug_objects; + int start_poll_irqsoff; const char *name; }; @@ -581,6 +582,7 @@ static struct rcu_torture_ops rcu_ops = { .can_boost = IS_ENABLED(CONFIG_RCU_BOOST), .extendables = RCUTORTURE_MAX_EXTEND, .debug_objects = 1, + .start_poll_irqsoff = 1, .name = "rcu" }; @@ -1695,14 +1697,22 @@ rcu_torture_fakewriter(void *arg) cur_ops->cond_sync_exp_full(&gp_snap_full); break; case RTWS_POLL_GET: + if (cur_ops->start_poll_irqsoff) + local_irq_disable(); gp_snap = cur_ops->start_gp_poll(); + if (cur_ops->start_poll_irqsoff) + local_irq_enable(); while (!cur_ops->poll_gp_state(gp_snap)) { torture_hrtimeout_jiffies(torture_random(&rand) % 16, &rand); } break; case RTWS_POLL_GET_FULL: + if (cur_ops->start_poll_irqsoff) + local_irq_disable(); cur_ops->start_gp_poll_full(&gp_snap_full); + if (cur_ops->start_poll_irqsoff) + local_irq_enable(); while (!cur_ops->poll_gp_state_full(&gp_snap_full)) { torture_hrtimeout_jiffies(torture_random(&rand) % 16, &rand); From patchwork Wed Oct 9 18:00:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828942 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60BD21E1029; Wed, 9 Oct 2024 18:00:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; cv=none; b=ZMzknIkVbrfTf7zS0TXhsH4U7WF6HTX0ZjwfqpnMWmatVLHjWW7CzjT51FpNB1yMrfUPeqAD0catzIZoq8TewkHKHv9riF+gcB7mGTmwjW6cuBP/w72Noz/S9sqb1lY6xSwGid5s5KxtE+PfhUbC+JaeJh+ZplRvoFFk/1lCvz0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; c=relaxed/simple; bh=azYSd/1FItgA8eDMglhxA/9kRsVFeYhI6Q2ye0pzl0I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TLooZnTusttCR+cRRPLNBw7QLGGL3oz9Mn6xKamwiCqdzxig+BnSS4DjLhZx6OnoV4KIuyGqV33JiI5kJazUZ9Y3hx5vKX0b4d8fNLJDFJ73tx3JYXoGDFogryWYQWTyKGZ/7nwPs1/bU6sGLf7LGCfbZR9Yy4NHO8L8UxiF3XY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jqtMcOxZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jqtMcOxZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 146B2C4CED6; Wed, 9 Oct 2024 18:00:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496847; bh=azYSd/1FItgA8eDMglhxA/9kRsVFeYhI6Q2ye0pzl0I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jqtMcOxZFauyLdBhGtTU6TwMB66yHev4zgKdN9z8f5K15+Wezny/IBHIWighLakQA grAEeYOyWXTN6ih46N6sLeLCOSKt9D8SMDjnHD5mcrzmWYDCFHV3octz233QR92y7+ 8vk5+r11EFt12r8F5hylIP4EQByWWBJQ/9/s9P37nu2pfjM08yGkDLvtWPYUnwZxt8 Na2CBHNPrPMnq7lNpbPizjZ7X+xYNiPu3xPvD4vFW7cdEnrWngFaCdOnTz2DdrsA57 doO6pL4vHAjo/5MOMRpuoJITb6jI94pYAKTh8Z7ucnwcPWXpAatrvhGoY0IqkjWmd6 Avq5+d4v8Pzdg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 769F1CE10CB; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Alexei Starovoitov , Andrii Nakryiko , Peter Zijlstra , Kent Overstreet , bpf@vger.kernel.org Subject: [PATCH rcu 6/7] doc: Remove kernel-parameters.txt entry for rcutorture.read_exit Date: Wed, 9 Oct 2024 11:00:44 -0700 Message-Id: <20241009180045.777721-6-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 There is only ever the one read-exit task, and there is no module parameter named rcutorture.read_exit, so remove the bogus documentation. Instead, use rcutorture.read_exit_burst to enable/disable read-exit race testing. Signed-off-by: Paul E. McKenney Cc: Alexei Starovoitov Cc: Andrii Nakryiko Cc: Peter Zijlstra Cc: Kent Overstreet Cc: --- Documentation/admin-guide/kernel-parameters.txt | 5 ----- 1 file changed, 5 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 1518343bbe223..7edc5a5ba9c98 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5412,11 +5412,6 @@ Set time (jiffies) between CPU-hotplug operations, or zero to disable CPU-hotplug testing. - rcutorture.read_exit= [KNL] - Set the number of read-then-exit kthreads used - to test the interaction of RCU updaters and - task-exit processing. - rcutorture.read_exit_burst= [KNL] The number of times in a given read-then-exit episode that a set of read-then-exit kthreads From patchwork Wed Oct 9 18:00:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13828941 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62F401E133D; Wed, 9 Oct 2024 18:00:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; cv=none; b=e2FOnXyXXvLJKAGKGxDPeJXqA0rNFXBxzadbVf/FqB+soKdM3p4JV73IQTFDhKJJXKYcRyXQq8THvYDX3wgFkzMN/4G4JZ8SUFVe7Sa54nvjwRFanKPgnQvXZ2c+U6o+JiyNnJad221vwBZ/uCrRk+/go6qPoJTHKt6FofvbsM8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728496847; c=relaxed/simple; bh=BE0VZPoXza7bZDtTDVSPUqFCnHF2OgmKDYBLxOWLPvQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uhEMFH/9Q3KPnTlJQ9cY9CKt5Q/PO+StT1JzU1nep8U8NhounOoEUow2AxNUq4ZAw9tojNCXGqqsETQ9xJz7gXDkhWamrRfNscmTxHoE8FU8fcAKbcL/JXHbm4FZXKntG5GUWxfRGOVa2bJnUMhfv7Vs2PrimhQkv8DFiA6kA88= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cSgb6JLC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cSgb6JLC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2241FC4CED7; Wed, 9 Oct 2024 18:00:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728496847; bh=BE0VZPoXza7bZDtTDVSPUqFCnHF2OgmKDYBLxOWLPvQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cSgb6JLCD6/eZ6VcL/J+ttiZoKnA5InYMWxGf6IZNG58AcF56ux1pTX+3yxV0ETPm mAMIJyl5Vn3EIprs4C5J27iNr1aQGVFo/pFLTkW//STjNPDAl0w0Eaiu4FR/UfBg4k n673YC9AU7ubzxYb+cy7MHGwy8kTpR3HMgNEFpYozYuTRJZfIGZXsnny6rPvtsITGq on3guSxt3/Y3UC6wXnA6GqVEcdudq0N2sJxBHRzxt3UEO16gS73IZgartBEjypvBqI AHCd/xZJNtNmO14X/nWysa+8krybY+HOZfJajf/OpnBghzZBgdVHLcJmZJcrL2rri2 LQ5VGRda2LrnA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 790CCCE1290; Wed, 9 Oct 2024 11:00:46 -0700 (PDT) From: "Paul E. McKenney" To: frederic@kernel.org, rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu 7/7] rcu-tasks: Remove open-coded one-byte cmpxchg() emulation Date: Wed, 9 Oct 2024 11:00:45 -0700 Message-Id: <20241009180045.777721-7-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> References: <2dc6de07-374d-44da-82e9-fa9d9c516b46@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit removes the open-coded one-byte cmpxchg() emulation from rcu_trc_cmpxchg_need_qs(), replacing it with just cmpxchg() given the latter's new-found ability to handle single-byte arguments across all architectures. Signed-off-by: Paul E. McKenney --- kernel/rcu/tasks.h | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index dd9730fd44fbe..c789d994e7ebc 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -1541,22 +1541,7 @@ static void rcu_st_need_qs(struct task_struct *t, u8 v) */ u8 rcu_trc_cmpxchg_need_qs(struct task_struct *t, u8 old, u8 new) { - union rcu_special ret; - union rcu_special trs_old = READ_ONCE(t->trc_reader_special); - union rcu_special trs_new = trs_old; - - if (trs_old.b.need_qs != old) - return trs_old.b.need_qs; - trs_new.b.need_qs = new; - - // Although cmpxchg() appears to KCSAN to update all four bytes, - // only the .b.need_qs byte actually changes. - instrument_atomic_read_write(&t->trc_reader_special.b.need_qs, - sizeof(t->trc_reader_special.b.need_qs)); - // Avoid false-positive KCSAN failures. - ret.s = data_race(cmpxchg(&t->trc_reader_special.s, trs_old.s, trs_new.s)); - - return ret.b.need_qs; + return cmpxchg(&t->trc_reader_special.b.need_qs, old, new); } EXPORT_SYMBOL_GPL(rcu_trc_cmpxchg_need_qs);