From patchwork Wed Nov 6 14:59:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF28D202F89; Wed, 6 Nov 2024 14:59:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905160; cv=none; b=NOwOUkMxQTz1ZvsNnFtxNX5vyOxHsl1qU+FCcX1SPiqq6jVM7tuXftuWZRCxiQQbbuqOGPuSajxXyM4gprEMFC+sVB/RdvmNkeYntJJvXXT5X6oUDQXbviLlPEL95Npgo7Her6wQkobUt2Mm50pljZ7v4LuRu8DITUOncy5xyTE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905160; c=relaxed/simple; bh=n40xhh0z9NziEa64CxH5rGePLUJObsn4LcYAfAzqI9o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gfznepEeOJwjA4XMHxqvhc9LywWTfphJSBdk7Bw8bl6m1rjs2Y4lfSXFHRB/vB+eZf8D5WjpExeC8T1sSuZttAHhEuwaLwkxpt0DI4MSz7Jqrq+hv8YC++R/GupIJtOQHo4zV9OIfWT6ONZUIW9bgdJ91n9KFs3cuqK/705Io3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=df/OymoN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="df/OymoN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 102F2C4CED4; Wed, 6 Nov 2024 14:59:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905160; bh=n40xhh0z9NziEa64CxH5rGePLUJObsn4LcYAfAzqI9o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=df/OymoNcVHVKFvXcpxp8QpKO87gY9UmfjHUoIPanFV4Eq1R21/bnhO175jRfoX4O SNCkHgAWUGJKFgrnkdaMCLndb8BH8emxi38GAqROXc+ML3qZSgOFZ0NVGLIWnR8z1A pyDXUZJC2AXyK2Ar4QazCAL7GcVk261Ry7LgYmxauOJDqmq+CqnrL4N2eqrVAgyZXh s8njYZ2BQiG20TpBIfV3nqKR/nZsA67kK7m1Je2WvC65FI8MQ63bBAS26HhBmMr2jT QucXDp1sUHi3AKEgDkA3ZR2LM9urhgAABLChW+brmEzbYZbbqVOvYwIuRcF6C5HIrv CVJbnAj1dg89g== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Frederic Weisbecker Subject: [PATCH 1/8] doc: Add rcuog kthreads to kernel-per-CPU-kthreads.rst Date: Wed, 6 Nov 2024 15:59:04 +0100 Message-ID: <20241106145911.35503-2-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" This commit adds the rcuog kthreads to the list of callback-offloading kthreads that can be affinitied away from worker CPUs. Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- Documentation/admin-guide/kernel-per-CPU-kthreads.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-per-CPU-kthreads.rst b/Documentation/admin-guide/kernel-per-CPU-kthreads.rst index b6aeae3327ce..ea7fa2a8bbf0 100644 --- a/Documentation/admin-guide/kernel-per-CPU-kthreads.rst +++ b/Documentation/admin-guide/kernel-per-CPU-kthreads.rst @@ -315,7 +315,7 @@ To reduce its OS jitter, do at least one of the following: to do. Name: - rcuop/%d and rcuos/%d + rcuop/%d, rcuos/%d, and rcuog/%d Purpose: Offload RCU callbacks from the corresponding CPU. From patchwork Wed Nov 6 14:59:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10412202625; Wed, 6 Nov 2024 14:59:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905164; cv=none; b=ZpJ2NTNfakG0Xy0UueCnnh4EvCzyR7uKTnVPa1eecIF1CZ5yhvtccP17apkNYlfylsM0SbPZSenzydC8GHpImeVDOJXi6k9b7fCNI6FEtCICA2CeaZCoUqSNLo8LfK0dtM5ApIR7V4sF2jQcWi9JkwTha1w/3qeWlLubD1F/uWg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905164; c=relaxed/simple; bh=+ZiepWhi94NGo7vByz0jA+dg9Rjr5KkBmO7yVbgAD8I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=k2wDcELkjEhkiHRsjd1f+ZjdW5RmDNS29Y/P2zt/HEAoFuA4HFasqY0nAe44d3539yfcQTIWXu3xwUpxh79XKuYLwg99M11Aw94EC/Qx92n4sB9rCcawUVLMlbVtCXOb1XjnvXdQOS4WDwfHHrRpt+e6Bj1RkxCMnXHrMhi8xmA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=S1sGyXur; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="S1sGyXur" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C5789C4CED5; Wed, 6 Nov 2024 14:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905163; bh=+ZiepWhi94NGo7vByz0jA+dg9Rjr5KkBmO7yVbgAD8I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S1sGyXurEsjXhdxOTwtcP5dAZR3ha1gmmQYBTLGYuoVLWop6t74Vi/celdwHeWkpJ xUBU734TXM8UwCwnpQPfPybrFRpVholT+ut5DJmpN8H0TyXCNx/i2tvoOaOon6z4Ah alBXGnGdEOqh7fN0lzWHMF1tOMhiEUheITMZAPEBKyiuLILR8NgQSf6J6iJjlGDZwb 9GFRyNshlDnCfHtGW95b6ns+1/VbGg2nlnQPto51ALVtCtu3E++CFAdugsJd765SzK 2eUZSkwl90Q8NawA+RXxuZC7SrGBJ/bfMVORkcVtTSh/1zk+XCr1jXSFTSg+G7WHR0 XAZq3Kclxnybg== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Peter Zijlstra , Neeraj Upadhyay , Frederic Weisbecker Subject: [PATCH 2/8] rcu: Allow short-circuiting of synchronize_rcu_tasks_rude() Date: Wed, 6 Nov 2024 15:59:05 +0100 Message-ID: <20241106145911.35503-3-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" There are now architectures for which all deep-idle and entry-exit functions are properly inlined or marked noinstr. Such architectures do not need synchronize_rcu_tasks_rude(), or will not once RCU Tasks has been modified to pay attention to idle tasks. This commit therefore allows a CONFIG_ARCH_HAS_NOINSTR_MARKINGS Kconfig option to turn synchronize_rcu_tasks_rude() into a no-op. To facilitate testing, kernels built by rcutorture scripting will enable RCU Tasks Trace even on systems that do not need it. [ paulmck: Apply Peter Zijlstra feedback. ] Signed-off-by: Paul E. McKenney Cc: Peter Zijlstra Signed-off-by: Neeraj Upadhyay Signed-off-by: Frederic Weisbecker --- kernel/rcu/tasks.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index 6333f4ccf024..dd9730fd44fb 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -1398,7 +1398,8 @@ static void call_rcu_tasks_rude(struct rcu_head *rhp, rcu_callback_t func) */ void synchronize_rcu_tasks_rude(void) { - synchronize_rcu_tasks_generic(&rcu_tasks_rude); + if (!IS_ENABLED(CONFIG_ARCH_WANTS_NO_INSTR) || IS_ENABLED(CONFIG_FORCE_TASKS_RUDE_RCU)) + synchronize_rcu_tasks_generic(&rcu_tasks_rude); } EXPORT_SYMBOL_GPL(synchronize_rcu_tasks_rude); From patchwork Wed Nov 6 14:59:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05795202625; Wed, 6 Nov 2024 14:59:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905168; cv=none; b=QVktJGZZ4X9aNgXoCOCZuc5o7HsR1bvb/RWQgBEezd7N1TMncIRMzRkJEw4OEpkObz74HHnz54bTwD5gB1enD5CkzMlJOnV9hlX2b9w5sJSnfEGnWoFnqm822Q5gjCkUHVymHV56Ib4T7Mjv9+TKZa+hM1TC+KELJt50Vwof8SA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905168; c=relaxed/simple; bh=qA/c0N3rIn1d+527roOwF7bl5h7TXeVRuH3DxgNRebw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pf2sAHjoCUhREIK2chhTSiRqY6TcscfdTLw9vLhpW9JjvbsXteXLGUSqknPHUXQIffRh7JhMvwYxgySUY4Cyor/IeRYO57kOLWj8wGlYh4d/g2F7sxmO18woeI6g/EfbLV6wV6rU887WaVvDXu0OOGfvHqFTvF24YXUDryjbx8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kiq8Et49; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kiq8Et49" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C369C4CEC6; Wed, 6 Nov 2024 14:59:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905166; bh=qA/c0N3rIn1d+527roOwF7bl5h7TXeVRuH3DxgNRebw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kiq8Et49D6w50kLceRxq1pVCsNKk+lrWWZpW+ScNk4nVPEwlWESeMhuHe4c5dLEK4 RvXRaXOTbrPgOgN/lCXkF3kiPmjbWrOy6lv5BJeiMs+unBf6Z4ybnmPvFHa1NLrJ4B TQ/MN6SksZjvxa1f9jQhXGCtqBMmbhfllIROHvvrmGqWRuJiuDaPzFrqPhY6aKPT8+ WKJzYDngpAxPXmc3txEbpQIltaooKSIYjhpy+1wFH5Nk0WBXdJQd7XcJ6Fg7XBZ8dP G9MvZjix/EGr4vhrU4p/2Gr2LgECwlfN5+VDyacYmpa9FTkgXpijBoMz72z+tuT/uz WVhiUsVB6q86g== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Kent Overstreet , Frederic Weisbecker Subject: [PATCH 3/8] rcu: Permit start_poll_synchronize_rcu*() with interrupts disabled Date: Wed, 6 Nov 2024 15:59:06 +0100 Message-ID: <20241106145911.35503-4-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" The header comment for both start_poll_synchronize_rcu() and start_poll_synchronize_rcu_full() state that interrupts must be enabled when calling these two functions, and there is a lockdep assertion in start_poll_synchronize_rcu_common() enforcing this restriction. However, there is no need for this restrictions, as can be seen in call_rcu(), which does wakeups when interrupts are disabled. This commit therefore removes the lockdep assertion and the comments. Reported-by: Kent Overstreet Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 38b121a77366..13829cf38f52 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4194,7 +4194,6 @@ static void start_poll_synchronize_rcu_common(void) struct rcu_data *rdp; struct rcu_node *rnp; - lockdep_assert_irqs_enabled(); local_irq_save(flags); rdp = this_cpu_ptr(&rcu_data); rnp = rdp->mynode; @@ -4219,9 +4218,6 @@ static void start_poll_synchronize_rcu_common(void) * grace period has elapsed in the meantime. If the needed grace period * is not already slated to start, notifies RCU core of the need for that * grace period. - * - * Interrupts must be enabled for the case where it is necessary to awaken - * the grace-period kthread. */ unsigned long start_poll_synchronize_rcu(void) { @@ -4242,9 +4238,6 @@ EXPORT_SYMBOL_GPL(start_poll_synchronize_rcu); * grace period (whether normal or expedited) has elapsed in the meantime. * If the needed grace period is not already slated to start, notifies * RCU core of the need for that grace period. - * - * Interrupts must be enabled for the case where it is necessary to awaken - * the grace-period kthread. */ void start_poll_synchronize_rcu_full(struct rcu_gp_oldstate *rgosp) { From patchwork Wed Nov 6 14:59:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865027 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7BFA20402B; Wed, 6 Nov 2024 14:59:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905170; cv=none; b=Izfq3nRx6tON2ErMI5+VIZyZxdQKy3NpQr1S9bB8L9TaLljV6rICqrHG0kzyMwulZL/mYoznJm0VtDAXeVqKu4AJTcOdnhn1P5K+1PdArjtqFxde4kpQ54iRHzpeURvqHSXjqIz+ajJ2+rfUeBnyMaFJ4exXIhZyClL4J+g/c6I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905170; c=relaxed/simple; bh=KfEzAK0yWwsc73/eBMwT2Rzo8Q/O9jbmqLzgwx89IvU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hoGXYYPJecIkrwrMu1r1aXmp0ReXN8k96BXk7jASgQWzmtaedrYw7Fc/erLpENq1lBaIpQbbVhsYHq9nZRR6XfCGYAE+LWR3UCGsE7MmG42urB66YvT3dFaa33ocEiTVZcfySyi2cFZCDt2oqD9ziiiD5hEeN1GwYKt9siT/TF4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lXqcBxcX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lXqcBxcX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9551C4CED4; Wed, 6 Nov 2024 14:59:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905169; bh=KfEzAK0yWwsc73/eBMwT2Rzo8Q/O9jbmqLzgwx89IvU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lXqcBxcXkyn27Puwj3HRxr8vRWOX/LJt+EyevkZ32fB6o36U9w6tYd+y7fwA60i0f ixoUOvFQ+uoQyFHSUQL1iFoZgNrB/GkZZd1TzuE0TJ3n/Ip/k8zK00j00WYwF/17cB rY3438l1WpUAJ2zmUjr3uOxMv45pPBI2aYXp7IaIGDRI0ouPQOeI00bYmtVGGWMzhD /AV6nJfaIXqFYOOHDK7sLacADfIpCUA/CwNq7+CuWFmnwE2BqlM2BLfEGLPrtMD+0S aGIcsGyd85CgGqhjqOOZXoEFGoHHtXGbHdU6ZWVmcG3/7zDbZUF7NSWChlRUUr3cfI klYFlwEqLh3PA== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Frederic Weisbecker Subject: [PATCH 4/8] rcutorture: Test start-poll primitives with interrupts disabled Date: Wed, 6 Nov 2024 15:59:07 +0100 Message-ID: <20241106145911.35503-5-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" This commit tests the ->start_poll() and ->start_poll_full() functions with interrupts disabled, but only for RCU variants setting the ->start_poll_irqsoff flag. Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/rcutorture.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c index bb75dbf5c800..b4cb7623a8bf 100644 --- a/kernel/rcu/rcutorture.c +++ b/kernel/rcu/rcutorture.c @@ -393,6 +393,7 @@ struct rcu_torture_ops { int slow_gps; int no_pi_lock; int debug_objects; + int start_poll_irqsoff; const char *name; }; @@ -581,6 +582,7 @@ static struct rcu_torture_ops rcu_ops = { .can_boost = IS_ENABLED(CONFIG_RCU_BOOST), .extendables = RCUTORTURE_MAX_EXTEND, .debug_objects = 1, + .start_poll_irqsoff = 1, .name = "rcu" }; @@ -1695,14 +1697,22 @@ rcu_torture_fakewriter(void *arg) cur_ops->cond_sync_exp_full(&gp_snap_full); break; case RTWS_POLL_GET: + if (cur_ops->start_poll_irqsoff) + local_irq_disable(); gp_snap = cur_ops->start_gp_poll(); + if (cur_ops->start_poll_irqsoff) + local_irq_enable(); while (!cur_ops->poll_gp_state(gp_snap)) { torture_hrtimeout_jiffies(torture_random(&rand) % 16, &rand); } break; case RTWS_POLL_GET_FULL: + if (cur_ops->start_poll_irqsoff) + local_irq_disable(); cur_ops->start_gp_poll_full(&gp_snap_full); + if (cur_ops->start_poll_irqsoff) + local_irq_enable(); while (!cur_ops->poll_gp_state_full(&gp_snap_full)) { torture_hrtimeout_jiffies(torture_random(&rand) % 16, &rand); From patchwork Wed Nov 6 14:59:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865028 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48FF6202F8A; Wed, 6 Nov 2024 14:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905173; cv=none; b=X7gjsKBC0ZBv6vPfNe9JywQFAd9bfV4x6+WBUCqP/C2bEUUcXbxuGUVfg9DWx0A6S2WqPDWNtRIycuu5eL3eaMPCbgpZDp6diErY+07zBlNLSg3ECt5J0YAsAVUUGQGwrlUvtEBmACGWkIiAIXepk/xzw57sk5wLjzNcIidHj5c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905173; c=relaxed/simple; bh=oY9T9tNg446qMb0aOnJdQRe0Ot6PCpEIBaCDhmC46zo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=D1mICmmGKlsvwGb2QkZD5xvPkYdrgrRJemOYoN8pQ8jKJbrBoKYHh/LFsQbWtepKPHUrrnAwoNHgef+QEopNZSNP9RZf+atybh8pV009O1Kk1rx0MFP47JDh9HH5XZnczvIZUSSRjjw+b7Hzzi+u2e9JcWG08FeXpo+FsTXbFAI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LURUJwI2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LURUJwI2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF11CC4CED3; Wed, 6 Nov 2024 14:59:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905173; bh=oY9T9tNg446qMb0aOnJdQRe0Ot6PCpEIBaCDhmC46zo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LURUJwI23qRr870S7hjbwt/ofN95Wh8dC6YJCCpcHLfmp/OBIVTtw6/M/cNRSwjFb lXOjZVApBfPE2SA+3xS+pmBoQ9gNmNQz8RKeewsplPgjNi5ilG6AW13hrMuBbrO3Ux j0TyA8UMJoSw3bBmA6E0y68rK/aIgju4B1MVIlq44XgxnugknPUh9R4tSkq0UQvIrG CcVcR8gIEVzO5QIw38D5ZElsu4nu8yIM4RYa8NLzH352A3guBVXVG9zv/pcQZA8O6X YHFdbovfyad/Ti43O7gb4UAXdqucyaK8m/k/LOFlNVW0tnenFDqu2vztpPKsEulZhO qGrOXGBbtMzsA== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Alexei Starovoitov , Andrii Nakryiko , Peter Zijlstra , Kent Overstreet , bpf@vger.kernel.org, Frederic Weisbecker Subject: [PATCH 5/8] doc: Remove kernel-parameters.txt entry for rcutorture.read_exit Date: Wed, 6 Nov 2024 15:59:08 +0100 Message-ID: <20241106145911.35503-6-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" There is only ever the one read-exit task, and there is no module parameter named rcutorture.read_exit, so remove the bogus documentation. Instead, use rcutorture.read_exit_burst to enable/disable read-exit race testing. Signed-off-by: Paul E. McKenney Cc: Alexei Starovoitov Cc: Andrii Nakryiko Cc: Peter Zijlstra Cc: Kent Overstreet Cc: Signed-off-by: Frederic Weisbecker --- Documentation/admin-guide/kernel-parameters.txt | 5 ----- 1 file changed, 5 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 1518343bbe22..7edc5a5ba9c9 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5412,11 +5412,6 @@ Set time (jiffies) between CPU-hotplug operations, or zero to disable CPU-hotplug testing. - rcutorture.read_exit= [KNL] - Set the number of read-then-exit kthreads used - to test the interaction of RCU updaters and - task-exit processing. - rcutorture.read_exit_burst= [KNL] The number of times in a given read-then-exit episode that a set of read-then-exit kthreads From patchwork Wed Nov 6 14:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865029 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B6CB204F96; Wed, 6 Nov 2024 14:59:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905176; cv=none; b=bSPBbjEI51dSr/PS5+9Qxx98z8lz3it54QfFiJdtGPusrzMuAZ+zmkAzeBZyr64ikQr6yc3kZe+1t01OkuRlMwmX2N6xQ3mSpNWD7LRjkAWEvEHYB+oEbYGkr3lqPP8997QRLLGidGgHn1NZR5cSAD0qPODXSmetl2slLDH6NYw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905176; c=relaxed/simple; bh=mtdPPDok0LE6Co+ZnXJpGWxQCPVzxrwR26bqD2LgnTI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PESlHCKa5WiYPMlEkoY130LRo9BLlM7b1p2YZGRCAbW2ZAym8jre3u+ibMiMgJmOlJd4CZwjw9GTSvlZ/EWzc5kkXH0ENuiATCwOYNyICtJWUydFB81X/qXg/07pfD46Qzaa2Y/XQtEyOUECNYxau38/+59/H73fRIAqqmVJTCA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NBoMPArA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NBoMPArA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8B796C4CECD; Wed, 6 Nov 2024 14:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905175; bh=mtdPPDok0LE6Co+ZnXJpGWxQCPVzxrwR26bqD2LgnTI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NBoMPArArXziFiODkWMo3Q4cjM0j3yc+9XzDF5qPCwia2cAz4I92VHl4TL1Ufah9/ Zs1ys+mghx86xoubxR+0D3ljwHezb6YFMR91K7A7WPcvt50jLTeV0saO0TgtkcfORw bmmb/pVjXz3RL5IOKG78967AuV7L9+L21SPBQXeeTif/b0c9IlEzZIGfoSAq8VqNu9 TKvrBDj2a/QDQetwXcGbPhwwPccJXPm0JJyA991ebL4Dx8V8/B/2+lNnjd3/TvDrSm Wkyjjt2jnFxJp1eQRKIsWtH8Sl5knTatY7dHdyzRX9BusW5BKPTo6MIW6lVFO+GbTN xuvh8vT0xLXLg== From: Frederic Weisbecker To: LKML Cc: "Paul E. McKenney" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Frederic Weisbecker Subject: [PATCH 6/8] rcu-tasks: Remove open-coded one-byte cmpxchg() emulation Date: Wed, 6 Nov 2024 15:59:09 +0100 Message-ID: <20241106145911.35503-7-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Paul E. McKenney" This commit removes the open-coded one-byte cmpxchg() emulation from rcu_trc_cmpxchg_need_qs(), replacing it with just cmpxchg() given the latter's new-found ability to handle single-byte arguments across all architectures. Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/rcu/tasks.h | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index dd9730fd44fb..c789d994e7eb 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -1541,22 +1541,7 @@ static void rcu_st_need_qs(struct task_struct *t, u8 v) */ u8 rcu_trc_cmpxchg_need_qs(struct task_struct *t, u8 old, u8 new) { - union rcu_special ret; - union rcu_special trs_old = READ_ONCE(t->trc_reader_special); - union rcu_special trs_new = trs_old; - - if (trs_old.b.need_qs != old) - return trs_old.b.need_qs; - trs_new.b.need_qs = new; - - // Although cmpxchg() appears to KCSAN to update all four bytes, - // only the .b.need_qs byte actually changes. - instrument_atomic_read_write(&t->trc_reader_special.b.need_qs, - sizeof(t->trc_reader_special.b.need_qs)); - // Avoid false-positive KCSAN failures. - ret.s = data_race(cmpxchg(&t->trc_reader_special.s, trs_old.s, trs_new.s)); - - return ret.b.need_qs; + return cmpxchg(&t->trc_reader_special.b.need_qs, old, new); } EXPORT_SYMBOL_GPL(rcu_trc_cmpxchg_need_qs); From patchwork Wed Nov 6 14:59:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865030 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0DF2201244; Wed, 6 Nov 2024 14:59:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905179; cv=none; b=NYysOaEbcyt7fH9oaq4p5XJpbWsZPLjWPyRaHZSP/VbQwmsVh795fq2D7EQ05vaCDdJbpxVWQfrHLdGwWdPTCA9E/2DYBYZLDLjjcaL8uYvsvJBWq/lh7DyzTQd6fVwRtlk7gs8t49RPcnlOMUGce4f293xd6RHdxFu6U/EmGLw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905179; c=relaxed/simple; bh=NCzmn7UNI9ILCBFE/qKxLNFeFkK0TChbvdeR5cTakVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IFf2ec3EH1mHDBOMcFd2sGSuErET+KDSovWv6bC2gs2fwjazlrVaL2f8QgHsnUDX2WwhbjC5ss8Ik60lmOdsIacHUAf7Sl3Yj/P6ZQzDo5cK7qs4u4L+I8LmdPBh0p/kZw40Z9J1lzV1s/vws3KCKrnglAllF3hzc85vdDmy1Fs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZdsYGEP0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZdsYGEP0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 56083C4CECC; Wed, 6 Nov 2024 14:59:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905178; bh=NCzmn7UNI9ILCBFE/qKxLNFeFkK0TChbvdeR5cTakVE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZdsYGEP07gLWRwz+PSqs+MGoqrU4Z405Le/QRS3thhITgodfGdlcxwlSm0qlhOyNz PU9G+xEKjhtJk7iTK0r5J3FD8o+kG1LNT5zZsyEgIE95g9biS7SZJ6EmLX8Hx5gl6m yRRhs9a/xN4G1o3kUNbqSCwfoyZm6yn0nK7Y6sZFEN3XXrK0jdm8fjL/OdtInAs+wZ YP64xEKj0eyD3HUuO6m+cNd1l7BuuRrVNhdXgyNB3bvXMdr7SYcKeE+5WuG0aRAxDy 1QG7DBgO43qAjcIW4KytjVifNfoigNCCap9DbZK+cl/9qO401OAgTlqv6k99UAS1CY MB5YgjJNUXTzA== From: Frederic Weisbecker To: LKML Cc: Michal Schmidt , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , Frederic Weisbecker Subject: [PATCH 7/8] rcu/srcutiny: don't return before reenabling preemption Date: Wed, 6 Nov 2024 15:59:10 +0100 Message-ID: <20241106145911.35503-8-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michal Schmidt Code after the return statement is dead. Enable preemption before returning from srcu_drive_gp(). This will be important when/if PREEMPT_AUTO (lazy resched) gets merged. Fixes: 65b4a59557f6 ("srcu: Make Tiny SRCU explicitly disable preemption") Reviewed-by: Paul E. McKenney Signed-off-by: Michal Schmidt Signed-off-by: Frederic Weisbecker --- kernel/rcu/srcutiny.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/srcutiny.c b/kernel/rcu/srcutiny.c index 549c03336ee9..4dcbf8aa80ff 100644 --- a/kernel/rcu/srcutiny.c +++ b/kernel/rcu/srcutiny.c @@ -122,8 +122,8 @@ void srcu_drive_gp(struct work_struct *wp) ssp = container_of(wp, struct srcu_struct, srcu_work); preempt_disable(); // Needed for PREEMPT_AUTO if (ssp->srcu_gp_running || ULONG_CMP_GE(ssp->srcu_idx, READ_ONCE(ssp->srcu_idx_max))) { + preempt_enable(); return; /* Already running or nothing to do. */ - preempt_enable(); } /* Remove recently arrived callbacks and wait for readers. */ From patchwork Wed Nov 6 14:59:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 13865031 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F008206045; Wed, 6 Nov 2024 14:59:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905182; cv=none; b=RxSqPfCqEET3StDIN8etLugEQSqeWH0GkZkaCrRh0bSsGCeTybqwp9tG79+IcAkNs1tlEgox5TwbE0JTh7RPtX2L8kkhn0mFt74p9oSBWiivYXfkTcSrdPuEI/tG5RaMAfGlkZ/OfrHybh8rfx0SI61gFSwwPcgAt7HrCEjUWRg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730905182; c=relaxed/simple; bh=EKZK3XBXXX+F3Ao8jAgqUC+OMm945u1cnOH3t6tzhJ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GJ/wIpjnChix6KHXZZ6hDs5KMmB8oAwQ//gCpiAI88lAdfpsTLDM8ymN+RDWivqOqInrmsg/r0aDvgBjdibXdAXB/NqaabphS7rMqg4kVQYT5Ox4KHk5VwktamDNpJjP/S8+36Y1XaW4HpdSZ8aV/JigOxrdHIucNib2fHxZaEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UFP7uzod; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UFP7uzod" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 56D47C4CECD; Wed, 6 Nov 2024 14:59:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730905181; bh=EKZK3XBXXX+F3Ao8jAgqUC+OMm945u1cnOH3t6tzhJ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UFP7uzodl03B5rjR8BcUJOub3klrGH4MH+KuAYjOvnDDzwSFTtpZvA/8JDO7pcTCB XNkuWYpYUbent8R4X2ZEcxguzOZBb5HenQEuUr3nkhoB/w+K7e2rvoz/R7WvywDMfn SGO8SWzqJX6k1puLBVd6d/QCQlS6zuD3QS/TWTi68s0/iDPQEhXCleBgob9KKMxta/ SVclWI55E20ptx/ccLPO16mJ/q/B48MYRToZNU1ebYr63lE+s6UUI1cEdNfIODsoO9 +2qu/EZDJM4Qs3Dgpws/ZSsLJ4wwMVMZlfYbqazIMbWe9ig1UbXNh904nt0L/D5Gn9 lw9O2pz2+Hfcw== From: Frederic Weisbecker To: LKML Cc: "Uladzislau Rezki (Sony)" , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Zqiang , rcu , syzbot+061d370693bdd99f9d34@syzkaller.appspotmail.com, Frederic Weisbecker Subject: [PATCH 8/8] rcu/kvfree: Fix data-race in __mod_timer / kvfree_call_rcu Date: Wed, 6 Nov 2024 15:59:11 +0100 Message-ID: <20241106145911.35503-9-frederic@kernel.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241106145911.35503-1-frederic@kernel.org> References: <20241106145911.35503-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Uladzislau Rezki (Sony)" KCSAN reports a data race when access the krcp->monitor_work.timer.expires variable in the schedule_delayed_monitor_work() function: BUG: KCSAN: data-race in __mod_timer / kvfree_call_rcu read to 0xffff888237d1cce8 of 8 bytes by task 10149 on cpu 1: schedule_delayed_monitor_work kernel/rcu/tree.c:3520 [inline] kvfree_call_rcu+0x3b8/0x510 kernel/rcu/tree.c:3839 trie_update_elem+0x47c/0x620 kernel/bpf/lpm_trie.c:441 bpf_map_update_value+0x324/0x350 kernel/bpf/syscall.c:203 generic_map_update_batch+0x401/0x520 kernel/bpf/syscall.c:1849 bpf_map_do_batch+0x28c/0x3f0 kernel/bpf/syscall.c:5143 __sys_bpf+0x2e5/0x7a0 __do_sys_bpf kernel/bpf/syscall.c:5741 [inline] __se_sys_bpf kernel/bpf/syscall.c:5739 [inline] __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5739 x64_sys_call+0x2625/0x2d60 arch/x86/include/generated/asm/syscalls_64.h:322 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xc9/0x1c0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f write to 0xffff888237d1cce8 of 8 bytes by task 56 on cpu 0: __mod_timer+0x578/0x7f0 kernel/time/timer.c:1173 add_timer_global+0x51/0x70 kernel/time/timer.c:1330 __queue_delayed_work+0x127/0x1a0 kernel/workqueue.c:2523 queue_delayed_work_on+0xdf/0x190 kernel/workqueue.c:2552 queue_delayed_work include/linux/workqueue.h:677 [inline] schedule_delayed_monitor_work kernel/rcu/tree.c:3525 [inline] kfree_rcu_monitor+0x5e8/0x660 kernel/rcu/tree.c:3643 process_one_work kernel/workqueue.c:3229 [inline] process_scheduled_works+0x483/0x9a0 kernel/workqueue.c:3310 worker_thread+0x51d/0x6f0 kernel/workqueue.c:3391 kthread+0x1d1/0x210 kernel/kthread.c:389 ret_from_fork+0x4b/0x60 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 Reported by Kernel Concurrency Sanitizer on: CPU: 0 UID: 0 PID: 56 Comm: kworker/u8:4 Not tainted 6.12.0-rc2-syzkaller-00050-g5b7c893ed5ed #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024 Workqueue: events_unbound kfree_rcu_monitor kfree_rcu_monitor() rearms the work if a "krcp" has to be still offloaded and this is done without holding krcp->lock, whereas the kvfree_call_rcu() holds it. Fix it by acquiring the "krcp->lock" for kfree_rcu_monitor() so both functions do not race anymore. Reported-by: syzbot+061d370693bdd99f9d34@syzkaller.appspotmail.com Link: https://lore.kernel.org/lkml/ZxZ68KmHDQYU0yfD@pc636/T/ Fixes: 8fc5494ad5fa ("rcu/kvfree: Move need_offload_krc() out of krcp->lock") Signed-off-by: Uladzislau Rezki (Sony) Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 13829cf38f52..ff98233d4aa5 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3511,7 +3511,7 @@ static int krc_count(struct kfree_rcu_cpu *krcp) } static void -schedule_delayed_monitor_work(struct kfree_rcu_cpu *krcp) +__schedule_delayed_monitor_work(struct kfree_rcu_cpu *krcp) { long delay, delay_left; @@ -3525,6 +3525,16 @@ schedule_delayed_monitor_work(struct kfree_rcu_cpu *krcp) queue_delayed_work(system_unbound_wq, &krcp->monitor_work, delay); } +static void +schedule_delayed_monitor_work(struct kfree_rcu_cpu *krcp) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&krcp->lock, flags); + __schedule_delayed_monitor_work(krcp); + raw_spin_unlock_irqrestore(&krcp->lock, flags); +} + static void kvfree_rcu_drain_ready(struct kfree_rcu_cpu *krcp) { @@ -3836,7 +3846,7 @@ void kvfree_call_rcu(struct rcu_head *head, void *ptr) // Set timer to drain after KFREE_DRAIN_JIFFIES. if (rcu_scheduler_active == RCU_SCHEDULER_RUNNING) - schedule_delayed_monitor_work(krcp); + __schedule_delayed_monitor_work(krcp); unlock_return: krc_this_cpu_unlock(krcp, flags);