From patchwork Mon Jun 20 22:20:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 12888325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3401CCA480 for ; Mon, 20 Jun 2022 22:20:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243664AbiFTWUp (ORCPT ); Mon, 20 Jun 2022 18:20:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244095AbiFTWUj (ORCPT ); Mon, 20 Jun 2022 18:20:39 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0ACE1B792; Mon, 20 Jun 2022 15:20:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E484F611FA; Mon, 20 Jun 2022 22:20:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49EC6C341C5; Mon, 20 Jun 2022 22:20:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655763634; bh=Ev7MJB6+VbSr95NwkxJsx9nVPnwQqYzMCUqVwoydx3A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b4YeM8PlT4eNafI0SiBQ7QwlQtyA/YRjo9Scwk4seglPe/QhehGEGvXXQokLvspsw 2KYNYpltfwkTkOyhDhlqpB/ynVQs2yWgYBRnRI6wTYX/RGI9EAmFHFc8up3Fnx+/PS CODH6o+apFAqzE3JRJYL/RZS3SlBym6xi2Al3cd+UVF9rxkkozuATx6nGZ6CPZ5hTa aeoy3amLtgg9klHtQNF5yX++gCNXbt4NdUwgzm3qANWeVK+KAC0BlnrpIE4AYwjLJI 7zpcelurrXCEk16saDoTgSSAsbX0hjehg9UeTKH9qJytC/X+Nf9EHgZRiUXWl8M5in /vVm/jda2tTcg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 01D095C0A15; Mon, 20 Jun 2022 15:20:34 -0700 (PDT) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@fb.com, rostedt@goodmis.org, Zqiang , "Paul E . McKenney" Subject: [PATCH rcu 03/12] rcu: Add rnp->cbovldmask check in rcutree_migrate_callbacks() Date: Mon, 20 Jun 2022 15:20:23 -0700 Message-Id: <20220620222032.3839547-3-paulmck@kernel.org> X-Mailer: git-send-email 2.31.1.189.g2e36527f23 In-Reply-To: <20220620222022.GA3839466@paulmck-ThinkPad-P17-Gen-1> References: <20220620222022.GA3839466@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Zqiang Currently, the rcu_node structure's ->cbovlmask field is set in call_rcu() when a given CPU is suffering from callback overload. But if that CPU goes offline, the outgoing CPU's callbacks is migrated to the running CPU, which is likely to overload the running CPU. However, that CPU's bit in its leaf rcu_node structure's ->cbovlmask field remains zero. Initially, this is OK because the outgoing CPU's bit remains set. However, that bit will be cleared at the next end of a grace period, at which time it is quite possible that the running CPU will still be overloaded. If the running CPU invokes call_rcu(), then overload will be checked for and the bit will be set. Except that there is no guarantee that the running CPU will invoke call_rcu(), in which case the next grace period will fail to take the running CPU's overload condition into account. Plus, because the bit is not set, the end of the grace period won't check for overload on this CPU. This commit therefore adds a call to check_cb_ovld_locked() in check_cb_ovld_locked() to set the running CPU's ->cbovlmask bit appropriately. Signed-off-by: Zqiang Signed-off-by: Paul E. McKenney Reviewed-by: Neeraj Upadhyay --- kernel/rcu/tree.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index c19d5926886fb..f4a37f2032664 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4491,6 +4491,7 @@ void rcutree_migrate_callbacks(int cpu) needwake = needwake || rcu_advance_cbs(my_rnp, my_rdp); rcu_segcblist_disable(&rdp->cblist); WARN_ON_ONCE(rcu_segcblist_empty(&my_rdp->cblist) != !rcu_segcblist_n_cbs(&my_rdp->cblist)); + check_cb_ovld_locked(my_rdp, my_rnp); if (rcu_rdp_is_offloaded(my_rdp)) { raw_spin_unlock_rcu_node(my_rnp); /* irqs remain disabled. */ __call_rcu_nocb_wake(my_rdp, true, flags);