From patchwork Sun Nov 20 03:40:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 13049890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFAA2C4332F for ; Sun, 20 Nov 2022 03:40:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229485AbiKTDkf (ORCPT ); Sat, 19 Nov 2022 22:40:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229447AbiKTDke (ORCPT ); Sat, 19 Nov 2022 22:40:34 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71F30A8C24 for ; Sat, 19 Nov 2022 19:40:33 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id p12so7820550plq.4 for ; Sat, 19 Nov 2022 19:40:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jY1YP/yfTIErWUpERknfxBPt8SSGSNGDSkDLWbEyYCg=; b=b6330JsqAY/qj8cJ/m9McnCdEJKDano2X02R4XFkeSjwA+2uDS07r6ap6roOWS46GY zO71il5pzX/cvhdqrGTjui7NKbcOUiOb1V/zrsPD5J1JZudHsIWKUjMs3XNTQCf+zr4M E8WE2sSHNn1KFu9UYzPY/nuQ1rz3H2eOw0heAvoT7d4M0nU5+Tm2X0U6RfUaoys5E8jR iF26NEx0tRpOOfNNusXIBNDbgri0UrIHjCW41YTiXGY8Qw+sPc93RgGL6m88NKglB+Bz +BSHBtw0+fyU5WMIXQmwrmZDzoWVfktuASzL5KG6Clh0oW6nuCp06issFOUQqPqG4A1A QXcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jY1YP/yfTIErWUpERknfxBPt8SSGSNGDSkDLWbEyYCg=; b=OOwrwmzKv2Xkk+MR51qP601NR6AwNWNynS+9TXpKLhU9u06VeBFs/nyljwjemU04GM 9nxzABxefvFzjxEvo8DR1n7msFyNw/Xdd5Uc+0uMvRm7mIGXNKBsasLqG5twL3OEktr8 4VSPOp4rBuYhrVWPdaxZnLhhWjksrcrzVrah1aFm9PGIZtcgQt4hn79y9oROkwhANcSU 3XQDFqUyI1O11fw/gMfJg46C1JIklIEonl/wEdrg8T8VN5LKam8eL9A48MolvSmuGCzu 2T3GnDMUzAYy0WHSz0kfUdgmzjgmPASj55EDs+bMZhvQez+gkIdFH1YawDK5StRm6jlb vZ3A== X-Gm-Message-State: ANoB5pkV19yB0e2r24PfYCr2h+b7f91NyvdSkm/uu9/CSWYU1e+x3BCf 0fJukTZ7AD4gLptwCsrBvVDf725BDQ== X-Google-Smtp-Source: AA0mqf4OXD6+5bo0ImbI/KDamIa/SACamTO4Yx0a33h5wmFikXk9bYUAhJx4DJ5a+RczoylWyplecA== X-Received: by 2002:a17:903:3295:b0:188:5dbc:9685 with SMTP id jh21-20020a170903329500b001885dbc9685mr6504960plb.48.1668915632575; Sat, 19 Nov 2022 19:40:32 -0800 (PST) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id gq18-20020a17090b105200b0021864cf062dsm5297260pjb.21.2022.11.19.19.40.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Nov 2022 19:40:32 -0800 (PST) From: Pingfan Liu To: rcu@vger.kernel.org Cc: Pingfan Liu , Lai Jiangshan , "Paul E. McKenney" , Frederic Weisbecker , Josh Triplett , Steven Rostedt , Mathieu Desnoyers Subject: [PATCH 1/2] srcu: Remove needless rcu_seq_done() check while holding read lock Date: Sun, 20 Nov 2022 11:40:13 +0800 Message-Id: <20221120034014.7390-2-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221120034014.7390-1-kernelfans@gmail.com> References: <20221120034014.7390-1-kernelfans@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org As the code changes, now, srcu_gp_start_if_needed() holds the srcu read lock, gets the gp_seq snap and call srcu_funnel_gp_start() passing that snap value. As the rcu_seq_snap() promises "a full grace period has elapsed since the current time". In srcu_funnel_gp_start(), the statement rcu_seq_done(&ssp->srcu_gp_seq, s) always return false. The same condition stands for srcu_funnel_exp_start(). Hence removing all the checks of rcu_seq_done(). Test info: Running "tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 10h --configs 18*SRCU-P" without any failure. Signed-off-by: Pingfan Liu Cc: Lai Jiangshan Cc: "Paul E. McKenney" Cc: Frederic Weisbecker Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers To: rcu@vger.kernel.org --- kernel/rcu/srcutree.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 1c304fec89c0..7df59fc8073e 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -851,8 +851,7 @@ static void srcu_funnel_exp_start(struct srcu_struct *ssp, struct srcu_node *snp if (snp) for (; snp != NULL; snp = snp->srcu_parent) { sgsne = READ_ONCE(snp->srcu_gp_seq_needed_exp); - if (rcu_seq_done(&ssp->srcu_gp_seq, s) || - (!srcu_invl_snp_seq(sgsne) && ULONG_CMP_GE(sgsne, s))) + if (!srcu_invl_snp_seq(sgsne) && ULONG_CMP_GE(sgsne, s)) return; spin_lock_irqsave_rcu_node(snp, flags); sgsne = snp->srcu_gp_seq_needed_exp; @@ -878,6 +877,9 @@ static void srcu_funnel_exp_start(struct srcu_struct *ssp, struct srcu_node *snp * * Note that this function also does the work of srcu_funnel_exp_start(), * in some cases by directly invoking it. + * + * The srcu read lock should be hold around this function. And s is a seq snap + * after holding that lock. */ static void srcu_funnel_gp_start(struct srcu_struct *ssp, struct srcu_data *sdp, unsigned long s, bool do_norm) @@ -898,8 +900,6 @@ static void srcu_funnel_gp_start(struct srcu_struct *ssp, struct srcu_data *sdp, if (snp_leaf) /* Each pass through the loop does one level of the srcu_node tree. */ for (snp = snp_leaf; snp != NULL; snp = snp->srcu_parent) { - if (rcu_seq_done(&ssp->srcu_gp_seq, s) && snp != snp_leaf) - return; /* GP already done and CBs recorded. */ spin_lock_irqsave_rcu_node(snp, flags); snp_seq = snp->srcu_have_cbs[idx]; if (!srcu_invl_snp_seq(snp_seq) && ULONG_CMP_GE(snp_seq, s)) { @@ -935,9 +935,8 @@ static void srcu_funnel_gp_start(struct srcu_struct *ssp, struct srcu_data *sdp, if (!do_norm && ULONG_CMP_LT(ssp->srcu_gp_seq_needed_exp, s)) WRITE_ONCE(ssp->srcu_gp_seq_needed_exp, s); - /* If grace period not already done and none in progress, start it. */ - if (!rcu_seq_done(&ssp->srcu_gp_seq, s) && - rcu_seq_state(ssp->srcu_gp_seq) == SRCU_STATE_IDLE) { + /* If grace period not already in progress, start it. */ + if (rcu_seq_state(ssp->srcu_gp_seq) == SRCU_STATE_IDLE) { WARN_ON_ONCE(ULONG_CMP_GE(ssp->srcu_gp_seq, ssp->srcu_gp_seq_needed)); srcu_gp_start(ssp); From patchwork Sun Nov 20 03:40:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 13049891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25EFFC433FE for ; Sun, 20 Nov 2022 03:40:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229456AbiKTDkk (ORCPT ); Sat, 19 Nov 2022 22:40:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229447AbiKTDkj (ORCPT ); Sat, 19 Nov 2022 22:40:39 -0500 Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6D5BA8C24 for ; Sat, 19 Nov 2022 19:40:38 -0800 (PST) Received: by mail-pg1-x530.google.com with SMTP id r18so8305498pgr.12 for ; Sat, 19 Nov 2022 19:40:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WiNXBMi5a60FPiTGLznN6GlG311S8i8YBj6N2/PptqE=; b=GylSTi3Itl1GH+rQGnpcDazb7EmawFLZtc15U43Xiojv+08ATT2ZkASGRFnHB6TK6r I5Up36C+P50b6TmUT1zkZlDx38GErhIekTqpsZIASOUWq1ZM5M+fpAd/3R24AYrKY2gN dBANmbYRvxGmatUVxhjrrc/dmruRyJucdpfkyGWf2EhbDr8U4P9xMuelJ0yqCZM61wPD XXjf0vPB+yRuYB+vry5Acrd0JywsXfzrsiTvgjP2mefReXIcVv/U16sAby55qKcOG8Pp i7JaqvadkPFiWIGOb+F5pfvbN0QAGfEq/N0elbbKOeQsn+uwd9S5jdrXiqSl+UgCJE42 MqTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WiNXBMi5a60FPiTGLznN6GlG311S8i8YBj6N2/PptqE=; b=QgZzoQe2osin2ioznBNEOKGQWC2YvWaV/EMxe7LICQbt6scLYMAtRAxAKwGytEvrHt XC8TKyUdk36JfuG/hSSkRGV+3B/OsOBnzmj2YZ4wmud77BoXaSQh1zghSfhLGdIDhhXE YoMdhWFYB0h4CQ9RXB2Lw+/oT6N9gxkFdWAb2Q32HMZtxyqotii6cT/W7UJmJLYDh5tL GIwODmFl71ZlYIlC4zHyfMUoGwEUL/LSMA0vNOP0LOwI+Yv2C9wyPb4u3w99Skg5/6Uq qcdb7HiarCu2I5JGkE+FdaTlE980PJHGhs/K8ps+JbZrCgVaLWFYiMtkM3D/gKLyuDoe ogaQ== X-Gm-Message-State: ANoB5pm0NHisK01mc7s+Bp0B68KNYFvB1SVzsCHDv2lT8ZPv6nwYNm2l ZNZGTuQz60yzh4RlzOth8rR8K5I+kg== X-Google-Smtp-Source: AA0mqf5RfY9u8NnwvWhsJmPCCPJeiiLb2U4SSlCWQ1uR2jwO88y727KTwTnwtUGDMK2XmSZBDX+H8Q== X-Received: by 2002:a63:1514:0:b0:46e:a4ed:467e with SMTP id v20-20020a631514000000b0046ea4ed467emr1037122pgl.319.1668915637829; Sat, 19 Nov 2022 19:40:37 -0800 (PST) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id gq18-20020a17090b105200b0021864cf062dsm5297260pjb.21.2022.11.19.19.40.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Nov 2022 19:40:37 -0800 (PST) From: Pingfan Liu To: rcu@vger.kernel.org Cc: Pingfan Liu , Lai Jiangshan , "Paul E. McKenney" , Frederic Weisbecker , Josh Triplett , Steven Rostedt , Mathieu Desnoyers Subject: [PATCH 2/2] srcu: Remove needless updating of srcu_have_cbs in srcu_gp_end() Date: Sun, 20 Nov 2022 11:40:14 +0800 Message-Id: <20221120034014.7390-3-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221120034014.7390-1-kernelfans@gmail.com> References: <20221120034014.7390-1-kernelfans@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org At present, snp->srcu_have_cbs[idx] is updated by either srcu_funnel_gp_start() or srcu_gp_end(). But as the code changes, now, srcu_funnel_gp_start() is called with srcu read lock held. And its input parameter s, s = rcu_seq_snap(&ssp->srcu_gp_seq), whose counter field always proceeds that of the srcu_gp_seq by one or two. If the seq snap only proceeds by one, the state machine should be in state SRCU_STATE_IDLE, the held srcu read lock will prevent the state machine from moving ahead. So when srcu_gp_end() updates snp->srcu_have_cbs[idx], the idx must be the past idx for srcu_funnel_gp_start() that is cared by nobody. So removing the unnecessary updating in srcu_gp_end(). Test info: Running "tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 10h --configs 18*SRCU-P" without any failure. Signed-off-by: Pingfan Liu Cc: Lai Jiangshan Cc: "Paul E. McKenney" Cc: Frederic Weisbecker Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers To: rcu@vger.kernel.org --- kernel/rcu/srcutree.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 7df59fc8073e..c54d6c04751f 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -783,8 +783,6 @@ static void srcu_gp_end(struct srcu_struct *ssp) last_lvl = snp >= ssp->level[rcu_num_lvls - 1]; if (last_lvl) cbs = ss_state < SRCU_SIZE_BIG || snp->srcu_have_cbs[idx] == gpseq; - snp->srcu_have_cbs[idx] = gpseq; - rcu_seq_set_state(&snp->srcu_have_cbs[idx], 1); sgsne = snp->srcu_gp_seq_needed_exp; if (srcu_invl_snp_seq(sgsne) || ULONG_CMP_LT(sgsne, gpseq)) WRITE_ONCE(snp->srcu_gp_seq_needed_exp, gpseq);