| Message ID | 20220622103549.2840087-1-qiang1.zhang@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | rcu: Add exp QS check in rcu_exp_handler() for no-preemptible expedited RCU | expand |
On 6/22/2022 4:05 PM, Zqiang wrote: > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > grace period begins, if detected current CPU enters idle in > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > current cpu, at this time, maybe not being in an RCU read-side critical > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > occurs on current CPU to check and report exp QS. > > This commit add a exp QS check in rcu_exp_handler(), when not being > in an RCU read-side critical section, report exp QS earlier. > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > --- I am a bit unsure on this change. For non-preempt kernel, technically (and by design) context switch points are considered RCU read side critical section end? This change, changes that behavior between CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT_COUNT=n case, for non-preemptive kernels. However, I could be missing something here, and would wait for Paul's review. Thanks Neeraj > kernel/rcu/tree_exp.h | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > index be667583a554..34f08267410f 100644 > --- a/kernel/rcu/tree_exp.h > +++ b/kernel/rcu/tree_exp.h > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > { > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > struct rcu_node *rnp = rdp->mynode; > + bool preempt_bh_disabled = > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > return; > - if (rcu_is_cpu_rrupt_from_idle()) { > + if (rcu_is_cpu_rrupt_from_idle() || > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > return; > }
Hi, On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > grace period begins, if detected current CPU enters idle in > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > current cpu, at this time, maybe not being in an RCU read-side critical > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > occurs on current CPU to check and report exp QS. > I think the idea is OK, however, this "optimization" is based on the implementation detail that rcu_read_lock() counts preempt_count when CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly for debugging purposes IIUC, and in other words, _it could be gone_. Also I'm not aware of any but there could be someone assuming that RCU read-side critical sections can be formed without rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there might be "creative" code like the following: void do_something_only_in_nonpreempt(void) { int *p; // This function only gets called in PREEMPT=n kernel, // which means everywhere is a RCU read-side critical // section, let's save some lines of code. p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); ... // of course no schedule() here. <access p> } Again, I'm not aware of any existing code that does this but we need to be sure. Regards, Boqun > This commit add a exp QS check in rcu_exp_handler(), when not being > in an RCU read-side critical section, report exp QS earlier. > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > --- > kernel/rcu/tree_exp.h | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > index be667583a554..34f08267410f 100644 > --- a/kernel/rcu/tree_exp.h > +++ b/kernel/rcu/tree_exp.h > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > { > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > struct rcu_node *rnp = rdp->mynode; > + bool preempt_bh_disabled = > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > return; > - if (rcu_is_cpu_rrupt_from_idle()) { > + if (rcu_is_cpu_rrupt_from_idle() || > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > return; > } > -- > 2.25.1 >
On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > grace period begins, if detected current CPU enters idle in > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > current cpu, at this time, maybe not being in an RCU read-side critical > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > occurs on current CPU to check and report exp QS. > >I think the idea is OK, however, this "optimization" is based on the >implementation detail that rcu_read_lock() counts preempt_count when >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly >for debugging purposes IIUC, and in other words, _it could be gone_. > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the preempt-count is exists, so can report exp QS when not being an RCU read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). in IPI handler. For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, The rcu_read_lock/unlock is just barrier(). So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the preempt-count is also checked /* Report any deferred quiescent states if preemption enabled. */ if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { rcu_preempt_deferred_qs(current); but the RCU softirq may not be triggered in time and reported exp QS, for example a kernel loop exist on NO_HZ_FULL CPU this change, It is to capture the exp QS state earlier and report it. >Also I'm not aware of any but there could be someone assuming that RCU >read-side critical sections can be formed without >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there >might be "creative" code like the following: > > void do_something_only_in_nonpreempt(void) > { > int *p; > > // This function only gets called in PREEMPT=n kernel, > // which means everywhere is a RCU read-side critical > // section, let's save some lines of code. > rcu_read_lock(); > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > ... // of course no schedule() here. > <access p> rcu_read_unlock(); > } > Usually access to pointers of type rcu needs to be protected. Any thoughts? >Again, I'm not aware of any existing code that does this but we need to >be sure. > >Regards, >Boqun > > This commit add a exp QS check in rcu_exp_handler(), when not being > in an RCU read-side critical section, report exp QS earlier. > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > --- > kernel/rcu/tree_exp.h | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > index be667583a554..34f08267410f 100644 > --- a/kernel/rcu/tree_exp.h > +++ b/kernel/rcu/tree_exp.h > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > { > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > struct rcu_node *rnp = rdp->mynode; > + bool preempt_bh_disabled = > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > return; > - if (rcu_is_cpu_rrupt_from_idle()) { > + if (rcu_is_cpu_rrupt_from_idle() || > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > return; > } > -- > 2.25.1 >
On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > grace period begins, if detected current CPU enters idle in > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > current cpu, at this time, maybe not being in an RCU read-side critical > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > occurs on current CPU to check and report exp QS. > >I think the idea is OK, however, this "optimization" is based on the >implementation detail that rcu_read_lock() counts preempt_count when >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly >for debugging purposes IIUC, and in other words, _it could be gone_. > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the preempt-count is exists, so can report exp QS when not being an RCU read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). in IPI handler. For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, The rcu_read_lock/unlock is just barrier(). So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. Of course, for CONFIG_PREEMPT_COUNT=y kernel, in RCU softirq, the preempt-count is also checked /* Report any deferred quiescent states if preemption enabled. */ if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { rcu_preempt_deferred_qs(current); but the RCU softirq may not be triggered in time and reported exp QS, for example a kernel loop exist on NO_HZ_FULL CPU this change, It is to capture the exp QS state earlier and report it. >Also I'm not aware of any but there could be someone assuming that RCU >read-side critical sections can be formed without >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there >might be "creative" code like the following: > > void do_something_only_in_nonpreempt(void) > { > int *p; > > // This function only gets called in PREEMPT=n kernel, > // which means everywhere is a RCU read-side critical > // section, let's save some lines of code. > rcu_read_lock(); > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > ... // of course no schedule() here. > <access p> rcu_read_unlock(); > } > Usually access to pointers of type rcu needs to be protected. Any thoughts? >Again, I'm not aware of any existing code that does this but we need to >be sure. > >Regards, >Boqun > > This commit add a exp QS check in rcu_exp_handler(), when not being > in an RCU read-side critical section, report exp QS earlier. > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > --- > kernel/rcu/tree_exp.h | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > index be667583a554..34f08267410f 100644 > --- a/kernel/rcu/tree_exp.h > +++ b/kernel/rcu/tree_exp.h > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > { > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > struct rcu_node *rnp = rdp->mynode; > + bool preempt_bh_disabled = > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > return; > - if (rcu_is_cpu_rrupt_from_idle()) { > + if (rcu_is_cpu_rrupt_from_idle() || > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > return; > } > -- > 2.25.1 >
On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > grace period begins, if detected current CPU enters idle in > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > current cpu, at this time, maybe not being in an RCU read-side critical > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > occurs on current CPU to check and report exp QS. > > > > >I think the idea is OK, however, this "optimization" is based on the > >implementation detail that rcu_read_lock() counts preempt_count when > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > preempt-count is exists, so can report exp QS when not being an RCU > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > in IPI handler. > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > The rcu_read_lock/unlock is just barrier(). > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > preempt-count is also checked > > /* Report any deferred quiescent states if preemption enabled. */ > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > rcu_preempt_deferred_qs(current); > > but the RCU softirq may not be triggered in time and reported exp QS, for > example a kernel loop exist on NO_HZ_FULL CPU > > this change, It is to capture the exp QS state earlier and report it. > > > >Also I'm not aware of any but there could be someone assuming that RCU > >read-side critical sections can be formed without > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > >might be "creative" code like the following: > > > > void do_something_only_in_nonpreempt(void) > > { > > int *p; > > > > // This function only gets called in PREEMPT=n kernel, > > // which means everywhere is a RCU read-side critical > > // section, let's save some lines of code. > > > rcu_read_lock(); > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > ... // of course no schedule() here. > > <access p> > rcu_read_unlock(); > > } > > > > Usually access to pointers of type rcu needs to be protected. Indeed, lockdep would normally complain about this sort of thing. But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > Any thoughts? It would be good to have some performance data on this change to expedited grace periods. It is adding code, so it needs some real motivation. So, does this change make a system-level difference in (say) expedited RCU grace-period latency, and if so, under what conditions? Thanx, Paul > >Again, I'm not aware of any existing code that does this but we need to > >be sure. > > > >Regards, > >Boqun > > > > This commit add a exp QS check in rcu_exp_handler(), when not being > > in an RCU read-side critical section, report exp QS earlier. > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > --- > > kernel/rcu/tree_exp.h | 5 ++++- > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > > index be667583a554..34f08267410f 100644 > > --- a/kernel/rcu/tree_exp.h > > +++ b/kernel/rcu/tree_exp.h > > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > > { > > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > > struct rcu_node *rnp = rdp->mynode; > > + bool preempt_bh_disabled = > > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > > return; > > - if (rcu_is_cpu_rrupt_from_idle()) { > > + if (rcu_is_cpu_rrupt_from_idle() || > > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > > return; > > } > > -- > > 2.25.1 > >
On Wed, Jun 22, 2022 at 05:34:09PM -0700, Paul E. McKenney wrote: > On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > > grace period begins, if detected current CPU enters idle in > > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > > current cpu, at this time, maybe not being in an RCU read-side critical > > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > > occurs on current CPU to check and report exp QS. > > > > > > > >I think the idea is OK, however, this "optimization" is based on the > > >implementation detail that rcu_read_lock() counts preempt_count when > > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > > preempt-count is exists, so can report exp QS when not being an RCU > > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > > in IPI handler. > > > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > > The rcu_read_lock/unlock is just barrier(). > > > > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > > preempt-count is also checked > > > > /* Report any deferred quiescent states if preemption enabled. */ > > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > > rcu_preempt_deferred_qs(current); > > > > but the RCU softirq may not be triggered in time and reported exp QS, for > > example a kernel loop exist on NO_HZ_FULL CPU > > > > this change, It is to capture the exp QS state earlier and report it. > > > > > > >Also I'm not aware of any but there could be someone assuming that RCU > > >read-side critical sections can be formed without > > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > > >might be "creative" code like the following: > > > > > > void do_something_only_in_nonpreempt(void) > > > { > > > int *p; > > > > > > // This function only gets called in PREEMPT=n kernel, > > > // which means everywhere is a RCU read-side critical > > > // section, let's save some lines of code. > > > > > rcu_read_lock(); > > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > > ... // of course no schedule() here. > > > <access p> > > rcu_read_unlock(); > > > } > > > > > > > Usually access to pointers of type rcu needs to be protected. Yes, _ususally_ they are, but what about the special cases? Because in PREEMPT=n kernel, almost everywhere is a RCU read-side critical section, some one might have been "creative" enough to omit these rcu_read_lock() and rcu_read_unlock(). > > Indeed, lockdep would normally complain about this sort of thing. > But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without > CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > Actually with the rcu_dereference_check() above, lockdep cannot detect even CONFIG_PREEMPT_COUNT=y, that rcu_dereference_check() basically says "I know I'm in a read-side critical section if it's a non-preempt kernel, so don't bother to check". ;-) > > Any thoughts? > > It would be good to have some performance data on this change to expedited > grace periods. It is adding code, so it needs some real motivation. Agreed. Regards, Boqun > So, does this change make a system-level difference in (say) expedited > RCU grace-period latency, and if so, under what conditions? > > Thanx, Paul > > > >Again, I'm not aware of any existing code that does this but we need to > > >be sure. > > > > > >Regards, > > >Boqun > > > > > > This commit add a exp QS check in rcu_exp_handler(), when not being > > > in an RCU read-side critical section, report exp QS earlier. > > > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > > --- > > > kernel/rcu/tree_exp.h | 5 ++++- > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > > > index be667583a554..34f08267410f 100644 > > > --- a/kernel/rcu/tree_exp.h > > > +++ b/kernel/rcu/tree_exp.h > > > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > > > { > > > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > > > struct rcu_node *rnp = rdp->mynode; > > > + bool preempt_bh_disabled = > > > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > > > > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > > > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > > > return; > > > - if (rcu_is_cpu_rrupt_from_idle()) { > > > + if (rcu_is_cpu_rrupt_from_idle() || > > > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > > > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > > > return; > > > } > > > -- > > > 2.25.1 > > >
On Wed, Jun 22, 2022 at 05:34:09PM -0700, Paul E. McKenney wrote: > On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > > grace period begins, if detected current CPU enters idle in > > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > > current cpu, at this time, maybe not being in an RCU read-side critical > > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > > occurs on current CPU to check and report exp QS. > > > > > > > >I think the idea is OK, however, this "optimization" is based on the > > >implementation detail that rcu_read_lock() counts preempt_count when > > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > > preempt-count is exists, so can report exp QS when not being an RCU > > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > > in IPI handler. > > > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > > The rcu_read_lock/unlock is just barrier(). > > > > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > > preempt-count is also checked > > > > /* Report any deferred quiescent states if preemption enabled. */ > > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > > rcu_preempt_deferred_qs(current); > > > > but the RCU softirq may not be triggered in time and reported exp QS, for > > example a kernel loop exist on NO_HZ_FULL CPU > > > > this change, It is to capture the exp QS state earlier and report it. > > > > > > >Also I'm not aware of any but there could be someone assuming that RCU > > >read-side critical sections can be formed without > > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > > >might be "creative" code like the following: > > > > > > void do_something_only_in_nonpreempt(void) > > > { > > > int *p; > > > > > > // This function only gets called in PREEMPT=n kernel, > > > // which means everywhere is a RCU read-side critical > > > // section, let's save some lines of code. > > > > > rcu_read_lock(); > > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > > ... // of course no schedule() here. > > > <access p> > > rcu_read_unlock(); > > > } > > > > > > > Usually access to pointers of type rcu needs to be protected. Yes, _ususally_ they are, but what about the special cases? Because in PREEMPT=n kernel, almost everywhere is a RCU read-side critical section, some one might have been "creative" enough to omit these rcu_read_lock() and rcu_read_unlock(). > > Indeed, lockdep would normally complain about this sort of thing. > But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without > CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > >Actually with the rcu_dereference_check() above, lockdep cannot detect >even CONFIG_PREEMPT_COUNT=y, that rcu_dereference_check() basically says >"I know I'm in a read-side critical section if it's a non-preempt >kernel, so don't bother to check". ;-) > > Any thoughts? > > It would be good to have some performance data on this change to expedited > grace periods. It is adding code, so it needs some real motivation. I used rcuscale to test it, and count the average time of writer-duration no applied patch(org.txt) writer avg time 29690.39 29670.78 29770.65 29423.25 applied patch(new.txt) writer avg time 28989.99 29003.54 29281.39 28986.58 or Is there a better way to test? Thanks Zqiang >Agreed. > >Regards, >Boqun > So, does this change make a system-level difference in (say) expedited > RCU grace-period latency, and if so, under what conditions? > > Thanx, Paul > > > >Again, I'm not aware of any existing code that does this but we need to > > >be sure. > > > > > >Regards, > > >Boqun > > > > > > This commit add a exp QS check in rcu_exp_handler(), when not being > > > in an RCU read-side critical section, report exp QS earlier. > > > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > > --- > > > kernel/rcu/tree_exp.h | 5 ++++- > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > > > index be667583a554..34f08267410f 100644 > > > --- a/kernel/rcu/tree_exp.h > > > +++ b/kernel/rcu/tree_exp.h > > > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > > > { > > > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > > > struct rcu_node *rnp = rdp->mynode; > > > + bool preempt_bh_disabled = > > > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > > > > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > > > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > > > return; > > > - if (rcu_is_cpu_rrupt_from_idle()) { > > > + if (rcu_is_cpu_rrupt_from_idle() || > > > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > > > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > > > return; > > > } > > > -- > > > 2.25.1 > > > [ 28.907109] rcu-scale: rcu_scale_writer 3 has 100 measurements [ 28.907133] rcu-scale: rcu_scale_writer 0 has 100 measurements [ 28.907135] rcu-scale: rcu_scale_writer 1 has 100 measurements [ 28.907138] rcu-scale: rcu_scale_writer 2 has 100 measurements [ 28.917241] Dumping ftrace buffer: [ 28.917256] (ftrace buffer empty) [ 28.917261] rcu-scale: Test complete [ 49.336302] rcu-scale:!!! All grace periods expedited, no normal ones to measure! [ 49.336329] rcu-torture: Stopping rcu_scale_reader task [ 49.336340] rcu-torture: rcu_scale_reader is stopping [ 49.336351] rcu-torture: Stopping rcu_scale_reader task [ 49.336361] rcu-torture: rcu_scale_reader is stopping [ 49.336370] rcu-torture: Stopping rcu_scale_reader task [ 49.336405] rcu-torture: rcu_scale_reader is stopping [ 49.336426] rcu-torture: Stopping rcu_scale_reader task [ 49.336432] rcu-torture: rcu_scale_reader is stopping [ 49.336443] rcu-torture: Stopping rcu_scale_writer task [ 49.336450] rcu-torture: rcu_scale_writer is stopping [ 49.336458] rcu-scale: writer 0 gps: 102 [ 49.336460] rcu-torture: Stopping rcu_scale_writer task [ 49.336465] rcu-torture: rcu_scale_writer is stopping [ 49.336517] rcu-scale: writer 1 gps: 102 [ 49.336519] rcu-torture: Stopping rcu_scale_writer task [ 49.336524] rcu-torture: rcu_scale_writer is stopping [ 49.336532] rcu-scale: writer 2 gps: 101 [ 49.336533] rcu-torture: Stopping rcu_scale_writer task [ 49.336539] rcu-torture: rcu_scale_writer is stopping [ 49.336546] rcu-scale: writer 3 gps: 102 [ 49.336548] rcu-scale: start: 28841772292 end: 28844718545 duration: 2946253 gps: 407 batches: 2 [ 49.336550] rcu-scale: 0 writer-duration: 0 25552 [ 49.339264] rcu-scale: 0 writer-duration: 1 42012 [ 49.339274] rcu-scale: 0 writer-duration: 2 24377 [ 49.339280] rcu-scale: 0 writer-duration: 3 34458 [ 49.339299] rcu-scale: 0 writer-duration: 4 27007 [ 49.339305] rcu-scale: 0 writer-duration: 5 36494 [ 49.339310] rcu-scale: 0 writer-duration: 6 24278 [ 49.339315] rcu-scale: 0 writer-duration: 7 29383 [ 49.339321] rcu-scale: 0 writer-duration: 8 26494 [ 49.339326] rcu-scale: 0 writer-duration: 9 35430 [ 49.339331] rcu-scale: 0 writer-duration: 10 23986 [ 49.339337] rcu-scale: 0 writer-duration: 11 28663 [ 49.339342] rcu-scale: 0 writer-duration: 12 26329 [ 49.339347] rcu-scale: 0 writer-duration: 13 35645 [ 49.339353] rcu-scale: 0 writer-duration: 14 23761 [ 49.339358] rcu-scale: 0 writer-duration: 15 28889 [ 49.339364] rcu-scale: 0 writer-duration: 16 26519 [ 49.339369] rcu-scale: 0 writer-duration: 17 35863 [ 49.339374] rcu-scale: 0 writer-duration: 18 23984 [ 49.339380] rcu-scale: 0 writer-duration: 19 28618 [ 49.339385] rcu-scale: 0 writer-duration: 20 26547 [ 49.339390] rcu-scale: 0 writer-duration: 21 35515 [ 49.339395] rcu-scale: 0 writer-duration: 22 23913 [ 49.339400] rcu-scale: 0 writer-duration: 23 28510 [ 49.339405] rcu-scale: 0 writer-duration: 24 26746 [ 49.339410] rcu-scale: 0 writer-duration: 25 35670 [ 49.339416] rcu-scale: 0 writer-duration: 26 23963 [ 49.339421] rcu-scale: 0 writer-duration: 27 28561 [ 49.339427] rcu-scale: 0 writer-duration: 28 26380 [ 49.339432] rcu-scale: 0 writer-duration: 29 36039 [ 49.339438] rcu-scale: 0 writer-duration: 30 24215 [ 49.339443] rcu-scale: 0 writer-duration: 31 28891 [ 49.339449] rcu-scale: 0 writer-duration: 32 26578 [ 49.339454] rcu-scale: 0 writer-duration: 33 40498 [ 49.339459] rcu-scale: 0 writer-duration: 34 24245 [ 49.339464] rcu-scale: 0 writer-duration: 35 29465 [ 49.339469] rcu-scale: 0 writer-duration: 36 26946 [ 49.339475] rcu-scale: 0 writer-duration: 37 35955 [ 49.339480] rcu-scale: 0 writer-duration: 38 24059 [ 49.339486] rcu-scale: 0 writer-duration: 39 28795 [ 49.339491] rcu-scale: 0 writer-duration: 40 26536 [ 49.339496] rcu-scale: 0 writer-duration: 41 35879 [ 49.339501] rcu-scale: 0 writer-duration: 42 24181 [ 49.339506] rcu-scale: 0 writer-duration: 43 28639 [ 49.339512] rcu-scale: 0 writer-duration: 44 26546 [ 49.339517] rcu-scale: 0 writer-duration: 45 35884 [ 49.339522] rcu-scale: 0 writer-duration: 46 23986 [ 49.339527] rcu-scale: 0 writer-duration: 47 28668 [ 49.339532] rcu-scale: 0 writer-duration: 48 26454 [ 49.339537] rcu-scale: 0 writer-duration: 49 35772 [ 49.339542] rcu-scale: 0 writer-duration: 50 23942 [ 49.339547] rcu-scale: 0 writer-duration: 51 28803 [ 49.339553] rcu-scale: 0 writer-duration: 52 26545 [ 49.339558] rcu-scale: 0 writer-duration: 53 35615 [ 49.339563] rcu-scale: 0 writer-duration: 54 23569 [ 49.339568] rcu-scale: 0 writer-duration: 55 28427 [ 49.339573] rcu-scale: 0 writer-duration: 56 26489 [ 49.339578] rcu-scale: 0 writer-duration: 57 35516 [ 49.339584] rcu-scale: 0 writer-duration: 58 23722 [ 49.339589] rcu-scale: 0 writer-duration: 59 28435 [ 49.339594] rcu-scale: 0 writer-duration: 60 26401 [ 49.339599] rcu-scale: 0 writer-duration: 61 35488 [ 49.339604] rcu-scale: 0 writer-duration: 62 23855 [ 49.339610] rcu-scale: 0 writer-duration: 63 28583 [ 49.339615] rcu-scale: 0 writer-duration: 64 26629 [ 49.339620] rcu-scale: 0 writer-duration: 65 36168 [ 49.339625] rcu-scale: 0 writer-duration: 66 24191 [ 49.339630] rcu-scale: 0 writer-duration: 67 28802 [ 49.339635] rcu-scale: 0 writer-duration: 68 32436 [ 49.339640] rcu-scale: 0 writer-duration: 69 38444 [ 49.339646] rcu-scale: 0 writer-duration: 70 24417 [ 49.339651] rcu-scale: 0 writer-duration: 71 29128 [ 49.339656] rcu-scale: 0 writer-duration: 72 26480 [ 49.339661] rcu-scale: 0 writer-duration: 73 35881 [ 49.339666] rcu-scale: 0 writer-duration: 74 24123 [ 49.339671] rcu-scale: 0 writer-duration: 75 28735 [ 49.339676] rcu-scale: 0 writer-duration: 76 26359 [ 49.339681] rcu-scale: 0 writer-duration: 77 35715 [ 49.339687] rcu-scale: 0 writer-duration: 78 24092 [ 49.339692] rcu-scale: 0 writer-duration: 79 28716 [ 49.339698] rcu-scale: 0 writer-duration: 80 26448 [ 49.339703] rcu-scale: 0 writer-duration: 81 35935 [ 49.339708] rcu-scale: 0 writer-duration: 82 23966 [ 49.339713] rcu-scale: 0 writer-duration: 83 28552 [ 49.339718] rcu-scale: 0 writer-duration: 84 26487 [ 49.339723] rcu-scale: 0 writer-duration: 85 35490 [ 49.339729] rcu-scale: 0 writer-duration: 86 23976 [ 49.339734] rcu-scale: 0 writer-duration: 87 28679 [ 49.339740] rcu-scale: 0 writer-duration: 88 26525 [ 49.339745] rcu-scale: 0 writer-duration: 89 35764 [ 49.339750] rcu-scale: 0 writer-duration: 90 24050 [ 49.339756] rcu-scale: 0 writer-duration: 91 28777 [ 49.339761] rcu-scale: 0 writer-duration: 92 26412 [ 49.339767] rcu-scale: 0 writer-duration: 93 35707 [ 49.339772] rcu-scale: 0 writer-duration: 94 23913 [ 49.339777] rcu-scale: 0 writer-duration: 95 28450 [ 49.339782] rcu-scale: 0 writer-duration: 96 26665 [ 49.339788] rcu-scale: 0 writer-duration: 97 35636 [ 49.339793] rcu-scale: 0 writer-duration: 98 23953 [ 49.339798] rcu-scale: 0 writer-duration: 99 28466 [ 49.339804] rcu-scale: 0 writer-duration: 100 26654 [ 49.341227] rcu-scale: 0 writer-duration: 101 225635 [ 49.341246] rcu-scale: 1 writer-duration: 0 27277 [ 49.343227] rcu-scale: 1 writer-duration: 1 34458 [ 49.343236] rcu-scale: 1 writer-duration: 2 27340 [ 49.343242] rcu-scale: 1 writer-duration: 3 41758 [ 49.343248] rcu-scale: 1 writer-duration: 4 24161 [ 49.343254] rcu-scale: 1 writer-duration: 5 29250 [ 49.343260] rcu-scale: 1 writer-duration: 6 26996 [ 49.343266] rcu-scale: 1 writer-duration: 7 36482 [ 49.343272] rcu-scale: 1 writer-duration: 8 23782 [ 49.343277] rcu-scale: 1 writer-duration: 9 28433 [ 49.343283] rcu-scale: 1 writer-duration: 10 26577 [ 49.343289] rcu-scale: 1 writer-duration: 11 35616 [ 49.343295] rcu-scale: 1 writer-duration: 12 23821 [ 49.343301] rcu-scale: 1 writer-duration: 13 28678 [ 49.343307] rcu-scale: 1 writer-duration: 14 26517 [ 49.343313] rcu-scale: 1 writer-duration: 15 35548 [ 49.343319] rcu-scale: 1 writer-duration: 16 24196 [ 49.343325] rcu-scale: 1 writer-duration: 17 28626 [ 49.343332] rcu-scale: 1 writer-duration: 18 26772 [ 49.343338] rcu-scale: 1 writer-duration: 19 35539 [ 49.343344] rcu-scale: 1 writer-duration: 20 23897 [ 49.343350] rcu-scale: 1 writer-duration: 21 28635 [ 49.343356] rcu-scale: 1 writer-duration: 22 26453 [ 49.343362] rcu-scale: 1 writer-duration: 23 35416 [ 49.343368] rcu-scale: 1 writer-duration: 24 24096 [ 49.343373] rcu-scale: 1 writer-duration: 25 28733 [ 49.343379] rcu-scale: 1 writer-duration: 26 26538 [ 49.343385] rcu-scale: 1 writer-duration: 27 35590 [ 49.343390] rcu-scale: 1 writer-duration: 28 23739 [ 49.343396] rcu-scale: 1 writer-duration: 29 29450 [ 49.343402] rcu-scale: 1 writer-duration: 30 26452 [ 49.343408] rcu-scale: 1 writer-duration: 31 35807 [ 49.343413] rcu-scale: 1 writer-duration: 32 24145 [ 49.343419] rcu-scale: 1 writer-duration: 33 33452 [ 49.343425] rcu-scale: 1 writer-duration: 34 26767 [ 49.343431] rcu-scale: 1 writer-duration: 35 36677 [ 49.343438] rcu-scale: 1 writer-duration: 36 24102 [ 49.343444] rcu-scale: 1 writer-duration: 37 28937 [ 49.343449] rcu-scale: 1 writer-duration: 38 26622 [ 49.343454] rcu-scale: 1 writer-duration: 39 35903 [ 49.343460] rcu-scale: 1 writer-duration: 40 23956 [ 49.343465] rcu-scale: 1 writer-duration: 41 28684 [ 49.343470] rcu-scale: 1 writer-duration: 42 26901 [ 49.343475] rcu-scale: 1 writer-duration: 43 35717 [ 49.343480] rcu-scale: 1 writer-duration: 44 23957 [ 49.343485] rcu-scale: 1 writer-duration: 45 28780 [ 49.343490] rcu-scale: 1 writer-duration: 46 26747 [ 49.343495] rcu-scale: 1 writer-duration: 47 35538 [ 49.343501] rcu-scale: 1 writer-duration: 48 23963 [ 49.343506] rcu-scale: 1 writer-duration: 49 28727 [ 49.343511] rcu-scale: 1 writer-duration: 50 26519 [ 49.343516] rcu-scale: 1 writer-duration: 51 35733 [ 49.343521] rcu-scale: 1 writer-duration: 52 24098 [ 49.343527] rcu-scale: 1 writer-duration: 53 28557 [ 49.343532] rcu-scale: 1 writer-duration: 54 26208 [ 49.343537] rcu-scale: 1 writer-duration: 55 35315 [ 49.343543] rcu-scale: 1 writer-duration: 56 23932 [ 49.343548] rcu-scale: 1 writer-duration: 57 28507 [ 49.343553] rcu-scale: 1 writer-duration: 58 26443 [ 49.343559] rcu-scale: 1 writer-duration: 59 35306 [ 49.343564] rcu-scale: 1 writer-duration: 60 23868 [ 49.343570] rcu-scale: 1 writer-duration: 61 28606 [ 49.343575] rcu-scale: 1 writer-duration: 62 26308 [ 49.343580] rcu-scale: 1 writer-duration: 63 35693 [ 49.343586] rcu-scale: 1 writer-duration: 64 23934 [ 49.343591] rcu-scale: 1 writer-duration: 65 29456 [ 49.343596] rcu-scale: 1 writer-duration: 66 26600 [ 49.343602] rcu-scale: 1 writer-duration: 67 35682 [ 49.343607] rcu-scale: 1 writer-duration: 68 29849 [ 49.343612] rcu-scale: 1 writer-duration: 69 31555 [ 49.343618] rcu-scale: 1 writer-duration: 70 26942 [ 49.343623] rcu-scale: 1 writer-duration: 71 36037 [ 49.343629] rcu-scale: 1 writer-duration: 72 23965 [ 49.343634] rcu-scale: 1 writer-duration: 73 28811 [ 49.343640] rcu-scale: 1 writer-duration: 74 26734 [ 49.343645] rcu-scale: 1 writer-duration: 75 35712 [ 49.343650] rcu-scale: 1 writer-duration: 76 23845 [ 49.343656] rcu-scale: 1 writer-duration: 77 28619 [ 49.343661] rcu-scale: 1 writer-duration: 78 26637 [ 49.343666] rcu-scale: 1 writer-duration: 79 35773 [ 49.343672] rcu-scale: 1 writer-duration: 80 23910 [ 49.343677] rcu-scale: 1 writer-duration: 81 28873 [ 49.343682] rcu-scale: 1 writer-duration: 82 26581 [ 49.343688] rcu-scale: 1 writer-duration: 83 35517 [ 49.343694] rcu-scale: 1 writer-duration: 84 23928 [ 49.343699] rcu-scale: 1 writer-duration: 85 28565 [ 49.343704] rcu-scale: 1 writer-duration: 86 26464 [ 49.343710] rcu-scale: 1 writer-duration: 87 35715 [ 49.343715] rcu-scale: 1 writer-duration: 88 23968 [ 49.343721] rcu-scale: 1 writer-duration: 89 28784 [ 49.343726] rcu-scale: 1 writer-duration: 90 26576 [ 49.343731] rcu-scale: 1 writer-duration: 91 35800 [ 49.343737] rcu-scale: 1 writer-duration: 92 23918 [ 49.343742] rcu-scale: 1 writer-duration: 93 28487 [ 49.343747] rcu-scale: 1 writer-duration: 94 26491 [ 49.343753] rcu-scale: 1 writer-duration: 95 35565 [ 49.343758] rcu-scale: 1 writer-duration: 96 24092 [ 49.343763] rcu-scale: 1 writer-duration: 97 28469 [ 49.343768] rcu-scale: 1 writer-duration: 98 26865 [ 49.343774] rcu-scale: 1 writer-duration: 99 35343 [ 49.343779] rcu-scale: 1 writer-duration: 100 23995 [ 49.345221] rcu-scale: 1 writer-duration: 101 60524 [ 49.345235] rcu-scale: 2 writer-duration: 0 56581 [ 49.347314] rcu-scale: 2 writer-duration: 1 32281 [ 49.347329] rcu-scale: 2 writer-duration: 2 29516 [ 49.347336] rcu-scale: 2 writer-duration: 3 29979 [ 49.347342] rcu-scale: 2 writer-duration: 4 36232 [ 49.347348] rcu-scale: 2 writer-duration: 5 26878 [ 49.347354] rcu-scale: 2 writer-duration: 6 29099 [ 49.347359] rcu-scale: 2 writer-duration: 7 24890 [ 49.347365] rcu-scale: 2 writer-duration: 8 35614 [ 49.347371] rcu-scale: 2 writer-duration: 9 26166 [ 49.347377] rcu-scale: 2 writer-duration: 10 28552 [ 49.347384] rcu-scale: 2 writer-duration: 11 24335 [ 49.347390] rcu-scale: 2 writer-duration: 12 35614 [ 49.347396] rcu-scale: 2 writer-duration: 13 25978 [ 49.347402] rcu-scale: 2 writer-duration: 14 28541 [ 49.347408] rcu-scale: 2 writer-duration: 15 24634 [ 49.347413] rcu-scale: 2 writer-duration: 16 35640 [ 49.347419] rcu-scale: 2 writer-duration: 17 26235 [ 49.347425] rcu-scale: 2 writer-duration: 18 28782 [ 49.347431] rcu-scale: 2 writer-duration: 19 24288 [ 49.347437] rcu-scale: 2 writer-duration: 20 35649 [ 49.347442] rcu-scale: 2 writer-duration: 21 25989 [ 49.347448] rcu-scale: 2 writer-duration: 22 28527 [ 49.347454] rcu-scale: 2 writer-duration: 23 24150 [ 49.347460] rcu-scale: 2 writer-duration: 24 35960 [ 49.347465] rcu-scale: 2 writer-duration: 25 26133 [ 49.347471] rcu-scale: 2 writer-duration: 26 28714 [ 49.347477] rcu-scale: 2 writer-duration: 27 24192 [ 49.347482] rcu-scale: 2 writer-duration: 28 35545 [ 49.347488] rcu-scale: 2 writer-duration: 29 26893 [ 49.347494] rcu-scale: 2 writer-duration: 30 28621 [ 49.347499] rcu-scale: 2 writer-duration: 31 24453 [ 49.347505] rcu-scale: 2 writer-duration: 32 35750 [ 49.347510] rcu-scale: 2 writer-duration: 33 31021 [ 49.347516] rcu-scale: 2 writer-duration: 34 29061 [ 49.347522] rcu-scale: 2 writer-duration: 35 25015 [ 49.347528] rcu-scale: 2 writer-duration: 36 36107 [ 49.347534] rcu-scale: 2 writer-duration: 37 26370 [ 49.347540] rcu-scale: 2 writer-duration: 38 28888 [ 49.347546] rcu-scale: 2 writer-duration: 39 24418 [ 49.347552] rcu-scale: 2 writer-duration: 40 35675 [ 49.347558] rcu-scale: 2 writer-duration: 41 26284 [ 49.347564] rcu-scale: 2 writer-duration: 42 28924 [ 49.347570] rcu-scale: 2 writer-duration: 43 24244 [ 49.347576] rcu-scale: 2 writer-duration: 44 35835 [ 49.347582] rcu-scale: 2 writer-duration: 45 26194 [ 49.347587] rcu-scale: 2 writer-duration: 46 29080 [ 49.347593] rcu-scale: 2 writer-duration: 47 24185 [ 49.347599] rcu-scale: 2 writer-duration: 48 35543 [ 49.347605] rcu-scale: 2 writer-duration: 49 26122 [ 49.347610] rcu-scale: 2 writer-duration: 50 28822 [ 49.347616] rcu-scale: 2 writer-duration: 51 24285 [ 49.347622] rcu-scale: 2 writer-duration: 52 35720 [ 49.347627] rcu-scale: 2 writer-duration: 53 26063 [ 49.347633] rcu-scale: 2 writer-duration: 54 28508 [ 49.347639] rcu-scale: 2 writer-duration: 55 24019 [ 49.347645] rcu-scale: 2 writer-duration: 56 35499 [ 49.347651] rcu-scale: 2 writer-duration: 57 26079 [ 49.347657] rcu-scale: 2 writer-duration: 58 28502 [ 49.347663] rcu-scale: 2 writer-duration: 59 24016 [ 49.347668] rcu-scale: 2 writer-duration: 60 35515 [ 49.347673] rcu-scale: 2 writer-duration: 61 26095 [ 49.347677] rcu-scale: 2 writer-duration: 62 28602 [ 49.347681] rcu-scale: 2 writer-duration: 63 24246 [ 49.347686] rcu-scale: 2 writer-duration: 64 35675 [ 49.347690] rcu-scale: 2 writer-duration: 65 26761 [ 49.347694] rcu-scale: 2 writer-duration: 66 28833 [ 49.347699] rcu-scale: 2 writer-duration: 67 24524 [ 49.347703] rcu-scale: 2 writer-duration: 68 43780 [ 49.347707] rcu-scale: 2 writer-duration: 69 26612 [ 49.347712] rcu-scale: 2 writer-duration: 70 29290 [ 49.347716] rcu-scale: 2 writer-duration: 71 24670 [ 49.347720] rcu-scale: 2 writer-duration: 72 35526 [ 49.347725] rcu-scale: 2 writer-duration: 73 26393 [ 49.347729] rcu-scale: 2 writer-duration: 74 28967 [ 49.347733] rcu-scale: 2 writer-duration: 75 24125 [ 49.347738] rcu-scale: 2 writer-duration: 76 35713 [ 49.347742] rcu-scale: 2 writer-duration: 77 26062 [ 49.347746] rcu-scale: 2 writer-duration: 78 28888 [ 49.347751] rcu-scale: 2 writer-duration: 79 24344 [ 49.347755] rcu-scale: 2 writer-duration: 80 35858 [ 49.347760] rcu-scale: 2 writer-duration: 81 26330 [ 49.347764] rcu-scale: 2 writer-duration: 82 28661 [ 49.347769] rcu-scale: 2 writer-duration: 83 24180 [ 49.347773] rcu-scale: 2 writer-duration: 84 35464 [ 49.347778] rcu-scale: 2 writer-duration: 85 25886 [ 49.347782] rcu-scale: 2 writer-duration: 86 28916 [ 49.347786] rcu-scale: 2 writer-duration: 87 24149 [ 49.347791] rcu-scale: 2 writer-duration: 88 35860 [ 49.347795] rcu-scale: 2 writer-duration: 89 26100 [ 49.347800] rcu-scale: 2 writer-duration: 90 28999 [ 49.347804] rcu-scale: 2 writer-duration: 91 24105 [ 49.347809] rcu-scale: 2 writer-duration: 92 35720 [ 49.347813] rcu-scale: 2 writer-duration: 93 26093 [ 49.347817] rcu-scale: 2 writer-duration: 94 28729 [ 49.347822] rcu-scale: 2 writer-duration: 95 24063 [ 49.347826] rcu-scale: 2 writer-duration: 96 35819 [ 49.347831] rcu-scale: 2 writer-duration: 97 26119 [ 49.347835] rcu-scale: 2 writer-duration: 98 28648 [ 49.347840] rcu-scale: 2 writer-duration: 99 24175 [ 49.347844] rcu-scale: 2 writer-duration: 100 34035 [ 49.349299] rcu-scale: 3 writer-duration: 0 26939 [ 49.351268] rcu-scale: 3 writer-duration: 1 27152 [ 49.351277] rcu-scale: 3 writer-duration: 2 30299 [ 49.351281] rcu-scale: 3 writer-duration: 3 36662 [ 49.351284] rcu-scale: 3 writer-duration: 4 31847 [ 49.351287] rcu-scale: 3 writer-duration: 5 29203 [ 49.351290] rcu-scale: 3 writer-duration: 6 24861 [ 49.351294] rcu-scale: 3 writer-duration: 7 36222 [ 49.351297] rcu-scale: 3 writer-duration: 8 26790 [ 49.351300] rcu-scale: 3 writer-duration: 9 28626 [ 49.351303] rcu-scale: 3 writer-duration: 10 24007 [ 49.351307] rcu-scale: 3 writer-duration: 11 35765 [ 49.351310] rcu-scale: 3 writer-duration: 12 26226 [ 49.351314] rcu-scale: 3 writer-duration: 13 28495 [ 49.351317] rcu-scale: 3 writer-duration: 14 24188 [ 49.351321] rcu-scale: 3 writer-duration: 15 35686 [ 49.351324] rcu-scale: 3 writer-duration: 16 26290 [ 49.351327] rcu-scale: 3 writer-duration: 17 28740 [ 49.351331] rcu-scale: 3 writer-duration: 18 24201 [ 49.351334] rcu-scale: 3 writer-duration: 19 36000 [ 49.351337] rcu-scale: 3 writer-duration: 20 25995 [ 49.351341] rcu-scale: 3 writer-duration: 21 28649 [ 49.351344] rcu-scale: 3 writer-duration: 22 24081 [ 49.351348] rcu-scale: 3 writer-duration: 23 35718 [ 49.351351] rcu-scale: 3 writer-duration: 24 26077 [ 49.351354] rcu-scale: 3 writer-duration: 25 28773 [ 49.351357] rcu-scale: 3 writer-duration: 26 24280 [ 49.351360] rcu-scale: 3 writer-duration: 27 35767 [ 49.351364] rcu-scale: 3 writer-duration: 28 25956 [ 49.351367] rcu-scale: 3 writer-duration: 29 28632 [ 49.351370] rcu-scale: 3 writer-duration: 30 24996 [ 49.351374] rcu-scale: 3 writer-duration: 31 35673 [ 49.351377] rcu-scale: 3 writer-duration: 32 26217 [ 49.351380] rcu-scale: 3 writer-duration: 33 28855 [ 49.351383] rcu-scale: 3 writer-duration: 34 28655 [ 49.351387] rcu-scale: 3 writer-duration: 35 36525 [ 49.351390] rcu-scale: 3 writer-duration: 36 27052 [ 49.351394] rcu-scale: 3 writer-duration: 37 28841 [ 49.351397] rcu-scale: 3 writer-duration: 38 24600 [ 49.351400] rcu-scale: 3 writer-duration: 39 35743 [ 49.351404] rcu-scale: 3 writer-duration: 40 26391 [ 49.351407] rcu-scale: 3 writer-duration: 41 28642 [ 49.351410] rcu-scale: 3 writer-duration: 42 24334 [ 49.351413] rcu-scale: 3 writer-duration: 43 36152 [ 49.351417] rcu-scale: 3 writer-duration: 44 26133 [ 49.351420] rcu-scale: 3 writer-duration: 45 28732 [ 49.351423] rcu-scale: 3 writer-duration: 46 24235 [ 49.351426] rcu-scale: 3 writer-duration: 47 36069 [ 49.351429] rcu-scale: 3 writer-duration: 48 25993 [ 49.351433] rcu-scale: 3 writer-duration: 49 28658 [ 49.351436] rcu-scale: 3 writer-duration: 50 24225 [ 49.351439] rcu-scale: 3 writer-duration: 51 35909 [ 49.351442] rcu-scale: 3 writer-duration: 52 26176 [ 49.351446] rcu-scale: 3 writer-duration: 53 28702 [ 49.351449] rcu-scale: 3 writer-duration: 54 24169 [ 49.351453] rcu-scale: 3 writer-duration: 55 35462 [ 49.351456] rcu-scale: 3 writer-duration: 56 25771 [ 49.351459] rcu-scale: 3 writer-duration: 57 28677 [ 49.351462] rcu-scale: 3 writer-duration: 58 24089 [ 49.351466] rcu-scale: 3 writer-duration: 59 35538 [ 49.351469] rcu-scale: 3 writer-duration: 60 25785 [ 49.351472] rcu-scale: 3 writer-duration: 61 28669 [ 49.351475] rcu-scale: 3 writer-duration: 62 24075 [ 49.351478] rcu-scale: 3 writer-duration: 63 35720 [ 49.351482] rcu-scale: 3 writer-duration: 64 26017 [ 49.351485] rcu-scale: 3 writer-duration: 65 28726 [ 49.351488] rcu-scale: 3 writer-duration: 66 24990 [ 49.351491] rcu-scale: 3 writer-duration: 67 35814 [ 49.351494] rcu-scale: 3 writer-duration: 68 26215 [ 49.351498] rcu-scale: 3 writer-duration: 69 34468 [ 49.351501] rcu-scale: 3 writer-duration: 70 27200 [ 49.351504] rcu-scale: 3 writer-duration: 71 36288 [ 49.351507] rcu-scale: 3 writer-duration: 72 26445 [ 49.351511] rcu-scale: 3 writer-duration: 73 28593 [ 49.351514] rcu-scale: 3 writer-duration: 74 24317 [ 49.351517] rcu-scale: 3 writer-duration: 75 36153 [ 49.351521] rcu-scale: 3 writer-duration: 76 26164 [ 49.351524] rcu-scale: 3 writer-duration: 77 28523 [ 49.351527] rcu-scale: 3 writer-duration: 78 24252 [ 49.351530] rcu-scale: 3 writer-duration: 79 35939 [ 49.351533] rcu-scale: 3 writer-duration: 80 26058 [ 49.351536] rcu-scale: 3 writer-duration: 81 28892 [ 49.351540] rcu-scale: 3 writer-duration: 82 24342 [ 49.351543] rcu-scale: 3 writer-duration: 83 35751 [ 49.351546] rcu-scale: 3 writer-duration: 84 25932 [ 49.351549] rcu-scale: 3 writer-duration: 85 28682 [ 49.351552] rcu-scale: 3 writer-duration: 86 24086 [ 49.351555] rcu-scale: 3 writer-duration: 87 35791 [ 49.351559] rcu-scale: 3 writer-duration: 88 26189 [ 49.351562] rcu-scale: 3 writer-duration: 89 28754 [ 49.351565] rcu-scale: 3 writer-duration: 90 24250 [ 49.351568] rcu-scale: 3 writer-duration: 91 35768 [ 49.351572] rcu-scale: 3 writer-duration: 92 26160 [ 49.351575] rcu-scale: 3 writer-duration: 93 28896 [ 49.351578] rcu-scale: 3 writer-duration: 94 24020 [ 49.351581] rcu-scale: 3 writer-duration: 95 35727 [ 49.351584] rcu-scale: 3 writer-duration: 96 26099 [ 49.351587] rcu-scale: 3 writer-duration: 97 28768 [ 49.351590] rcu-scale: 3 writer-duration: 98 24011 [ 49.351593] rcu-scale: 3 writer-duration: 99 35786 [ 49.351597] rcu-scale: 3 writer-duration: 100 25989 [ 49.353257] rcu-scale: 3 writer-duration: 101 46411 [ 47.850402] rcu-scale: rcu_scale_writer 3 has 100 measurements [ 47.850427] rcu-scale: rcu_scale_writer 0 has 100 measurements [ 47.850429] rcu-scale: rcu_scale_writer 1 has 100 measurements [ 47.850432] rcu-scale: rcu_scale_writer 2 has 100 measurements [ 47.861470] Dumping ftrace buffer: [ 47.861486] (ftrace buffer empty) [ 47.861491] rcu-scale: Test complete [ 63.668029] rcu-scale:!!! All grace periods expedited, no normal ones to measure! [ 63.668051] rcu-torture: Stopping rcu_scale_reader task [ 63.668074] rcu-torture: rcu_scale_reader is stopping [ 63.668089] rcu-torture: Stopping rcu_scale_reader task [ 63.668138] rcu-torture: rcu_scale_reader is stopping [ 63.668151] rcu-torture: Stopping rcu_scale_reader task [ 63.668191] rcu-torture: rcu_scale_reader is stopping [ 63.668204] rcu-torture: Stopping rcu_scale_reader task [ 63.668209] rcu-torture: rcu_scale_reader is stopping [ 63.668231] rcu-torture: Stopping rcu_scale_writer task [ 63.668241] rcu-torture: rcu_scale_writer is stopping [ 63.668249] rcu-scale: writer 0 gps: 102 [ 63.668251] rcu-torture: Stopping rcu_scale_writer task [ 63.668260] rcu-torture: rcu_scale_writer is stopping [ 63.668268] rcu-scale: writer 1 gps: 102 [ 63.668270] rcu-torture: Stopping rcu_scale_writer task [ 63.668279] rcu-torture: rcu_scale_writer is stopping [ 63.668287] rcu-scale: writer 2 gps: 101 [ 63.668288] rcu-torture: Stopping rcu_scale_writer task [ 63.668293] rcu-torture: rcu_scale_writer is stopping [ 63.668310] rcu-scale: writer 3 gps: 102 [ 63.668311] rcu-scale: start: 47784890510 end: 47787901801 duration: 3011291 gps: 407 batches: 2 [ 63.668314] rcu-scale: 0 writer-duration: 0 55830 [ 63.669543] rcu-scale: 0 writer-duration: 1 40421 [ 63.669546] rcu-scale: 0 writer-duration: 2 25268 [ 63.669548] rcu-scale: 0 writer-duration: 3 32703 [ 63.669549] rcu-scale: 0 writer-duration: 4 28624 [ 63.669550] rcu-scale: 0 writer-duration: 5 36967 [ 63.669551] rcu-scale: 0 writer-duration: 6 24686 [ 63.669552] rcu-scale: 0 writer-duration: 7 29598 [ 63.669554] rcu-scale: 0 writer-duration: 8 27026 [ 63.669555] rcu-scale: 0 writer-duration: 9 38789 [ 63.669556] rcu-scale: 0 writer-duration: 10 24398 [ 63.669557] rcu-scale: 0 writer-duration: 11 29562 [ 63.669559] rcu-scale: 0 writer-duration: 12 27348 [ 63.669560] rcu-scale: 0 writer-duration: 13 36315 [ 63.669561] rcu-scale: 0 writer-duration: 14 24233 [ 63.669562] rcu-scale: 0 writer-duration: 15 29035 [ 63.669564] rcu-scale: 0 writer-duration: 16 26754 [ 63.669606] rcu-scale: 0 writer-duration: 17 36272 [ 63.669608] rcu-scale: 0 writer-duration: 18 24237 [ 63.669609] rcu-scale: 0 writer-duration: 19 28834 [ 63.669610] rcu-scale: 0 writer-duration: 20 26993 [ 63.669611] rcu-scale: 0 writer-duration: 21 36432 [ 63.669612] rcu-scale: 0 writer-duration: 22 24178 [ 63.669614] rcu-scale: 0 writer-duration: 23 28838 [ 63.669615] rcu-scale: 0 writer-duration: 24 26733 [ 63.669616] rcu-scale: 0 writer-duration: 25 36087 [ 63.669617] rcu-scale: 0 writer-duration: 26 24237 [ 63.669618] rcu-scale: 0 writer-duration: 27 28656 [ 63.669620] rcu-scale: 0 writer-duration: 28 26921 [ 63.669621] rcu-scale: 0 writer-duration: 29 36423 [ 63.669622] rcu-scale: 0 writer-duration: 30 24201 [ 63.669623] rcu-scale: 0 writer-duration: 31 28748 [ 63.669625] rcu-scale: 0 writer-duration: 32 31925 [ 63.669626] rcu-scale: 0 writer-duration: 33 36673 [ 63.669627] rcu-scale: 0 writer-duration: 34 24630 [ 63.669628] rcu-scale: 0 writer-duration: 35 29033 [ 63.669629] rcu-scale: 0 writer-duration: 36 26761 [ 63.669631] rcu-scale: 0 writer-duration: 37 36495 [ 63.669632] rcu-scale: 0 writer-duration: 38 24209 [ 63.669633] rcu-scale: 0 writer-duration: 39 29468 [ 63.669634] rcu-scale: 0 writer-duration: 40 27137 [ 63.669635] rcu-scale: 0 writer-duration: 41 36193 [ 63.669637] rcu-scale: 0 writer-duration: 42 24531 [ 63.669638] rcu-scale: 0 writer-duration: 43 29113 [ 63.669639] rcu-scale: 0 writer-duration: 44 26915 [ 63.669640] rcu-scale: 0 writer-duration: 45 36394 [ 63.669642] rcu-scale: 0 writer-duration: 46 24247 [ 63.669643] rcu-scale: 0 writer-duration: 47 28823 [ 63.669644] rcu-scale: 0 writer-duration: 48 27029 [ 63.669645] rcu-scale: 0 writer-duration: 49 36042 [ 63.669646] rcu-scale: 0 writer-duration: 50 24164 [ 63.669648] rcu-scale: 0 writer-duration: 51 28972 [ 63.669649] rcu-scale: 0 writer-duration: 52 26858 [ 63.669650] rcu-scale: 0 writer-duration: 53 36319 [ 63.669651] rcu-scale: 0 writer-duration: 54 24233 [ 63.669652] rcu-scale: 0 writer-duration: 55 28986 [ 63.669654] rcu-scale: 0 writer-duration: 56 26895 [ 63.669655] rcu-scale: 0 writer-duration: 57 36213 [ 63.669656] rcu-scale: 0 writer-duration: 58 24156 [ 63.669657] rcu-scale: 0 writer-duration: 59 28859 [ 63.669658] rcu-scale: 0 writer-duration: 60 26917 [ 63.669660] rcu-scale: 0 writer-duration: 61 36174 [ 63.669661] rcu-scale: 0 writer-duration: 62 24237 [ 63.669662] rcu-scale: 0 writer-duration: 63 28873 [ 63.669663] rcu-scale: 0 writer-duration: 64 27045 [ 63.669664] rcu-scale: 0 writer-duration: 65 36167 [ 63.669666] rcu-scale: 0 writer-duration: 66 29556 [ 63.669667] rcu-scale: 0 writer-duration: 67 28900 [ 63.669668] rcu-scale: 0 writer-duration: 68 27072 [ 63.669669] rcu-scale: 0 writer-duration: 69 36733 [ 63.669670] rcu-scale: 0 writer-duration: 70 24327 [ 63.669672] rcu-scale: 0 writer-duration: 71 28842 [ 63.669673] rcu-scale: 0 writer-duration: 72 27027 [ 63.669674] rcu-scale: 0 writer-duration: 73 36228 [ 63.669675] rcu-scale: 0 writer-duration: 74 24137 [ 63.669677] rcu-scale: 0 writer-duration: 75 29924 [ 63.669678] rcu-scale: 0 writer-duration: 76 26775 [ 63.669679] rcu-scale: 0 writer-duration: 77 36300 [ 63.669680] rcu-scale: 0 writer-duration: 78 24547 [ 63.669681] rcu-scale: 0 writer-duration: 79 28852 [ 63.669683] rcu-scale: 0 writer-duration: 80 26945 [ 63.669684] rcu-scale: 0 writer-duration: 81 36127 [ 63.669685] rcu-scale: 0 writer-duration: 82 24373 [ 63.669686] rcu-scale: 0 writer-duration: 83 28853 [ 63.669687] rcu-scale: 0 writer-duration: 84 26652 [ 63.669689] rcu-scale: 0 writer-duration: 85 35952 [ 63.669690] rcu-scale: 0 writer-duration: 86 24379 [ 63.669691] rcu-scale: 0 writer-duration: 87 28948 [ 63.669692] rcu-scale: 0 writer-duration: 88 26958 [ 63.669693] rcu-scale: 0 writer-duration: 89 36302 [ 63.669695] rcu-scale: 0 writer-duration: 90 24181 [ 63.669696] rcu-scale: 0 writer-duration: 91 29120 [ 63.669697] rcu-scale: 0 writer-duration: 92 27032 [ 63.669698] rcu-scale: 0 writer-duration: 93 36083 [ 63.669699] rcu-scale: 0 writer-duration: 94 24248 [ 63.669701] rcu-scale: 0 writer-duration: 95 29205 [ 63.669702] rcu-scale: 0 writer-duration: 96 26577 [ 63.669703] rcu-scale: 0 writer-duration: 97 36187 [ 63.669704] rcu-scale: 0 writer-duration: 98 24472 [ 63.669705] rcu-scale: 0 writer-duration: 99 28883 [ 63.669707] rcu-scale: 0 writer-duration: 100 31999 [ 63.671532] rcu-scale: 0 writer-duration: 101 48271 [ 63.671556] rcu-scale: 1 writer-duration: 0 53537 [ 63.673442] rcu-scale: 1 writer-duration: 1 33311 [ 63.673451] rcu-scale: 1 writer-duration: 2 28033 [ 63.673457] rcu-scale: 1 writer-duration: 3 40200 [ 63.673464] rcu-scale: 1 writer-duration: 4 25479 [ 63.673470] rcu-scale: 1 writer-duration: 5 29908 [ 63.673475] rcu-scale: 1 writer-duration: 6 27336 [ 63.673481] rcu-scale: 1 writer-duration: 7 36948 [ 63.673487] rcu-scale: 1 writer-duration: 8 24216 [ 63.673494] rcu-scale: 1 writer-duration: 9 31639 [ 63.673499] rcu-scale: 1 writer-duration: 10 27175 [ 63.673506] rcu-scale: 1 writer-duration: 11 36695 [ 63.673512] rcu-scale: 1 writer-duration: 12 24609 [ 63.673518] rcu-scale: 1 writer-duration: 13 29140 [ 63.673524] rcu-scale: 1 writer-duration: 14 26955 [ 63.673530] rcu-scale: 1 writer-duration: 15 36122 [ 63.673535] rcu-scale: 1 writer-duration: 16 24236 [ 63.673541] rcu-scale: 1 writer-duration: 17 29137 [ 63.673546] rcu-scale: 1 writer-duration: 18 26766 [ 63.673552] rcu-scale: 1 writer-duration: 19 36207 [ 63.673558] rcu-scale: 1 writer-duration: 20 24215 [ 63.673563] rcu-scale: 1 writer-duration: 21 29239 [ 63.673569] rcu-scale: 1 writer-duration: 22 26866 [ 63.673575] rcu-scale: 1 writer-duration: 23 36054 [ 63.673581] rcu-scale: 1 writer-duration: 24 24012 [ 63.673586] rcu-scale: 1 writer-duration: 25 28914 [ 63.673592] rcu-scale: 1 writer-duration: 26 26891 [ 63.673597] rcu-scale: 1 writer-duration: 27 36102 [ 63.673603] rcu-scale: 1 writer-duration: 28 24039 [ 63.673609] rcu-scale: 1 writer-duration: 29 29125 [ 63.673614] rcu-scale: 1 writer-duration: 30 26926 [ 63.673620] rcu-scale: 1 writer-duration: 31 40754 [ 63.673625] rcu-scale: 1 writer-duration: 32 24519 [ 63.673631] rcu-scale: 1 writer-duration: 33 29397 [ 63.673637] rcu-scale: 1 writer-duration: 34 27267 [ 63.673642] rcu-scale: 1 writer-duration: 35 36358 [ 63.673648] rcu-scale: 1 writer-duration: 36 24007 [ 63.673654] rcu-scale: 1 writer-duration: 37 29326 [ 63.673659] rcu-scale: 1 writer-duration: 38 26854 [ 63.673665] rcu-scale: 1 writer-duration: 39 36625 [ 63.673671] rcu-scale: 1 writer-duration: 40 24437 [ 63.673676] rcu-scale: 1 writer-duration: 41 29314 [ 63.673682] rcu-scale: 1 writer-duration: 42 26955 [ 63.673688] rcu-scale: 1 writer-duration: 43 36415 [ 63.673693] rcu-scale: 1 writer-duration: 44 24098 [ 63.673699] rcu-scale: 1 writer-duration: 45 29216 [ 63.673705] rcu-scale: 1 writer-duration: 46 26904 [ 63.673710] rcu-scale: 1 writer-duration: 47 36286 [ 63.673716] rcu-scale: 1 writer-duration: 48 24127 [ 63.673722] rcu-scale: 1 writer-duration: 49 28887 [ 63.673727] rcu-scale: 1 writer-duration: 50 26894 [ 63.673733] rcu-scale: 1 writer-duration: 51 36145 [ 63.673739] rcu-scale: 1 writer-duration: 52 24238 [ 63.673745] rcu-scale: 1 writer-duration: 53 29136 [ 63.673750] rcu-scale: 1 writer-duration: 54 26895 [ 63.673756] rcu-scale: 1 writer-duration: 55 36087 [ 63.673761] rcu-scale: 1 writer-duration: 56 24215 [ 63.673767] rcu-scale: 1 writer-duration: 57 29034 [ 63.673773] rcu-scale: 1 writer-duration: 58 26817 [ 63.673778] rcu-scale: 1 writer-duration: 59 36069 [ 63.673784] rcu-scale: 1 writer-duration: 60 24255 [ 63.673790] rcu-scale: 1 writer-duration: 61 29004 [ 63.673795] rcu-scale: 1 writer-duration: 62 26878 [ 63.673801] rcu-scale: 1 writer-duration: 63 36186 [ 63.673806] rcu-scale: 1 writer-duration: 64 24355 [ 63.673812] rcu-scale: 1 writer-duration: 65 28884 [ 63.673818] rcu-scale: 1 writer-duration: 66 32061 [ 63.673823] rcu-scale: 1 writer-duration: 67 36261 [ 63.673829] rcu-scale: 1 writer-duration: 68 24453 [ 63.673835] rcu-scale: 1 writer-duration: 69 29508 [ 63.673840] rcu-scale: 1 writer-duration: 70 27004 [ 63.673846] rcu-scale: 1 writer-duration: 71 36240 [ 63.673851] rcu-scale: 1 writer-duration: 72 23958 [ 63.673857] rcu-scale: 1 writer-duration: 73 29276 [ 63.673863] rcu-scale: 1 writer-duration: 74 26793 [ 63.673868] rcu-scale: 1 writer-duration: 75 36779 [ 63.673874] rcu-scale: 1 writer-duration: 76 24462 [ 63.673880] rcu-scale: 1 writer-duration: 77 29156 [ 63.673885] rcu-scale: 1 writer-duration: 78 27128 [ 63.673891] rcu-scale: 1 writer-duration: 79 36182 [ 63.673896] rcu-scale: 1 writer-duration: 80 24073 [ 63.673902] rcu-scale: 1 writer-duration: 81 28955 [ 63.673908] rcu-scale: 1 writer-duration: 82 27009 [ 63.673914] rcu-scale: 1 writer-duration: 83 36169 [ 63.673919] rcu-scale: 1 writer-duration: 84 23906 [ 63.673925] rcu-scale: 1 writer-duration: 85 28946 [ 63.673931] rcu-scale: 1 writer-duration: 86 26866 [ 63.673937] rcu-scale: 1 writer-duration: 87 36249 [ 63.673942] rcu-scale: 1 writer-duration: 88 24096 [ 63.673948] rcu-scale: 1 writer-duration: 89 29290 [ 63.673954] rcu-scale: 1 writer-duration: 90 26844 [ 63.673959] rcu-scale: 1 writer-duration: 91 36353 [ 63.673965] rcu-scale: 1 writer-duration: 92 24166 [ 63.673971] rcu-scale: 1 writer-duration: 93 29072 [ 63.673976] rcu-scale: 1 writer-duration: 94 26862 [ 63.673982] rcu-scale: 1 writer-duration: 95 36348 [ 63.673988] rcu-scale: 1 writer-duration: 96 23907 [ 63.673993] rcu-scale: 1 writer-duration: 97 29122 [ 63.673999] rcu-scale: 1 writer-duration: 98 27072 [ 63.674004] rcu-scale: 1 writer-duration: 99 36188 [ 63.674010] rcu-scale: 1 writer-duration: 100 29125 [ 63.675441] rcu-scale: 1 writer-duration: 101 48063 [ 63.675458] rcu-scale: 2 writer-duration: 0 65839 [ 63.677455] rcu-scale: 2 writer-duration: 1 30102 [ 63.677470] rcu-scale: 2 writer-duration: 2 30481 [ 63.677477] rcu-scale: 2 writer-duration: 3 28129 [ 63.677483] rcu-scale: 2 writer-duration: 4 37897 [ 63.677489] rcu-scale: 2 writer-duration: 5 27420 [ 63.677496] rcu-scale: 2 writer-duration: 6 29475 [ 63.677501] rcu-scale: 2 writer-duration: 7 25128 [ 63.677507] rcu-scale: 2 writer-duration: 8 36449 [ 63.677512] rcu-scale: 2 writer-duration: 9 29093 [ 63.677518] rcu-scale: 2 writer-duration: 10 29369 [ 63.677524] rcu-scale: 2 writer-duration: 11 24938 [ 63.677530] rcu-scale: 2 writer-duration: 12 36634 [ 63.677535] rcu-scale: 2 writer-duration: 13 26628 [ 63.677541] rcu-scale: 2 writer-duration: 14 29116 [ 63.677547] rcu-scale: 2 writer-duration: 15 24320 [ 63.677552] rcu-scale: 2 writer-duration: 16 36351 [ 63.677558] rcu-scale: 2 writer-duration: 17 26613 [ 63.677564] rcu-scale: 2 writer-duration: 18 29039 [ 63.677570] rcu-scale: 2 writer-duration: 19 24353 [ 63.677575] rcu-scale: 2 writer-duration: 20 36335 [ 63.677581] rcu-scale: 2 writer-duration: 21 26706 [ 63.677587] rcu-scale: 2 writer-duration: 22 29200 [ 63.677592] rcu-scale: 2 writer-duration: 23 24211 [ 63.677598] rcu-scale: 2 writer-duration: 24 36036 [ 63.677604] rcu-scale: 2 writer-duration: 25 26464 [ 63.677610] rcu-scale: 2 writer-duration: 26 29181 [ 63.677616] rcu-scale: 2 writer-duration: 27 24021 [ 63.677621] rcu-scale: 2 writer-duration: 28 36169 [ 63.677627] rcu-scale: 2 writer-duration: 29 26697 [ 63.677633] rcu-scale: 2 writer-duration: 30 29213 [ 63.677638] rcu-scale: 2 writer-duration: 31 24181 [ 63.677644] rcu-scale: 2 writer-duration: 32 41287 [ 63.677650] rcu-scale: 2 writer-duration: 33 27016 [ 63.677656] rcu-scale: 2 writer-duration: 34 29455 [ 63.677662] rcu-scale: 2 writer-duration: 35 24571 [ 63.677667] rcu-scale: 2 writer-duration: 36 36144 [ 63.677673] rcu-scale: 2 writer-duration: 37 26673 [ 63.677678] rcu-scale: 2 writer-duration: 38 29031 [ 63.677684] rcu-scale: 2 writer-duration: 39 25127 [ 63.677690] rcu-scale: 2 writer-duration: 40 36453 [ 63.677695] rcu-scale: 2 writer-duration: 41 26455 [ 63.677701] rcu-scale: 2 writer-duration: 42 29460 [ 63.677707] rcu-scale: 2 writer-duration: 43 24698 [ 63.677712] rcu-scale: 2 writer-duration: 44 36093 [ 63.677718] rcu-scale: 2 writer-duration: 45 26573 [ 63.677723] rcu-scale: 2 writer-duration: 46 29157 [ 63.677729] rcu-scale: 2 writer-duration: 47 24356 [ 63.677735] rcu-scale: 2 writer-duration: 48 36504 [ 63.677741] rcu-scale: 2 writer-duration: 49 26157 [ 63.677746] rcu-scale: 2 writer-duration: 50 29103 [ 63.677752] rcu-scale: 2 writer-duration: 51 24577 [ 63.677758] rcu-scale: 2 writer-duration: 52 36218 [ 63.677763] rcu-scale: 2 writer-duration: 53 26553 [ 63.677769] rcu-scale: 2 writer-duration: 54 29224 [ 63.677775] rcu-scale: 2 writer-duration: 55 24290 [ 63.677780] rcu-scale: 2 writer-duration: 56 36322 [ 63.677786] rcu-scale: 2 writer-duration: 57 26509 [ 63.677792] rcu-scale: 2 writer-duration: 58 28951 [ 63.677797] rcu-scale: 2 writer-duration: 59 24316 [ 63.677803] rcu-scale: 2 writer-duration: 60 36402 [ 63.677809] rcu-scale: 2 writer-duration: 61 26453 [ 63.677814] rcu-scale: 2 writer-duration: 62 29023 [ 63.677820] rcu-scale: 2 writer-duration: 63 24538 [ 63.677826] rcu-scale: 2 writer-duration: 64 36279 [ 63.677831] rcu-scale: 2 writer-duration: 65 26277 [ 63.677837] rcu-scale: 2 writer-duration: 66 34442 [ 63.677843] rcu-scale: 2 writer-duration: 67 24564 [ 63.677849] rcu-scale: 2 writer-duration: 68 36429 [ 63.677855] rcu-scale: 2 writer-duration: 69 27018 [ 63.677861] rcu-scale: 2 writer-duration: 70 29207 [ 63.677866] rcu-scale: 2 writer-duration: 71 24488 [ 63.677872] rcu-scale: 2 writer-duration: 72 36187 [ 63.677878] rcu-scale: 2 writer-duration: 73 26564 [ 63.677883] rcu-scale: 2 writer-duration: 74 29127 [ 63.677889] rcu-scale: 2 writer-duration: 75 25213 [ 63.677894] rcu-scale: 2 writer-duration: 76 36202 [ 63.677900] rcu-scale: 2 writer-duration: 77 26709 [ 63.677905] rcu-scale: 2 writer-duration: 78 29292 [ 63.677911] rcu-scale: 2 writer-duration: 79 24338 [ 63.677917] rcu-scale: 2 writer-duration: 80 36322 [ 63.677922] rcu-scale: 2 writer-duration: 81 26289 [ 63.677928] rcu-scale: 2 writer-duration: 82 29386 [ 63.677934] rcu-scale: 2 writer-duration: 83 24300 [ 63.677939] rcu-scale: 2 writer-duration: 84 36011 [ 63.677945] rcu-scale: 2 writer-duration: 85 26219 [ 63.677951] rcu-scale: 2 writer-duration: 86 29250 [ 63.677957] rcu-scale: 2 writer-duration: 87 24484 [ 63.677963] rcu-scale: 2 writer-duration: 88 36254 [ 63.677969] rcu-scale: 2 writer-duration: 89 26631 [ 63.677975] rcu-scale: 2 writer-duration: 90 29105 [ 63.677980] rcu-scale: 2 writer-duration: 91 24502 [ 63.677986] rcu-scale: 2 writer-duration: 92 36409 [ 63.677992] rcu-scale: 2 writer-duration: 93 26414 [ 63.677997] rcu-scale: 2 writer-duration: 94 29130 [ 63.678003] rcu-scale: 2 writer-duration: 95 24586 [ 63.678009] rcu-scale: 2 writer-duration: 96 36159 [ 63.678014] rcu-scale: 2 writer-duration: 97 26263 [ 63.678021] rcu-scale: 2 writer-duration: 98 29479 [ 63.678027] rcu-scale: 2 writer-duration: 99 24352 [ 63.678032] rcu-scale: 2 writer-duration: 100 39478 [ 63.679533] rcu-scale: 3 writer-duration: 0 31164 [ 63.681440] rcu-scale: 3 writer-duration: 1 31690 [ 63.681451] rcu-scale: 3 writer-duration: 2 28398 [ 63.681457] rcu-scale: 3 writer-duration: 3 37524 [ 63.681462] rcu-scale: 3 writer-duration: 4 30331 [ 63.681468] rcu-scale: 3 writer-duration: 5 30588 [ 63.681473] rcu-scale: 3 writer-duration: 6 25430 [ 63.681479] rcu-scale: 3 writer-duration: 7 36628 [ 63.681485] rcu-scale: 3 writer-duration: 8 27195 [ 63.681490] rcu-scale: 3 writer-duration: 9 29267 [ 63.681496] rcu-scale: 3 writer-duration: 10 26967 [ 63.681503] rcu-scale: 3 writer-duration: 11 36610 [ 63.681509] rcu-scale: 3 writer-duration: 12 27017 [ 63.681514] rcu-scale: 3 writer-duration: 13 29493 [ 63.681520] rcu-scale: 3 writer-duration: 14 24647 [ 63.681526] rcu-scale: 3 writer-duration: 15 36099 [ 63.681531] rcu-scale: 3 writer-duration: 16 26574 [ 63.681537] rcu-scale: 3 writer-duration: 17 28995 [ 63.681542] rcu-scale: 3 writer-duration: 18 24627 [ 63.681548] rcu-scale: 3 writer-duration: 19 36099 [ 63.681554] rcu-scale: 3 writer-duration: 20 26580 [ 63.681559] rcu-scale: 3 writer-duration: 21 29133 [ 63.681565] rcu-scale: 3 writer-duration: 22 24684 [ 63.681570] rcu-scale: 3 writer-duration: 23 36089 [ 63.681576] rcu-scale: 3 writer-duration: 24 26508 [ 63.681582] rcu-scale: 3 writer-duration: 25 28821 [ 63.681587] rcu-scale: 3 writer-duration: 26 24382 [ 63.681593] rcu-scale: 3 writer-duration: 27 36151 [ 63.681599] rcu-scale: 3 writer-duration: 28 26336 [ 63.681605] rcu-scale: 3 writer-duration: 29 28946 [ 63.681610] rcu-scale: 3 writer-duration: 30 24735 [ 63.681616] rcu-scale: 3 writer-duration: 31 36237 [ 63.681622] rcu-scale: 3 writer-duration: 32 26239 [ 63.681627] rcu-scale: 3 writer-duration: 33 34242 [ 63.681633] rcu-scale: 3 writer-duration: 34 24917 [ 63.681639] rcu-scale: 3 writer-duration: 35 36459 [ 63.681645] rcu-scale: 3 writer-duration: 36 26774 [ 63.681650] rcu-scale: 3 writer-duration: 37 28911 [ 63.681656] rcu-scale: 3 writer-duration: 38 24797 [ 63.681662] rcu-scale: 3 writer-duration: 39 36099 [ 63.681667] rcu-scale: 3 writer-duration: 40 27206 [ 63.681673] rcu-scale: 3 writer-duration: 41 29201 [ 63.681678] rcu-scale: 3 writer-duration: 42 24676 [ 63.681684] rcu-scale: 3 writer-duration: 43 36260 [ 63.681690] rcu-scale: 3 writer-duration: 44 26859 [ 63.681695] rcu-scale: 3 writer-duration: 45 29087 [ 63.681701] rcu-scale: 3 writer-duration: 46 24531 [ 63.681707] rcu-scale: 3 writer-duration: 47 36144 [ 63.681712] rcu-scale: 3 writer-duration: 48 26415 [ 63.681718] rcu-scale: 3 writer-duration: 49 29230 [ 63.681724] rcu-scale: 3 writer-duration: 50 24371 [ 63.681729] rcu-scale: 3 writer-duration: 51 36045 [ 63.681735] rcu-scale: 3 writer-duration: 52 26707 [ 63.681740] rcu-scale: 3 writer-duration: 53 29079 [ 63.681746] rcu-scale: 3 writer-duration: 54 24622 [ 63.681752] rcu-scale: 3 writer-duration: 55 36202 [ 63.681757] rcu-scale: 3 writer-duration: 56 26489 [ 63.681763] rcu-scale: 3 writer-duration: 57 29016 [ 63.681769] rcu-scale: 3 writer-duration: 58 24632 [ 63.681775] rcu-scale: 3 writer-duration: 59 35968 [ 63.681780] rcu-scale: 3 writer-duration: 60 26480 [ 63.681786] rcu-scale: 3 writer-duration: 61 29171 [ 63.681792] rcu-scale: 3 writer-duration: 62 24542 [ 63.681797] rcu-scale: 3 writer-duration: 63 35972 [ 63.681803] rcu-scale: 3 writer-duration: 64 26459 [ 63.681809] rcu-scale: 3 writer-duration: 65 29248 [ 63.681814] rcu-scale: 3 writer-duration: 66 24403 [ 63.681820] rcu-scale: 3 writer-duration: 67 41353 [ 63.681826] rcu-scale: 3 writer-duration: 68 26759 [ 63.681831] rcu-scale: 3 writer-duration: 69 29167 [ 63.681837] rcu-scale: 3 writer-duration: 70 25051 [ 63.681843] rcu-scale: 3 writer-duration: 71 36298 [ 63.681848] rcu-scale: 3 writer-duration: 72 26557 [ 63.681854] rcu-scale: 3 writer-duration: 73 28940 [ 63.681860] rcu-scale: 3 writer-duration: 74 24726 [ 63.681865] rcu-scale: 3 writer-duration: 75 36109 [ 63.681871] rcu-scale: 3 writer-duration: 76 27216 [ 63.681877] rcu-scale: 3 writer-duration: 77 29170 [ 63.681882] rcu-scale: 3 writer-duration: 78 24606 [ 63.681888] rcu-scale: 3 writer-duration: 79 36459 [ 63.681893] rcu-scale: 3 writer-duration: 80 26470 [ 63.681899] rcu-scale: 3 writer-duration: 81 28939 [ 63.681905] rcu-scale: 3 writer-duration: 82 24758 [ 63.681910] rcu-scale: 3 writer-duration: 83 36139 [ 63.681916] rcu-scale: 3 writer-duration: 84 26535 [ 63.681922] rcu-scale: 3 writer-duration: 85 28824 [ 63.681927] rcu-scale: 3 writer-duration: 86 24272 [ 63.681933] rcu-scale: 3 writer-duration: 87 36187 [ 63.681939] rcu-scale: 3 writer-duration: 88 26565 [ 63.681944] rcu-scale: 3 writer-duration: 89 29050 [ 63.681950] rcu-scale: 3 writer-duration: 90 24686 [ 63.681955] rcu-scale: 3 writer-duration: 91 36216 [ 63.681961] rcu-scale: 3 writer-duration: 92 26589 [ 63.681967] rcu-scale: 3 writer-duration: 93 29076 [ 63.681973] rcu-scale: 3 writer-duration: 94 24683 [ 63.681978] rcu-scale: 3 writer-duration: 95 36081 [ 63.681984] rcu-scale: 3 writer-duration: 96 26710 [ 63.681989] rcu-scale: 3 writer-duration: 97 28881 [ 63.681995] rcu-scale: 3 writer-duration: 98 24494 [ 63.682001] rcu-scale: 3 writer-duration: 99 36297 [ 63.682006] rcu-scale: 3 writer-duration: 100 26487 [ 63.683440] rcu-scale: 3 writer-duration: 101 48416
On Thu, Jun 23, 2022 at 03:23:19AM +0000, Zhang, Qiang1 wrote: > On Wed, Jun 22, 2022 at 05:34:09PM -0700, Paul E. McKenney wrote: > > On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > > > > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > > > grace period begins, if detected current CPU enters idle in > > > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > > > current cpu, at this time, maybe not being in an RCU read-side critical > > > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > > > occurs on current CPU to check and report exp QS. > > > > > > > > > > >I think the idea is OK, however, this "optimization" is based on the > > > >implementation detail that rcu_read_lock() counts preempt_count when > > > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > > > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > > > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > > > > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > > > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > > > preempt-count is exists, so can report exp QS when not being an RCU > > > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > > > in IPI handler. > > > > > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > > > The rcu_read_lock/unlock is just barrier(). > > > > > > > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > > > > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > > > preempt-count is also checked > > > > > > /* Report any deferred quiescent states if preemption enabled. */ > > > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > > > rcu_preempt_deferred_qs(current); > > > > > > but the RCU softirq may not be triggered in time and reported exp QS, for > > > example a kernel loop exist on NO_HZ_FULL CPU > > > > > > this change, It is to capture the exp QS state earlier and report it. > > > > > > > > > >Also I'm not aware of any but there could be someone assuming that RCU > > > >read-side critical sections can be formed without > > > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > > > >might be "creative" code like the following: > > > > > > > > void do_something_only_in_nonpreempt(void) > > > > { > > > > int *p; > > > > > > > > // This function only gets called in PREEMPT=n kernel, > > > > // which means everywhere is a RCU read-side critical > > > > // section, let's save some lines of code. > > > > > > > rcu_read_lock(); > > > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > > > ... // of course no schedule() here. > > > > <access p> > > > rcu_read_unlock(); > > > > } > > > > > > > > > > Usually access to pointers of type rcu needs to be protected. > > Yes, _ususally_ they are, but what about the special cases? Because > in PREEMPT=n kernel, almost everywhere is a RCU read-side critical > section, some one might have been "creative" enough to omit these > rcu_read_lock() and rcu_read_unlock(). > > > > > Indeed, lockdep would normally complain about this sort of thing. > > But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without > > CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > > > > >Actually with the rcu_dereference_check() above, lockdep cannot detect > >even CONFIG_PREEMPT_COUNT=y, that rcu_dereference_check() basically says > >"I know I'm in a read-side critical section if it's a non-preempt > >kernel, so don't bother to check". ;-) > > > > Any thoughts? > > > > It would be good to have some performance data on this change to expedited > > grace periods. It is adding code, so it needs some real motivation. > > I used rcuscale to test it, and count the average time of writer-duration > > no applied patch(org.txt) > writer avg time 29690.39 29670.78 29770.65 29423.25 > > applied patch(new.txt) > writer avg time 28989.99 29003.54 29281.39 28986.58 > > or Is there a better way to test? If I understand your measurments correctly, you are getting about a 2% improvement in expedited grace-period latency in !PREEMPT kernels. Do we have a situation for which that 2% improvement is important? We would be taking some risk due to the issues Boqun points out or we would be adding some complexity to avoid those issues. So this 2% has a definite cost associated with it. What am I missing here? Thanx, Paul > Thanks > Zqiang > > > >Agreed. > > > >Regards, > >Boqun > > > So, does this change make a system-level difference in (say) expedited > > RCU grace-period latency, and if so, under what conditions? > > > > Thanx, Paul > > > > > >Again, I'm not aware of any existing code that does this but we need to > > > >be sure. > > > > > > > >Regards, > > > >Boqun > > > > > > > > This commit add a exp QS check in rcu_exp_handler(), when not being > > > > in an RCU read-side critical section, report exp QS earlier. > > > > > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > > > --- > > > > kernel/rcu/tree_exp.h | 5 ++++- > > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > > > > index be667583a554..34f08267410f 100644 > > > > --- a/kernel/rcu/tree_exp.h > > > > +++ b/kernel/rcu/tree_exp.h > > > > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > > > > { > > > > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > > > > struct rcu_node *rnp = rdp->mynode; > > > > + bool preempt_bh_disabled = > > > > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > > > > > > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > > > > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > > > > return; > > > > - if (rcu_is_cpu_rrupt_from_idle()) { > > > > + if (rcu_is_cpu_rrupt_from_idle() || > > > > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > > > > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > > > > return; > > > > } > > > > -- > > > > 2.25.1 > > > > > [ 28.907109] rcu-scale: rcu_scale_writer 3 has 100 measurements > [ 28.907133] rcu-scale: rcu_scale_writer 0 has 100 measurements > [ 28.907135] rcu-scale: rcu_scale_writer 1 has 100 measurements > [ 28.907138] rcu-scale: rcu_scale_writer 2 has 100 measurements > [ 28.917241] Dumping ftrace buffer: > [ 28.917256] (ftrace buffer empty) > [ 28.917261] rcu-scale: Test complete > [ 49.336302] rcu-scale:!!! All grace periods expedited, no normal ones to measure! > [ 49.336329] rcu-torture: Stopping rcu_scale_reader task > [ 49.336340] rcu-torture: rcu_scale_reader is stopping > [ 49.336351] rcu-torture: Stopping rcu_scale_reader task > [ 49.336361] rcu-torture: rcu_scale_reader is stopping > [ 49.336370] rcu-torture: Stopping rcu_scale_reader task > [ 49.336405] rcu-torture: rcu_scale_reader is stopping > [ 49.336426] rcu-torture: Stopping rcu_scale_reader task > [ 49.336432] rcu-torture: rcu_scale_reader is stopping > [ 49.336443] rcu-torture: Stopping rcu_scale_writer task > [ 49.336450] rcu-torture: rcu_scale_writer is stopping > [ 49.336458] rcu-scale: writer 0 gps: 102 > [ 49.336460] rcu-torture: Stopping rcu_scale_writer task > [ 49.336465] rcu-torture: rcu_scale_writer is stopping > [ 49.336517] rcu-scale: writer 1 gps: 102 > [ 49.336519] rcu-torture: Stopping rcu_scale_writer task > [ 49.336524] rcu-torture: rcu_scale_writer is stopping > [ 49.336532] rcu-scale: writer 2 gps: 101 > [ 49.336533] rcu-torture: Stopping rcu_scale_writer task > [ 49.336539] rcu-torture: rcu_scale_writer is stopping > [ 49.336546] rcu-scale: writer 3 gps: 102 > [ 49.336548] rcu-scale: start: 28841772292 end: 28844718545 duration: 2946253 gps: 407 batches: 2 > [ 49.336550] rcu-scale: 0 writer-duration: 0 25552 > [ 49.339264] rcu-scale: 0 writer-duration: 1 42012 > [ 49.339274] rcu-scale: 0 writer-duration: 2 24377 > [ 49.339280] rcu-scale: 0 writer-duration: 3 34458 > [ 49.339299] rcu-scale: 0 writer-duration: 4 27007 > [ 49.339305] rcu-scale: 0 writer-duration: 5 36494 > [ 49.339310] rcu-scale: 0 writer-duration: 6 24278 > [ 49.339315] rcu-scale: 0 writer-duration: 7 29383 > [ 49.339321] rcu-scale: 0 writer-duration: 8 26494 > [ 49.339326] rcu-scale: 0 writer-duration: 9 35430 > [ 49.339331] rcu-scale: 0 writer-duration: 10 23986 > [ 49.339337] rcu-scale: 0 writer-duration: 11 28663 > [ 49.339342] rcu-scale: 0 writer-duration: 12 26329 > [ 49.339347] rcu-scale: 0 writer-duration: 13 35645 > [ 49.339353] rcu-scale: 0 writer-duration: 14 23761 > [ 49.339358] rcu-scale: 0 writer-duration: 15 28889 > [ 49.339364] rcu-scale: 0 writer-duration: 16 26519 > [ 49.339369] rcu-scale: 0 writer-duration: 17 35863 > [ 49.339374] rcu-scale: 0 writer-duration: 18 23984 > [ 49.339380] rcu-scale: 0 writer-duration: 19 28618 > [ 49.339385] rcu-scale: 0 writer-duration: 20 26547 > [ 49.339390] rcu-scale: 0 writer-duration: 21 35515 > [ 49.339395] rcu-scale: 0 writer-duration: 22 23913 > [ 49.339400] rcu-scale: 0 writer-duration: 23 28510 > [ 49.339405] rcu-scale: 0 writer-duration: 24 26746 > [ 49.339410] rcu-scale: 0 writer-duration: 25 35670 > [ 49.339416] rcu-scale: 0 writer-duration: 26 23963 > [ 49.339421] rcu-scale: 0 writer-duration: 27 28561 > [ 49.339427] rcu-scale: 0 writer-duration: 28 26380 > [ 49.339432] rcu-scale: 0 writer-duration: 29 36039 > [ 49.339438] rcu-scale: 0 writer-duration: 30 24215 > [ 49.339443] rcu-scale: 0 writer-duration: 31 28891 > [ 49.339449] rcu-scale: 0 writer-duration: 32 26578 > [ 49.339454] rcu-scale: 0 writer-duration: 33 40498 > [ 49.339459] rcu-scale: 0 writer-duration: 34 24245 > [ 49.339464] rcu-scale: 0 writer-duration: 35 29465 > [ 49.339469] rcu-scale: 0 writer-duration: 36 26946 > [ 49.339475] rcu-scale: 0 writer-duration: 37 35955 > [ 49.339480] rcu-scale: 0 writer-duration: 38 24059 > [ 49.339486] rcu-scale: 0 writer-duration: 39 28795 > [ 49.339491] rcu-scale: 0 writer-duration: 40 26536 > [ 49.339496] rcu-scale: 0 writer-duration: 41 35879 > [ 49.339501] rcu-scale: 0 writer-duration: 42 24181 > [ 49.339506] rcu-scale: 0 writer-duration: 43 28639 > [ 49.339512] rcu-scale: 0 writer-duration: 44 26546 > [ 49.339517] rcu-scale: 0 writer-duration: 45 35884 > [ 49.339522] rcu-scale: 0 writer-duration: 46 23986 > [ 49.339527] rcu-scale: 0 writer-duration: 47 28668 > [ 49.339532] rcu-scale: 0 writer-duration: 48 26454 > [ 49.339537] rcu-scale: 0 writer-duration: 49 35772 > [ 49.339542] rcu-scale: 0 writer-duration: 50 23942 > [ 49.339547] rcu-scale: 0 writer-duration: 51 28803 > [ 49.339553] rcu-scale: 0 writer-duration: 52 26545 > [ 49.339558] rcu-scale: 0 writer-duration: 53 35615 > [ 49.339563] rcu-scale: 0 writer-duration: 54 23569 > [ 49.339568] rcu-scale: 0 writer-duration: 55 28427 > [ 49.339573] rcu-scale: 0 writer-duration: 56 26489 > [ 49.339578] rcu-scale: 0 writer-duration: 57 35516 > [ 49.339584] rcu-scale: 0 writer-duration: 58 23722 > [ 49.339589] rcu-scale: 0 writer-duration: 59 28435 > [ 49.339594] rcu-scale: 0 writer-duration: 60 26401 > [ 49.339599] rcu-scale: 0 writer-duration: 61 35488 > [ 49.339604] rcu-scale: 0 writer-duration: 62 23855 > [ 49.339610] rcu-scale: 0 writer-duration: 63 28583 > [ 49.339615] rcu-scale: 0 writer-duration: 64 26629 > [ 49.339620] rcu-scale: 0 writer-duration: 65 36168 > [ 49.339625] rcu-scale: 0 writer-duration: 66 24191 > [ 49.339630] rcu-scale: 0 writer-duration: 67 28802 > [ 49.339635] rcu-scale: 0 writer-duration: 68 32436 > [ 49.339640] rcu-scale: 0 writer-duration: 69 38444 > [ 49.339646] rcu-scale: 0 writer-duration: 70 24417 > [ 49.339651] rcu-scale: 0 writer-duration: 71 29128 > [ 49.339656] rcu-scale: 0 writer-duration: 72 26480 > [ 49.339661] rcu-scale: 0 writer-duration: 73 35881 > [ 49.339666] rcu-scale: 0 writer-duration: 74 24123 > [ 49.339671] rcu-scale: 0 writer-duration: 75 28735 > [ 49.339676] rcu-scale: 0 writer-duration: 76 26359 > [ 49.339681] rcu-scale: 0 writer-duration: 77 35715 > [ 49.339687] rcu-scale: 0 writer-duration: 78 24092 > [ 49.339692] rcu-scale: 0 writer-duration: 79 28716 > [ 49.339698] rcu-scale: 0 writer-duration: 80 26448 > [ 49.339703] rcu-scale: 0 writer-duration: 81 35935 > [ 49.339708] rcu-scale: 0 writer-duration: 82 23966 > [ 49.339713] rcu-scale: 0 writer-duration: 83 28552 > [ 49.339718] rcu-scale: 0 writer-duration: 84 26487 > [ 49.339723] rcu-scale: 0 writer-duration: 85 35490 > [ 49.339729] rcu-scale: 0 writer-duration: 86 23976 > [ 49.339734] rcu-scale: 0 writer-duration: 87 28679 > [ 49.339740] rcu-scale: 0 writer-duration: 88 26525 > [ 49.339745] rcu-scale: 0 writer-duration: 89 35764 > [ 49.339750] rcu-scale: 0 writer-duration: 90 24050 > [ 49.339756] rcu-scale: 0 writer-duration: 91 28777 > [ 49.339761] rcu-scale: 0 writer-duration: 92 26412 > [ 49.339767] rcu-scale: 0 writer-duration: 93 35707 > [ 49.339772] rcu-scale: 0 writer-duration: 94 23913 > [ 49.339777] rcu-scale: 0 writer-duration: 95 28450 > [ 49.339782] rcu-scale: 0 writer-duration: 96 26665 > [ 49.339788] rcu-scale: 0 writer-duration: 97 35636 > [ 49.339793] rcu-scale: 0 writer-duration: 98 23953 > [ 49.339798] rcu-scale: 0 writer-duration: 99 28466 > [ 49.339804] rcu-scale: 0 writer-duration: 100 26654 > [ 49.341227] rcu-scale: 0 writer-duration: 101 225635 > > > > > [ 49.341246] rcu-scale: 1 writer-duration: 0 27277 > [ 49.343227] rcu-scale: 1 writer-duration: 1 34458 > [ 49.343236] rcu-scale: 1 writer-duration: 2 27340 > [ 49.343242] rcu-scale: 1 writer-duration: 3 41758 > [ 49.343248] rcu-scale: 1 writer-duration: 4 24161 > [ 49.343254] rcu-scale: 1 writer-duration: 5 29250 > [ 49.343260] rcu-scale: 1 writer-duration: 6 26996 > [ 49.343266] rcu-scale: 1 writer-duration: 7 36482 > [ 49.343272] rcu-scale: 1 writer-duration: 8 23782 > [ 49.343277] rcu-scale: 1 writer-duration: 9 28433 > [ 49.343283] rcu-scale: 1 writer-duration: 10 26577 > [ 49.343289] rcu-scale: 1 writer-duration: 11 35616 > [ 49.343295] rcu-scale: 1 writer-duration: 12 23821 > [ 49.343301] rcu-scale: 1 writer-duration: 13 28678 > [ 49.343307] rcu-scale: 1 writer-duration: 14 26517 > [ 49.343313] rcu-scale: 1 writer-duration: 15 35548 > [ 49.343319] rcu-scale: 1 writer-duration: 16 24196 > [ 49.343325] rcu-scale: 1 writer-duration: 17 28626 > [ 49.343332] rcu-scale: 1 writer-duration: 18 26772 > [ 49.343338] rcu-scale: 1 writer-duration: 19 35539 > [ 49.343344] rcu-scale: 1 writer-duration: 20 23897 > [ 49.343350] rcu-scale: 1 writer-duration: 21 28635 > [ 49.343356] rcu-scale: 1 writer-duration: 22 26453 > [ 49.343362] rcu-scale: 1 writer-duration: 23 35416 > [ 49.343368] rcu-scale: 1 writer-duration: 24 24096 > [ 49.343373] rcu-scale: 1 writer-duration: 25 28733 > [ 49.343379] rcu-scale: 1 writer-duration: 26 26538 > [ 49.343385] rcu-scale: 1 writer-duration: 27 35590 > [ 49.343390] rcu-scale: 1 writer-duration: 28 23739 > [ 49.343396] rcu-scale: 1 writer-duration: 29 29450 > [ 49.343402] rcu-scale: 1 writer-duration: 30 26452 > [ 49.343408] rcu-scale: 1 writer-duration: 31 35807 > [ 49.343413] rcu-scale: 1 writer-duration: 32 24145 > [ 49.343419] rcu-scale: 1 writer-duration: 33 33452 > [ 49.343425] rcu-scale: 1 writer-duration: 34 26767 > [ 49.343431] rcu-scale: 1 writer-duration: 35 36677 > [ 49.343438] rcu-scale: 1 writer-duration: 36 24102 > [ 49.343444] rcu-scale: 1 writer-duration: 37 28937 > [ 49.343449] rcu-scale: 1 writer-duration: 38 26622 > [ 49.343454] rcu-scale: 1 writer-duration: 39 35903 > [ 49.343460] rcu-scale: 1 writer-duration: 40 23956 > [ 49.343465] rcu-scale: 1 writer-duration: 41 28684 > [ 49.343470] rcu-scale: 1 writer-duration: 42 26901 > [ 49.343475] rcu-scale: 1 writer-duration: 43 35717 > [ 49.343480] rcu-scale: 1 writer-duration: 44 23957 > [ 49.343485] rcu-scale: 1 writer-duration: 45 28780 > [ 49.343490] rcu-scale: 1 writer-duration: 46 26747 > [ 49.343495] rcu-scale: 1 writer-duration: 47 35538 > [ 49.343501] rcu-scale: 1 writer-duration: 48 23963 > [ 49.343506] rcu-scale: 1 writer-duration: 49 28727 > [ 49.343511] rcu-scale: 1 writer-duration: 50 26519 > [ 49.343516] rcu-scale: 1 writer-duration: 51 35733 > [ 49.343521] rcu-scale: 1 writer-duration: 52 24098 > [ 49.343527] rcu-scale: 1 writer-duration: 53 28557 > [ 49.343532] rcu-scale: 1 writer-duration: 54 26208 > [ 49.343537] rcu-scale: 1 writer-duration: 55 35315 > [ 49.343543] rcu-scale: 1 writer-duration: 56 23932 > [ 49.343548] rcu-scale: 1 writer-duration: 57 28507 > [ 49.343553] rcu-scale: 1 writer-duration: 58 26443 > [ 49.343559] rcu-scale: 1 writer-duration: 59 35306 > [ 49.343564] rcu-scale: 1 writer-duration: 60 23868 > [ 49.343570] rcu-scale: 1 writer-duration: 61 28606 > [ 49.343575] rcu-scale: 1 writer-duration: 62 26308 > [ 49.343580] rcu-scale: 1 writer-duration: 63 35693 > [ 49.343586] rcu-scale: 1 writer-duration: 64 23934 > [ 49.343591] rcu-scale: 1 writer-duration: 65 29456 > [ 49.343596] rcu-scale: 1 writer-duration: 66 26600 > [ 49.343602] rcu-scale: 1 writer-duration: 67 35682 > [ 49.343607] rcu-scale: 1 writer-duration: 68 29849 > [ 49.343612] rcu-scale: 1 writer-duration: 69 31555 > [ 49.343618] rcu-scale: 1 writer-duration: 70 26942 > [ 49.343623] rcu-scale: 1 writer-duration: 71 36037 > [ 49.343629] rcu-scale: 1 writer-duration: 72 23965 > [ 49.343634] rcu-scale: 1 writer-duration: 73 28811 > [ 49.343640] rcu-scale: 1 writer-duration: 74 26734 > [ 49.343645] rcu-scale: 1 writer-duration: 75 35712 > [ 49.343650] rcu-scale: 1 writer-duration: 76 23845 > [ 49.343656] rcu-scale: 1 writer-duration: 77 28619 > [ 49.343661] rcu-scale: 1 writer-duration: 78 26637 > [ 49.343666] rcu-scale: 1 writer-duration: 79 35773 > [ 49.343672] rcu-scale: 1 writer-duration: 80 23910 > [ 49.343677] rcu-scale: 1 writer-duration: 81 28873 > [ 49.343682] rcu-scale: 1 writer-duration: 82 26581 > [ 49.343688] rcu-scale: 1 writer-duration: 83 35517 > [ 49.343694] rcu-scale: 1 writer-duration: 84 23928 > [ 49.343699] rcu-scale: 1 writer-duration: 85 28565 > [ 49.343704] rcu-scale: 1 writer-duration: 86 26464 > [ 49.343710] rcu-scale: 1 writer-duration: 87 35715 > [ 49.343715] rcu-scale: 1 writer-duration: 88 23968 > [ 49.343721] rcu-scale: 1 writer-duration: 89 28784 > [ 49.343726] rcu-scale: 1 writer-duration: 90 26576 > [ 49.343731] rcu-scale: 1 writer-duration: 91 35800 > [ 49.343737] rcu-scale: 1 writer-duration: 92 23918 > [ 49.343742] rcu-scale: 1 writer-duration: 93 28487 > [ 49.343747] rcu-scale: 1 writer-duration: 94 26491 > [ 49.343753] rcu-scale: 1 writer-duration: 95 35565 > [ 49.343758] rcu-scale: 1 writer-duration: 96 24092 > [ 49.343763] rcu-scale: 1 writer-duration: 97 28469 > [ 49.343768] rcu-scale: 1 writer-duration: 98 26865 > [ 49.343774] rcu-scale: 1 writer-duration: 99 35343 > [ 49.343779] rcu-scale: 1 writer-duration: 100 23995 > [ 49.345221] rcu-scale: 1 writer-duration: 101 60524 > > > > [ 49.345235] rcu-scale: 2 writer-duration: 0 56581 > [ 49.347314] rcu-scale: 2 writer-duration: 1 32281 > [ 49.347329] rcu-scale: 2 writer-duration: 2 29516 > [ 49.347336] rcu-scale: 2 writer-duration: 3 29979 > [ 49.347342] rcu-scale: 2 writer-duration: 4 36232 > [ 49.347348] rcu-scale: 2 writer-duration: 5 26878 > [ 49.347354] rcu-scale: 2 writer-duration: 6 29099 > [ 49.347359] rcu-scale: 2 writer-duration: 7 24890 > [ 49.347365] rcu-scale: 2 writer-duration: 8 35614 > [ 49.347371] rcu-scale: 2 writer-duration: 9 26166 > [ 49.347377] rcu-scale: 2 writer-duration: 10 28552 > [ 49.347384] rcu-scale: 2 writer-duration: 11 24335 > [ 49.347390] rcu-scale: 2 writer-duration: 12 35614 > [ 49.347396] rcu-scale: 2 writer-duration: 13 25978 > [ 49.347402] rcu-scale: 2 writer-duration: 14 28541 > [ 49.347408] rcu-scale: 2 writer-duration: 15 24634 > [ 49.347413] rcu-scale: 2 writer-duration: 16 35640 > [ 49.347419] rcu-scale: 2 writer-duration: 17 26235 > [ 49.347425] rcu-scale: 2 writer-duration: 18 28782 > [ 49.347431] rcu-scale: 2 writer-duration: 19 24288 > [ 49.347437] rcu-scale: 2 writer-duration: 20 35649 > [ 49.347442] rcu-scale: 2 writer-duration: 21 25989 > [ 49.347448] rcu-scale: 2 writer-duration: 22 28527 > [ 49.347454] rcu-scale: 2 writer-duration: 23 24150 > [ 49.347460] rcu-scale: 2 writer-duration: 24 35960 > [ 49.347465] rcu-scale: 2 writer-duration: 25 26133 > [ 49.347471] rcu-scale: 2 writer-duration: 26 28714 > [ 49.347477] rcu-scale: 2 writer-duration: 27 24192 > [ 49.347482] rcu-scale: 2 writer-duration: 28 35545 > [ 49.347488] rcu-scale: 2 writer-duration: 29 26893 > [ 49.347494] rcu-scale: 2 writer-duration: 30 28621 > [ 49.347499] rcu-scale: 2 writer-duration: 31 24453 > [ 49.347505] rcu-scale: 2 writer-duration: 32 35750 > [ 49.347510] rcu-scale: 2 writer-duration: 33 31021 > [ 49.347516] rcu-scale: 2 writer-duration: 34 29061 > [ 49.347522] rcu-scale: 2 writer-duration: 35 25015 > [ 49.347528] rcu-scale: 2 writer-duration: 36 36107 > [ 49.347534] rcu-scale: 2 writer-duration: 37 26370 > [ 49.347540] rcu-scale: 2 writer-duration: 38 28888 > [ 49.347546] rcu-scale: 2 writer-duration: 39 24418 > [ 49.347552] rcu-scale: 2 writer-duration: 40 35675 > [ 49.347558] rcu-scale: 2 writer-duration: 41 26284 > [ 49.347564] rcu-scale: 2 writer-duration: 42 28924 > [ 49.347570] rcu-scale: 2 writer-duration: 43 24244 > [ 49.347576] rcu-scale: 2 writer-duration: 44 35835 > [ 49.347582] rcu-scale: 2 writer-duration: 45 26194 > [ 49.347587] rcu-scale: 2 writer-duration: 46 29080 > [ 49.347593] rcu-scale: 2 writer-duration: 47 24185 > [ 49.347599] rcu-scale: 2 writer-duration: 48 35543 > [ 49.347605] rcu-scale: 2 writer-duration: 49 26122 > [ 49.347610] rcu-scale: 2 writer-duration: 50 28822 > [ 49.347616] rcu-scale: 2 writer-duration: 51 24285 > [ 49.347622] rcu-scale: 2 writer-duration: 52 35720 > [ 49.347627] rcu-scale: 2 writer-duration: 53 26063 > [ 49.347633] rcu-scale: 2 writer-duration: 54 28508 > [ 49.347639] rcu-scale: 2 writer-duration: 55 24019 > [ 49.347645] rcu-scale: 2 writer-duration: 56 35499 > [ 49.347651] rcu-scale: 2 writer-duration: 57 26079 > [ 49.347657] rcu-scale: 2 writer-duration: 58 28502 > [ 49.347663] rcu-scale: 2 writer-duration: 59 24016 > [ 49.347668] rcu-scale: 2 writer-duration: 60 35515 > [ 49.347673] rcu-scale: 2 writer-duration: 61 26095 > [ 49.347677] rcu-scale: 2 writer-duration: 62 28602 > [ 49.347681] rcu-scale: 2 writer-duration: 63 24246 > [ 49.347686] rcu-scale: 2 writer-duration: 64 35675 > [ 49.347690] rcu-scale: 2 writer-duration: 65 26761 > [ 49.347694] rcu-scale: 2 writer-duration: 66 28833 > [ 49.347699] rcu-scale: 2 writer-duration: 67 24524 > [ 49.347703] rcu-scale: 2 writer-duration: 68 43780 > [ 49.347707] rcu-scale: 2 writer-duration: 69 26612 > [ 49.347712] rcu-scale: 2 writer-duration: 70 29290 > [ 49.347716] rcu-scale: 2 writer-duration: 71 24670 > [ 49.347720] rcu-scale: 2 writer-duration: 72 35526 > [ 49.347725] rcu-scale: 2 writer-duration: 73 26393 > [ 49.347729] rcu-scale: 2 writer-duration: 74 28967 > [ 49.347733] rcu-scale: 2 writer-duration: 75 24125 > [ 49.347738] rcu-scale: 2 writer-duration: 76 35713 > [ 49.347742] rcu-scale: 2 writer-duration: 77 26062 > [ 49.347746] rcu-scale: 2 writer-duration: 78 28888 > [ 49.347751] rcu-scale: 2 writer-duration: 79 24344 > [ 49.347755] rcu-scale: 2 writer-duration: 80 35858 > [ 49.347760] rcu-scale: 2 writer-duration: 81 26330 > [ 49.347764] rcu-scale: 2 writer-duration: 82 28661 > [ 49.347769] rcu-scale: 2 writer-duration: 83 24180 > [ 49.347773] rcu-scale: 2 writer-duration: 84 35464 > [ 49.347778] rcu-scale: 2 writer-duration: 85 25886 > [ 49.347782] rcu-scale: 2 writer-duration: 86 28916 > [ 49.347786] rcu-scale: 2 writer-duration: 87 24149 > [ 49.347791] rcu-scale: 2 writer-duration: 88 35860 > [ 49.347795] rcu-scale: 2 writer-duration: 89 26100 > [ 49.347800] rcu-scale: 2 writer-duration: 90 28999 > [ 49.347804] rcu-scale: 2 writer-duration: 91 24105 > [ 49.347809] rcu-scale: 2 writer-duration: 92 35720 > [ 49.347813] rcu-scale: 2 writer-duration: 93 26093 > [ 49.347817] rcu-scale: 2 writer-duration: 94 28729 > [ 49.347822] rcu-scale: 2 writer-duration: 95 24063 > [ 49.347826] rcu-scale: 2 writer-duration: 96 35819 > [ 49.347831] rcu-scale: 2 writer-duration: 97 26119 > [ 49.347835] rcu-scale: 2 writer-duration: 98 28648 > [ 49.347840] rcu-scale: 2 writer-duration: 99 24175 > [ 49.347844] rcu-scale: 2 writer-duration: 100 34035 > > > > > [ 49.349299] rcu-scale: 3 writer-duration: 0 26939 > [ 49.351268] rcu-scale: 3 writer-duration: 1 27152 > [ 49.351277] rcu-scale: 3 writer-duration: 2 30299 > [ 49.351281] rcu-scale: 3 writer-duration: 3 36662 > [ 49.351284] rcu-scale: 3 writer-duration: 4 31847 > [ 49.351287] rcu-scale: 3 writer-duration: 5 29203 > [ 49.351290] rcu-scale: 3 writer-duration: 6 24861 > [ 49.351294] rcu-scale: 3 writer-duration: 7 36222 > [ 49.351297] rcu-scale: 3 writer-duration: 8 26790 > [ 49.351300] rcu-scale: 3 writer-duration: 9 28626 > [ 49.351303] rcu-scale: 3 writer-duration: 10 24007 > [ 49.351307] rcu-scale: 3 writer-duration: 11 35765 > [ 49.351310] rcu-scale: 3 writer-duration: 12 26226 > [ 49.351314] rcu-scale: 3 writer-duration: 13 28495 > [ 49.351317] rcu-scale: 3 writer-duration: 14 24188 > [ 49.351321] rcu-scale: 3 writer-duration: 15 35686 > [ 49.351324] rcu-scale: 3 writer-duration: 16 26290 > [ 49.351327] rcu-scale: 3 writer-duration: 17 28740 > [ 49.351331] rcu-scale: 3 writer-duration: 18 24201 > [ 49.351334] rcu-scale: 3 writer-duration: 19 36000 > [ 49.351337] rcu-scale: 3 writer-duration: 20 25995 > [ 49.351341] rcu-scale: 3 writer-duration: 21 28649 > [ 49.351344] rcu-scale: 3 writer-duration: 22 24081 > [ 49.351348] rcu-scale: 3 writer-duration: 23 35718 > [ 49.351351] rcu-scale: 3 writer-duration: 24 26077 > [ 49.351354] rcu-scale: 3 writer-duration: 25 28773 > [ 49.351357] rcu-scale: 3 writer-duration: 26 24280 > [ 49.351360] rcu-scale: 3 writer-duration: 27 35767 > [ 49.351364] rcu-scale: 3 writer-duration: 28 25956 > [ 49.351367] rcu-scale: 3 writer-duration: 29 28632 > [ 49.351370] rcu-scale: 3 writer-duration: 30 24996 > [ 49.351374] rcu-scale: 3 writer-duration: 31 35673 > [ 49.351377] rcu-scale: 3 writer-duration: 32 26217 > [ 49.351380] rcu-scale: 3 writer-duration: 33 28855 > [ 49.351383] rcu-scale: 3 writer-duration: 34 28655 > [ 49.351387] rcu-scale: 3 writer-duration: 35 36525 > [ 49.351390] rcu-scale: 3 writer-duration: 36 27052 > [ 49.351394] rcu-scale: 3 writer-duration: 37 28841 > [ 49.351397] rcu-scale: 3 writer-duration: 38 24600 > [ 49.351400] rcu-scale: 3 writer-duration: 39 35743 > [ 49.351404] rcu-scale: 3 writer-duration: 40 26391 > [ 49.351407] rcu-scale: 3 writer-duration: 41 28642 > [ 49.351410] rcu-scale: 3 writer-duration: 42 24334 > [ 49.351413] rcu-scale: 3 writer-duration: 43 36152 > [ 49.351417] rcu-scale: 3 writer-duration: 44 26133 > [ 49.351420] rcu-scale: 3 writer-duration: 45 28732 > [ 49.351423] rcu-scale: 3 writer-duration: 46 24235 > [ 49.351426] rcu-scale: 3 writer-duration: 47 36069 > [ 49.351429] rcu-scale: 3 writer-duration: 48 25993 > [ 49.351433] rcu-scale: 3 writer-duration: 49 28658 > [ 49.351436] rcu-scale: 3 writer-duration: 50 24225 > [ 49.351439] rcu-scale: 3 writer-duration: 51 35909 > [ 49.351442] rcu-scale: 3 writer-duration: 52 26176 > [ 49.351446] rcu-scale: 3 writer-duration: 53 28702 > [ 49.351449] rcu-scale: 3 writer-duration: 54 24169 > [ 49.351453] rcu-scale: 3 writer-duration: 55 35462 > [ 49.351456] rcu-scale: 3 writer-duration: 56 25771 > [ 49.351459] rcu-scale: 3 writer-duration: 57 28677 > [ 49.351462] rcu-scale: 3 writer-duration: 58 24089 > [ 49.351466] rcu-scale: 3 writer-duration: 59 35538 > [ 49.351469] rcu-scale: 3 writer-duration: 60 25785 > [ 49.351472] rcu-scale: 3 writer-duration: 61 28669 > [ 49.351475] rcu-scale: 3 writer-duration: 62 24075 > [ 49.351478] rcu-scale: 3 writer-duration: 63 35720 > [ 49.351482] rcu-scale: 3 writer-duration: 64 26017 > [ 49.351485] rcu-scale: 3 writer-duration: 65 28726 > [ 49.351488] rcu-scale: 3 writer-duration: 66 24990 > [ 49.351491] rcu-scale: 3 writer-duration: 67 35814 > [ 49.351494] rcu-scale: 3 writer-duration: 68 26215 > [ 49.351498] rcu-scale: 3 writer-duration: 69 34468 > [ 49.351501] rcu-scale: 3 writer-duration: 70 27200 > [ 49.351504] rcu-scale: 3 writer-duration: 71 36288 > [ 49.351507] rcu-scale: 3 writer-duration: 72 26445 > [ 49.351511] rcu-scale: 3 writer-duration: 73 28593 > [ 49.351514] rcu-scale: 3 writer-duration: 74 24317 > [ 49.351517] rcu-scale: 3 writer-duration: 75 36153 > [ 49.351521] rcu-scale: 3 writer-duration: 76 26164 > [ 49.351524] rcu-scale: 3 writer-duration: 77 28523 > [ 49.351527] rcu-scale: 3 writer-duration: 78 24252 > [ 49.351530] rcu-scale: 3 writer-duration: 79 35939 > [ 49.351533] rcu-scale: 3 writer-duration: 80 26058 > [ 49.351536] rcu-scale: 3 writer-duration: 81 28892 > [ 49.351540] rcu-scale: 3 writer-duration: 82 24342 > [ 49.351543] rcu-scale: 3 writer-duration: 83 35751 > [ 49.351546] rcu-scale: 3 writer-duration: 84 25932 > [ 49.351549] rcu-scale: 3 writer-duration: 85 28682 > [ 49.351552] rcu-scale: 3 writer-duration: 86 24086 > [ 49.351555] rcu-scale: 3 writer-duration: 87 35791 > [ 49.351559] rcu-scale: 3 writer-duration: 88 26189 > [ 49.351562] rcu-scale: 3 writer-duration: 89 28754 > [ 49.351565] rcu-scale: 3 writer-duration: 90 24250 > [ 49.351568] rcu-scale: 3 writer-duration: 91 35768 > [ 49.351572] rcu-scale: 3 writer-duration: 92 26160 > [ 49.351575] rcu-scale: 3 writer-duration: 93 28896 > [ 49.351578] rcu-scale: 3 writer-duration: 94 24020 > [ 49.351581] rcu-scale: 3 writer-duration: 95 35727 > [ 49.351584] rcu-scale: 3 writer-duration: 96 26099 > [ 49.351587] rcu-scale: 3 writer-duration: 97 28768 > [ 49.351590] rcu-scale: 3 writer-duration: 98 24011 > [ 49.351593] rcu-scale: 3 writer-duration: 99 35786 > [ 49.351597] rcu-scale: 3 writer-duration: 100 25989 > [ 49.353257] rcu-scale: 3 writer-duration: 101 46411 > > > [ 47.850402] rcu-scale: rcu_scale_writer 3 has 100 measurements > [ 47.850427] rcu-scale: rcu_scale_writer 0 has 100 measurements > [ 47.850429] rcu-scale: rcu_scale_writer 1 has 100 measurements > [ 47.850432] rcu-scale: rcu_scale_writer 2 has 100 measurements > [ 47.861470] Dumping ftrace buffer: > [ 47.861486] (ftrace buffer empty) > [ 47.861491] rcu-scale: Test complete > [ 63.668029] rcu-scale:!!! All grace periods expedited, no normal ones to measure! > [ 63.668051] rcu-torture: Stopping rcu_scale_reader task > [ 63.668074] rcu-torture: rcu_scale_reader is stopping > [ 63.668089] rcu-torture: Stopping rcu_scale_reader task > [ 63.668138] rcu-torture: rcu_scale_reader is stopping > [ 63.668151] rcu-torture: Stopping rcu_scale_reader task > [ 63.668191] rcu-torture: rcu_scale_reader is stopping > [ 63.668204] rcu-torture: Stopping rcu_scale_reader task > [ 63.668209] rcu-torture: rcu_scale_reader is stopping > [ 63.668231] rcu-torture: Stopping rcu_scale_writer task > [ 63.668241] rcu-torture: rcu_scale_writer is stopping > [ 63.668249] rcu-scale: writer 0 gps: 102 > [ 63.668251] rcu-torture: Stopping rcu_scale_writer task > [ 63.668260] rcu-torture: rcu_scale_writer is stopping > [ 63.668268] rcu-scale: writer 1 gps: 102 > [ 63.668270] rcu-torture: Stopping rcu_scale_writer task > [ 63.668279] rcu-torture: rcu_scale_writer is stopping > [ 63.668287] rcu-scale: writer 2 gps: 101 > [ 63.668288] rcu-torture: Stopping rcu_scale_writer task > [ 63.668293] rcu-torture: rcu_scale_writer is stopping > [ 63.668310] rcu-scale: writer 3 gps: 102 > [ 63.668311] rcu-scale: start: 47784890510 end: 47787901801 duration: 3011291 gps: 407 batches: 2 > [ 63.668314] rcu-scale: 0 writer-duration: 0 55830 > [ 63.669543] rcu-scale: 0 writer-duration: 1 40421 > [ 63.669546] rcu-scale: 0 writer-duration: 2 25268 > [ 63.669548] rcu-scale: 0 writer-duration: 3 32703 > [ 63.669549] rcu-scale: 0 writer-duration: 4 28624 > [ 63.669550] rcu-scale: 0 writer-duration: 5 36967 > [ 63.669551] rcu-scale: 0 writer-duration: 6 24686 > [ 63.669552] rcu-scale: 0 writer-duration: 7 29598 > [ 63.669554] rcu-scale: 0 writer-duration: 8 27026 > [ 63.669555] rcu-scale: 0 writer-duration: 9 38789 > [ 63.669556] rcu-scale: 0 writer-duration: 10 24398 > [ 63.669557] rcu-scale: 0 writer-duration: 11 29562 > [ 63.669559] rcu-scale: 0 writer-duration: 12 27348 > [ 63.669560] rcu-scale: 0 writer-duration: 13 36315 > [ 63.669561] rcu-scale: 0 writer-duration: 14 24233 > [ 63.669562] rcu-scale: 0 writer-duration: 15 29035 > [ 63.669564] rcu-scale: 0 writer-duration: 16 26754 > [ 63.669606] rcu-scale: 0 writer-duration: 17 36272 > [ 63.669608] rcu-scale: 0 writer-duration: 18 24237 > [ 63.669609] rcu-scale: 0 writer-duration: 19 28834 > [ 63.669610] rcu-scale: 0 writer-duration: 20 26993 > [ 63.669611] rcu-scale: 0 writer-duration: 21 36432 > [ 63.669612] rcu-scale: 0 writer-duration: 22 24178 > [ 63.669614] rcu-scale: 0 writer-duration: 23 28838 > [ 63.669615] rcu-scale: 0 writer-duration: 24 26733 > [ 63.669616] rcu-scale: 0 writer-duration: 25 36087 > [ 63.669617] rcu-scale: 0 writer-duration: 26 24237 > [ 63.669618] rcu-scale: 0 writer-duration: 27 28656 > [ 63.669620] rcu-scale: 0 writer-duration: 28 26921 > [ 63.669621] rcu-scale: 0 writer-duration: 29 36423 > [ 63.669622] rcu-scale: 0 writer-duration: 30 24201 > [ 63.669623] rcu-scale: 0 writer-duration: 31 28748 > [ 63.669625] rcu-scale: 0 writer-duration: 32 31925 > [ 63.669626] rcu-scale: 0 writer-duration: 33 36673 > [ 63.669627] rcu-scale: 0 writer-duration: 34 24630 > [ 63.669628] rcu-scale: 0 writer-duration: 35 29033 > [ 63.669629] rcu-scale: 0 writer-duration: 36 26761 > [ 63.669631] rcu-scale: 0 writer-duration: 37 36495 > [ 63.669632] rcu-scale: 0 writer-duration: 38 24209 > [ 63.669633] rcu-scale: 0 writer-duration: 39 29468 > [ 63.669634] rcu-scale: 0 writer-duration: 40 27137 > [ 63.669635] rcu-scale: 0 writer-duration: 41 36193 > [ 63.669637] rcu-scale: 0 writer-duration: 42 24531 > [ 63.669638] rcu-scale: 0 writer-duration: 43 29113 > [ 63.669639] rcu-scale: 0 writer-duration: 44 26915 > [ 63.669640] rcu-scale: 0 writer-duration: 45 36394 > [ 63.669642] rcu-scale: 0 writer-duration: 46 24247 > [ 63.669643] rcu-scale: 0 writer-duration: 47 28823 > [ 63.669644] rcu-scale: 0 writer-duration: 48 27029 > [ 63.669645] rcu-scale: 0 writer-duration: 49 36042 > [ 63.669646] rcu-scale: 0 writer-duration: 50 24164 > [ 63.669648] rcu-scale: 0 writer-duration: 51 28972 > [ 63.669649] rcu-scale: 0 writer-duration: 52 26858 > [ 63.669650] rcu-scale: 0 writer-duration: 53 36319 > [ 63.669651] rcu-scale: 0 writer-duration: 54 24233 > [ 63.669652] rcu-scale: 0 writer-duration: 55 28986 > [ 63.669654] rcu-scale: 0 writer-duration: 56 26895 > [ 63.669655] rcu-scale: 0 writer-duration: 57 36213 > [ 63.669656] rcu-scale: 0 writer-duration: 58 24156 > [ 63.669657] rcu-scale: 0 writer-duration: 59 28859 > [ 63.669658] rcu-scale: 0 writer-duration: 60 26917 > [ 63.669660] rcu-scale: 0 writer-duration: 61 36174 > [ 63.669661] rcu-scale: 0 writer-duration: 62 24237 > [ 63.669662] rcu-scale: 0 writer-duration: 63 28873 > [ 63.669663] rcu-scale: 0 writer-duration: 64 27045 > [ 63.669664] rcu-scale: 0 writer-duration: 65 36167 > [ 63.669666] rcu-scale: 0 writer-duration: 66 29556 > [ 63.669667] rcu-scale: 0 writer-duration: 67 28900 > [ 63.669668] rcu-scale: 0 writer-duration: 68 27072 > [ 63.669669] rcu-scale: 0 writer-duration: 69 36733 > [ 63.669670] rcu-scale: 0 writer-duration: 70 24327 > [ 63.669672] rcu-scale: 0 writer-duration: 71 28842 > [ 63.669673] rcu-scale: 0 writer-duration: 72 27027 > [ 63.669674] rcu-scale: 0 writer-duration: 73 36228 > [ 63.669675] rcu-scale: 0 writer-duration: 74 24137 > [ 63.669677] rcu-scale: 0 writer-duration: 75 29924 > [ 63.669678] rcu-scale: 0 writer-duration: 76 26775 > [ 63.669679] rcu-scale: 0 writer-duration: 77 36300 > [ 63.669680] rcu-scale: 0 writer-duration: 78 24547 > [ 63.669681] rcu-scale: 0 writer-duration: 79 28852 > [ 63.669683] rcu-scale: 0 writer-duration: 80 26945 > [ 63.669684] rcu-scale: 0 writer-duration: 81 36127 > [ 63.669685] rcu-scale: 0 writer-duration: 82 24373 > [ 63.669686] rcu-scale: 0 writer-duration: 83 28853 > [ 63.669687] rcu-scale: 0 writer-duration: 84 26652 > [ 63.669689] rcu-scale: 0 writer-duration: 85 35952 > [ 63.669690] rcu-scale: 0 writer-duration: 86 24379 > [ 63.669691] rcu-scale: 0 writer-duration: 87 28948 > [ 63.669692] rcu-scale: 0 writer-duration: 88 26958 > [ 63.669693] rcu-scale: 0 writer-duration: 89 36302 > [ 63.669695] rcu-scale: 0 writer-duration: 90 24181 > [ 63.669696] rcu-scale: 0 writer-duration: 91 29120 > [ 63.669697] rcu-scale: 0 writer-duration: 92 27032 > [ 63.669698] rcu-scale: 0 writer-duration: 93 36083 > [ 63.669699] rcu-scale: 0 writer-duration: 94 24248 > [ 63.669701] rcu-scale: 0 writer-duration: 95 29205 > [ 63.669702] rcu-scale: 0 writer-duration: 96 26577 > [ 63.669703] rcu-scale: 0 writer-duration: 97 36187 > [ 63.669704] rcu-scale: 0 writer-duration: 98 24472 > [ 63.669705] rcu-scale: 0 writer-duration: 99 28883 > [ 63.669707] rcu-scale: 0 writer-duration: 100 31999 > [ 63.671532] rcu-scale: 0 writer-duration: 101 48271 > > > [ 63.671556] rcu-scale: 1 writer-duration: 0 53537 > [ 63.673442] rcu-scale: 1 writer-duration: 1 33311 > [ 63.673451] rcu-scale: 1 writer-duration: 2 28033 > [ 63.673457] rcu-scale: 1 writer-duration: 3 40200 > [ 63.673464] rcu-scale: 1 writer-duration: 4 25479 > [ 63.673470] rcu-scale: 1 writer-duration: 5 29908 > [ 63.673475] rcu-scale: 1 writer-duration: 6 27336 > [ 63.673481] rcu-scale: 1 writer-duration: 7 36948 > [ 63.673487] rcu-scale: 1 writer-duration: 8 24216 > [ 63.673494] rcu-scale: 1 writer-duration: 9 31639 > [ 63.673499] rcu-scale: 1 writer-duration: 10 27175 > [ 63.673506] rcu-scale: 1 writer-duration: 11 36695 > [ 63.673512] rcu-scale: 1 writer-duration: 12 24609 > [ 63.673518] rcu-scale: 1 writer-duration: 13 29140 > [ 63.673524] rcu-scale: 1 writer-duration: 14 26955 > [ 63.673530] rcu-scale: 1 writer-duration: 15 36122 > [ 63.673535] rcu-scale: 1 writer-duration: 16 24236 > [ 63.673541] rcu-scale: 1 writer-duration: 17 29137 > [ 63.673546] rcu-scale: 1 writer-duration: 18 26766 > [ 63.673552] rcu-scale: 1 writer-duration: 19 36207 > [ 63.673558] rcu-scale: 1 writer-duration: 20 24215 > [ 63.673563] rcu-scale: 1 writer-duration: 21 29239 > [ 63.673569] rcu-scale: 1 writer-duration: 22 26866 > [ 63.673575] rcu-scale: 1 writer-duration: 23 36054 > [ 63.673581] rcu-scale: 1 writer-duration: 24 24012 > [ 63.673586] rcu-scale: 1 writer-duration: 25 28914 > [ 63.673592] rcu-scale: 1 writer-duration: 26 26891 > [ 63.673597] rcu-scale: 1 writer-duration: 27 36102 > [ 63.673603] rcu-scale: 1 writer-duration: 28 24039 > [ 63.673609] rcu-scale: 1 writer-duration: 29 29125 > [ 63.673614] rcu-scale: 1 writer-duration: 30 26926 > [ 63.673620] rcu-scale: 1 writer-duration: 31 40754 > [ 63.673625] rcu-scale: 1 writer-duration: 32 24519 > [ 63.673631] rcu-scale: 1 writer-duration: 33 29397 > [ 63.673637] rcu-scale: 1 writer-duration: 34 27267 > [ 63.673642] rcu-scale: 1 writer-duration: 35 36358 > [ 63.673648] rcu-scale: 1 writer-duration: 36 24007 > [ 63.673654] rcu-scale: 1 writer-duration: 37 29326 > [ 63.673659] rcu-scale: 1 writer-duration: 38 26854 > [ 63.673665] rcu-scale: 1 writer-duration: 39 36625 > [ 63.673671] rcu-scale: 1 writer-duration: 40 24437 > [ 63.673676] rcu-scale: 1 writer-duration: 41 29314 > [ 63.673682] rcu-scale: 1 writer-duration: 42 26955 > [ 63.673688] rcu-scale: 1 writer-duration: 43 36415 > [ 63.673693] rcu-scale: 1 writer-duration: 44 24098 > [ 63.673699] rcu-scale: 1 writer-duration: 45 29216 > [ 63.673705] rcu-scale: 1 writer-duration: 46 26904 > [ 63.673710] rcu-scale: 1 writer-duration: 47 36286 > [ 63.673716] rcu-scale: 1 writer-duration: 48 24127 > [ 63.673722] rcu-scale: 1 writer-duration: 49 28887 > [ 63.673727] rcu-scale: 1 writer-duration: 50 26894 > [ 63.673733] rcu-scale: 1 writer-duration: 51 36145 > [ 63.673739] rcu-scale: 1 writer-duration: 52 24238 > [ 63.673745] rcu-scale: 1 writer-duration: 53 29136 > [ 63.673750] rcu-scale: 1 writer-duration: 54 26895 > [ 63.673756] rcu-scale: 1 writer-duration: 55 36087 > [ 63.673761] rcu-scale: 1 writer-duration: 56 24215 > [ 63.673767] rcu-scale: 1 writer-duration: 57 29034 > [ 63.673773] rcu-scale: 1 writer-duration: 58 26817 > [ 63.673778] rcu-scale: 1 writer-duration: 59 36069 > [ 63.673784] rcu-scale: 1 writer-duration: 60 24255 > [ 63.673790] rcu-scale: 1 writer-duration: 61 29004 > [ 63.673795] rcu-scale: 1 writer-duration: 62 26878 > [ 63.673801] rcu-scale: 1 writer-duration: 63 36186 > [ 63.673806] rcu-scale: 1 writer-duration: 64 24355 > [ 63.673812] rcu-scale: 1 writer-duration: 65 28884 > [ 63.673818] rcu-scale: 1 writer-duration: 66 32061 > [ 63.673823] rcu-scale: 1 writer-duration: 67 36261 > [ 63.673829] rcu-scale: 1 writer-duration: 68 24453 > [ 63.673835] rcu-scale: 1 writer-duration: 69 29508 > [ 63.673840] rcu-scale: 1 writer-duration: 70 27004 > [ 63.673846] rcu-scale: 1 writer-duration: 71 36240 > [ 63.673851] rcu-scale: 1 writer-duration: 72 23958 > [ 63.673857] rcu-scale: 1 writer-duration: 73 29276 > [ 63.673863] rcu-scale: 1 writer-duration: 74 26793 > [ 63.673868] rcu-scale: 1 writer-duration: 75 36779 > [ 63.673874] rcu-scale: 1 writer-duration: 76 24462 > [ 63.673880] rcu-scale: 1 writer-duration: 77 29156 > [ 63.673885] rcu-scale: 1 writer-duration: 78 27128 > [ 63.673891] rcu-scale: 1 writer-duration: 79 36182 > [ 63.673896] rcu-scale: 1 writer-duration: 80 24073 > [ 63.673902] rcu-scale: 1 writer-duration: 81 28955 > [ 63.673908] rcu-scale: 1 writer-duration: 82 27009 > [ 63.673914] rcu-scale: 1 writer-duration: 83 36169 > [ 63.673919] rcu-scale: 1 writer-duration: 84 23906 > [ 63.673925] rcu-scale: 1 writer-duration: 85 28946 > [ 63.673931] rcu-scale: 1 writer-duration: 86 26866 > [ 63.673937] rcu-scale: 1 writer-duration: 87 36249 > [ 63.673942] rcu-scale: 1 writer-duration: 88 24096 > [ 63.673948] rcu-scale: 1 writer-duration: 89 29290 > [ 63.673954] rcu-scale: 1 writer-duration: 90 26844 > [ 63.673959] rcu-scale: 1 writer-duration: 91 36353 > [ 63.673965] rcu-scale: 1 writer-duration: 92 24166 > [ 63.673971] rcu-scale: 1 writer-duration: 93 29072 > [ 63.673976] rcu-scale: 1 writer-duration: 94 26862 > [ 63.673982] rcu-scale: 1 writer-duration: 95 36348 > [ 63.673988] rcu-scale: 1 writer-duration: 96 23907 > [ 63.673993] rcu-scale: 1 writer-duration: 97 29122 > [ 63.673999] rcu-scale: 1 writer-duration: 98 27072 > [ 63.674004] rcu-scale: 1 writer-duration: 99 36188 > [ 63.674010] rcu-scale: 1 writer-duration: 100 29125 > [ 63.675441] rcu-scale: 1 writer-duration: 101 48063 > > > > [ 63.675458] rcu-scale: 2 writer-duration: 0 65839 > [ 63.677455] rcu-scale: 2 writer-duration: 1 30102 > [ 63.677470] rcu-scale: 2 writer-duration: 2 30481 > [ 63.677477] rcu-scale: 2 writer-duration: 3 28129 > [ 63.677483] rcu-scale: 2 writer-duration: 4 37897 > [ 63.677489] rcu-scale: 2 writer-duration: 5 27420 > [ 63.677496] rcu-scale: 2 writer-duration: 6 29475 > [ 63.677501] rcu-scale: 2 writer-duration: 7 25128 > [ 63.677507] rcu-scale: 2 writer-duration: 8 36449 > [ 63.677512] rcu-scale: 2 writer-duration: 9 29093 > [ 63.677518] rcu-scale: 2 writer-duration: 10 29369 > [ 63.677524] rcu-scale: 2 writer-duration: 11 24938 > [ 63.677530] rcu-scale: 2 writer-duration: 12 36634 > [ 63.677535] rcu-scale: 2 writer-duration: 13 26628 > [ 63.677541] rcu-scale: 2 writer-duration: 14 29116 > [ 63.677547] rcu-scale: 2 writer-duration: 15 24320 > [ 63.677552] rcu-scale: 2 writer-duration: 16 36351 > [ 63.677558] rcu-scale: 2 writer-duration: 17 26613 > [ 63.677564] rcu-scale: 2 writer-duration: 18 29039 > [ 63.677570] rcu-scale: 2 writer-duration: 19 24353 > [ 63.677575] rcu-scale: 2 writer-duration: 20 36335 > [ 63.677581] rcu-scale: 2 writer-duration: 21 26706 > [ 63.677587] rcu-scale: 2 writer-duration: 22 29200 > [ 63.677592] rcu-scale: 2 writer-duration: 23 24211 > [ 63.677598] rcu-scale: 2 writer-duration: 24 36036 > [ 63.677604] rcu-scale: 2 writer-duration: 25 26464 > [ 63.677610] rcu-scale: 2 writer-duration: 26 29181 > [ 63.677616] rcu-scale: 2 writer-duration: 27 24021 > [ 63.677621] rcu-scale: 2 writer-duration: 28 36169 > [ 63.677627] rcu-scale: 2 writer-duration: 29 26697 > [ 63.677633] rcu-scale: 2 writer-duration: 30 29213 > [ 63.677638] rcu-scale: 2 writer-duration: 31 24181 > [ 63.677644] rcu-scale: 2 writer-duration: 32 41287 > [ 63.677650] rcu-scale: 2 writer-duration: 33 27016 > [ 63.677656] rcu-scale: 2 writer-duration: 34 29455 > [ 63.677662] rcu-scale: 2 writer-duration: 35 24571 > [ 63.677667] rcu-scale: 2 writer-duration: 36 36144 > [ 63.677673] rcu-scale: 2 writer-duration: 37 26673 > [ 63.677678] rcu-scale: 2 writer-duration: 38 29031 > [ 63.677684] rcu-scale: 2 writer-duration: 39 25127 > [ 63.677690] rcu-scale: 2 writer-duration: 40 36453 > [ 63.677695] rcu-scale: 2 writer-duration: 41 26455 > [ 63.677701] rcu-scale: 2 writer-duration: 42 29460 > [ 63.677707] rcu-scale: 2 writer-duration: 43 24698 > [ 63.677712] rcu-scale: 2 writer-duration: 44 36093 > [ 63.677718] rcu-scale: 2 writer-duration: 45 26573 > [ 63.677723] rcu-scale: 2 writer-duration: 46 29157 > [ 63.677729] rcu-scale: 2 writer-duration: 47 24356 > [ 63.677735] rcu-scale: 2 writer-duration: 48 36504 > [ 63.677741] rcu-scale: 2 writer-duration: 49 26157 > [ 63.677746] rcu-scale: 2 writer-duration: 50 29103 > [ 63.677752] rcu-scale: 2 writer-duration: 51 24577 > [ 63.677758] rcu-scale: 2 writer-duration: 52 36218 > [ 63.677763] rcu-scale: 2 writer-duration: 53 26553 > [ 63.677769] rcu-scale: 2 writer-duration: 54 29224 > [ 63.677775] rcu-scale: 2 writer-duration: 55 24290 > [ 63.677780] rcu-scale: 2 writer-duration: 56 36322 > [ 63.677786] rcu-scale: 2 writer-duration: 57 26509 > [ 63.677792] rcu-scale: 2 writer-duration: 58 28951 > [ 63.677797] rcu-scale: 2 writer-duration: 59 24316 > [ 63.677803] rcu-scale: 2 writer-duration: 60 36402 > [ 63.677809] rcu-scale: 2 writer-duration: 61 26453 > [ 63.677814] rcu-scale: 2 writer-duration: 62 29023 > [ 63.677820] rcu-scale: 2 writer-duration: 63 24538 > [ 63.677826] rcu-scale: 2 writer-duration: 64 36279 > [ 63.677831] rcu-scale: 2 writer-duration: 65 26277 > [ 63.677837] rcu-scale: 2 writer-duration: 66 34442 > [ 63.677843] rcu-scale: 2 writer-duration: 67 24564 > [ 63.677849] rcu-scale: 2 writer-duration: 68 36429 > [ 63.677855] rcu-scale: 2 writer-duration: 69 27018 > [ 63.677861] rcu-scale: 2 writer-duration: 70 29207 > [ 63.677866] rcu-scale: 2 writer-duration: 71 24488 > [ 63.677872] rcu-scale: 2 writer-duration: 72 36187 > [ 63.677878] rcu-scale: 2 writer-duration: 73 26564 > [ 63.677883] rcu-scale: 2 writer-duration: 74 29127 > [ 63.677889] rcu-scale: 2 writer-duration: 75 25213 > [ 63.677894] rcu-scale: 2 writer-duration: 76 36202 > [ 63.677900] rcu-scale: 2 writer-duration: 77 26709 > [ 63.677905] rcu-scale: 2 writer-duration: 78 29292 > [ 63.677911] rcu-scale: 2 writer-duration: 79 24338 > [ 63.677917] rcu-scale: 2 writer-duration: 80 36322 > [ 63.677922] rcu-scale: 2 writer-duration: 81 26289 > [ 63.677928] rcu-scale: 2 writer-duration: 82 29386 > [ 63.677934] rcu-scale: 2 writer-duration: 83 24300 > [ 63.677939] rcu-scale: 2 writer-duration: 84 36011 > [ 63.677945] rcu-scale: 2 writer-duration: 85 26219 > [ 63.677951] rcu-scale: 2 writer-duration: 86 29250 > [ 63.677957] rcu-scale: 2 writer-duration: 87 24484 > [ 63.677963] rcu-scale: 2 writer-duration: 88 36254 > [ 63.677969] rcu-scale: 2 writer-duration: 89 26631 > [ 63.677975] rcu-scale: 2 writer-duration: 90 29105 > [ 63.677980] rcu-scale: 2 writer-duration: 91 24502 > [ 63.677986] rcu-scale: 2 writer-duration: 92 36409 > [ 63.677992] rcu-scale: 2 writer-duration: 93 26414 > [ 63.677997] rcu-scale: 2 writer-duration: 94 29130 > [ 63.678003] rcu-scale: 2 writer-duration: 95 24586 > [ 63.678009] rcu-scale: 2 writer-duration: 96 36159 > [ 63.678014] rcu-scale: 2 writer-duration: 97 26263 > [ 63.678021] rcu-scale: 2 writer-duration: 98 29479 > [ 63.678027] rcu-scale: 2 writer-duration: 99 24352 > [ 63.678032] rcu-scale: 2 writer-duration: 100 39478 > > > > > [ 63.679533] rcu-scale: 3 writer-duration: 0 31164 > [ 63.681440] rcu-scale: 3 writer-duration: 1 31690 > [ 63.681451] rcu-scale: 3 writer-duration: 2 28398 > [ 63.681457] rcu-scale: 3 writer-duration: 3 37524 > [ 63.681462] rcu-scale: 3 writer-duration: 4 30331 > [ 63.681468] rcu-scale: 3 writer-duration: 5 30588 > [ 63.681473] rcu-scale: 3 writer-duration: 6 25430 > [ 63.681479] rcu-scale: 3 writer-duration: 7 36628 > [ 63.681485] rcu-scale: 3 writer-duration: 8 27195 > [ 63.681490] rcu-scale: 3 writer-duration: 9 29267 > [ 63.681496] rcu-scale: 3 writer-duration: 10 26967 > [ 63.681503] rcu-scale: 3 writer-duration: 11 36610 > [ 63.681509] rcu-scale: 3 writer-duration: 12 27017 > [ 63.681514] rcu-scale: 3 writer-duration: 13 29493 > [ 63.681520] rcu-scale: 3 writer-duration: 14 24647 > [ 63.681526] rcu-scale: 3 writer-duration: 15 36099 > [ 63.681531] rcu-scale: 3 writer-duration: 16 26574 > [ 63.681537] rcu-scale: 3 writer-duration: 17 28995 > [ 63.681542] rcu-scale: 3 writer-duration: 18 24627 > [ 63.681548] rcu-scale: 3 writer-duration: 19 36099 > [ 63.681554] rcu-scale: 3 writer-duration: 20 26580 > [ 63.681559] rcu-scale: 3 writer-duration: 21 29133 > [ 63.681565] rcu-scale: 3 writer-duration: 22 24684 > [ 63.681570] rcu-scale: 3 writer-duration: 23 36089 > [ 63.681576] rcu-scale: 3 writer-duration: 24 26508 > [ 63.681582] rcu-scale: 3 writer-duration: 25 28821 > [ 63.681587] rcu-scale: 3 writer-duration: 26 24382 > [ 63.681593] rcu-scale: 3 writer-duration: 27 36151 > [ 63.681599] rcu-scale: 3 writer-duration: 28 26336 > [ 63.681605] rcu-scale: 3 writer-duration: 29 28946 > [ 63.681610] rcu-scale: 3 writer-duration: 30 24735 > [ 63.681616] rcu-scale: 3 writer-duration: 31 36237 > [ 63.681622] rcu-scale: 3 writer-duration: 32 26239 > [ 63.681627] rcu-scale: 3 writer-duration: 33 34242 > [ 63.681633] rcu-scale: 3 writer-duration: 34 24917 > [ 63.681639] rcu-scale: 3 writer-duration: 35 36459 > [ 63.681645] rcu-scale: 3 writer-duration: 36 26774 > [ 63.681650] rcu-scale: 3 writer-duration: 37 28911 > [ 63.681656] rcu-scale: 3 writer-duration: 38 24797 > [ 63.681662] rcu-scale: 3 writer-duration: 39 36099 > [ 63.681667] rcu-scale: 3 writer-duration: 40 27206 > [ 63.681673] rcu-scale: 3 writer-duration: 41 29201 > [ 63.681678] rcu-scale: 3 writer-duration: 42 24676 > [ 63.681684] rcu-scale: 3 writer-duration: 43 36260 > [ 63.681690] rcu-scale: 3 writer-duration: 44 26859 > [ 63.681695] rcu-scale: 3 writer-duration: 45 29087 > [ 63.681701] rcu-scale: 3 writer-duration: 46 24531 > [ 63.681707] rcu-scale: 3 writer-duration: 47 36144 > [ 63.681712] rcu-scale: 3 writer-duration: 48 26415 > [ 63.681718] rcu-scale: 3 writer-duration: 49 29230 > [ 63.681724] rcu-scale: 3 writer-duration: 50 24371 > [ 63.681729] rcu-scale: 3 writer-duration: 51 36045 > [ 63.681735] rcu-scale: 3 writer-duration: 52 26707 > [ 63.681740] rcu-scale: 3 writer-duration: 53 29079 > [ 63.681746] rcu-scale: 3 writer-duration: 54 24622 > [ 63.681752] rcu-scale: 3 writer-duration: 55 36202 > [ 63.681757] rcu-scale: 3 writer-duration: 56 26489 > [ 63.681763] rcu-scale: 3 writer-duration: 57 29016 > [ 63.681769] rcu-scale: 3 writer-duration: 58 24632 > [ 63.681775] rcu-scale: 3 writer-duration: 59 35968 > [ 63.681780] rcu-scale: 3 writer-duration: 60 26480 > [ 63.681786] rcu-scale: 3 writer-duration: 61 29171 > [ 63.681792] rcu-scale: 3 writer-duration: 62 24542 > [ 63.681797] rcu-scale: 3 writer-duration: 63 35972 > [ 63.681803] rcu-scale: 3 writer-duration: 64 26459 > [ 63.681809] rcu-scale: 3 writer-duration: 65 29248 > [ 63.681814] rcu-scale: 3 writer-duration: 66 24403 > [ 63.681820] rcu-scale: 3 writer-duration: 67 41353 > [ 63.681826] rcu-scale: 3 writer-duration: 68 26759 > [ 63.681831] rcu-scale: 3 writer-duration: 69 29167 > [ 63.681837] rcu-scale: 3 writer-duration: 70 25051 > [ 63.681843] rcu-scale: 3 writer-duration: 71 36298 > [ 63.681848] rcu-scale: 3 writer-duration: 72 26557 > [ 63.681854] rcu-scale: 3 writer-duration: 73 28940 > [ 63.681860] rcu-scale: 3 writer-duration: 74 24726 > [ 63.681865] rcu-scale: 3 writer-duration: 75 36109 > [ 63.681871] rcu-scale: 3 writer-duration: 76 27216 > [ 63.681877] rcu-scale: 3 writer-duration: 77 29170 > [ 63.681882] rcu-scale: 3 writer-duration: 78 24606 > [ 63.681888] rcu-scale: 3 writer-duration: 79 36459 > [ 63.681893] rcu-scale: 3 writer-duration: 80 26470 > [ 63.681899] rcu-scale: 3 writer-duration: 81 28939 > [ 63.681905] rcu-scale: 3 writer-duration: 82 24758 > [ 63.681910] rcu-scale: 3 writer-duration: 83 36139 > [ 63.681916] rcu-scale: 3 writer-duration: 84 26535 > [ 63.681922] rcu-scale: 3 writer-duration: 85 28824 > [ 63.681927] rcu-scale: 3 writer-duration: 86 24272 > [ 63.681933] rcu-scale: 3 writer-duration: 87 36187 > [ 63.681939] rcu-scale: 3 writer-duration: 88 26565 > [ 63.681944] rcu-scale: 3 writer-duration: 89 29050 > [ 63.681950] rcu-scale: 3 writer-duration: 90 24686 > [ 63.681955] rcu-scale: 3 writer-duration: 91 36216 > [ 63.681961] rcu-scale: 3 writer-duration: 92 26589 > [ 63.681967] rcu-scale: 3 writer-duration: 93 29076 > [ 63.681973] rcu-scale: 3 writer-duration: 94 24683 > [ 63.681978] rcu-scale: 3 writer-duration: 95 36081 > [ 63.681984] rcu-scale: 3 writer-duration: 96 26710 > [ 63.681989] rcu-scale: 3 writer-duration: 97 28881 > [ 63.681995] rcu-scale: 3 writer-duration: 98 24494 > [ 63.682001] rcu-scale: 3 writer-duration: 99 36297 > [ 63.682006] rcu-scale: 3 writer-duration: 100 26487 > > > [ 63.683440] rcu-scale: 3 writer-duration: 101 48416
On Thu, Jun 23, 2022 at 03:23:19AM +0000, Zhang, Qiang1 wrote: > On Wed, Jun 22, 2022 at 05:34:09PM -0700, Paul E. McKenney wrote: > > On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > > > > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > > > grace period begins, if detected current CPU enters idle in > > > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > > > current cpu, at this time, maybe not being in an RCU read-side critical > > > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > > > occurs on current CPU to check and report exp QS. > > > > > > > > > > >I think the idea is OK, however, this "optimization" is based on the > > > >implementation detail that rcu_read_lock() counts preempt_count when > > > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > > > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > > > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > > > > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > > > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > > > preempt-count is exists, so can report exp QS when not being an RCU > > > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > > > in IPI handler. > > > > > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > > > The rcu_read_lock/unlock is just barrier(). > > > > > > > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > > > > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > > > preempt-count is also checked > > > > > > /* Report any deferred quiescent states if preemption enabled. */ > > > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > > > rcu_preempt_deferred_qs(current); > > > > > > but the RCU softirq may not be triggered in time and reported exp QS, for > > > example a kernel loop exist on NO_HZ_FULL CPU > > > > > > this change, It is to capture the exp QS state earlier and report it. > > > > > > > > > >Also I'm not aware of any but there could be someone assuming that RCU > > > >read-side critical sections can be formed without > > > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > > > >might be "creative" code like the following: > > > > > > > > void do_something_only_in_nonpreempt(void) > > > > { > > > > int *p; > > > > > > > > // This function only gets called in PREEMPT=n kernel, > > > > // which means everywhere is a RCU read-side critical > > > > // section, let's save some lines of code. > > > > > > > rcu_read_lock(); > > > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > > > ... // of course no schedule() here. > > > > <access p> > > > rcu_read_unlock(); > > > > } > > > > > > > > > > Usually access to pointers of type rcu needs to be protected. > > Yes, _ususally_ they are, but what about the special cases? Because > in PREEMPT=n kernel, almost everywhere is a RCU read-side critical > section, some one might have been "creative" enough to omit these > rcu_read_lock() and rcu_read_unlock(). > > > > > Indeed, lockdep would normally complain about this sort of thing. > > But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without > > CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > > > > >Actually with the rcu_dereference_check() above, lockdep cannot detect > >even CONFIG_PREEMPT_COUNT=y, that rcu_dereference_check() basically says > >"I know I'm in a read-side critical section if it's a non-preempt > >kernel, so don't bother to check". ;-) > > > > Any thoughts? > > > > It would be good to have some performance data on this change to expedited > > grace periods. It is adding code, so it needs some real motivation. > > I used rcuscale to test it, and count the average time of writer-duration > > no applied patch(org.txt) > writer avg time 29690.39 29670.78 29770.65 29423.25 > > applied patch(new.txt) > writer avg time 28989.99 29003.54 29281.39 28986.58 > > or Is there a better way to test? >If I understand your measurments correctly, you are getting about a 2% >improvement in expedited grace-period latency in !PREEMPT kernels. >Do we have a situation for which that 2% improvement is important? To be honest, it will also be affected by the work scheduling delay. Indeed, my current test results are not very good, but it is beneficial to detect the QS state in advance and report it in time. of course, I will do more tests in the future >We would be taking some risk due to the issues Boqun points out or we >would be adding some complexity to avoid those issues. So this As for the code similar to the above scenario proposed by Boqun, it is true that lockdep will not complain, I do not deny that someone will write this way. If there really is such a situation as Boqun mentioned, There is also a risk in the following code segment in PREEMPT=n and PREEMPT_COUNT=y /* Report any deferred quiescent states if preemption enabled. */ if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) rcu_preempt_deferred_qs(current); Thanks Zqiang >2% has a definite cost associated with it. > >What am I missing here? > > Thanx, Paul > > Thanks > Zqiang > > > >Agreed. > > > >Regards, > >Boqun > > > So, does this change make a system-level difference in (say) expedited > > RCU grace-period latency, and if so, under what conditions? > > > > Thanx, Paul > > > > > >Again, I'm not aware of any existing code that does this but we need to > > > >be sure. > > > > > > > >Regards, > > > >Boqun > > > > > > > > This commit add a exp QS check in rcu_exp_handler(), when not being > > > > in an RCU read-side critical section, report exp QS earlier. > > > > > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > > > --- > > > > kernel/rcu/tree_exp.h | 5 ++++- > > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h > > > > index be667583a554..34f08267410f 100644 > > > > --- a/kernel/rcu/tree_exp.h > > > > +++ b/kernel/rcu/tree_exp.h > > > > @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) > > > > { > > > > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > > > > struct rcu_node *rnp = rdp->mynode; > > > > + bool preempt_bh_disabled = > > > > + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > > > > > > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > > > > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > > > > return; > > > > - if (rcu_is_cpu_rrupt_from_idle()) { > > > > + if (rcu_is_cpu_rrupt_from_idle() || > > > > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { > > > > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > > > > return; > > > > } > > > > -- > > > > 2.25.1 > > > > > [ 28.907109] rcu-scale: rcu_scale_writer 3 has 100 measurements > [ 28.907133] rcu-scale: rcu_scale_writer 0 has 100 measurements > [ 28.907135] rcu-scale: rcu_scale_writer 1 has 100 measurements > [ 28.907138] rcu-scale: rcu_scale_writer 2 has 100 measurements > [ 28.917241] Dumping ftrace buffer: > [ 28.917256] (ftrace buffer empty) > [ 28.917261] rcu-scale: Test complete > [ 49.336302] rcu-scale:!!! All grace periods expedited, no normal ones to measure! > [ 49.336329] rcu-torture: Stopping rcu_scale_reader task > [ 49.336340] rcu-torture: rcu_scale_reader is stopping > [ 49.336351] rcu-torture: Stopping rcu_scale_reader task > [ 49.336361] rcu-torture: rcu_scale_reader is stopping > [ 49.336370] rcu-torture: Stopping rcu_scale_reader task > [ 49.336405] rcu-torture: rcu_scale_reader is stopping > [ 49.336426] rcu-torture: Stopping rcu_scale_reader task > [ 49.336432] rcu-torture: rcu_scale_reader is stopping > [ 49.336443] rcu-torture: Stopping rcu_scale_writer task > [ 49.336450] rcu-torture: rcu_scale_writer is stopping > [ 49.336458] rcu-scale: writer 0 gps: 102 > [ 49.336460] rcu-torture: Stopping rcu_scale_writer task > [ 49.336465] rcu-torture: rcu_scale_writer is stopping > [ 49.336517] rcu-scale: writer 1 gps: 102 > [ 49.336519] rcu-torture: Stopping rcu_scale_writer task > [ 49.336524] rcu-torture: rcu_scale_writer is stopping > [ 49.336532] rcu-scale: writer 2 gps: 101 > [ 49.336533] rcu-torture: Stopping rcu_scale_writer task > [ 49.336539] rcu-torture: rcu_scale_writer is stopping > [ 49.336546] rcu-scale: writer 3 gps: 102 > [ 49.336548] rcu-scale: start: 28841772292 end: 28844718545 duration: 2946253 gps: 407 batches: 2 > [ 49.336550] rcu-scale: 0 writer-duration: 0 25552 > [ 49.339264] rcu-scale: 0 writer-duration: 1 42012 > [ 49.339274] rcu-scale: 0 writer-duration: 2 24377 > [ 49.339280] rcu-scale: 0 writer-duration: 3 34458 > [ 49.339299] rcu-scale: 0 writer-duration: 4 27007 > [ 49.339305] rcu-scale: 0 writer-duration: 5 36494 > [ 49.339310] rcu-scale: 0 writer-duration: 6 24278 > [ 49.339315] rcu-scale: 0 writer-duration: 7 29383 > [ 49.339321] rcu-scale: 0 writer-duration: 8 26494 > [ 49.339326] rcu-scale: 0 writer-duration: 9 35430 > [ 49.339331] rcu-scale: 0 writer-duration: 10 23986 > [ 49.339337] rcu-scale: 0 writer-duration: 11 28663 > [ 49.339342] rcu-scale: 0 writer-duration: 12 26329 > [ 49.339347] rcu-scale: 0 writer-duration: 13 35645 > [ 49.339353] rcu-scale: 0 writer-duration: 14 23761 > [ 49.339358] rcu-scale: 0 writer-duration: 15 28889 > [ 49.339364] rcu-scale: 0 writer-duration: 16 26519 > [ 49.339369] rcu-scale: 0 writer-duration: 17 35863 > [ 49.339374] rcu-scale: 0 writer-duration: 18 23984 > [ 49.339380] rcu-scale: 0 writer-duration: 19 28618 > [ 49.339385] rcu-scale: 0 writer-duration: 20 26547 > [ 49.339390] rcu-scale: 0 writer-duration: 21 35515 > [ 49.339395] rcu-scale: 0 writer-duration: 22 23913 > [ 49.339400] rcu-scale: 0 writer-duration: 23 28510 > [ 49.339405] rcu-scale: 0 writer-duration: 24 26746 > [ 49.339410] rcu-scale: 0 writer-duration: 25 35670 > [ 49.339416] rcu-scale: 0 writer-duration: 26 23963 > [ 49.339421] rcu-scale: 0 writer-duration: 27 28561 > [ 49.339427] rcu-scale: 0 writer-duration: 28 26380 > [ 49.339432] rcu-scale: 0 writer-duration: 29 36039 > [ 49.339438] rcu-scale: 0 writer-duration: 30 24215 > [ 49.339443] rcu-scale: 0 writer-duration: 31 28891 > [ 49.339449] rcu-scale: 0 writer-duration: 32 26578 > [ 49.339454] rcu-scale: 0 writer-duration: 33 40498 > [ 49.339459] rcu-scale: 0 writer-duration: 34 24245 > [ 49.339464] rcu-scale: 0 writer-duration: 35 29465 > [ 49.339469] rcu-scale: 0 writer-duration: 36 26946 > [ 49.339475] rcu-scale: 0 writer-duration: 37 35955 > [ 49.339480] rcu-scale: 0 writer-duration: 38 24059 > [ 49.339486] rcu-scale: 0 writer-duration: 39 28795 > [ 49.339491] rcu-scale: 0 writer-duration: 40 26536 > [ 49.339496] rcu-scale: 0 writer-duration: 41 35879 > [ 49.339501] rcu-scale: 0 writer-duration: 42 24181 > [ 49.339506] rcu-scale: 0 writer-duration: 43 28639 > [ 49.339512] rcu-scale: 0 writer-duration: 44 26546 > [ 49.339517] rcu-scale: 0 writer-duration: 45 35884 > [ 49.339522] rcu-scale: 0 writer-duration: 46 23986 > [ 49.339527] rcu-scale: 0 writer-duration: 47 28668 > [ 49.339532] rcu-scale: 0 writer-duration: 48 26454 > [ 49.339537] rcu-scale: 0 writer-duration: 49 35772 > [ 49.339542] rcu-scale: 0 writer-duration: 50 23942 > [ 49.339547] rcu-scale: 0 writer-duration: 51 28803 > [ 49.339553] rcu-scale: 0 writer-duration: 52 26545 > [ 49.339558] rcu-scale: 0 writer-duration: 53 35615 > [ 49.339563] rcu-scale: 0 writer-duration: 54 23569 > [ 49.339568] rcu-scale: 0 writer-duration: 55 28427 > [ 49.339573] rcu-scale: 0 writer-duration: 56 26489 > [ 49.339578] rcu-scale: 0 writer-duration: 57 35516 > [ 49.339584] rcu-scale: 0 writer-duration: 58 23722 > [ 49.339589] rcu-scale: 0 writer-duration: 59 28435 > [ 49.339594] rcu-scale: 0 writer-duration: 60 26401 > [ 49.339599] rcu-scale: 0 writer-duration: 61 35488 > [ 49.339604] rcu-scale: 0 writer-duration: 62 23855 > [ 49.339610] rcu-scale: 0 writer-duration: 63 28583 > [ 49.339615] rcu-scale: 0 writer-duration: 64 26629 > [ 49.339620] rcu-scale: 0 writer-duration: 65 36168 > [ 49.339625] rcu-scale: 0 writer-duration: 66 24191 > [ 49.339630] rcu-scale: 0 writer-duration: 67 28802 > [ 49.339635] rcu-scale: 0 writer-duration: 68 32436 > [ 49.339640] rcu-scale: 0 writer-duration: 69 38444 > [ 49.339646] rcu-scale: 0 writer-duration: 70 24417 > [ 49.339651] rcu-scale: 0 writer-duration: 71 29128 > [ 49.339656] rcu-scale: 0 writer-duration: 72 26480 > [ 49.339661] rcu-scale: 0 writer-duration: 73 35881 > [ 49.339666] rcu-scale: 0 writer-duration: 74 24123 > [ 49.339671] rcu-scale: 0 writer-duration: 75 28735 > [ 49.339676] rcu-scale: 0 writer-duration: 76 26359 > [ 49.339681] rcu-scale: 0 writer-duration: 77 35715 > [ 49.339687] rcu-scale: 0 writer-duration: 78 24092 > [ 49.339692] rcu-scale: 0 writer-duration: 79 28716 > [ 49.339698] rcu-scale: 0 writer-duration: 80 26448 > [ 49.339703] rcu-scale: 0 writer-duration: 81 35935 > [ 49.339708] rcu-scale: 0 writer-duration: 82 23966 > [ 49.339713] rcu-scale: 0 writer-duration: 83 28552 > [ 49.339718] rcu-scale: 0 writer-duration: 84 26487 > [ 49.339723] rcu-scale: 0 writer-duration: 85 35490 > [ 49.339729] rcu-scale: 0 writer-duration: 86 23976 > [ 49.339734] rcu-scale: 0 writer-duration: 87 28679 > [ 49.339740] rcu-scale: 0 writer-duration: 88 26525 > [ 49.339745] rcu-scale: 0 writer-duration: 89 35764 > [ 49.339750] rcu-scale: 0 writer-duration: 90 24050 > [ 49.339756] rcu-scale: 0 writer-duration: 91 28777 > [ 49.339761] rcu-scale: 0 writer-duration: 92 26412 > [ 49.339767] rcu-scale: 0 writer-duration: 93 35707 > [ 49.339772] rcu-scale: 0 writer-duration: 94 23913 > [ 49.339777] rcu-scale: 0 writer-duration: 95 28450 > [ 49.339782] rcu-scale: 0 writer-duration: 96 26665 > [ 49.339788] rcu-scale: 0 writer-duration: 97 35636 > [ 49.339793] rcu-scale: 0 writer-duration: 98 23953 > [ 49.339798] rcu-scale: 0 writer-duration: 99 28466 > [ 49.339804] rcu-scale: 0 writer-duration: 100 26654 > [ 49.341227] rcu-scale: 0 writer-duration: 101 225635 > > > > > [ 49.341246] rcu-scale: 1 writer-duration: 0 27277 > [ 49.343227] rcu-scale: 1 writer-duration: 1 34458 > [ 49.343236] rcu-scale: 1 writer-duration: 2 27340 > [ 49.343242] rcu-scale: 1 writer-duration: 3 41758 > [ 49.343248] rcu-scale: 1 writer-duration: 4 24161 > [ 49.343254] rcu-scale: 1 writer-duration: 5 29250 > [ 49.343260] rcu-scale: 1 writer-duration: 6 26996 > [ 49.343266] rcu-scale: 1 writer-duration: 7 36482 > [ 49.343272] rcu-scale: 1 writer-duration: 8 23782 > [ 49.343277] rcu-scale: 1 writer-duration: 9 28433 > [ 49.343283] rcu-scale: 1 writer-duration: 10 26577 > [ 49.343289] rcu-scale: 1 writer-duration: 11 35616 > [ 49.343295] rcu-scale: 1 writer-duration: 12 23821 > [ 49.343301] rcu-scale: 1 writer-duration: 13 28678 > [ 49.343307] rcu-scale: 1 writer-duration: 14 26517 > [ 49.343313] rcu-scale: 1 writer-duration: 15 35548 > [ 49.343319] rcu-scale: 1 writer-duration: 16 24196 > [ 49.343325] rcu-scale: 1 writer-duration: 17 28626 > [ 49.343332] rcu-scale: 1 writer-duration: 18 26772 > [ 49.343338] rcu-scale: 1 writer-duration: 19 35539 > [ 49.343344] rcu-scale: 1 writer-duration: 20 23897 > [ 49.343350] rcu-scale: 1 writer-duration: 21 28635 > [ 49.343356] rcu-scale: 1 writer-duration: 22 26453 > [ 49.343362] rcu-scale: 1 writer-duration: 23 35416 > [ 49.343368] rcu-scale: 1 writer-duration: 24 24096 > [ 49.343373] rcu-scale: 1 writer-duration: 25 28733 > [ 49.343379] rcu-scale: 1 writer-duration: 26 26538 > [ 49.343385] rcu-scale: 1 writer-duration: 27 35590 > [ 49.343390] rcu-scale: 1 writer-duration: 28 23739 > [ 49.343396] rcu-scale: 1 writer-duration: 29 29450 > [ 49.343402] rcu-scale: 1 writer-duration: 30 26452 > [ 49.343408] rcu-scale: 1 writer-duration: 31 35807 > [ 49.343413] rcu-scale: 1 writer-duration: 32 24145 > [ 49.343419] rcu-scale: 1 writer-duration: 33 33452 > [ 49.343425] rcu-scale: 1 writer-duration: 34 26767 > [ 49.343431] rcu-scale: 1 writer-duration: 35 36677 > [ 49.343438] rcu-scale: 1 writer-duration: 36 24102 > [ 49.343444] rcu-scale: 1 writer-duration: 37 28937 > [ 49.343449] rcu-scale: 1 writer-duration: 38 26622 > [ 49.343454] rcu-scale: 1 writer-duration: 39 35903 > [ 49.343460] rcu-scale: 1 writer-duration: 40 23956 > [ 49.343465] rcu-scale: 1 writer-duration: 41 28684 > [ 49.343470] rcu-scale: 1 writer-duration: 42 26901 > [ 49.343475] rcu-scale: 1 writer-duration: 43 35717 > [ 49.343480] rcu-scale: 1 writer-duration: 44 23957 > [ 49.343485] rcu-scale: 1 writer-duration: 45 28780 > [ 49.343490] rcu-scale: 1 writer-duration: 46 26747 > [ 49.343495] rcu-scale: 1 writer-duration: 47 35538 > [ 49.343501] rcu-scale: 1 writer-duration: 48 23963 > [ 49.343506] rcu-scale: 1 writer-duration: 49 28727 > [ 49.343511] rcu-scale: 1 writer-duration: 50 26519 > [ 49.343516] rcu-scale: 1 writer-duration: 51 35733 > [ 49.343521] rcu-scale: 1 writer-duration: 52 24098 > [ 49.343527] rcu-scale: 1 writer-duration: 53 28557 > [ 49.343532] rcu-scale: 1 writer-duration: 54 26208 > [ 49.343537] rcu-scale: 1 writer-duration: 55 35315 > [ 49.343543] rcu-scale: 1 writer-duration: 56 23932 > [ 49.343548] rcu-scale: 1 writer-duration: 57 28507 > [ 49.343553] rcu-scale: 1 writer-duration: 58 26443 > [ 49.343559] rcu-scale: 1 writer-duration: 59 35306 > [ 49.343564] rcu-scale: 1 writer-duration: 60 23868 > [ 49.343570] rcu-scale: 1 writer-duration: 61 28606 > [ 49.343575] rcu-scale: 1 writer-duration: 62 26308 > [ 49.343580] rcu-scale: 1 writer-duration: 63 35693 > [ 49.343586] rcu-scale: 1 writer-duration: 64 23934 > [ 49.343591] rcu-scale: 1 writer-duration: 65 29456 > [ 49.343596] rcu-scale: 1 writer-duration: 66 26600 > [ 49.343602] rcu-scale: 1 writer-duration: 67 35682 > [ 49.343607] rcu-scale: 1 writer-duration: 68 29849 > [ 49.343612] rcu-scale: 1 writer-duration: 69 31555 > [ 49.343618] rcu-scale: 1 writer-duration: 70 26942 > [ 49.343623] rcu-scale: 1 writer-duration: 71 36037 > [ 49.343629] rcu-scale: 1 writer-duration: 72 23965 > [ 49.343634] rcu-scale: 1 writer-duration: 73 28811 > [ 49.343640] rcu-scale: 1 writer-duration: 74 26734 > [ 49.343645] rcu-scale: 1 writer-duration: 75 35712 > [ 49.343650] rcu-scale: 1 writer-duration: 76 23845 > [ 49.343656] rcu-scale: 1 writer-duration: 77 28619 > [ 49.343661] rcu-scale: 1 writer-duration: 78 26637 > [ 49.343666] rcu-scale: 1 writer-duration: 79 35773 > [ 49.343672] rcu-scale: 1 writer-duration: 80 23910 > [ 49.343677] rcu-scale: 1 writer-duration: 81 28873 > [ 49.343682] rcu-scale: 1 writer-duration: 82 26581 > [ 49.343688] rcu-scale: 1 writer-duration: 83 35517 > [ 49.343694] rcu-scale: 1 writer-duration: 84 23928 > [ 49.343699] rcu-scale: 1 writer-duration: 85 28565 > [ 49.343704] rcu-scale: 1 writer-duration: 86 26464 > [ 49.343710] rcu-scale: 1 writer-duration: 87 35715 > [ 49.343715] rcu-scale: 1 writer-duration: 88 23968 > [ 49.343721] rcu-scale: 1 writer-duration: 89 28784 > [ 49.343726] rcu-scale: 1 writer-duration: 90 26576 > [ 49.343731] rcu-scale: 1 writer-duration: 91 35800 > [ 49.343737] rcu-scale: 1 writer-duration: 92 23918 > [ 49.343742] rcu-scale: 1 writer-duration: 93 28487 > [ 49.343747] rcu-scale: 1 writer-duration: 94 26491 > [ 49.343753] rcu-scale: 1 writer-duration: 95 35565 > [ 49.343758] rcu-scale: 1 writer-duration: 96 24092 > [ 49.343763] rcu-scale: 1 writer-duration: 97 28469 > [ 49.343768] rcu-scale: 1 writer-duration: 98 26865 > [ 49.343774] rcu-scale: 1 writer-duration: 99 35343 > [ 49.343779] rcu-scale: 1 writer-duration: 100 23995 > [ 49.345221] rcu-scale: 1 writer-duration: 101 60524 > > > > [ 49.345235] rcu-scale: 2 writer-duration: 0 56581 > [ 49.347314] rcu-scale: 2 writer-duration: 1 32281 > [ 49.347329] rcu-scale: 2 writer-duration: 2 29516 > [ 49.347336] rcu-scale: 2 writer-duration: 3 29979 > [ 49.347342] rcu-scale: 2 writer-duration: 4 36232 > [ 49.347348] rcu-scale: 2 writer-duration: 5 26878 > [ 49.347354] rcu-scale: 2 writer-duration: 6 29099 > [ 49.347359] rcu-scale: 2 writer-duration: 7 24890 > [ 49.347365] rcu-scale: 2 writer-duration: 8 35614 > [ 49.347371] rcu-scale: 2 writer-duration: 9 26166 > [ 49.347377] rcu-scale: 2 writer-duration: 10 28552 > [ 49.347384] rcu-scale: 2 writer-duration: 11 24335 > [ 49.347390] rcu-scale: 2 writer-duration: 12 35614 > [ 49.347396] rcu-scale: 2 writer-duration: 13 25978 > [ 49.347402] rcu-scale: 2 writer-duration: 14 28541 > [ 49.347408] rcu-scale: 2 writer-duration: 15 24634 > [ 49.347413] rcu-scale: 2 writer-duration: 16 35640 > [ 49.347419] rcu-scale: 2 writer-duration: 17 26235 > [ 49.347425] rcu-scale: 2 writer-duration: 18 28782 > [ 49.347431] rcu-scale: 2 writer-duration: 19 24288 > [ 49.347437] rcu-scale: 2 writer-duration: 20 35649 > [ 49.347442] rcu-scale: 2 writer-duration: 21 25989 > [ 49.347448] rcu-scale: 2 writer-duration: 22 28527 > [ 49.347454] rcu-scale: 2 writer-duration: 23 24150 > [ 49.347460] rcu-scale: 2 writer-duration: 24 35960 > [ 49.347465] rcu-scale: 2 writer-duration: 25 26133 > [ 49.347471] rcu-scale: 2 writer-duration: 26 28714 > [ 49.347477] rcu-scale: 2 writer-duration: 27 24192 > [ 49.347482] rcu-scale: 2 writer-duration: 28 35545 > [ 49.347488] rcu-scale: 2 writer-duration: 29 26893 > [ 49.347494] rcu-scale: 2 writer-duration: 30 28621 > [ 49.347499] rcu-scale: 2 writer-duration: 31 24453 > [ 49.347505] rcu-scale: 2 writer-duration: 32 35750 > [ 49.347510] rcu-scale: 2 writer-duration: 33 31021 > [ 49.347516] rcu-scale: 2 writer-duration: 34 29061 > [ 49.347522] rcu-scale: 2 writer-duration: 35 25015 > [ 49.347528] rcu-scale: 2 writer-duration: 36 36107 > [ 49.347534] rcu-scale: 2 writer-duration: 37 26370 > [ 49.347540] rcu-scale: 2 writer-duration: 38 28888 > [ 49.347546] rcu-scale: 2 writer-duration: 39 24418 > [ 49.347552] rcu-scale: 2 writer-duration: 40 35675 > [ 49.347558] rcu-scale: 2 writer-duration: 41 26284 > [ 49.347564] rcu-scale: 2 writer-duration: 42 28924 > [ 49.347570] rcu-scale: 2 writer-duration: 43 24244 > [ 49.347576] rcu-scale: 2 writer-duration: 44 35835 > [ 49.347582] rcu-scale: 2 writer-duration: 45 26194 > [ 49.347587] rcu-scale: 2 writer-duration: 46 29080 > [ 49.347593] rcu-scale: 2 writer-duration: 47 24185 > [ 49.347599] rcu-scale: 2 writer-duration: 48 35543 > [ 49.347605] rcu-scale: 2 writer-duration: 49 26122 > [ 49.347610] rcu-scale: 2 writer-duration: 50 28822 > [ 49.347616] rcu-scale: 2 writer-duration: 51 24285 > [ 49.347622] rcu-scale: 2 writer-duration: 52 35720 > [ 49.347627] rcu-scale: 2 writer-duration: 53 26063 > [ 49.347633] rcu-scale: 2 writer-duration: 54 28508 > [ 49.347639] rcu-scale: 2 writer-duration: 55 24019 > [ 49.347645] rcu-scale: 2 writer-duration: 56 35499 > [ 49.347651] rcu-scale: 2 writer-duration: 57 26079 > [ 49.347657] rcu-scale: 2 writer-duration: 58 28502 > [ 49.347663] rcu-scale: 2 writer-duration: 59 24016 > [ 49.347668] rcu-scale: 2 writer-duration: 60 35515 > [ 49.347673] rcu-scale: 2 writer-duration: 61 26095 > [ 49.347677] rcu-scale: 2 writer-duration: 62 28602 > [ 49.347681] rcu-scale: 2 writer-duration: 63 24246 > [ 49.347686] rcu-scale: 2 writer-duration: 64 35675 > [ 49.347690] rcu-scale: 2 writer-duration: 65 26761 > [ 49.347694] rcu-scale: 2 writer-duration: 66 28833 > [ 49.347699] rcu-scale: 2 writer-duration: 67 24524 > [ 49.347703] rcu-scale: 2 writer-duration: 68 43780 > [ 49.347707] rcu-scale: 2 writer-duration: 69 26612 > [ 49.347712] rcu-scale: 2 writer-duration: 70 29290 > [ 49.347716] rcu-scale: 2 writer-duration: 71 24670 > [ 49.347720] rcu-scale: 2 writer-duration: 72 35526 > [ 49.347725] rcu-scale: 2 writer-duration: 73 26393 > [ 49.347729] rcu-scale: 2 writer-duration: 74 28967 > [ 49.347733] rcu-scale: 2 writer-duration: 75 24125 > [ 49.347738] rcu-scale: 2 writer-duration: 76 35713 > [ 49.347742] rcu-scale: 2 writer-duration: 77 26062 > [ 49.347746] rcu-scale: 2 writer-duration: 78 28888 > [ 49.347751] rcu-scale: 2 writer-duration: 79 24344 > [ 49.347755] rcu-scale: 2 writer-duration: 80 35858 > [ 49.347760] rcu-scale: 2 writer-duration: 81 26330 > [ 49.347764] rcu-scale: 2 writer-duration: 82 28661 > [ 49.347769] rcu-scale: 2 writer-duration: 83 24180 > [ 49.347773] rcu-scale: 2 writer-duration: 84 35464 > [ 49.347778] rcu-scale: 2 writer-duration: 85 25886 > [ 49.347782] rcu-scale: 2 writer-duration: 86 28916 > [ 49.347786] rcu-scale: 2 writer-duration: 87 24149 > [ 49.347791] rcu-scale: 2 writer-duration: 88 35860 > [ 49.347795] rcu-scale: 2 writer-duration: 89 26100 > [ 49.347800] rcu-scale: 2 writer-duration: 90 28999 > [ 49.347804] rcu-scale: 2 writer-duration: 91 24105 > [ 49.347809] rcu-scale: 2 writer-duration: 92 35720 > [ 49.347813] rcu-scale: 2 writer-duration: 93 26093 > [ 49.347817] rcu-scale: 2 writer-duration: 94 28729 > [ 49.347822] rcu-scale: 2 writer-duration: 95 24063 > [ 49.347826] rcu-scale: 2 writer-duration: 96 35819 > [ 49.347831] rcu-scale: 2 writer-duration: 97 26119 > [ 49.347835] rcu-scale: 2 writer-duration: 98 28648 > [ 49.347840] rcu-scale: 2 writer-duration: 99 24175 > [ 49.347844] rcu-scale: 2 writer-duration: 100 34035 > > > > > [ 49.349299] rcu-scale: 3 writer-duration: 0 26939 > [ 49.351268] rcu-scale: 3 writer-duration: 1 27152 > [ 49.351277] rcu-scale: 3 writer-duration: 2 30299 > [ 49.351281] rcu-scale: 3 writer-duration: 3 36662 > [ 49.351284] rcu-scale: 3 writer-duration: 4 31847 > [ 49.351287] rcu-scale: 3 writer-duration: 5 29203 > [ 49.351290] rcu-scale: 3 writer-duration: 6 24861 > [ 49.351294] rcu-scale: 3 writer-duration: 7 36222 > [ 49.351297] rcu-scale: 3 writer-duration: 8 26790 > [ 49.351300] rcu-scale: 3 writer-duration: 9 28626 > [ 49.351303] rcu-scale: 3 writer-duration: 10 24007 > [ 49.351307] rcu-scale: 3 writer-duration: 11 35765 > [ 49.351310] rcu-scale: 3 writer-duration: 12 26226 > [ 49.351314] rcu-scale: 3 writer-duration: 13 28495 > [ 49.351317] rcu-scale: 3 writer-duration: 14 24188 > [ 49.351321] rcu-scale: 3 writer-duration: 15 35686 > [ 49.351324] rcu-scale: 3 writer-duration: 16 26290 > [ 49.351327] rcu-scale: 3 writer-duration: 17 28740 > [ 49.351331] rcu-scale: 3 writer-duration: 18 24201 > [ 49.351334] rcu-scale: 3 writer-duration: 19 36000 > [ 49.351337] rcu-scale: 3 writer-duration: 20 25995 > [ 49.351341] rcu-scale: 3 writer-duration: 21 28649 > [ 49.351344] rcu-scale: 3 writer-duration: 22 24081 > [ 49.351348] rcu-scale: 3 writer-duration: 23 35718 > [ 49.351351] rcu-scale: 3 writer-duration: 24 26077 > [ 49.351354] rcu-scale: 3 writer-duration: 25 28773 > [ 49.351357] rcu-scale: 3 writer-duration: 26 24280 > [ 49.351360] rcu-scale: 3 writer-duration: 27 35767 > [ 49.351364] rcu-scale: 3 writer-duration: 28 25956 > [ 49.351367] rcu-scale: 3 writer-duration: 29 28632 > [ 49.351370] rcu-scale: 3 writer-duration: 30 24996 > [ 49.351374] rcu-scale: 3 writer-duration: 31 35673 > [ 49.351377] rcu-scale: 3 writer-duration: 32 26217 > [ 49.351380] rcu-scale: 3 writer-duration: 33 28855 > [ 49.351383] rcu-scale: 3 writer-duration: 34 28655 > [ 49.351387] rcu-scale: 3 writer-duration: 35 36525 > [ 49.351390] rcu-scale: 3 writer-duration: 36 27052 > [ 49.351394] rcu-scale: 3 writer-duration: 37 28841 > [ 49.351397] rcu-scale: 3 writer-duration: 38 24600 > [ 49.351400] rcu-scale: 3 writer-duration: 39 35743 > [ 49.351404] rcu-scale: 3 writer-duration: 40 26391 > [ 49.351407] rcu-scale: 3 writer-duration: 41 28642 > [ 49.351410] rcu-scale: 3 writer-duration: 42 24334 > [ 49.351413] rcu-scale: 3 writer-duration: 43 36152 > [ 49.351417] rcu-scale: 3 writer-duration: 44 26133 > [ 49.351420] rcu-scale: 3 writer-duration: 45 28732 > [ 49.351423] rcu-scale: 3 writer-duration: 46 24235 > [ 49.351426] rcu-scale: 3 writer-duration: 47 36069 > [ 49.351429] rcu-scale: 3 writer-duration: 48 25993 > [ 49.351433] rcu-scale: 3 writer-duration: 49 28658 > [ 49.351436] rcu-scale: 3 writer-duration: 50 24225 > [ 49.351439] rcu-scale: 3 writer-duration: 51 35909 > [ 49.351442] rcu-scale: 3 writer-duration: 52 26176 > [ 49.351446] rcu-scale: 3 writer-duration: 53 28702 > [ 49.351449] rcu-scale: 3 writer-duration: 54 24169 > [ 49.351453] rcu-scale: 3 writer-duration: 55 35462 > [ 49.351456] rcu-scale: 3 writer-duration: 56 25771 > [ 49.351459] rcu-scale: 3 writer-duration: 57 28677 > [ 49.351462] rcu-scale: 3 writer-duration: 58 24089 > [ 49.351466] rcu-scale: 3 writer-duration: 59 35538 > [ 49.351469] rcu-scale: 3 writer-duration: 60 25785 > [ 49.351472] rcu-scale: 3 writer-duration: 61 28669 > [ 49.351475] rcu-scale: 3 writer-duration: 62 24075 > [ 49.351478] rcu-scale: 3 writer-duration: 63 35720 > [ 49.351482] rcu-scale: 3 writer-duration: 64 26017 > [ 49.351485] rcu-scale: 3 writer-duration: 65 28726 > [ 49.351488] rcu-scale: 3 writer-duration: 66 24990 > [ 49.351491] rcu-scale: 3 writer-duration: 67 35814 > [ 49.351494] rcu-scale: 3 writer-duration: 68 26215 > [ 49.351498] rcu-scale: 3 writer-duration: 69 34468 > [ 49.351501] rcu-scale: 3 writer-duration: 70 27200 > [ 49.351504] rcu-scale: 3 writer-duration: 71 36288 > [ 49.351507] rcu-scale: 3 writer-duration: 72 26445 > [ 49.351511] rcu-scale: 3 writer-duration: 73 28593 > [ 49.351514] rcu-scale: 3 writer-duration: 74 24317 > [ 49.351517] rcu-scale: 3 writer-duration: 75 36153 > [ 49.351521] rcu-scale: 3 writer-duration: 76 26164 > [ 49.351524] rcu-scale: 3 writer-duration: 77 28523 > [ 49.351527] rcu-scale: 3 writer-duration: 78 24252 > [ 49.351530] rcu-scale: 3 writer-duration: 79 35939 > [ 49.351533] rcu-scale: 3 writer-duration: 80 26058 > [ 49.351536] rcu-scale: 3 writer-duration: 81 28892 > [ 49.351540] rcu-scale: 3 writer-duration: 82 24342 > [ 49.351543] rcu-scale: 3 writer-duration: 83 35751 > [ 49.351546] rcu-scale: 3 writer-duration: 84 25932 > [ 49.351549] rcu-scale: 3 writer-duration: 85 28682 > [ 49.351552] rcu-scale: 3 writer-duration: 86 24086 > [ 49.351555] rcu-scale: 3 writer-duration: 87 35791 > [ 49.351559] rcu-scale: 3 writer-duration: 88 26189 > [ 49.351562] rcu-scale: 3 writer-duration: 89 28754 > [ 49.351565] rcu-scale: 3 writer-duration: 90 24250 > [ 49.351568] rcu-scale: 3 writer-duration: 91 35768 > [ 49.351572] rcu-scale: 3 writer-duration: 92 26160 > [ 49.351575] rcu-scale: 3 writer-duration: 93 28896 > [ 49.351578] rcu-scale: 3 writer-duration: 94 24020 > [ 49.351581] rcu-scale: 3 writer-duration: 95 35727 > [ 49.351584] rcu-scale: 3 writer-duration: 96 26099 > [ 49.351587] rcu-scale: 3 writer-duration: 97 28768 > [ 49.351590] rcu-scale: 3 writer-duration: 98 24011 > [ 49.351593] rcu-scale: 3 writer-duration: 99 35786 > [ 49.351597] rcu-scale: 3 writer-duration: 100 25989 > [ 49.353257] rcu-scale: 3 writer-duration: 101 46411 > > > [ 47.850402] rcu-scale: rcu_scale_writer 3 has 100 measurements > [ 47.850427] rcu-scale: rcu_scale_writer 0 has 100 measurements > [ 47.850429] rcu-scale: rcu_scale_writer 1 has 100 measurements > [ 47.850432] rcu-scale: rcu_scale_writer 2 has 100 measurements > [ 47.861470] Dumping ftrace buffer: > [ 47.861486] (ftrace buffer empty) > [ 47.861491] rcu-scale: Test complete > [ 63.668029] rcu-scale:!!! All grace periods expedited, no normal ones to measure! > [ 63.668051] rcu-torture: Stopping rcu_scale_reader task > [ 63.668074] rcu-torture: rcu_scale_reader is stopping > [ 63.668089] rcu-torture: Stopping rcu_scale_reader task > [ 63.668138] rcu-torture: rcu_scale_reader is stopping > [ 63.668151] rcu-torture: Stopping rcu_scale_reader task > [ 63.668191] rcu-torture: rcu_scale_reader is stopping > [ 63.668204] rcu-torture: Stopping rcu_scale_reader task > [ 63.668209] rcu-torture: rcu_scale_reader is stopping > [ 63.668231] rcu-torture: Stopping rcu_scale_writer task > [ 63.668241] rcu-torture: rcu_scale_writer is stopping > [ 63.668249] rcu-scale: writer 0 gps: 102 > [ 63.668251] rcu-torture: Stopping rcu_scale_writer task > [ 63.668260] rcu-torture: rcu_scale_writer is stopping > [ 63.668268] rcu-scale: writer 1 gps: 102 > [ 63.668270] rcu-torture: Stopping rcu_scale_writer task > [ 63.668279] rcu-torture: rcu_scale_writer is stopping > [ 63.668287] rcu-scale: writer 2 gps: 101 > [ 63.668288] rcu-torture: Stopping rcu_scale_writer task > [ 63.668293] rcu-torture: rcu_scale_writer is stopping > [ 63.668310] rcu-scale: writer 3 gps: 102 > [ 63.668311] rcu-scale: start: 47784890510 end: 47787901801 duration: 3011291 gps: 407 batches: 2 > [ 63.668314] rcu-scale: 0 writer-duration: 0 55830 > [ 63.669543] rcu-scale: 0 writer-duration: 1 40421 > [ 63.669546] rcu-scale: 0 writer-duration: 2 25268 > [ 63.669548] rcu-scale: 0 writer-duration: 3 32703 > [ 63.669549] rcu-scale: 0 writer-duration: 4 28624 > [ 63.669550] rcu-scale: 0 writer-duration: 5 36967 > [ 63.669551] rcu-scale: 0 writer-duration: 6 24686 > [ 63.669552] rcu-scale: 0 writer-duration: 7 29598 > [ 63.669554] rcu-scale: 0 writer-duration: 8 27026 > [ 63.669555] rcu-scale: 0 writer-duration: 9 38789 > [ 63.669556] rcu-scale: 0 writer-duration: 10 24398 > [ 63.669557] rcu-scale: 0 writer-duration: 11 29562 > [ 63.669559] rcu-scale: 0 writer-duration: 12 27348 > [ 63.669560] rcu-scale: 0 writer-duration: 13 36315 > [ 63.669561] rcu-scale: 0 writer-duration: 14 24233 > [ 63.669562] rcu-scale: 0 writer-duration: 15 29035 > [ 63.669564] rcu-scale: 0 writer-duration: 16 26754 > [ 63.669606] rcu-scale: 0 writer-duration: 17 36272 > [ 63.669608] rcu-scale: 0 writer-duration: 18 24237 > [ 63.669609] rcu-scale: 0 writer-duration: 19 28834 > [ 63.669610] rcu-scale: 0 writer-duration: 20 26993 > [ 63.669611] rcu-scale: 0 writer-duration: 21 36432 > [ 63.669612] rcu-scale: 0 writer-duration: 22 24178 > [ 63.669614] rcu-scale: 0 writer-duration: 23 28838 > [ 63.669615] rcu-scale: 0 writer-duration: 24 26733 > [ 63.669616] rcu-scale: 0 writer-duration: 25 36087 > [ 63.669617] rcu-scale: 0 writer-duration: 26 24237 > [ 63.669618] rcu-scale: 0 writer-duration: 27 28656 > [ 63.669620] rcu-scale: 0 writer-duration: 28 26921 > [ 63.669621] rcu-scale: 0 writer-duration: 29 36423 > [ 63.669622] rcu-scale: 0 writer-duration: 30 24201 > [ 63.669623] rcu-scale: 0 writer-duration: 31 28748 > [ 63.669625] rcu-scale: 0 writer-duration: 32 31925 > [ 63.669626] rcu-scale: 0 writer-duration: 33 36673 > [ 63.669627] rcu-scale: 0 writer-duration: 34 24630 > [ 63.669628] rcu-scale: 0 writer-duration: 35 29033 > [ 63.669629] rcu-scale: 0 writer-duration: 36 26761 > [ 63.669631] rcu-scale: 0 writer-duration: 37 36495 > [ 63.669632] rcu-scale: 0 writer-duration: 38 24209 > [ 63.669633] rcu-scale: 0 writer-duration: 39 29468 > [ 63.669634] rcu-scale: 0 writer-duration: 40 27137 > [ 63.669635] rcu-scale: 0 writer-duration: 41 36193 > [ 63.669637] rcu-scale: 0 writer-duration: 42 24531 > [ 63.669638] rcu-scale: 0 writer-duration: 43 29113 > [ 63.669639] rcu-scale: 0 writer-duration: 44 26915 > [ 63.669640] rcu-scale: 0 writer-duration: 45 36394 > [ 63.669642] rcu-scale: 0 writer-duration: 46 24247 > [ 63.669643] rcu-scale: 0 writer-duration: 47 28823 > [ 63.669644] rcu-scale: 0 writer-duration: 48 27029 > [ 63.669645] rcu-scale: 0 writer-duration: 49 36042 > [ 63.669646] rcu-scale: 0 writer-duration: 50 24164 > [ 63.669648] rcu-scale: 0 writer-duration: 51 28972 > [ 63.669649] rcu-scale: 0 writer-duration: 52 26858 > [ 63.669650] rcu-scale: 0 writer-duration: 53 36319 > [ 63.669651] rcu-scale: 0 writer-duration: 54 24233 > [ 63.669652] rcu-scale: 0 writer-duration: 55 28986 > [ 63.669654] rcu-scale: 0 writer-duration: 56 26895 > [ 63.669655] rcu-scale: 0 writer-duration: 57 36213 > [ 63.669656] rcu-scale: 0 writer-duration: 58 24156 > [ 63.669657] rcu-scale: 0 writer-duration: 59 28859 > [ 63.669658] rcu-scale: 0 writer-duration: 60 26917 > [ 63.669660] rcu-scale: 0 writer-duration: 61 36174 > [ 63.669661] rcu-scale: 0 writer-duration: 62 24237 > [ 63.669662] rcu-scale: 0 writer-duration: 63 28873 > [ 63.669663] rcu-scale: 0 writer-duration: 64 27045 > [ 63.669664] rcu-scale: 0 writer-duration: 65 36167 > [ 63.669666] rcu-scale: 0 writer-duration: 66 29556 > [ 63.669667] rcu-scale: 0 writer-duration: 67 28900 > [ 63.669668] rcu-scale: 0 writer-duration: 68 27072 > [ 63.669669] rcu-scale: 0 writer-duration: 69 36733 > [ 63.669670] rcu-scale: 0 writer-duration: 70 24327 > [ 63.669672] rcu-scale: 0 writer-duration: 71 28842 > [ 63.669673] rcu-scale: 0 writer-duration: 72 27027 > [ 63.669674] rcu-scale: 0 writer-duration: 73 36228 > [ 63.669675] rcu-scale: 0 writer-duration: 74 24137 > [ 63.669677] rcu-scale: 0 writer-duration: 75 29924 > [ 63.669678] rcu-scale: 0 writer-duration: 76 26775 > [ 63.669679] rcu-scale: 0 writer-duration: 77 36300 > [ 63.669680] rcu-scale: 0 writer-duration: 78 24547 > [ 63.669681] rcu-scale: 0 writer-duration: 79 28852 > [ 63.669683] rcu-scale: 0 writer-duration: 80 26945 > [ 63.669684] rcu-scale: 0 writer-duration: 81 36127 > [ 63.669685] rcu-scale: 0 writer-duration: 82 24373 > [ 63.669686] rcu-scale: 0 writer-duration: 83 28853 > [ 63.669687] rcu-scale: 0 writer-duration: 84 26652 > [ 63.669689] rcu-scale: 0 writer-duration: 85 35952 > [ 63.669690] rcu-scale: 0 writer-duration: 86 24379 > [ 63.669691] rcu-scale: 0 writer-duration: 87 28948 > [ 63.669692] rcu-scale: 0 writer-duration: 88 26958 > [ 63.669693] rcu-scale: 0 writer-duration: 89 36302 > [ 63.669695] rcu-scale: 0 writer-duration: 90 24181 > [ 63.669696] rcu-scale: 0 writer-duration: 91 29120 > [ 63.669697] rcu-scale: 0 writer-duration: 92 27032 > [ 63.669698] rcu-scale: 0 writer-duration: 93 36083 > [ 63.669699] rcu-scale: 0 writer-duration: 94 24248 > [ 63.669701] rcu-scale: 0 writer-duration: 95 29205 > [ 63.669702] rcu-scale: 0 writer-duration: 96 26577 > [ 63.669703] rcu-scale: 0 writer-duration: 97 36187 > [ 63.669704] rcu-scale: 0 writer-duration: 98 24472 > [ 63.669705] rcu-scale: 0 writer-duration: 99 28883 > [ 63.669707] rcu-scale: 0 writer-duration: 100 31999 > [ 63.671532] rcu-scale: 0 writer-duration: 101 48271 > > > [ 63.671556] rcu-scale: 1 writer-duration: 0 53537 > [ 63.673442] rcu-scale: 1 writer-duration: 1 33311 > [ 63.673451] rcu-scale: 1 writer-duration: 2 28033 > [ 63.673457] rcu-scale: 1 writer-duration: 3 40200 > [ 63.673464] rcu-scale: 1 writer-duration: 4 25479 > [ 63.673470] rcu-scale: 1 writer-duration: 5 29908 > [ 63.673475] rcu-scale: 1 writer-duration: 6 27336 > [ 63.673481] rcu-scale: 1 writer-duration: 7 36948 > [ 63.673487] rcu-scale: 1 writer-duration: 8 24216 > [ 63.673494] rcu-scale: 1 writer-duration: 9 31639 > [ 63.673499] rcu-scale: 1 writer-duration: 10 27175 > [ 63.673506] rcu-scale: 1 writer-duration: 11 36695 > [ 63.673512] rcu-scale: 1 writer-duration: 12 24609 > [ 63.673518] rcu-scale: 1 writer-duration: 13 29140 > [ 63.673524] rcu-scale: 1 writer-duration: 14 26955 > [ 63.673530] rcu-scale: 1 writer-duration: 15 36122 > [ 63.673535] rcu-scale: 1 writer-duration: 16 24236 > [ 63.673541] rcu-scale: 1 writer-duration: 17 29137 > [ 63.673546] rcu-scale: 1 writer-duration: 18 26766 > [ 63.673552] rcu-scale: 1 writer-duration: 19 36207 > [ 63.673558] rcu-scale: 1 writer-duration: 20 24215 > [ 63.673563] rcu-scale: 1 writer-duration: 21 29239 > [ 63.673569] rcu-scale: 1 writer-duration: 22 26866 > [ 63.673575] rcu-scale: 1 writer-duration: 23 36054 > [ 63.673581] rcu-scale: 1 writer-duration: 24 24012 > [ 63.673586] rcu-scale: 1 writer-duration: 25 28914 > [ 63.673592] rcu-scale: 1 writer-duration: 26 26891 > [ 63.673597] rcu-scale: 1 writer-duration: 27 36102 > [ 63.673603] rcu-scale: 1 writer-duration: 28 24039 > [ 63.673609] rcu-scale: 1 writer-duration: 29 29125 > [ 63.673614] rcu-scale: 1 writer-duration: 30 26926 > [ 63.673620] rcu-scale: 1 writer-duration: 31 40754 > [ 63.673625] rcu-scale: 1 writer-duration: 32 24519 > [ 63.673631] rcu-scale: 1 writer-duration: 33 29397 > [ 63.673637] rcu-scale: 1 writer-duration: 34 27267 > [ 63.673642] rcu-scale: 1 writer-duration: 35 36358 > [ 63.673648] rcu-scale: 1 writer-duration: 36 24007 > [ 63.673654] rcu-scale: 1 writer-duration: 37 29326 > [ 63.673659] rcu-scale: 1 writer-duration: 38 26854 > [ 63.673665] rcu-scale: 1 writer-duration: 39 36625 > [ 63.673671] rcu-scale: 1 writer-duration: 40 24437 > [ 63.673676] rcu-scale: 1 writer-duration: 41 29314 > [ 63.673682] rcu-scale: 1 writer-duration: 42 26955 > [ 63.673688] rcu-scale: 1 writer-duration: 43 36415 > [ 63.673693] rcu-scale: 1 writer-duration: 44 24098 > [ 63.673699] rcu-scale: 1 writer-duration: 45 29216 > [ 63.673705] rcu-scale: 1 writer-duration: 46 26904 > [ 63.673710] rcu-scale: 1 writer-duration: 47 36286 > [ 63.673716] rcu-scale: 1 writer-duration: 48 24127 > [ 63.673722] rcu-scale: 1 writer-duration: 49 28887 > [ 63.673727] rcu-scale: 1 writer-duration: 50 26894 > [ 63.673733] rcu-scale: 1 writer-duration: 51 36145 > [ 63.673739] rcu-scale: 1 writer-duration: 52 24238 > [ 63.673745] rcu-scale: 1 writer-duration: 53 29136 > [ 63.673750] rcu-scale: 1 writer-duration: 54 26895 > [ 63.673756] rcu-scale: 1 writer-duration: 55 36087 > [ 63.673761] rcu-scale: 1 writer-duration: 56 24215 > [ 63.673767] rcu-scale: 1 writer-duration: 57 29034 > [ 63.673773] rcu-scale: 1 writer-duration: 58 26817 > [ 63.673778] rcu-scale: 1 writer-duration: 59 36069 > [ 63.673784] rcu-scale: 1 writer-duration: 60 24255 > [ 63.673790] rcu-scale: 1 writer-duration: 61 29004 > [ 63.673795] rcu-scale: 1 writer-duration: 62 26878 > [ 63.673801] rcu-scale: 1 writer-duration: 63 36186 > [ 63.673806] rcu-scale: 1 writer-duration: 64 24355 > [ 63.673812] rcu-scale: 1 writer-duration: 65 28884 > [ 63.673818] rcu-scale: 1 writer-duration: 66 32061 > [ 63.673823] rcu-scale: 1 writer-duration: 67 36261 > [ 63.673829] rcu-scale: 1 writer-duration: 68 24453 > [ 63.673835] rcu-scale: 1 writer-duration: 69 29508 > [ 63.673840] rcu-scale: 1 writer-duration: 70 27004 > [ 63.673846] rcu-scale: 1 writer-duration: 71 36240 > [ 63.673851] rcu-scale: 1 writer-duration: 72 23958 > [ 63.673857] rcu-scale: 1 writer-duration: 73 29276 > [ 63.673863] rcu-scale: 1 writer-duration: 74 26793 > [ 63.673868] rcu-scale: 1 writer-duration: 75 36779 > [ 63.673874] rcu-scale: 1 writer-duration: 76 24462 > [ 63.673880] rcu-scale: 1 writer-duration: 77 29156 > [ 63.673885] rcu-scale: 1 writer-duration: 78 27128 > [ 63.673891] rcu-scale: 1 writer-duration: 79 36182 > [ 63.673896] rcu-scale: 1 writer-duration: 80 24073 > [ 63.673902] rcu-scale: 1 writer-duration: 81 28955 > [ 63.673908] rcu-scale: 1 writer-duration: 82 27009 > [ 63.673914] rcu-scale: 1 writer-duration: 83 36169 > [ 63.673919] rcu-scale: 1 writer-duration: 84 23906 > [ 63.673925] rcu-scale: 1 writer-duration: 85 28946 > [ 63.673931] rcu-scale: 1 writer-duration: 86 26866 > [ 63.673937] rcu-scale: 1 writer-duration: 87 36249 > [ 63.673942] rcu-scale: 1 writer-duration: 88 24096 > [ 63.673948] rcu-scale: 1 writer-duration: 89 29290 > [ 63.673954] rcu-scale: 1 writer-duration: 90 26844 > [ 63.673959] rcu-scale: 1 writer-duration: 91 36353 > [ 63.673965] rcu-scale: 1 writer-duration: 92 24166 > [ 63.673971] rcu-scale: 1 writer-duration: 93 29072 > [ 63.673976] rcu-scale: 1 writer-duration: 94 26862 > [ 63.673982] rcu-scale: 1 writer-duration: 95 36348 > [ 63.673988] rcu-scale: 1 writer-duration: 96 23907 > [ 63.673993] rcu-scale: 1 writer-duration: 97 29122 > [ 63.673999] rcu-scale: 1 writer-duration: 98 27072 > [ 63.674004] rcu-scale: 1 writer-duration: 99 36188 > [ 63.674010] rcu-scale: 1 writer-duration: 100 29125 > [ 63.675441] rcu-scale: 1 writer-duration: 101 48063 > > > > [ 63.675458] rcu-scale: 2 writer-duration: 0 65839 > [ 63.677455] rcu-scale: 2 writer-duration: 1 30102 > [ 63.677470] rcu-scale: 2 writer-duration: 2 30481 > [ 63.677477] rcu-scale: 2 writer-duration: 3 28129 > [ 63.677483] rcu-scale: 2 writer-duration: 4 37897 > [ 63.677489] rcu-scale: 2 writer-duration: 5 27420 > [ 63.677496] rcu-scale: 2 writer-duration: 6 29475 > [ 63.677501] rcu-scale: 2 writer-duration: 7 25128 > [ 63.677507] rcu-scale: 2 writer-duration: 8 36449 > [ 63.677512] rcu-scale: 2 writer-duration: 9 29093 > [ 63.677518] rcu-scale: 2 writer-duration: 10 29369 > [ 63.677524] rcu-scale: 2 writer-duration: 11 24938 > [ 63.677530] rcu-scale: 2 writer-duration: 12 36634 > [ 63.677535] rcu-scale: 2 writer-duration: 13 26628 > [ 63.677541] rcu-scale: 2 writer-duration: 14 29116 > [ 63.677547] rcu-scale: 2 writer-duration: 15 24320 > [ 63.677552] rcu-scale: 2 writer-duration: 16 36351 > [ 63.677558] rcu-scale: 2 writer-duration: 17 26613 > [ 63.677564] rcu-scale: 2 writer-duration: 18 29039 > [ 63.677570] rcu-scale: 2 writer-duration: 19 24353 > [ 63.677575] rcu-scale: 2 writer-duration: 20 36335 > [ 63.677581] rcu-scale: 2 writer-duration: 21 26706 > [ 63.677587] rcu-scale: 2 writer-duration: 22 29200 > [ 63.677592] rcu-scale: 2 writer-duration: 23 24211 > [ 63.677598] rcu-scale: 2 writer-duration: 24 36036 > [ 63.677604] rcu-scale: 2 writer-duration: 25 26464 > [ 63.677610] rcu-scale: 2 writer-duration: 26 29181 > [ 63.677616] rcu-scale: 2 writer-duration: 27 24021 > [ 63.677621] rcu-scale: 2 writer-duration: 28 36169 > [ 63.677627] rcu-scale: 2 writer-duration: 29 26697 > [ 63.677633] rcu-scale: 2 writer-duration: 30 29213 > [ 63.677638] rcu-scale: 2 writer-duration: 31 24181 > [ 63.677644] rcu-scale: 2 writer-duration: 32 41287 > [ 63.677650] rcu-scale: 2 writer-duration: 33 27016 > [ 63.677656] rcu-scale: 2 writer-duration: 34 29455 > [ 63.677662] rcu-scale: 2 writer-duration: 35 24571 > [ 63.677667] rcu-scale: 2 writer-duration: 36 36144 > [ 63.677673] rcu-scale: 2 writer-duration: 37 26673 > [ 63.677678] rcu-scale: 2 writer-duration: 38 29031 > [ 63.677684] rcu-scale: 2 writer-duration: 39 25127 > [ 63.677690] rcu-scale: 2 writer-duration: 40 36453 > [ 63.677695] rcu-scale: 2 writer-duration: 41 26455 > [ 63.677701] rcu-scale: 2 writer-duration: 42 29460 > [ 63.677707] rcu-scale: 2 writer-duration: 43 24698 > [ 63.677712] rcu-scale: 2 writer-duration: 44 36093 > [ 63.677718] rcu-scale: 2 writer-duration: 45 26573 > [ 63.677723] rcu-scale: 2 writer-duration: 46 29157 > [ 63.677729] rcu-scale: 2 writer-duration: 47 24356 > [ 63.677735] rcu-scale: 2 writer-duration: 48 36504 > [ 63.677741] rcu-scale: 2 writer-duration: 49 26157 > [ 63.677746] rcu-scale: 2 writer-duration: 50 29103 > [ 63.677752] rcu-scale: 2 writer-duration: 51 24577 > [ 63.677758] rcu-scale: 2 writer-duration: 52 36218 > [ 63.677763] rcu-scale: 2 writer-duration: 53 26553 > [ 63.677769] rcu-scale: 2 writer-duration: 54 29224 > [ 63.677775] rcu-scale: 2 writer-duration: 55 24290 > [ 63.677780] rcu-scale: 2 writer-duration: 56 36322 > [ 63.677786] rcu-scale: 2 writer-duration: 57 26509 > [ 63.677792] rcu-scale: 2 writer-duration: 58 28951 > [ 63.677797] rcu-scale: 2 writer-duration: 59 24316 > [ 63.677803] rcu-scale: 2 writer-duration: 60 36402 > [ 63.677809] rcu-scale: 2 writer-duration: 61 26453 > [ 63.677814] rcu-scale: 2 writer-duration: 62 29023 > [ 63.677820] rcu-scale: 2 writer-duration: 63 24538 > [ 63.677826] rcu-scale: 2 writer-duration: 64 36279 > [ 63.677831] rcu-scale: 2 writer-duration: 65 26277 > [ 63.677837] rcu-scale: 2 writer-duration: 66 34442 > [ 63.677843] rcu-scale: 2 writer-duration: 67 24564 > [ 63.677849] rcu-scale: 2 writer-duration: 68 36429 > [ 63.677855] rcu-scale: 2 writer-duration: 69 27018 > [ 63.677861] rcu-scale: 2 writer-duration: 70 29207 > [ 63.677866] rcu-scale: 2 writer-duration: 71 24488 > [ 63.677872] rcu-scale: 2 writer-duration: 72 36187 > [ 63.677878] rcu-scale: 2 writer-duration: 73 26564 > [ 63.677883] rcu-scale: 2 writer-duration: 74 29127 > [ 63.677889] rcu-scale: 2 writer-duration: 75 25213 > [ 63.677894] rcu-scale: 2 writer-duration: 76 36202 > [ 63.677900] rcu-scale: 2 writer-duration: 77 26709 > [ 63.677905] rcu-scale: 2 writer-duration: 78 29292 > [ 63.677911] rcu-scale: 2 writer-duration: 79 24338 > [ 63.677917] rcu-scale: 2 writer-duration: 80 36322 > [ 63.677922] rcu-scale: 2 writer-duration: 81 26289 > [ 63.677928] rcu-scale: 2 writer-duration: 82 29386 > [ 63.677934] rcu-scale: 2 writer-duration: 83 24300 > [ 63.677939] rcu-scale: 2 writer-duration: 84 36011 > [ 63.677945] rcu-scale: 2 writer-duration: 85 26219 > [ 63.677951] rcu-scale: 2 writer-duration: 86 29250 > [ 63.677957] rcu-scale: 2 writer-duration: 87 24484 > [ 63.677963] rcu-scale: 2 writer-duration: 88 36254 > [ 63.677969] rcu-scale: 2 writer-duration: 89 26631 > [ 63.677975] rcu-scale: 2 writer-duration: 90 29105 > [ 63.677980] rcu-scale: 2 writer-duration: 91 24502 > [ 63.677986] rcu-scale: 2 writer-duration: 92 36409 > [ 63.677992] rcu-scale: 2 writer-duration: 93 26414 > [ 63.677997] rcu-scale: 2 writer-duration: 94 29130 > [ 63.678003] rcu-scale: 2 writer-duration: 95 24586 > [ 63.678009] rcu-scale: 2 writer-duration: 96 36159 > [ 63.678014] rcu-scale: 2 writer-duration: 97 26263 > [ 63.678021] rcu-scale: 2 writer-duration: 98 29479 > [ 63.678027] rcu-scale: 2 writer-duration: 99 24352 > [ 63.678032] rcu-scale: 2 writer-duration: 100 39478 > > > > > [ 63.679533] rcu-scale: 3 writer-duration: 0 31164 > [ 63.681440] rcu-scale: 3 writer-duration: 1 31690 > [ 63.681451] rcu-scale: 3 writer-duration: 2 28398 > [ 63.681457] rcu-scale: 3 writer-duration: 3 37524 > [ 63.681462] rcu-scale: 3 writer-duration: 4 30331 > [ 63.681468] rcu-scale: 3 writer-duration: 5 30588 > [ 63.681473] rcu-scale: 3 writer-duration: 6 25430 > [ 63.681479] rcu-scale: 3 writer-duration: 7 36628 > [ 63.681485] rcu-scale: 3 writer-duration: 8 27195 > [ 63.681490] rcu-scale: 3 writer-duration: 9 29267 > [ 63.681496] rcu-scale: 3 writer-duration: 10 26967 > [ 63.681503] rcu-scale: 3 writer-duration: 11 36610 > [ 63.681509] rcu-scale: 3 writer-duration: 12 27017 > [ 63.681514] rcu-scale: 3 writer-duration: 13 29493 > [ 63.681520] rcu-scale: 3 writer-duration: 14 24647 > [ 63.681526] rcu-scale: 3 writer-duration: 15 36099 > [ 63.681531] rcu-scale: 3 writer-duration: 16 26574 > [ 63.681537] rcu-scale: 3 writer-duration: 17 28995 > [ 63.681542] rcu-scale: 3 writer-duration: 18 24627 > [ 63.681548] rcu-scale: 3 writer-duration: 19 36099 > [ 63.681554] rcu-scale: 3 writer-duration: 20 26580 > [ 63.681559] rcu-scale: 3 writer-duration: 21 29133 > [ 63.681565] rcu-scale: 3 writer-duration: 22 24684 > [ 63.681570] rcu-scale: 3 writer-duration: 23 36089 > [ 63.681576] rcu-scale: 3 writer-duration: 24 26508 > [ 63.681582] rcu-scale: 3 writer-duration: 25 28821 > [ 63.681587] rcu-scale: 3 writer-duration: 26 24382 > [ 63.681593] rcu-scale: 3 writer-duration: 27 36151 > [ 63.681599] rcu-scale: 3 writer-duration: 28 26336 > [ 63.681605] rcu-scale: 3 writer-duration: 29 28946 > [ 63.681610] rcu-scale: 3 writer-duration: 30 24735 > [ 63.681616] rcu-scale: 3 writer-duration: 31 36237 > [ 63.681622] rcu-scale: 3 writer-duration: 32 26239 > [ 63.681627] rcu-scale: 3 writer-duration: 33 34242 > [ 63.681633] rcu-scale: 3 writer-duration: 34 24917 > [ 63.681639] rcu-scale: 3 writer-duration: 35 36459 > [ 63.681645] rcu-scale: 3 writer-duration: 36 26774 > [ 63.681650] rcu-scale: 3 writer-duration: 37 28911 > [ 63.681656] rcu-scale: 3 writer-duration: 38 24797 > [ 63.681662] rcu-scale: 3 writer-duration: 39 36099 > [ 63.681667] rcu-scale: 3 writer-duration: 40 27206 > [ 63.681673] rcu-scale: 3 writer-duration: 41 29201 > [ 63.681678] rcu-scale: 3 writer-duration: 42 24676 > [ 63.681684] rcu-scale: 3 writer-duration: 43 36260 > [ 63.681690] rcu-scale: 3 writer-duration: 44 26859 > [ 63.681695] rcu-scale: 3 writer-duration: 45 29087 > [ 63.681701] rcu-scale: 3 writer-duration: 46 24531 > [ 63.681707] rcu-scale: 3 writer-duration: 47 36144 > [ 63.681712] rcu-scale: 3 writer-duration: 48 26415 > [ 63.681718] rcu-scale: 3 writer-duration: 49 29230 > [ 63.681724] rcu-scale: 3 writer-duration: 50 24371 > [ 63.681729] rcu-scale: 3 writer-duration: 51 36045 > [ 63.681735] rcu-scale: 3 writer-duration: 52 26707 > [ 63.681740] rcu-scale: 3 writer-duration: 53 29079 > [ 63.681746] rcu-scale: 3 writer-duration: 54 24622 > [ 63.681752] rcu-scale: 3 writer-duration: 55 36202 > [ 63.681757] rcu-scale: 3 writer-duration: 56 26489 > [ 63.681763] rcu-scale: 3 writer-duration: 57 29016 > [ 63.681769] rcu-scale: 3 writer-duration: 58 24632 > [ 63.681775] rcu-scale: 3 writer-duration: 59 35968 > [ 63.681780] rcu-scale: 3 writer-duration: 60 26480 > [ 63.681786] rcu-scale: 3 writer-duration: 61 29171 > [ 63.681792] rcu-scale: 3 writer-duration: 62 24542 > [ 63.681797] rcu-scale: 3 writer-duration: 63 35972 > [ 63.681803] rcu-scale: 3 writer-duration: 64 26459 > [ 63.681809] rcu-scale: 3 writer-duration: 65 29248 > [ 63.681814] rcu-scale: 3 writer-duration: 66 24403 > [ 63.681820] rcu-scale: 3 writer-duration: 67 41353 > [ 63.681826] rcu-scale: 3 writer-duration: 68 26759 > [ 63.681831] rcu-scale: 3 writer-duration: 69 29167 > [ 63.681837] rcu-scale: 3 writer-duration: 70 25051 > [ 63.681843] rcu-scale: 3 writer-duration: 71 36298 > [ 63.681848] rcu-scale: 3 writer-duration: 72 26557 > [ 63.681854] rcu-scale: 3 writer-duration: 73 28940 > [ 63.681860] rcu-scale: 3 writer-duration: 74 24726 > [ 63.681865] rcu-scale: 3 writer-duration: 75 36109 > [ 63.681871] rcu-scale: 3 writer-duration: 76 27216 > [ 63.681877] rcu-scale: 3 writer-duration: 77 29170 > [ 63.681882] rcu-scale: 3 writer-duration: 78 24606 > [ 63.681888] rcu-scale: 3 writer-duration: 79 36459 > [ 63.681893] rcu-scale: 3 writer-duration: 80 26470 > [ 63.681899] rcu-scale: 3 writer-duration: 81 28939 > [ 63.681905] rcu-scale: 3 writer-duration: 82 24758 > [ 63.681910] rcu-scale: 3 writer-duration: 83 36139 > [ 63.681916] rcu-scale: 3 writer-duration: 84 26535 > [ 63.681922] rcu-scale: 3 writer-duration: 85 28824 > [ 63.681927] rcu-scale: 3 writer-duration: 86 24272 > [ 63.681933] rcu-scale: 3 writer-duration: 87 36187 > [ 63.681939] rcu-scale: 3 writer-duration: 88 26565 > [ 63.681944] rcu-scale: 3 writer-duration: 89 29050 > [ 63.681950] rcu-scale: 3 writer-duration: 90 24686 > [ 63.681955] rcu-scale: 3 writer-duration: 91 36216 > [ 63.681961] rcu-scale: 3 writer-duration: 92 26589 > [ 63.681967] rcu-scale: 3 writer-duration: 93 29076 > [ 63.681973] rcu-scale: 3 writer-duration: 94 24683 > [ 63.681978] rcu-scale: 3 writer-duration: 95 36081 > [ 63.681984] rcu-scale: 3 writer-duration: 96 26710 > [ 63.681989] rcu-scale: 3 writer-duration: 97 28881 > [ 63.681995] rcu-scale: 3 writer-duration: 98 24494 > [ 63.682001] rcu-scale: 3 writer-duration: 99 36297 > [ 63.682006] rcu-scale: 3 writer-duration: 100 26487 > > > [ 63.683440] rcu-scale: 3 writer-duration: 101 48416
On Tue, Jun 28, 2022 at 04:32:58AM +0000, Zhang, Qiang1 wrote: > On Thu, Jun 23, 2022 at 03:23:19AM +0000, Zhang, Qiang1 wrote: > > On Wed, Jun 22, 2022 at 05:34:09PM -0700, Paul E. McKenney wrote: > > > On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > > > > > > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > > > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > > > > grace period begins, if detected current CPU enters idle in > > > > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > > > > current cpu, at this time, maybe not being in an RCU read-side critical > > > > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > > > > occurs on current CPU to check and report exp QS. > > > > > > > > > > > > > >I think the idea is OK, however, this "optimization" is based on the > > > > >implementation detail that rcu_read_lock() counts preempt_count when > > > > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > > > > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > > > > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > > > > > > > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > > > > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > > > > preempt-count is exists, so can report exp QS when not being an RCU > > > > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > > > > in IPI handler. > > > > > > > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > > > > The rcu_read_lock/unlock is just barrier(). > > > > > > > > > > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > > > > > > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > > > > preempt-count is also checked > > > > > > > > /* Report any deferred quiescent states if preemption enabled. */ > > > > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > > > > rcu_preempt_deferred_qs(current); > > > > > > > > but the RCU softirq may not be triggered in time and reported exp QS, for > > > > example a kernel loop exist on NO_HZ_FULL CPU > > > > > > > > this change, It is to capture the exp QS state earlier and report it. > > > > > > > > > > > > >Also I'm not aware of any but there could be someone assuming that RCU > > > > >read-side critical sections can be formed without > > > > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > > > > >might be "creative" code like the following: > > > > > > > > > > void do_something_only_in_nonpreempt(void) > > > > > { > > > > > int *p; > > > > > > > > > > // This function only gets called in PREEMPT=n kernel, > > > > > // which means everywhere is a RCU read-side critical > > > > > // section, let's save some lines of code. > > > > > > > > > rcu_read_lock(); > > > > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > > > > ... // of course no schedule() here. > > > > > <access p> > > > > rcu_read_unlock(); > > > > > } > > > > > > > > > > > > > Usually access to pointers of type rcu needs to be protected. > > > > Yes, _ususally_ they are, but what about the special cases? Because > > in PREEMPT=n kernel, almost everywhere is a RCU read-side critical > > section, some one might have been "creative" enough to omit these > > rcu_read_lock() and rcu_read_unlock(). > > > > > > > > Indeed, lockdep would normally complain about this sort of thing. > > > But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without > > > CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > > > > > > > >Actually with the rcu_dereference_check() above, lockdep cannot detect > > >even CONFIG_PREEMPT_COUNT=y, that rcu_dereference_check() basically says > > >"I know I'm in a read-side critical section if it's a non-preempt > > >kernel, so don't bother to check". ;-) > > > > > > Any thoughts? > > > > > > It would be good to have some performance data on this change to expedited > > > grace periods. It is adding code, so it needs some real motivation. > > > > I used rcuscale to test it, and count the average time of writer-duration > > > > no applied patch(org.txt) > > writer avg time 29690.39 29670.78 29770.65 29423.25 > > > > applied patch(new.txt) > > writer avg time 28989.99 29003.54 29281.39 28986.58 > > > > or Is there a better way to test? > > >If I understand your measurments correctly, you are getting about a 2% > >improvement in expedited grace-period latency in !PREEMPT kernels. > >Do we have a situation for which that 2% improvement is important? > > To be honest, it will also be affected by the work scheduling delay. Indeed, > my current test results are not very good, but it is beneficial to detect the > QS state in advance and report it in time. of course, I will do more tests in > the future > > >We would be taking some risk due to the issues Boqun points out or we > >would be adding some complexity to avoid those issues. So this > > As for the code similar to the above scenario proposed by Boqun, > it is true that lockdep will not complain, I do not deny that someone > will write this way. > > If there really is such a situation as Boqun mentioned, There is also > a risk in the following code segment in PREEMPT=n and PREEMPT_COUNT=y > > /* Report any deferred quiescent states if preemption enabled. */ > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) > rcu_preempt_deferred_qs(current); Fair point. I have applied the commit shown below. Does that look correct? Thanx, Paul ------------------------------------------------------------------------ commit 9db96a8acda6837e5d4cd6a1cd7323709810e418 Author: Zqiang <qiang1.zhang@intel.com> Date: Tue Jul 5 12:09:51 2022 -0700 rcu: Add QS check in rcu_exp_handler() for non-preemptible kernels Kernels built with CONFIG_PREEMPTION=n and CONFIG_PREEMPT_COUNT=y maintain preempt_count() state. Because such kernels map __rcu_read_lock() and __rcu_read_unlock() to preempt_disable() and preempt_enable(), respectively, this allows the expedited grace period's !CONFIG_PREEMPT_RCU version of the rcu_exp_handler() IPI handler function to use preempt_count() to detect quiescent states. This preempt_count() usage might seem to risk failures due to use of implicit RCU readers in portions of the kernel under #ifndef CONFIG_PREEMPTION, except that rcu_core() already disallows such implicit RCU readers. The moral of this story is that you must use explicit read-side markings such as rcu_read_lock() or preempt_disable() even if the code knows that this kernel does not support preemption. This commit therefore adds a preempt_count()-based check for a quiescent state in the !CONFIG_PREEMPT_RCU version of the rcu_exp_handler() function for kernels built with CONFIG_PREEMPT_COUNT=y, reporting an immediate quiescent state when the interrupted code had both preemption and softirqs enabled. This change results in about a 2% reduction in expedited grace-period latency in kernels built with both CONFIG_PREEMPT_RCU=n and CONFIG_PREEMPT_COUNT=y. Signed-off-by: Zqiang <qiang1.zhang@intel.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> Link: https://lore.kernel.org/all/20220622103549.2840087-1-qiang1.zhang@intel.com/ diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h index be667583a5547..b07998159d1fa 100644 --- a/kernel/rcu/tree_exp.h +++ b/kernel/rcu/tree_exp.h @@ -828,11 +828,13 @@ static void rcu_exp_handler(void *unused) { struct rcu_data *rdp = this_cpu_ptr(&rcu_data); struct rcu_node *rnp = rdp->mynode; + bool preempt_bh_enabled = !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) return; - if (rcu_is_cpu_rrupt_from_idle()) { + if (rcu_is_cpu_rrupt_from_idle() || + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && preempt_bh_enabled)) { rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); return; }
On Tue, Jun 28, 2022 at 04:32:58AM +0000, Zhang, Qiang1 wrote: > On Thu, Jun 23, 2022 at 03:23:19AM +0000, Zhang, Qiang1 wrote: > > On Wed, Jun 22, 2022 at 05:34:09PM -0700, Paul E. McKenney wrote: > > > On Wed, Jun 22, 2022 at 11:34:15PM +0000, Zhang, Qiang1 wrote: > > > > > > > > On Wed, Jun 22, 2022 at 06:35:49PM +0800, Zqiang wrote: > > > > > In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp > > > > > grace period begins, if detected current CPU enters idle in > > > > > rcu_exp_handler() IPI handler, will immediately report the exp QS of the > > > > > current cpu, at this time, maybe not being in an RCU read-side critical > > > > > section, but need wait until rcu-softirq or sched-clock irq or sched-switch > > > > > occurs on current CPU to check and report exp QS. > > > > > > > > > > > > > >I think the idea is OK, however, this "optimization" is based on the > > > > >implementation detail that rcu_read_lock() counts preempt_count when > > > > >CONFIG_PREEMPT_COUNT=y, right? It's a little bit dangerous because the > > > > >preempt_count when CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n is mostly > > > > >for debugging purposes IIUC, and in other words, _it could be gone_. > > > > > > > > > > > > > Yes, for CONFIG_PREEMPT_COUNT=y and CONFIG_PREEMPT=n kernel > > > > The rcu_read_lock/unlock are replaced by preempt_disbale/enable, and the > > > > preempt-count is exists, so can report exp QS when not being an RCU > > > > read-side critical(preempt_count & (PREEMPT_MASK | SOFTIRQ_MASK )return zero). > > > > in IPI handler. > > > > > > > > For CONFIG_PREEMPT_COUNT=n and CONFIG_PREEMPT=n kernel, > > > > The rcu_read_lock/unlock is just barrier(). > > > > > > > > > > > > So I add IS_ENABLED(CONFIG_PREEMPT_COUNT) check in code. > > > > > > > > Of course, for CONFIG_PREEMPT_COUNT=n kernel, in RCU softirq, the > > > > preempt-count is also checked > > > > > > > > /* Report any deferred quiescent states if preemption enabled. */ > > > > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) { > > > > rcu_preempt_deferred_qs(current); > > > > > > > > but the RCU softirq may not be triggered in time and reported exp QS, for > > > > example a kernel loop exist on NO_HZ_FULL CPU > > > > > > > > this change, It is to capture the exp QS state earlier and report it. > > > > > > > > > > > > >Also I'm not aware of any but there could be someone assuming that RCU > > > > >read-side critical sections can be formed without > > > > >rcu_read_{lock,unlock}() in CONFIG_PREEMPT=n kernel. For example, there > > > > >might be "creative" code like the following: > > > > > > > > > > void do_something_only_in_nonpreempt(void) > > > > > { > > > > > int *p; > > > > > > > > > > // This function only gets called in PREEMPT=n kernel, > > > > > // which means everywhere is a RCU read-side critical > > > > > // section, let's save some lines of code. > > > > > > > > > rcu_read_lock(); > > > > > p = rcu_dereference_check(gp, !IS_ENABLED(PREEMPT)); > > > > > ... // of course no schedule() here. > > > > > <access p> > > > > rcu_read_unlock(); > > > > > } > > > > > > > > > > > > > Usually access to pointers of type rcu needs to be protected. > > > > Yes, _ususally_ they are, but what about the special cases? Because > > in PREEMPT=n kernel, almost everywhere is a RCU read-side critical > > section, some one might have been "creative" enough to omit these > > rcu_read_lock() and rcu_read_unlock(). > > > > > > > > Indeed, lockdep would normally complain about this sort of thing. > > > But in kernels built with (say) CONFIG_PREEMPT_NONE=y but without > > > CONFIG_PREEMPT_COUNT=N, can lockdep really tell the difference? > > > > > > > >Actually with the rcu_dereference_check() above, lockdep cannot detect > > >even CONFIG_PREEMPT_COUNT=y, that rcu_dereference_check() basically says > > >"I know I'm in a read-side critical section if it's a non-preempt > > >kernel, so don't bother to check". ;-) > > > > > > Any thoughts? > > > > > > It would be good to have some performance data on this change to expedited > > > grace periods. It is adding code, so it needs some real motivation. > > > > I used rcuscale to test it, and count the average time of writer-duration > > > > no applied patch(org.txt) > > writer avg time 29690.39 29670.78 29770.65 29423.25 > > > > applied patch(new.txt) > > writer avg time 28989.99 29003.54 29281.39 28986.58 > > > > or Is there a better way to test? > > >If I understand your measurments correctly, you are getting about a 2% > >improvement in expedited grace-period latency in !PREEMPT kernels. > >Do we have a situation for which that 2% improvement is important? > > To be honest, it will also be affected by the work scheduling delay. Indeed, > my current test results are not very good, but it is beneficial to detect the > QS state in advance and report it in time. of course, I will do more tests in > the future > > >We would be taking some risk due to the issues Boqun points out or we > >would be adding some complexity to avoid those issues. So this > > As for the code similar to the above scenario proposed by Boqun, > it is true that lockdep will not complain, I do not deny that someone > will write this way. > > If there really is such a situation as Boqun mentioned, There is also > a risk in the following code segment in PREEMPT=n and PREEMPT_COUNT=y > > /* Report any deferred quiescent states if preemption enabled. */ > if (IS_ENABLED(CONFIG_PREEMPT_COUNT) && (!(preempt_count() & PREEMPT_MASK))) > rcu_preempt_deferred_qs(current); > >Fair point. > >I have applied the commit shown below. Does that look correct? Thanks, This description is sufficient. > > Thanx, Paul > >------------------------------------------------------------------------ > >commit 9db96a8acda6837e5d4cd6a1cd7323709810e418 >Author: Zqiang <qiang1.zhang@intel.com> >Date: Tue Jul 5 12:09:51 2022 -0700 > > rcu: Add QS check in rcu_exp_handler() for non-preemptible kernels > > Kernels built with CONFIG_PREEMPTION=n and CONFIG_PREEMPT_COUNT=y maintain > preempt_count() state. Because such kernels map __rcu_read_lock() > and __rcu_read_unlock() to preempt_disable() and preempt_enable(), > respectively, this allows the expedited grace period's !CONFIG_PREEMPT_RCU > version of the rcu_exp_handler() IPI handler function to use > preempt_count() to detect quiescent states. > > This preempt_count() usage might seem to risk failures due to > use of implicit RCU readers in portions of the kernel under #ifndef > CONFIG_PREEMPTION, except that rcu_core() already disallows such implicit > RCU readers. The moral of this story is that you must use explicit > read-side markings such as rcu_read_lock() or preempt_disable() even if > the code knows that this kernel does not support preemption. > > This commit therefore adds a preempt_count()-based check for a quiescent > state in the !CONFIG_PREEMPT_RCU version of the rcu_exp_handler() > function for kernels built with CONFIG_PREEMPT_COUNT=y, reporting an > immediate quiescent state when the interrupted code had both preemption > and softirqs enabled. > > This change results in about a 2% reduction in expedited grace-period > latency in kernels built with both CONFIG_PREEMPT_RCU=n and > CONFIG_PREEMPT_COUNT=y. > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > Signed-off-by: Paul E. McKenney <paulmck@kernel.org> > Link: https://lore.kernel.org/all/20220622103549.2840087-1-qiang1.zhang@intel.com/ > >diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h >index be667583a5547..b07998159d1fa 100644 >--- a/kernel/rcu/tree_exp.h >+++ b/kernel/rcu/tree_exp.h >@@ -828,11 +828,13 @@ static void rcu_exp_handler(void *unused) > { > struct rcu_data *rdp = this_cpu_ptr(&rcu_data); > struct rcu_node *rnp = rdp->mynode; >+ bool preempt_bh_enabled = !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); > > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || > __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) > return; >- if (rcu_is_cpu_rrupt_from_idle()) { >+ if (rcu_is_cpu_rrupt_from_idle() || >+ (IS_ENABLED(CONFIG_PREEMPT_COUNT) && preempt_bh_enabled)) { > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); > return; > }
diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h index be667583a554..34f08267410f 100644 --- a/kernel/rcu/tree_exp.h +++ b/kernel/rcu/tree_exp.h @@ -828,11 +828,14 @@ static void rcu_exp_handler(void *unused) { struct rcu_data *rdp = this_cpu_ptr(&rcu_data); struct rcu_node *rnp = rdp->mynode; + bool preempt_bh_disabled = + !!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK)); if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) || __this_cpu_read(rcu_data.cpu_no_qs.b.exp)) return; - if (rcu_is_cpu_rrupt_from_idle()) { + if (rcu_is_cpu_rrupt_from_idle() || + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && !preempt_bh_disabled)) { rcu_report_exp_rdp(this_cpu_ptr(&rcu_data)); return; }
In CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y kernel, after a exp grace period begins, if detected current CPU enters idle in rcu_exp_handler() IPI handler, will immediately report the exp QS of the current cpu, at this time, maybe not being in an RCU read-side critical section, but need wait until rcu-softirq or sched-clock irq or sched-switch occurs on current CPU to check and report exp QS. This commit add a exp QS check in rcu_exp_handler(), when not being in an RCU read-side critical section, report exp QS earlier. Signed-off-by: Zqiang <qiang1.zhang@intel.com> --- kernel/rcu/tree_exp.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)