[rcu,0/17] SRCU updates, including SRCU-fast

Message

Paul E. McKenney Jan. 16, 2025, 8:21 p.m. UTC

Hello!

This series contains SRCU updates, perhaps most notably the addition of
srcu_read_lock_fast() and srcu_read_unlock_fast(), which on my laptop are
about 20% faster than rcu_read_lock_trace() and rcu_read_unlock_trace().
There are of course drawbacks:

o	Lack of CPU stall warnings.
o	SRCU-fast readers permitted only where rcu_is_watching().
o	A pointer-sized return value from srcu_read_lock_fast() must
	be passed to the corresponding srcu_read_unlock_fast().
o	In the absence of readers, a synchronize_srcu() having _fast()
	readers will incur the latency of at least two normal RCU grace
	periods.
o	RCU Tasks Trace priority boosting could be easily added.
	Boosting SRCU readers is more difficult.

Whether or not this can replace RCU Tasks Trace, it should replace the
SRCU-lite API, where the only drawback is a pointer-sized return value
from srcu_read_lock_fast() compared to the int-sized return value from
srcu_read_lock_lite().  And yes, I should have thought harder before
doing that SRCU-lite...

The series is as follows:

1.	Make Tiny SRCU able to operate in preemptible kernels.

2.	Define SRCU_READ_FLAVOR_ALL in terms of symbols.

3.	Use ->srcu_gp_seq for rcutorture reader batch.

4.	Pull ->srcu_{un,}lock_count into a new srcu_ctr structure.

5.	Make SRCU readers use ->srcu_ctrs for counter selection.

6.	Make Tree SRCU updates independent of ->srcu_idx.

7.	Force synchronization for srcu_get_delay().

8.	Rename srcu_check_read_flavor_lite() to
	srcu_check_read_flavor_force().

9.	Add SRCU_READ_FLAVOR_SLOWGP to flag need for synchronize_rcu().

10.	Pull pointer-to-integer conversion into __srcu_ptr_to_ctr().

11.	Pull integer-to-pointer conversion into __srcu_ctr_to_ptr().

12.	Move SRCU Tree/Tiny definitions from srcu.h.

13.	Add SRCU-fast readers.

14.	Add ability to test srcu_read_{,un}lock_fast().

15.	Add srcu_read_lock_fast() support using "srcu-fast".

16.	Make scenario SRCU-P use srcu_read_lock_fast().

17.	Fix srcu_read_unlock_{lite,nmisafe}() kernel-doc.

						Thanx, Paul

------------------------------------------------------------------------

 b/include/linux/srcu.h                                       |    3 
 b/include/linux/srcutiny.h                                   |    2 
 b/include/linux/srcutree.h                                   |   13 
 b/kernel/rcu/rcu.h                                           |    9 
 b/kernel/rcu/rcutorture.c                                    |    2 
 b/kernel/rcu/refscale.c                                      |   32 +
 b/kernel/rcu/srcutiny.c                                      |    6 
 b/kernel/rcu/srcutree.c                                      |    2 
 b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-P.boot |    1 
 include/linux/srcu.h                                         |   66 ++-
 include/linux/srcutiny.h                                     |   25 +
 include/linux/srcutree.h                                     |   77 +++
 kernel/rcu/rcutorture.c                                      |    9 
 kernel/rcu/srcutree.c                                        |  233 +++++------
 14 files changed, 332 insertions(+), 148 deletions(-)