| Message ID | d65d6d60faf773293e5189bd2b95c7bbc034976b.1634759754.git.asml.silence@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | optimise blk_try_enter_queue() | expand |
On Wed, Oct 20, 2021 at 09:03:18PM +0100, Pavel Begunkov wrote: > +/** > + * percpu_ref_tryget_live_rcu - same as percpu_ref_tryget_live() but the > + * caller is responsible for taking RCU. > + * > + * This function is safe to call as long as @ref is between init and exit. > + */ > +static inline bool percpu_ref_tryget_live_rcu(struct percpu_ref *ref) > +{ > + unsigned long __percpu *percpu_count; > + bool ret = false; > + > + if (likely(__ref_is_percpu(ref, &percpu_count))) { > + this_cpu_inc(*percpu_count); > + ret = true; > + } else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) { > + ret = atomic_long_inc_not_zero(&ref->data->count); > + } > + return ret; > +} Can we please add rcu_read_lock_held() assertion? Other than that, looks fine to me. Thanks.
On 10/20/21 21:10, Tejun Heo wrote: > On Wed, Oct 20, 2021 at 09:03:18PM +0100, Pavel Begunkov wrote: >> +/** >> + * percpu_ref_tryget_live_rcu - same as percpu_ref_tryget_live() but the >> + * caller is responsible for taking RCU. >> + * >> + * This function is safe to call as long as @ref is between init and exit. >> + */ >> +static inline bool percpu_ref_tryget_live_rcu(struct percpu_ref *ref) >> +{ >> + unsigned long __percpu *percpu_count; >> + bool ret = false; >> + >> + if (likely(__ref_is_percpu(ref, &percpu_count))) { >> + this_cpu_inc(*percpu_count); >> + ret = true; >> + } else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) { >> + ret = atomic_long_inc_not_zero(&ref->data->count); >> + } >> + return ret; >> +} > > Can we please add rcu_read_lock_held() assertion? Other than that, looks > fine to me. Will add, thanks
diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refcount.h index ae16a9856305..13e26add7d5c 100644 --- a/include/linux/percpu-refcount.h +++ b/include/linux/percpu-refcount.h @@ -266,6 +266,26 @@ static inline bool percpu_ref_tryget(struct percpu_ref *ref) return percpu_ref_tryget_many(ref, 1); } +/** + * percpu_ref_tryget_live_rcu - same as percpu_ref_tryget_live() but the + * caller is responsible for taking RCU. + * + * This function is safe to call as long as @ref is between init and exit. + */ +static inline bool percpu_ref_tryget_live_rcu(struct percpu_ref *ref) +{ + unsigned long __percpu *percpu_count; + bool ret = false; + + if (likely(__ref_is_percpu(ref, &percpu_count))) { + this_cpu_inc(*percpu_count); + ret = true; + } else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) { + ret = atomic_long_inc_not_zero(&ref->data->count); + } + return ret; +} + /** * percpu_ref_tryget_live - try to increment a live percpu refcount * @ref: percpu_ref to try-get @@ -283,20 +303,11 @@ static inline bool percpu_ref_tryget(struct percpu_ref *ref) */ static inline bool percpu_ref_tryget_live(struct percpu_ref *ref) { - unsigned long __percpu *percpu_count; bool ret = false; rcu_read_lock(); - - if (__ref_is_percpu(ref, &percpu_count)) { - this_cpu_inc(*percpu_count); - ret = true; - } else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) { - ret = atomic_long_inc_not_zero(&ref->data->count); - } - + ret = percpu_ref_tryget_live_rcu(ref); rcu_read_unlock(); - return ret; }
Add percpu_ref_tryget_live_rcu(), which is a version of percpu_ref_tryget_live() but the user is responsible for enclosing it in a RCU read lock section. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- include/linux/percpu-refcount.h | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-)