| Message ID | 20200324153643.15527-2-will@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Improve list integrity checking | expand |
On Tue, Mar 24, 2020 at 03:36:23PM +0000, Will Deacon wrote: > Commit c54a2744497d ("list: Add hlist_unhashed_lockless()") added a > "lockless" variant of hlist_unhashed() that uses READ_ONCE() to access > the 'pprev' pointer of the 'hlist_node', the intention being that this > could be used by 'timer_pending()' to silence a KCSAN warning. As well > as forgetting to add the caller, the patch also sprinkles > {READ,WRITE}_ONCE() invocations over the standard (i.e. non-RCU) hlist > code, which is undesirable for a number of reasons: > > 1. It gives the misleading impression that the non-RCU hlist code is > in some way lock-free (despite the notable absence of any memory > barriers!) and silences KCSAN in such cases. > > 2. It unnecessarily penalises code generation for non-RCU hlist users > > 3. It makes it difficult to introduce list integrity checks because > of the possibility of concurrent callers. > > Retain the {READ,WRITE}_ONCE() invocations for the RCU hlist code, but > remove them from the non-RCU implementation. Remove the unused > 'hlist_unhashed_lockless()' function entirely and add the READ_ONCE() > to hlist_unhashed(), as we do already for hlist_empty() already. > > Cc: Eric Dumazet <edumazet@google.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Paul E. McKenney <paulmck@kernel.org> > Cc: Peter Zijlstra <peterz@infradead.org> > Signed-off-by: Will Deacon <will@kernel.org> > --- > include/linux/list.h | 33 ++++++++++----------------------- > 1 file changed, 10 insertions(+), 23 deletions(-) Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
On Tue, Mar 24, 2020 at 03:36:23PM +0000, Will Deacon wrote: > Commit c54a2744497d ("list: Add hlist_unhashed_lockless()") added a > "lockless" variant of hlist_unhashed() that uses READ_ONCE() to access > the 'pprev' pointer of the 'hlist_node', the intention being that this > could be used by 'timer_pending()' to silence a KCSAN warning. As well > as forgetting to add the caller, the patch also sprinkles > {READ,WRITE}_ONCE() invocations over the standard (i.e. non-RCU) hlist > code, which is undesirable for a number of reasons: > > 1. It gives the misleading impression that the non-RCU hlist code is > in some way lock-free (despite the notable absence of any memory > barriers!) and silences KCSAN in such cases. > > 2. It unnecessarily penalises code generation for non-RCU hlist users > > 3. It makes it difficult to introduce list integrity checks because > of the possibility of concurrent callers. > > Retain the {READ,WRITE}_ONCE() invocations for the RCU hlist code, but > remove them from the non-RCU implementation. Remove the unused > 'hlist_unhashed_lockless()' function entirely and add the READ_ONCE() > to hlist_unhashed(), as we do already for hlist_empty() already. > > Cc: Eric Dumazet <edumazet@google.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Paul E. McKenney <paulmck@kernel.org> > Cc: Peter Zijlstra <peterz@infradead.org> > Signed-off-by: Will Deacon <will@kernel.org> No objection, however 90c018942c2b ("timer: Use hlist_unhashed_lockless() in timer_pending()") in -tip must change from hlist_unhashed_lockless() to hlist_unhashed(). Easy fix, though, so: Acked-by: Paul E. McKenney <paulmck@kernel.org> > --- > include/linux/list.h | 33 ++++++++++----------------------- > 1 file changed, 10 insertions(+), 23 deletions(-) > > diff --git a/include/linux/list.h b/include/linux/list.h > index 884216db3246..4fed5a0f9b77 100644 > --- a/include/linux/list.h > +++ b/include/linux/list.h > @@ -777,19 +777,6 @@ static inline void INIT_HLIST_NODE(struct hlist_node *h) > * node in unhashed state, but hlist_nulls_del() does not. > */ > static inline int hlist_unhashed(const struct hlist_node *h) > -{ > - return !h->pprev; > -} > - > -/** > - * hlist_unhashed_lockless - Version of hlist_unhashed for lockless use > - * @h: Node to be checked > - * > - * This variant of hlist_unhashed() must be used in lockless contexts > - * to avoid potential load-tearing. The READ_ONCE() is paired with the > - * various WRITE_ONCE() in hlist helpers that are defined below. > - */ > -static inline int hlist_unhashed_lockless(const struct hlist_node *h) > { > return !READ_ONCE(h->pprev); > } > @@ -852,11 +839,11 @@ static inline void hlist_del_init(struct hlist_node *n) > static inline void hlist_add_head(struct hlist_node *n, struct hlist_head *h) > { > struct hlist_node *first = h->first; > - WRITE_ONCE(n->next, first); > + n->next = first; > if (first) > - WRITE_ONCE(first->pprev, &n->next); > + first->pprev = &n->next; > WRITE_ONCE(h->first, n); > - WRITE_ONCE(n->pprev, &h->first); > + n->pprev = &h->first; > } > > /** > @@ -867,9 +854,9 @@ static inline void hlist_add_head(struct hlist_node *n, struct hlist_head *h) > static inline void hlist_add_before(struct hlist_node *n, > struct hlist_node *next) > { > - WRITE_ONCE(n->pprev, next->pprev); > - WRITE_ONCE(n->next, next); > - WRITE_ONCE(next->pprev, &n->next); > + n->pprev = next->pprev; > + n->next = next; > + next->pprev = &n->next; > WRITE_ONCE(*(n->pprev), n); > } > > @@ -881,12 +868,12 @@ static inline void hlist_add_before(struct hlist_node *n, > static inline void hlist_add_behind(struct hlist_node *n, > struct hlist_node *prev) > { > - WRITE_ONCE(n->next, prev->next); > - WRITE_ONCE(prev->next, n); > - WRITE_ONCE(n->pprev, &prev->next); > + n->next = prev->next; > + prev->next = n; > + n->pprev = &prev->next; > > if (n->next) > - WRITE_ONCE(n->next->pprev, &n->next); > + n->next->pprev = &n->next; > } > > /** > -- > 2.20.1 >
diff --git a/include/linux/list.h b/include/linux/list.h index 884216db3246..4fed5a0f9b77 100644 --- a/include/linux/list.h +++ b/include/linux/list.h @@ -777,19 +777,6 @@ static inline void INIT_HLIST_NODE(struct hlist_node *h) * node in unhashed state, but hlist_nulls_del() does not. */ static inline int hlist_unhashed(const struct hlist_node *h) -{ - return !h->pprev; -} - -/** - * hlist_unhashed_lockless - Version of hlist_unhashed for lockless use - * @h: Node to be checked - * - * This variant of hlist_unhashed() must be used in lockless contexts - * to avoid potential load-tearing. The READ_ONCE() is paired with the - * various WRITE_ONCE() in hlist helpers that are defined below. - */ -static inline int hlist_unhashed_lockless(const struct hlist_node *h) { return !READ_ONCE(h->pprev); } @@ -852,11 +839,11 @@ static inline void hlist_del_init(struct hlist_node *n) static inline void hlist_add_head(struct hlist_node *n, struct hlist_head *h) { struct hlist_node *first = h->first; - WRITE_ONCE(n->next, first); + n->next = first; if (first) - WRITE_ONCE(first->pprev, &n->next); + first->pprev = &n->next; WRITE_ONCE(h->first, n); - WRITE_ONCE(n->pprev, &h->first); + n->pprev = &h->first; } /** @@ -867,9 +854,9 @@ static inline void hlist_add_head(struct hlist_node *n, struct hlist_head *h) static inline void hlist_add_before(struct hlist_node *n, struct hlist_node *next) { - WRITE_ONCE(n->pprev, next->pprev); - WRITE_ONCE(n->next, next); - WRITE_ONCE(next->pprev, &n->next); + n->pprev = next->pprev; + n->next = next; + next->pprev = &n->next; WRITE_ONCE(*(n->pprev), n); } @@ -881,12 +868,12 @@ static inline void hlist_add_before(struct hlist_node *n, static inline void hlist_add_behind(struct hlist_node *n, struct hlist_node *prev) { - WRITE_ONCE(n->next, prev->next); - WRITE_ONCE(prev->next, n); - WRITE_ONCE(n->pprev, &prev->next); + n->next = prev->next; + prev->next = n; + n->pprev = &prev->next; if (n->next) - WRITE_ONCE(n->next->pprev, &n->next); + n->next->pprev = &n->next; } /**
Commit c54a2744497d ("list: Add hlist_unhashed_lockless()") added a "lockless" variant of hlist_unhashed() that uses READ_ONCE() to access the 'pprev' pointer of the 'hlist_node', the intention being that this could be used by 'timer_pending()' to silence a KCSAN warning. As well as forgetting to add the caller, the patch also sprinkles {READ,WRITE}_ONCE() invocations over the standard (i.e. non-RCU) hlist code, which is undesirable for a number of reasons: 1. It gives the misleading impression that the non-RCU hlist code is in some way lock-free (despite the notable absence of any memory barriers!) and silences KCSAN in such cases. 2. It unnecessarily penalises code generation for non-RCU hlist users 3. It makes it difficult to introduce list integrity checks because of the possibility of concurrent callers. Retain the {READ,WRITE}_ONCE() invocations for the RCU hlist code, but remove them from the non-RCU implementation. Remove the unused 'hlist_unhashed_lockless()' function entirely and add the READ_ONCE() to hlist_unhashed(), as we do already for hlist_empty() already. Cc: Eric Dumazet <edumazet@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Paul E. McKenney <paulmck@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Will Deacon <will@kernel.org> --- include/linux/list.h | 33 ++++++++++----------------------- 1 file changed, 10 insertions(+), 23 deletions(-)