| Message ID | 20240813204716.842811-3-kinseyho@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Improve mem_cgroup_iter() | expand |
On Tue, Aug 13, 2024 at 08:47:12PM GMT, Kinsey Ho <kinseyho@google.com> wrote: > To obtain the pointer to the next memcg position, mem_cgroup_iter() > currently holds css->refcnt during memcg traversal only to put > css->refcnt at the end of the routine. This isn't necessary as an > rcu_read_lock is already held throughout the function. The use of > the RCU read lock with css_next_descendant_pre() guarantees that > sibling linkage is safe without holding a ref on the passed-in @css. > > Remove css->refcnt usage during traversal by leveraging RCU. > > Signed-off-by: Kinsey Ho <kinseyho@google.com> > --- > include/linux/memcontrol.h | 2 +- > mm/memcontrol.c | 18 +----------------- > 2 files changed, 2 insertions(+), 18 deletions(-) > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 90ecd2dbca06..1aaed2f1f6ae 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -75,7 +75,7 @@ struct lruvec_stats_percpu; > struct lruvec_stats; > > struct mem_cgroup_reclaim_iter { > - struct mem_cgroup *position; > + struct mem_cgroup __rcu *position; I'm not sure about this annotation. This pointer could be modified concurrently with RCU read sections with the cmpxchg which would assume that's equivalent with rcu_assign_pointer(). (Which it might be but it's not idiomatic, so it causes some head wrapping.) Isn't this situation covered with a regular pointer and READ_ONCE()+cmpxchg? Regards, Michal
Hi Michal, Thank you for reviewing this patchset! On Wed, Aug 14, 2024 at 5:00 AM Michal Koutný <mkoutny@suse.com> wrote: > > On Tue, Aug 13, 2024 at 08:47:12PM GMT, Kinsey Ho <kinseyho@google.com> wrote: > > To obtain the pointer to the next memcg position, mem_cgroup_iter() > > currently holds css->refcnt during memcg traversal only to put > > css->refcnt at the end of the routine. This isn't necessary as an > > rcu_read_lock is already held throughout the function. The use of > > the RCU read lock with css_next_descendant_pre() guarantees that > > sibling linkage is safe without holding a ref on the passed-in @css. > > > > Remove css->refcnt usage during traversal by leveraging RCU. > > > > Signed-off-by: Kinsey Ho <kinseyho@google.com> > > --- > > include/linux/memcontrol.h | 2 +- > > mm/memcontrol.c | 18 +----------------- > > 2 files changed, 2 insertions(+), 18 deletions(-) > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 90ecd2dbca06..1aaed2f1f6ae 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -75,7 +75,7 @@ struct lruvec_stats_percpu; > > struct lruvec_stats; > > > > struct mem_cgroup_reclaim_iter { > > - struct mem_cgroup *position; > > + struct mem_cgroup __rcu *position; > > I'm not sure about this annotation. > This pointer could be modified concurrently with RCU read sections with > the cmpxchg which would assume that's equivalent with > rcu_assign_pointer(). (Which it might be but it's not idiomatic, so it > causes some head wrapping.) > Isn't this situation covered with a regular pointer and > READ_ONCE()+cmpxchg? Yes, that's a good point – this situation is covered with a regular pointer and READ_ONCE() + cmpxchg(). I'll make the change to remove the __rcu tag and replace rcu_dereference() with READ_ONCE() and send it out in v3. (This also rids of the sparse errors seen in v1) Thanks for pointing this out. Best, Kinsey
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 90ecd2dbca06..1aaed2f1f6ae 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -75,7 +75,7 @@ struct lruvec_stats_percpu; struct lruvec_stats; struct mem_cgroup_reclaim_iter { - struct mem_cgroup *position; + struct mem_cgroup __rcu *position; /* scan generation, increased every round-trip */ unsigned int generation; }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dacf4fec4541..1688aae3b1b4 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1052,20 +1052,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, else if (reclaim->generation != iter->generation) goto out_unlock; - while (1) { - pos = READ_ONCE(iter->position); - if (!pos || css_tryget(&pos->css)) - break; - /* - * css reference reached zero, so iter->position will - * be cleared by ->css_released. However, we should not - * rely on this happening soon, because ->css_released - * is called from a work queue, and by busy-waiting we - * might block it. So we clear iter->position right - * away. - */ - (void)cmpxchg(&iter->position, pos, NULL); - } + pos = rcu_dereference(iter->position); } else if (prev) { pos = prev; } @@ -1106,9 +1093,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (pos) - css_put(&pos->css); - if (!memcg) iter->generation++; }
To obtain the pointer to the next memcg position, mem_cgroup_iter() currently holds css->refcnt during memcg traversal only to put css->refcnt at the end of the routine. This isn't necessary as an rcu_read_lock is already held throughout the function. The use of the RCU read lock with css_next_descendant_pre() guarantees that sibling linkage is safe without holding a ref on the passed-in @css. Remove css->refcnt usage during traversal by leveraging RCU. Signed-off-by: Kinsey Ho <kinseyho@google.com> --- include/linux/memcontrol.h | 2 +- mm/memcontrol.c | 18 +----------------- 2 files changed, 2 insertions(+), 18 deletions(-)