| Message ID | 20250123-slub-tiny-kfree_rcu-v1-0-0e386ef1541a@suse.cz (mailing list archive) |
|---|---|
| Headers | show |
| Series | slab, rcu: move and consolidate TINY_RCU kvfree_rcu() to SLAB | expand |
On Thu, Jan 23, 2025 at 11:37:17AM +0100, Vlastimil Babka wrote: > Following the move of the TREE_RCU batching kvfree_rcu() implementation > to slab, we still have the simple non-batching implementation in tiny > RCU, and RCU implementation specific ifdefs in slab code. > > Finish the move and integration into slab. Allow using the simple > call_rcu() based implementation also with tree RCU when SLUB_TINY is > enabled, as its goal is also to limit memory footprint with less concern > for top performance. > > In order to avoid RCU having to recognize the fake callback function > pointers (__is_kvfree_rcu_offset()) when handling call_rcu(), implement > a callback that can calculate the object's address from the embedded > rcu_head pointer without knowing the specific offset (previously SLOB > would not have made it possible, but it's gone now). > > After this series, AFAIK only the following kvfree_rcu specific code > remains in RCU: > > - a call to kfree_rcu_scheduler_running() from rcu_set_runtime_mode() > > - probably necessary and a generic registration interface would be > unnecessary bloat? > > - declarations of kfree_rcu() API in include/linux/rcupdate.h > > - could be moved to slab.h after checking for/fixing up potential > missing includes > > git tree: > https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/slub-tiny-kfree_rcu Looks good to me, except typo in third patch 'kmalloc of vmalloc allocation' should be 'kmalloc or vmalloc allocation'. Maybe also add some comments on top of 'kvfree_rcu_cb()'. Like: /* * Given an rcu_head embedded within an object at an offset < 4k, * free the object in question. */ I like the touch with the config option in the last patch! Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org> thanks, - Joel > To: Christoph Lameter <cl@linux.com> > To: David Rientjes <rientjes@google.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Cc: linux-mm@kvack.org > To: "Paul E. McKenney" <paulmck@kernel.org> > To: Joel Fernandes <joel@joelfernandes.org> > To: Josh Triplett <josh@joshtriplett.org> > To: Boqun Feng <boqun.feng@gmail.com> > To: Uladzislau Rezki <urezki@gmail.com> > Cc: Steven Rostedt <rostedt@goodmis.org> > Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > Cc: Lai Jiangshan <jiangshanlai@gmail.com> > Cc: Zqiang <qiang.zhang1211@gmail.com> > Cc: rcu@vger.kernel.org > > Signed-off-by: Vlastimil Babka <vbabka@suse.cz> > --- > Vlastimil Babka (4): > slab, rcu: move TINY_RCU variant of kvfree_rcu() to SLAB > rcu: remove trace_rcu_kvfree_callback > rcu, slab: use a regular callback function for kvfree_rcu > slab: don't batch kvfree_rcu() with SLUB_TINY > > include/linux/rcupdate.h | 29 ++++++++++++++--------------- > include/linux/rcutiny.h | 36 ------------------------------------ > include/linux/rcutree.h | 3 --- > include/linux/slab.h | 14 ++++++++++++++ > include/trace/events/rcu.h | 34 ---------------------------------- > kernel/rcu/tiny.c | 24 ------------------------ > kernel/rcu/tree.c | 9 ++------- > mm/Kconfig | 4 ++++ > mm/slab.h | 2 ++ > mm/slab_common.c | 32 +++++++++++++++++++++++++------- > mm/slub.c | 42 ++++++++++++++++++++++++++++++++++++++++++ > 11 files changed, 103 insertions(+), 126 deletions(-) > --- > base-commit: e492fac3657b60b8dd78a6e8ca26d1d14706c7b3 > change-id: 20250123-slub-tiny-kfree_rcu-bd65bfe222f2 > > Best regards, > -- > Vlastimil Babka <vbabka@suse.cz> >
On Thu, Jan 23, 2025 at 11:37:17AM +0100, Vlastimil Babka wrote: > Following the move of the TREE_RCU batching kvfree_rcu() implementation > to slab, we still have the simple non-batching implementation in tiny > RCU, and RCU implementation specific ifdefs in slab code. > > Finish the move and integration into slab. Allow using the simple > call_rcu() based implementation also with tree RCU when SLUB_TINY is > enabled, as its goal is also to limit memory footprint with less concern > for top performance. > > In order to avoid RCU having to recognize the fake callback function > pointers (__is_kvfree_rcu_offset()) when handling call_rcu(), implement > a callback that can calculate the object's address from the embedded > rcu_head pointer without knowing the specific offset (previously SLOB > would not have made it possible, but it's gone now). > > After this series, AFAIK only the following kvfree_rcu specific code > remains in RCU: > > - a call to kfree_rcu_scheduler_running() from rcu_set_runtime_mode() > > - probably necessary and a generic registration interface would be > unnecessary bloat? > > - declarations of kfree_rcu() API in include/linux/rcupdate.h > > - could be moved to slab.h after checking for/fixing up potential > missing includes > > git tree: > https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/slub-tiny-kfree_rcu > > To: Christoph Lameter <cl@linux.com> > To: David Rientjes <rientjes@google.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Cc: linux-mm@kvack.org > To: "Paul E. McKenney" <paulmck@kernel.org> > To: Joel Fernandes <joel@joelfernandes.org> > To: Josh Triplett <josh@joshtriplett.org> > To: Boqun Feng <boqun.feng@gmail.com> > To: Uladzislau Rezki <urezki@gmail.com> > Cc: Steven Rostedt <rostedt@goodmis.org> > Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > Cc: Lai Jiangshan <jiangshanlai@gmail.com> > Cc: Zqiang <qiang.zhang1211@gmail.com> > Cc: rcu@vger.kernel.org > > Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Tested-by: Paul E. McKenney <paulmck@kernel.org> > --- > Vlastimil Babka (4): > slab, rcu: move TINY_RCU variant of kvfree_rcu() to SLAB > rcu: remove trace_rcu_kvfree_callback > rcu, slab: use a regular callback function for kvfree_rcu > slab: don't batch kvfree_rcu() with SLUB_TINY > > include/linux/rcupdate.h | 29 ++++++++++++++--------------- > include/linux/rcutiny.h | 36 ------------------------------------ > include/linux/rcutree.h | 3 --- > include/linux/slab.h | 14 ++++++++++++++ > include/trace/events/rcu.h | 34 ---------------------------------- > kernel/rcu/tiny.c | 24 ------------------------ > kernel/rcu/tree.c | 9 ++------- > mm/Kconfig | 4 ++++ > mm/slab.h | 2 ++ > mm/slab_common.c | 32 +++++++++++++++++++++++++------- > mm/slub.c | 42 ++++++++++++++++++++++++++++++++++++++++++ > 11 files changed, 103 insertions(+), 126 deletions(-) > --- > base-commit: e492fac3657b60b8dd78a6e8ca26d1d14706c7b3 > change-id: 20250123-slub-tiny-kfree_rcu-bd65bfe222f2 > > Best regards, > -- > Vlastimil Babka <vbabka@suse.cz> >
Following the move of the TREE_RCU batching kvfree_rcu() implementation to slab, we still have the simple non-batching implementation in tiny RCU, and RCU implementation specific ifdefs in slab code. Finish the move and integration into slab. Allow using the simple call_rcu() based implementation also with tree RCU when SLUB_TINY is enabled, as its goal is also to limit memory footprint with less concern for top performance. In order to avoid RCU having to recognize the fake callback function pointers (__is_kvfree_rcu_offset()) when handling call_rcu(), implement a callback that can calculate the object's address from the embedded rcu_head pointer without knowing the specific offset (previously SLOB would not have made it possible, but it's gone now). After this series, AFAIK only the following kvfree_rcu specific code remains in RCU: - a call to kfree_rcu_scheduler_running() from rcu_set_runtime_mode() - probably necessary and a generic registration interface would be unnecessary bloat? - declarations of kfree_rcu() API in include/linux/rcupdate.h - could be moved to slab.h after checking for/fixing up potential missing includes git tree: https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/slub-tiny-kfree_rcu To: Christoph Lameter <cl@linux.com> To: David Rientjes <rientjes@google.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org To: "Paul E. McKenney" <paulmck@kernel.org> To: Joel Fernandes <joel@joelfernandes.org> To: Josh Triplett <josh@joshtriplett.org> To: Boqun Feng <boqun.feng@gmail.com> To: Uladzislau Rezki <urezki@gmail.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Lai Jiangshan <jiangshanlai@gmail.com> Cc: Zqiang <qiang.zhang1211@gmail.com> Cc: rcu@vger.kernel.org Signed-off-by: Vlastimil Babka <vbabka@suse.cz> --- Vlastimil Babka (4): slab, rcu: move TINY_RCU variant of kvfree_rcu() to SLAB rcu: remove trace_rcu_kvfree_callback rcu, slab: use a regular callback function for kvfree_rcu slab: don't batch kvfree_rcu() with SLUB_TINY include/linux/rcupdate.h | 29 ++++++++++++++--------------- include/linux/rcutiny.h | 36 ------------------------------------ include/linux/rcutree.h | 3 --- include/linux/slab.h | 14 ++++++++++++++ include/trace/events/rcu.h | 34 ---------------------------------- kernel/rcu/tiny.c | 24 ------------------------ kernel/rcu/tree.c | 9 ++------- mm/Kconfig | 4 ++++ mm/slab.h | 2 ++ mm/slab_common.c | 32 +++++++++++++++++++++++++------- mm/slub.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 11 files changed, 103 insertions(+), 126 deletions(-) --- base-commit: e492fac3657b60b8dd78a6e8ca26d1d14706c7b3 change-id: 20250123-slub-tiny-kfree_rcu-bd65bfe222f2 Best regards,