| Message ID | 20240819165939.745801-1-kent.overstreet@linux.dev (mailing list archive) |
|---|---|
| Headers | show |
| Series | rcu_pending | expand |
On Mon, Aug 19, 2024 at 12:59:26PM -0400, Kent Overstreet wrote: > New data structure for tracking objects waiting on an RCU grace period. > Supports regular RCU and SRCU, and possibly other RCU flavors in the > future. Uses radix trees for tracking pending objects, falling back to > linked lists on allocation failure. > > This gets us a more general replacement for SLAB_TYPESAFE_BY_RCU, and a > cleaner and slightly faster backend for kvfree_call_rcu(), and > in the future a faster backend for call_rcu() as well. > > There's still some small todo items, mentioned in the relevant patches. > > Paul - I'm considering putting this into 6.11 for bcachefs (not the > patch that switches kvfree_rcu, of course), as I need it rather > pressingly. Thoughts? I can put it in fs/bcachefs/ if you hate it :) I am having a hard time imagining it being ready for inclusion in kernel/rcu by the upcoming (v6.12) merge window, so if you need it then, you will need to pull it into fs/bcachefs. Don't get me wrong, it has improved since the June version. And we can always move it later, if/when appropriate. And yes, the kvfree_rcu() changes would need some serious performance evaluation by the people who benefitted from the changes that make kvfree_rcu() what it is today. So let's please defer that one. Having the equivalent of SLAB_TYPESAFE_BY_SRCU could be attractive, but I suspect that there are smaller changes that get us that with the existing slab allocator. Thanx, Paul > Kent Overstreet (9): > lib/generic-radix-tree.c: genradix_ptr_inlined() > lib/generic-radix-tree.c: add preallocation > darray: lift from bcachefs > vmalloc: is_vmalloc_addr_inlined() > rcu: delete lockdep_assert_irqs_enabled() assert in > start_poll_synchronize_rcu_common() > rcu: rcu_pending > bcachefs: Rip out freelists from btree key cache > bcachefs: key cache can now allocate from pending > rcu: Switch kvfree_rcu() to new rcu_pending > > MAINTAINERS | 7 + > fs/bcachefs/Makefile | 1 - > fs/bcachefs/btree_key_cache.c | 406 +++---------- > fs/bcachefs/btree_key_cache_types.h | 18 +- > fs/bcachefs/btree_node_scan_types.h | 2 +- > fs/bcachefs/btree_types.h | 5 +- > fs/bcachefs/btree_update.c | 2 + > fs/bcachefs/btree_write_buffer_types.h | 2 +- > fs/bcachefs/disk_accounting_types.h | 2 +- > fs/bcachefs/fsck.c | 2 +- > fs/bcachefs/journal_io.h | 2 +- > fs/bcachefs/journal_sb.c | 2 +- > fs/bcachefs/sb-downgrade.c | 3 +- > fs/bcachefs/sb-errors_types.h | 2 +- > fs/bcachefs/sb-members.h | 2 +- > fs/bcachefs/subvolume.h | 1 - > fs/bcachefs/subvolume_types.h | 2 +- > fs/bcachefs/thread_with_file_types.h | 2 +- > fs/bcachefs/util.h | 29 +- > {fs/bcachefs => include/linux}/darray.h | 59 +- > include/linux/darray_types.h | 22 + > include/linux/generic-radix-tree.h | 106 +++- > include/linux/mm.h | 7 + > include/linux/rcu_pending.h | 27 + > init/main.c | 2 + > kernel/rcu/Makefile | 2 +- > kernel/rcu/pending.c | 623 ++++++++++++++++++++ > kernel/rcu/tree.c | 747 ------------------------ > kernel/rcu/update.c | 1 - > lib/Makefile | 2 +- > {fs/bcachefs => lib}/darray.c | 12 +- > lib/generic-radix-tree.c | 80 +-- > mm/vmalloc.c | 4 +- > 33 files changed, 962 insertions(+), 1224 deletions(-) > rename {fs/bcachefs => include/linux}/darray.h (66%) > create mode 100644 include/linux/darray_types.h > create mode 100644 include/linux/rcu_pending.h > create mode 100644 kernel/rcu/pending.c > rename {fs/bcachefs => lib}/darray.c (57%) > > -- > 2.45.2 >
On Mon, Aug 19, 2024 at 04:07:16PM GMT, Paul E. McKenney wrote: > On Mon, Aug 19, 2024 at 12:59:26PM -0400, Kent Overstreet wrote: > > New data structure for tracking objects waiting on an RCU grace period. > > Supports regular RCU and SRCU, and possibly other RCU flavors in the > > future. Uses radix trees for tracking pending objects, falling back to > > linked lists on allocation failure. > > > > This gets us a more general replacement for SLAB_TYPESAFE_BY_RCU, and a > > cleaner and slightly faster backend for kvfree_call_rcu(), and > > in the future a faster backend for call_rcu() as well. > > > > There's still some small todo items, mentioned in the relevant patches. > > > > Paul - I'm considering putting this into 6.11 for bcachefs (not the > > patch that switches kvfree_rcu, of course), as I need it rather > > pressingly. Thoughts? I can put it in fs/bcachefs/ if you hate it :) > > I am having a hard time imagining it being ready for inclusion in > kernel/rcu by the upcoming (v6.12) merge window, so if you need it then, > you will need to pull it into fs/bcachefs. > > Don't get me wrong, it has improved since the June version. And we > can always move it later, if/when appropriate. > > And yes, the kvfree_rcu() changes would need some serious performance > evaluation by the people who benefitted from the changes that make > kvfree_rcu() what it is today. So let's please defer that one. > > Having the equivalent of SLAB_TYPESAFE_BY_SRCU could be attractive, > but I suspect that there are smaller changes that get us that with the > existing slab allocator. I seriously doubt that - where are you going to stash the srcu_struct pointer? This is small and self contained, which is a _major_ win vs. dumping more complex algorithmic stuff into slub.
New data structure for tracking objects waiting on an RCU grace period. Supports regular RCU and SRCU, and possibly other RCU flavors in the future. Uses radix trees for tracking pending objects, falling back to linked lists on allocation failure. This gets us a more general replacement for SLAB_TYPESAFE_BY_RCU, and a cleaner and slightly faster backend for kvfree_call_rcu(), and in the future a faster backend for call_rcu() as well. There's still some small todo items, mentioned in the relevant patches. Paul - I'm considering putting this into 6.11 for bcachefs (not the patch that switches kvfree_rcu, of course), as I need it rather pressingly. Thoughts? I can put it in fs/bcachefs/ if you hate it :) Kent Overstreet (9): lib/generic-radix-tree.c: genradix_ptr_inlined() lib/generic-radix-tree.c: add preallocation darray: lift from bcachefs vmalloc: is_vmalloc_addr_inlined() rcu: delete lockdep_assert_irqs_enabled() assert in start_poll_synchronize_rcu_common() rcu: rcu_pending bcachefs: Rip out freelists from btree key cache bcachefs: key cache can now allocate from pending rcu: Switch kvfree_rcu() to new rcu_pending MAINTAINERS | 7 + fs/bcachefs/Makefile | 1 - fs/bcachefs/btree_key_cache.c | 406 +++---------- fs/bcachefs/btree_key_cache_types.h | 18 +- fs/bcachefs/btree_node_scan_types.h | 2 +- fs/bcachefs/btree_types.h | 5 +- fs/bcachefs/btree_update.c | 2 + fs/bcachefs/btree_write_buffer_types.h | 2 +- fs/bcachefs/disk_accounting_types.h | 2 +- fs/bcachefs/fsck.c | 2 +- fs/bcachefs/journal_io.h | 2 +- fs/bcachefs/journal_sb.c | 2 +- fs/bcachefs/sb-downgrade.c | 3 +- fs/bcachefs/sb-errors_types.h | 2 +- fs/bcachefs/sb-members.h | 2 +- fs/bcachefs/subvolume.h | 1 - fs/bcachefs/subvolume_types.h | 2 +- fs/bcachefs/thread_with_file_types.h | 2 +- fs/bcachefs/util.h | 29 +- {fs/bcachefs => include/linux}/darray.h | 59 +- include/linux/darray_types.h | 22 + include/linux/generic-radix-tree.h | 106 +++- include/linux/mm.h | 7 + include/linux/rcu_pending.h | 27 + init/main.c | 2 + kernel/rcu/Makefile | 2 +- kernel/rcu/pending.c | 623 ++++++++++++++++++++ kernel/rcu/tree.c | 747 ------------------------ kernel/rcu/update.c | 1 - lib/Makefile | 2 +- {fs/bcachefs => lib}/darray.c | 12 +- lib/generic-radix-tree.c | 80 +-- mm/vmalloc.c | 4 +- 33 files changed, 962 insertions(+), 1224 deletions(-) rename {fs/bcachefs => include/linux}/darray.h (66%) create mode 100644 include/linux/darray_types.h create mode 100644 include/linux/rcu_pending.h create mode 100644 kernel/rcu/pending.c rename {fs/bcachefs => lib}/darray.c (57%)