[00/12] maple_tree: iterator state changes

Message

Liam R. Howlett Nov. 1, 2023, 5:16 p.m. UTC

Andrew,

These patches have some general cleanup and a change to separate the
maple state status tracking from the maple state node.

The maple state status change allows for walks to continue from previous
places when the status needs to be recorded to make logical sense for
the next call to the maple state.  For instance, it allows for prev/next
to function in a way that better resembles the linked list.  It also
allows switch statements to be used to detect missed states during
compile, and the addition of fast-path "active" state is cleaner as an
enum.

While making the status change, perf showed some very small (one line)
functions that were not inlined even with the inline key word.  Making
these small functions __always_inline is less expensive according to
perf.  As part of that change, some inlines have been dropped from
larger functions.

Perf also showed that the commonly used mas_for_each() iterator was
spending a lot of time finding the end of the node.  This series
introduces caching of the end of the node in the maple state (and
updating it during writes). This caching along with the inline changes
yielded at 23.25% improvement on the BENCH_MAS_FOR_EACH maple tree test
framework benchmark.

I've also included a change to mtree_range_walk and mtree_lookup_walk to
take advantage of Peng's change [1] to the initial pivot setup.

mmtests did not produce any significant gains.

[1] https://lore.kernel.org/all/[e-mail]/T/#u

Liam R. Howlett (12):
  maple_tree: Remove unnecessary default labels from switch statements
  maple_tree: Make mas_erase() more robust
  maple_tree: Move debug check to __mas_set_range()
  maple_tree: Add end of node tracking to the maple state
  maple_tree: Use cached node end in mas_next()
  maple_tree: Use cached node end in mas_destroy()
  maple_tree: Clean up inlines for some functions
  maple_tree: Separate ma_state node from status.
  maple_tree: Remove mas_searchable()
  maple_tree: Use maple state end for write operations
  maple_tree: Don't find node end in mtree_lookup_walk()
  maple_tree: mtree_range_walk() clean up

 include/linux/maple_tree.h                  | 342 +++++-----
 include/linux/mm_types.h                    |   3 +-
 lib/maple_tree.c                            | 680 +++++++++++---------
 lib/test_maple_tree.c                       | 201 +++---
 mm/internal.h                               |  10 +-
 tools/testing/radix-tree/linux/maple_tree.h |   2 +-
 tools/testing/radix-tree/maple.c            |  27 +-
 7 files changed, 679 insertions(+), 586 deletions(-)