[00/10] shrinker debugging, .to_text() report (resend)

Message

Kent Overstreet Aug. 24, 2024, 7:10 p.m. UTC

recently new OOMs have been cropping up, and reclaim is implicated, so
I've had to dust off these patches.

nothing significant has changed since the last time I posted, and they
have been valuable - Dave, think we can get them in?

Kent Overstreet (10):
  seq_buf: seq_buf_human_readable_u64()
  mm: shrinker: Add a .to_text() method for shrinkers
  mm: shrinker: Add new stats for .to_text()
  mm: Centralize & improve oom reporting in show_mem.c
  mm: shrinker: Add shrinker_to_text() to debugfs interface
  bcachefs: shrinker.to_text() methods
  percpu: per_cpu_sum()
  fs: Add super_block->s_inodes_nr
  fs/dcache: Add per-sb accounting for nr dentries
  fs: super_cache_to_text()

 fs/bcachefs/btree_cache.c     | 13 +++++
 fs/bcachefs/btree_key_cache.c | 14 ++++++
 fs/bcachefs/util.h            | 10 ----
 fs/dcache.c                   | 18 ++-----
 fs/inode.c                    |  2 +
 fs/super.c                    | 25 ++++++++++
 include/linux/fs.h            |  2 +
 include/linux/percpu.h        | 10 ++++
 include/linux/seq_buf.h       |  4 ++
 include/linux/shrinker.h      | 13 ++++-
 lib/seq_buf.c                 | 10 ++++
 mm/oom_kill.c                 | 23 ---------
 mm/show_mem.c                 | 43 ++++++++++++++++
 mm/shrinker.c                 | 94 ++++++++++++++++++++++++++++++++++-
 mm/shrinker_debug.c           | 18 +++++++
 mm/slab.h                     |  6 ++-
 mm/slab_common.c              | 52 +++++++++++++++----
 17 files changed, 298 insertions(+), 59 deletions(-)

Comments

Dave Chinner Aug. 28, 2024, 1:51 a.m. UTC | #1

On Sat, Aug 24, 2024 at 03:10:07PM -0400, Kent Overstreet wrote:
> recently new OOMs have been cropping up, and reclaim is implicated, so
> I've had to dust off these patches.
> 
> nothing significant has changed since the last time I posted, and they
> have been valuable - Dave, think we can get them in?

You need to describe what this does. What does the output look like?
Where does it go (console, dmesg, etc). When is it called, etc.
Links to previous review threads so people can get back up to speed
on what was discussed last time and determine whether issues raised
were solved. A changelog since the last posting is helpful, too...

-Dave.

Kent Overstreet Aug. 28, 2024, 2:23 a.m. UTC | #2

On Wed, Aug 28, 2024 at 11:51:34AM GMT, Dave Chinner wrote:
> On Sat, Aug 24, 2024 at 03:10:07PM -0400, Kent Overstreet wrote:
> > recently new OOMs have been cropping up, and reclaim is implicated, so
> > I've had to dust off these patches.
> > 
> > nothing significant has changed since the last time I posted, and they
> > have been valuable - Dave, think we can get them in?
> 
> You need to describe what this does. What does the output look like?
> Where does it go (console, dmesg, etc). When is it called, etc.
> Links to previous review threads so people can get back up to speed
> on what was discussed last time and determine whether issues raised
> were solved. A changelog since the last posting is helpful, too...

Not much has changed since last posting, besides tweaking/improving
counters...

objects, requested to free, objects freed: so we can tell if a shrinker
isn't freeing as requested

last scanned and last freed are new - so we can tell if a shrinker's
gotten stuck (perhaps it requires a lock that it isn't able to get)

bcachefs shrinkers additionally have counters for every distinct reason
an object wasn't freed - this has been quite useful in the past:

BTW - the next thing I want to get done is adding an interface for
shrinkers to report the amount of memory they own in bytes. This will
greatly aid the show_mem report, so it can skip reporting on shrinkers
entirely if they don't control much memory or pick a number to show more
intelligently.

It'll also let us finally fix the free command.

root@moria-kvm:/sys/kernel/debug/shrinker$ cat 05baba83-05ef-4d71-be0e-121bdabfee36-btree_key_cache-43/report 
05baba83-05ef-4d71-be0e-121bdabfee36-btree_key_cache
objects:             122411
requested to free:   63
objects freed:       0
last scanned:        198 sec ago
last freed:          42949578 sec ago
ns per object freed: 0
keys:                         129119
dirty:                          6580
table size:                   262144

shrinker:
requested_to_free:                 0
freed:                             0
skipped_dirty:                    21
skipped_accessed:                 42
skipped_lock_fail:                 0

pending:                      114727

root@moria-kvm:/sys/kernel/debug/shrinker# cat sb-bcachefs-44/report 
sb-bcachefs
objects:             1
requested to free:   0
objects freed:       0
last scanned:        42949693 sec ago
last freed:          42949693 sec ago
ns per object freed: 0
inodes:   total 2 shrinkable 0
dentries: toal 2 shrinkbale 1