mbox series

[00/11] btrfs: add a shrinker for extent maps

Message ID cover.1712748143.git.fdmanana@suse.com (mailing list archive)
Headers show
Series btrfs: add a shrinker for extent maps | expand

Message

Filipe Manana April 10, 2024, 11:28 a.m. UTC
From: Filipe Manana <fdmanana@suse.com>

Currently we don't limit the amount of extent maps we can have for inodes
from a subvolume tree, which can result in excessive use of memory and in
some cases in running into OOM situations. This was reported some time ago
by a user and it's specially easier to trigger with direct IO.

The shrinker itself is patch 9/11, what comes before is simple preparatory
work and the rest just trace events. More details in the change logs.

Filipe Manana (11):
  btrfs: pass an inode to btrfs_add_extent_mapping()
  btrfs: tests: error out on unexpected extent map reference count
  btrfs: simplify add_extent_mapping() by removing pointless label
  btrfs: pass the extent map tree's inode to add_extent_mapping()
  btrfs: pass the extent map tree's inode to clear_em_logging()
  btrfs: pass the extent map tree's inode to remove_extent_mapping()
  btrfs: pass the extent map tree's inode to replace_extent_mapping()
  btrfs: add a global per cpu counter to track number of used extent maps
  btrfs: add a shrinker for extent maps
  btrfs: update comment for btrfs_set_inode_full_sync() about locking
  btrfs: add tracepoints for extent map shrinker events

 fs/btrfs/btrfs_inode.h            |   8 +-
 fs/btrfs/disk-io.c                |   5 +
 fs/btrfs/extent_io.c              |   2 +-
 fs/btrfs/extent_map.c             | 340 +++++++++++++++++++++++++-----
 fs/btrfs/extent_map.h             |   9 +-
 fs/btrfs/fs.h                     |   4 +
 fs/btrfs/inode.c                  |   2 +-
 fs/btrfs/tests/extent-map-tests.c | 216 ++++++++++---------
 fs/btrfs/tree-log.c               |   4 +-
 include/trace/events/btrfs.h      |  92 ++++++++
 10 files changed, 524 insertions(+), 158 deletions(-)

Comments

Qu Wenruo April 11, 2024, 5:25 a.m. UTC | #1
在 2024/4/10 20:58, fdmanana@kernel.org 写道:
> From: Filipe Manana <fdmanana@suse.com>
>
> Currently we don't limit the amount of extent maps we can have for inodes
> from a subvolume tree, which can result in excessive use of memory and in
> some cases in running into OOM situations. This was reported some time ago
> by a user and it's specially easier to trigger with direct IO.
>
> The shrinker itself is patch 9/11, what comes before is simple preparatory
> work and the rest just trace events. More details in the change logs.
>
> Filipe Manana (11):
>    btrfs: pass an inode to btrfs_add_extent_mapping()
>    btrfs: tests: error out on unexpected extent map reference count
>    btrfs: simplify add_extent_mapping() by removing pointless label
>    btrfs: pass the extent map tree's inode to add_extent_mapping()
>    btrfs: pass the extent map tree's inode to clear_em_logging()
>    btrfs: pass the extent map tree's inode to remove_extent_mapping()
>    btrfs: pass the extent map tree's inode to replace_extent_mapping()

Those preparation are all fine even as independent patchset.

Reviewed-by: Qu Wenruo <wqu@suse.com>

>    btrfs: add a global per cpu counter to track number of used extent maps
>    btrfs: add a shrinker for extent maps

Unfortunately I'm not yet familiar enough on logged/pinned extent maps yet.
Thus no comprehensive review for the shrinker implementation yet.

Thanks,
Qu

>    btrfs: update comment for btrfs_set_inode_full_sync() about locking
>    btrfs: add tracepoints for extent map shrinker events
>
>   fs/btrfs/btrfs_inode.h            |   8 +-
>   fs/btrfs/disk-io.c                |   5 +
>   fs/btrfs/extent_io.c              |   2 +-
>   fs/btrfs/extent_map.c             | 340 +++++++++++++++++++++++++-----
>   fs/btrfs/extent_map.h             |   9 +-
>   fs/btrfs/fs.h                     |   4 +
>   fs/btrfs/inode.c                  |   2 +-
>   fs/btrfs/tests/extent-map-tests.c | 216 ++++++++++---------
>   fs/btrfs/tree-log.c               |   4 +-
>   include/trace/events/btrfs.h      |  92 ++++++++
>   10 files changed, 524 insertions(+), 158 deletions(-)
>