mbox series

[GIT,PULL] Btrfs updates for 6.13

Message ID cover.1731515050.git.dsterba@suse.com (mailing list archive)
State New
Headers show
Series [GIT,PULL] Btrfs updates for 6.13 | expand

Pull-request

git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git tags/for-6.13-tag

Message

David Sterba Nov. 13, 2024, 6:13 p.m. UTC 
Hi,

please pull the following updates for btrfs.

Changes outside of btrfs: add io_uring command flag to track a dying
task (the rest will go via the block git tree).

User visible changes:

- wire encoded read (ioctl) to io_uring commands, this can be used on
  itself, in the future this will allow 'send' to be asynchronous
  - as a consequence, the encoded read ioctl can also work in
    non-blocking mode

- new ioctl to wait for cleaned subvolumes, no need to use the generic
  and root-only SEARCH_TREE ioctl, will be used by "btrfs subvol sync"

- recognize different paths/symlinks for the same devices and don't
  report them during rescanning, this can be observed with LVM or DM

- seeding device use case change, the sprout device (the one capturing
  new writes) will not clear the read-only status of the super block;
  this prevents accumulating space from deleted snapshots

Performance improvements:

- reduce lock contention when traversing extent buffers

- reduce extent tree lock contention when searching for inline backref

- switch from rb-trees to xarray for delayed ref tracking, improvements
  due to better cache locality, branching factors and more compact data
  structures

- enable extent map shrinker again (prevent memory exhaustion under
  some types of IO load), reworked to run in a single worker thread
  (there used to be problems causing long stalls under memory pressure)

Core changes:

- raid-stripe-tree feature updates
  - make device replace and scrub work
  - implement partial deletion of stripe extents
  - new selftests

- split the config option BTRFS_DEBUG and add EXPERIMENTAL for features
  that are experimental or with known problems so we don't misuse
  debugging config for that

- subpage mode updates (sector < page)
  - update compression implementations
  - update writepage, writeback

- continued folio API conversions
  - buffered writes

- make buffered write copy one page at a time, preparatory work for
  future integration with large folios, may cause performance drop

- proper locking of root item regarding starting send

- error handling improvements

- code cleanups and refactoring
  - dead code removal
  - unused parameter reduction
  - lockdep assertions

----------------------------------------------------------------
The following changes since commit 2d5404caa8c7bb5c4e0435f94b28834ae5456623:

  Linux 6.12-rc7 (2024-11-10 14:19:35 -0800)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git tags/for-6.13-tag

for you to fetch changes up to e82c936293aafb4f33b153c684c37291b3eed377:

  btrfs: send: check for read-only send root under critical section (2024-11-11 14:34:23 +0100)

----------------------------------------------------------------
Anand Jain (1):
      btrfs: use filemap_get_folio() helper

Boris Burkov (1):
      btrfs: do not clear read-only when adding sprout device

Christoph Hellwig (1):
      btrfs: fix a typo in btrfs_use_zone_append

David Sterba (26):
      btrfs: zstd: assert the timer pointer in callback
      btrfs: drop unused parameter path from btrfs_tree_mod_log_rewind()
      btrfs: drop unused parameter ctx from batch_delete_dir_index_items()
      btrfs: drop unused parameter fs_info from wait_reserve_ticket()
      btrfs: drop unused parameter fs_info from do_reclaim_sweep()
      btrfs: send: drop unused parameter num from iterate_inode_ref_t callbacks
      btrfs: send: drop unused parameter index from iterate_inode_ref_t callbacks
      btrfs: scrub: drop unused parameter sctx from scrub_submit_extent_sector_read()
      btrfs: drop unused parameter map from scrub_simple_mirror()
      btrfs: qgroup: drop unused parameter fs_info from __del_qgroup_rb()
      btrfs: drop unused transaction parameter from btrfs_qgroup_add_swapped_blocks()
      btrfs: lzo: drop unused paramter level from lzo_alloc_workspace()
      btrfs: drop unused parameter argp from btrfs_ioctl_quota_rescan_wait()
      btrfs: drop unused parameter inode from read_inline_extent()
      btrfs: drop unused parameter offset from __cow_file_range_inline()
      btrfs: drop unused parameter file_offset from btrfs_encoded_read_regular_fill_pages()
      btrfs: drop unused parameter iov_iter from btrfs_write_check()
      btrfs: drop unused parameter refs from visit_node_for_delete()
      btrfs: drop unused parameter mask from try_release_extent_state()
      btrfs: drop unused parameter fs_info from folio_range_has_eb()
      btrfs: drop unused parameter options from open_ctree()
      btrfs: drop unused parameter data from btrfs_fill_super()
      btrfs: drop unused parameter transaction from alloc_log_tree()
      btrfs: drop unused parameter fs_info from btrfs_match_dir_item_name()
      btrfs: drop unused parameter level from alloc_heuristic_ws()
      btrfs: add new ioctl to wait for cleaned subvolumes

Dr. David Alan Gilbert (3):
      btrfs: remove unused btrfs_free_squota_rsv()
      btrfs: remove unused btrfs_is_parity_mirror()
      btrfs: remove unused btrfs_try_tree_write_lock()

Filipe Manana (40):
      btrfs: qgroups: remove bytenr field from struct btrfs_qgroup_extent_record
      btrfs: store fs_info in a local variable at btrfs_qgroup_trace_extent_post()
      btrfs: remove unnecessary delayed refs locking at btrfs_qgroup_trace_extent()
      btrfs: always use delayed_refs local variable at btrfs_qgroup_trace_extent()
      btrfs: remove pointless initialization at btrfs_qgroup_trace_extent()
      btrfs: qgroup: run delayed iputs after ordered extent completion
      btrfs: add and use helper to remove extent map from its inode's tree
      btrfs: make the extent map shrinker run asynchronously as a work queue job
      btrfs: simplify tracking progress for the extent map shrinker
      btrfs: rename extent map shrinker members from struct btrfs_fs_info
      btrfs: re-enable the extent map shrinker
      btrfs: remove redundant level argument from read_block_for_search()
      btrfs: simplify arguments for btrfs_verify_level_key()
      btrfs: remove redundant initializations for struct btrfs_tree_parent_check
      btrfs: remove local generation variable from read_block_for_search()
      btrfs: remove BUG_ON() at btrfs_destroy_delayed_refs()
      btrfs: move btrfs_destroy_delayed_refs() to delayed-ref.c
      btrfs: remove fs_info parameter from btrfs_destroy_delayed_refs()
      btrfs: remove fs_info parameter from btrfs_cleanup_one_transaction()
      btrfs: remove duplicated code to drop delayed ref during transaction abort
      btrfs: use helper to find first ref head at btrfs_destroy_delayed_refs()
      btrfs: remove num_entries atomic counter from delayed ref root
      btrfs: change return type of btrfs_delayed_ref_lock() to boolean
      btrfs: simplify obtaining a delayed ref head
      btrfs: move delayed ref head unselection to delayed-ref.c
      btrfs: pass fs_info to functions that search for delayed ref heads
      btrfs: pass fs_info to btrfs_delete_ref_head()
      btrfs: assert delayed refs lock is held at find_ref_head()
      btrfs: assert delayed refs lock is held at find_first_ref_head()
      btrfs: assert delayed refs lock is held at add_delayed_ref_head()
      btrfs: add comments regarding locking to struct btrfs_delayed_ref_root
      btrfs: track delayed ref heads in an xarray
      btrfs: remove no longer used delayed ref head search functionality
      btrfs: update stale comment for struct btrfs_delayed_ref_node::add_list
      btrfs: remove hole from struct btrfs_delayed_node
      btrfs: simplify logic to decrement snapshot counter at btrfs_mksnapshot()
      btrfs: fix warning on PTR_ERR() against NULL device at btrfs_control_ioctl()
      btrfs: remove check for NULL fs_info at btrfs_folio_end_lock_bitmap()
      btrfs: send: check for dead send root under critical section
      btrfs: send: check for read-only send root under critical section

Haisu Wang (1):
      btrfs: simplify range tracking in cow_file_range()

Johannes Thumshirn (8):
      btrfs: don't take dev_replace rwsem on task already holding it
      btrfs: remove code duplication in ordered extent finishing
      btrfs: tests: add selftests for raid-stripe-tree
      btrfs: handle empty list of NOCOW ordered extents with checksum list
      btrfs: return ENODATA in case RST lookup fails
      btrfs: scrub: skip initial RST lookup errors
      btrfs: implement partial deletion of RAID stripe extents
      btrfs: tests: implement case for partial RAID stripe-tree delete

Leo Martins (2):
      btrfs: push cleanup into btrfs_read_locked_inode()
      btrfs: remove conditional path allocation in btrfs_read_locked_inode()

Mark Harmstone (8):
      btrfs: fix wrong sizeof in btrfs_do_encoded_write()
      btrfs: remove pointless iocb::ki_pos addition in btrfs_encoded_read()
      btrfs: change btrfs_encoded_read() so that reading of extent is done by caller
      btrfs: don't sleep in btrfs_encoded_read() if IOCB_NOWAIT is set
      btrfs: move priv off stack in btrfs_encoded_read_regular_fill_pages()
      btrfs: add io_uring command for encoded reads (ENCODED_READ ioctl)
      btrfs: add struct io_btrfs_cmd as type for io_uring_cmd_to_pdu()
      btrfs: avoid superfluous calls to free_extent_map() in btrfs_encoded_read()

Pavel Begunkov (1):
      io_uring/cmd: let cmds to know about dying task

Qu Wenruo (21):
      btrfs: make assert_rbio() to only check CONFIG_BTRFS_ASSERT
      btrfs: split out CONFIG_BTRFS_EXPERIMENTAL from CONFIG_BTRFS_DEBUG
      btrfs: zlib: make the compression path to handle sector size < page size
      btrfs: zstd: make the compression path to handle sector size < page size
      btrfs: compression: add an ASSERT() to ensure the read-in length is sane
      btrfs: wait for writeback if sector size is smaller than page size
      btrfs: make extent_range_clear_dirty_for_io() to handle sector size < page size cases
      btrfs: do not assume the full page range is not dirty in extent_writepage_io()
      btrfs: move the delalloc range bitmap search into extent_io.c
      btrfs: mark all dirty sectors as locked inside writepage_delalloc()
      btrfs: allow compression even if the range is not page aligned
      btrfs: avoid unnecessary device path update for the same device
      btrfs: canonicalize the device path before adding it
      btrfs: remove the dirty_page local variable
      btrfs: simplify the page uptodate preparation for prepare_pages()
      btrfs: remove btrfs_set_range_writeback()
      btrfs: remove unused btrfs_folio_start_writer_lock()
      btrfs: unify to use writer locks for subpage locking
      btrfs: rename btrfs_folio_(set|start|end)_writer_lock()
      btrfs: make buffered write to copy one page a time
      btrfs: convert btrfs_buffered_write() to use folios

Riyan Dhiman (1):
      btrfs: remove redundant stop_loop variable in scrub_stripe()

Robbie Ko (2):
      btrfs: reduce lock contention when eb cache miss for btree search
      btrfs: reduce extent tree lock contention when searching for inline backref

Shen Lichuan (1):
      btrfs: correct typos in multiple comments across various files

Thorsten Blum (1):
      btrfs: use str_yes_no() helper function in btrfs_dump_free_space()

Youling Tang (1):
      btrfs: remove unused page_to_inode and page_to_fs_info macros

 fs/btrfs/Kconfig                        |  26 ++
 fs/btrfs/Makefile                       |   3 +-
 fs/btrfs/backref.c                      |   3 +-
 fs/btrfs/bio.c                          |   2 +-
 fs/btrfs/block-group.c                  |   2 +-
 fs/btrfs/btrfs_inode.h                  |  15 +-
 fs/btrfs/compression.c                  |  14 +-
 fs/btrfs/compression.h                  |   2 +-
 fs/btrfs/ctree.c                        | 148 +++++----
 fs/btrfs/delayed-inode.h                |   2 +-
 fs/btrfs/delayed-ref.c                  | 327 +++++++++++--------
 fs/btrfs/delayed-ref.h                  |  64 ++--
 fs/btrfs/dev-replace.c                  |   4 +-
 fs/btrfs/dir-item.c                     |  11 +-
 fs/btrfs/dir-item.h                     |   3 +-
 fs/btrfs/direct-io.c                    |   2 +-
 fs/btrfs/disk-io.c                      |  93 +-----
 fs/btrfs/disk-io.h                      |   6 +-
 fs/btrfs/extent-tree.c                  | 104 +++---
 fs/btrfs/extent_io.c                    | 111 +++++--
 fs/btrfs/extent_map.c                   | 122 ++++----
 fs/btrfs/extent_map.h                   |   3 +-
 fs/btrfs/fiemap.c                       |   6 +-
 fs/btrfs/file.c                         | 351 ++++++++-------------
 fs/btrfs/file.h                         |   7 +-
 fs/btrfs/free-space-cache.c             |  22 +-
 fs/btrfs/fs.h                           |  16 +-
 fs/btrfs/inode.c                        | 495 +++++++++++++++--------------
 fs/btrfs/ioctl.c                        | 478 +++++++++++++++++++++++++++-
 fs/btrfs/ioctl.h                        |   2 +
 fs/btrfs/locking.c                      |  15 -
 fs/btrfs/locking.h                      |   1 -
 fs/btrfs/lzo.c                          |   2 +-
 fs/btrfs/qgroup.c                       |  90 +++---
 fs/btrfs/qgroup.h                       |  17 +-
 fs/btrfs/raid-stripe-tree.c             |  92 +++++-
 fs/btrfs/raid-stripe-tree.h             |   5 +
 fs/btrfs/raid56.c                       |   3 +-
 fs/btrfs/relocation.c                   |   2 +-
 fs/btrfs/scrub.c                        |  37 +--
 fs/btrfs/send.c                         |  61 ++--
 fs/btrfs/send.h                         |   2 +-
 fs/btrfs/space-info.c                   |  12 +-
 fs/btrfs/subpage.c                      | 204 ++----------
 fs/btrfs/subpage.h                      |  39 +--
 fs/btrfs/super.c                        |  33 +-
 fs/btrfs/sysfs.c                        |   4 +-
 fs/btrfs/tests/btrfs-tests.c            |   4 +
 fs/btrfs/tests/btrfs-tests.h            |   2 +
 fs/btrfs/tests/raid-stripe-tree-tests.c | 538 ++++++++++++++++++++++++++++++++
 fs/btrfs/transaction.c                  |   8 +-
 fs/btrfs/transaction.h                  |   2 +-
 fs/btrfs/tree-checker.c                 |  16 +-
 fs/btrfs/tree-checker.h                 |   4 +-
 fs/btrfs/tree-log.c                     |   3 +-
 fs/btrfs/tree-mod-log.c                 |   1 -
 fs/btrfs/tree-mod-log.h                 |   1 -
 fs/btrfs/volumes.c                      | 163 ++++++++--
 fs/btrfs/volumes.h                      |  11 +-
 fs/btrfs/xattr.c                        |   5 +-
 fs/btrfs/zlib.c                         |   2 +-
 fs/btrfs/zoned.c                        |   4 +-
 fs/btrfs/zstd.c                         |   4 +-
 include/linux/io_uring_types.h          |   1 +
 include/trace/events/btrfs.h            |  39 +--
 include/uapi/linux/btrfs.h              |  25 ++
 io_uring/uring_cmd.c                    |   6 +-
 67 files changed, 2470 insertions(+), 1432 deletions(-)
 create mode 100644 fs/btrfs/tests/raid-stripe-tree-tests.c