[GIT,PULL] xfs: new features for 5.3
mbox series

Message ID 20190712180205.GA5347@magnolia
State New
Headers show
Series
  • [GIT,PULL] xfs: new features for 5.3
Related show

Pull-request

git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-5.3-merge-12

Message

Darrick J. Wong July 12, 2019, 6:02 p.m. UTC
Hi Linus,

Here's all the new code for the 5.3 merge window.  In this release there
are a significant amounts of consolidations and cleanups in the log
code; restructuring of the log to issue struct bios directly; new
bulkstat ioctls to return v5 fs inode information (and fix all the
padding problems of the old ioctl); the beginnings of multithreaded
inode walks (e.g. quotacheck); and a reduction in memory usage in the
online scrub code leading to reduced runtimes.

The branch merges cleanly against this morning's HEAD and survived an
overnight run of xfstests.  The merge was completely straightforward, so
please let me know if you run into anything weird.

--D

The following changes since commit d1fdb6d8f6a4109a4263176c84b899076a5f8008:

  Linux 5.2-rc4 (2019-06-08 20:24:46 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-5.3-merge-12

for you to fetch changes up to 488ca3d8d088ec4658c87aaec6a91e98acccdd54:

  xfs: chain bios the right way around in xfs_rw_bdev (2019-07-10 10:04:16 -0700)

----------------------------------------------------------------
New stuff for 5.3:
- Refactor inode geometry calculation into a single structure instead of
  open-coding pieces everywhere.
- Add online repair to build options.
- Remove unnecessary function call flags and functions.
- Claim maintainership of various loose xfs documentation and header
  files.
- Use struct bio directly for log buffer IOs instead of struct xfs_buf.
- Reduce log item boilerplate code requirements.
- Merge log item code spread across too many files.
- Further distinguish between log item commits and cancellations.
- Various small cleanups to the ag small allocator.
- Support cgroup-aware writeback
- libxfs refactoring for mkfs cleanup
- Remove unneeded #includes
- Fix a memory allocation miscalculation in the new log bio code
- Fix bisection problems
- Fix a crash in ioend processing caused by tripping over freeing of
  preallocated transactions
- Split out a generic inode walk mechanism from the bulkstat code, hook
  up all the internal users to use the walking code, then clean up
  bulkstat to serve only the bulkstat ioctls.
- Add a multithreaded iwalk implementation to speed up quotacheck on
  fast storage with many CPUs.
- Remove unnecessary return values in logging teardown functions.
- Supplement the bstat and inogrp structures with new bulkstat and
  inumbers structures that have all the fields we need for v5
  filesystem features and none of the padding problems of their
  predecessors.
- Wire up new ioctls that use the new structures with a much simpler
  bulk_ireq structure at the head instead of the pointerhappy mess we
  had before.
- Enable userspace to constrain bulkstat returns to a single AG or a
  single special inode so that we can phase out a lot of geometry
  guesswork in userspace.
- Reduce memory consumption and zeroing overhead in extended attribute
  scrub code.
- Fix some behavioral regressions in the new bulkstat backend code.
- Fix some behavioral regressions in the new log bio code.

----------------------------------------------------------------
Amir Goldstein (7):
      vfs: introduce generic_file_rw_checks()
      vfs: remove redundant checks from generic_remap_checks()
      vfs: add missing checks to copy_file_range
      vfs: introduce file_modified() helper
      xfs: use file_modified() helper
      vfs: allow copy_file_range to copy across devices
      fuse: copy_file_range needs to strip setuid bits and update timestamps

Brian Foster (4):
      xfs: clean up small allocation helper
      xfs: move small allocation helper
      xfs: skip small alloc cntbt logic on NULL cursor
      xfs: always update params on small allocation

Christoph Hellwig (54):
      xfs: merge xfs_buf_zero and xfs_buf_iomove
      xfs: remove the debug-only q_transp field from struct xfs_dquot
      xfs: remove the no-op spinlock_destroy stub
      xfs: remove the never used _XBF_COMPOUND flag
      xfs: renumber XBF_WRITE_FAIL
      xfs: make mem_to_page available outside of xfs_buf.c
      xfs: remove the l_iclog_size_log field from struct xlog
      xfs: cleanup xlog_get_iclog_buffer_size
      xfs: reformat xlog_get_lowest_lsn
      xfs: remove XLOG_STATE_IOABORT
      xfs: don't use REQ_PREFLUSH for split log writes
      xfs: factor out log buffer writing from xlog_sync
      xfs: factor out splitting of an iclog from xlog_sync
      xfs: factor out iclog size calculation from xlog_sync
      xfs: update both stat counters together in xlog_sync
      xfs: remove the syncing argument from xlog_verify_iclog
      xfs: make use of the l_targ field in struct xlog
      xfs: use bios directly to write log buffers
      xfs: move the log ioend workqueue to struct xlog
      xfs: return an offset instead of a pointer from xlog_align
      xfs: use bios directly to read and write the log recovery buffers
      xfs: stop using bp naming for log recovery buffers
      xfs: remove unused buffer cache APIs
      xfs: properly type the b_log_item field in struct xfs_buf
      xfs: remove the b_io_length field in struct xfs_buf
      xfs: add struct xfs_mount pointer to struct xfs_buf
      xfs: fix a trivial comment typo in xfs_trans_committed_bulk
      xfs: stop using XFS_LI_ABORTED as a parameter flag
      xfs: don't require log items to implement optional methods
      xfs: remove the dummy iop_push implementation for inode creation items
      xfs: don't use xfs_trans_free_items in the commit path
      xfs: split iop_unlock
      xfs: add a flag to release log items on commit
      xfs: don't cast inode_log_items to get the log_item
      xfs: remove the xfs_log_item_t typedef
      xfs: use a list_head for iclog callbacks
      xfs: remove a pointless comment duplicated above all xfs_item_ops instances
      xfs: merge xfs_efd_init into xfs_trans_get_efd
      xfs: merge xfs_cud_init into xfs_trans_get_cud
      xfs: merge xfs_rud_init into xfs_trans_get_rud
      xfs: merge xfs_bud_init into xfs_trans_get_bud
      xfs: merge xfs_trans_extfree.c into xfs_extfree_item.c
      xfs: merge xfs_trans_refcount.c into xfs_refcount_item.c
      xfs: merge xfs_trans_rmap.c into xfs_rmap_item.c
      xfs: merge xfs_trans_bmap.c into xfs_bmap_item.c
      xfs: simplify xfs_chain_bio
      xfs: implement cgroup aware writeback
      xfs: fix iclog allocation size
      xfs: remove the unused xfs_count_page_state declaration
      xfs: fix a comment typo in xfs_submit_ioend
      xfs: allow merging ioends over append boundaries
      xfs: simplify xfs_ioend_can_merge
      xfs: remove XFS_TRANS_NOFS
      xfs: chain bios the right way around in xfs_rw_bdev

Darrick J. Wong (37):
      xfs: separate inode geometry
      xfs: refactor inode geometry setup routines
      xfs: fix inode_cluster_size rounding mayhem
      xfs: finish converting to inodes_per_cluster
      xfs: claim maintainership of loose files
      xfs: move xfs_ino_geometry to xfs_shared.h
      xfs: refactor free space btree record initialization
      xfs: account for log space when formatting new AGs
      xfs: create iterator error codes
      xfs: create simplified inode walk function
      xfs: convert quotacheck to use the new iwalk functions
      xfs: bulkstat should copy lastip whenever userspace supplies one
      xfs: convert bulkstat to new iwalk infrastructure
      xfs: calculate inode walk prefetch more carefully
      xfs: move bulkstat ichunk helpers to iwalk code
      xfs: change xfs_iwalk_grab_ichunk to use startino, not lastino
      xfs: clean up long conditionals in xfs_iwalk_ichunk_ra
      xfs: refactor xfs_iwalk_grab_ichunk
      xfs: refactor iwalk code to handle walking inobt records
      xfs: refactor INUMBERS to use iwalk functions
      xfs: multithreaded iwalk implementation
      xfs: poll waiting for quotacheck
      xfs: remove various bulk request typedef usage
      xfs: rename bulkstat functions
      xfs: introduce new v5 bulkstat structure
      xfs: introduce v5 inode group structure
      xfs: wire up new v5 bulkstat ioctls
      xfs: wire up the v5 inumbers ioctl
      xfs: specify AG in bulk req
      xfs: allow single bulkstat of special inodes
      xfs: attribute scrub should use seen_enough to pass error values
      xfs: refactor extended attribute buffer pointer functions
      xfs: refactor attr scrub memory allocation function
      xfs: only allocate memory for scrubbing attributes when we need it
      xfs: online scrub needn't bother zeroing its temporary buffer
      xfs: don't update lastino for FSBULKSTAT_SINGLE
      xfs: bump INUMBERS cursor correctly in xfs_inumbers_walk

Dave Chinner (2):
      vfs: introduce generic_copy_file_range()
      vfs: no fallback for ->copy_file_range

Eric Sandeen (4):
      xfs: include WARN, REPAIR build options in XFS_BUILD_OPTIONS
      xfs: remove unused flags arg from getsb interfaces
      xfs: remove unused flag arguments
      xfs: remove unused header files

Hariprasad Kelam (1):
      fs: xfs: xfs_log: Change return type from int to void

 .../filesystems/xfs-self-describing-metadata.txt   |   8 +-
 MAINTAINERS                                        |   6 +
 fs/ceph/file.c                                     |  23 +-
 fs/cifs/cifsfs.c                                   |   4 +
 fs/fuse/file.c                                     |  29 +-
 fs/inode.c                                         |  20 +
 fs/nfs/nfs4file.c                                  |  23 +-
 fs/read_write.c                                    | 124 ++--
 fs/xfs/Makefile                                    |   9 +-
 fs/xfs/kmem.c                                      |   5 -
 fs/xfs/kmem.h                                      |   8 +
 fs/xfs/libxfs/xfs_ag.c                             | 100 ++-
 fs/xfs/libxfs/xfs_ag_resv.c                        |   8 -
 fs/xfs/libxfs/xfs_alloc.c                          | 227 +++----
 fs/xfs/libxfs/xfs_alloc_btree.c                    |   3 +-
 fs/xfs/libxfs/xfs_attr.c                           |   5 -
 fs/xfs/libxfs/xfs_attr.h                           |   8 +-
 fs/xfs/libxfs/xfs_attr_leaf.c                      |  15 +-
 fs/xfs/libxfs/xfs_attr_remote.c                    |  14 +-
 fs/xfs/libxfs/xfs_bit.c                            |   1 -
 fs/xfs/libxfs/xfs_bmap.c                           |  19 +-
 fs/xfs/libxfs/xfs_bmap_btree.c                     |   5 +-
 fs/xfs/libxfs/xfs_btree.c                          |  49 +-
 fs/xfs/libxfs/xfs_btree.h                          |  14 +-
 fs/xfs/libxfs/xfs_da_btree.c                       |  12 +-
 fs/xfs/libxfs/xfs_da_format.c                      |   3 -
 fs/xfs/libxfs/xfs_defer.c                          |   2 -
 fs/xfs/libxfs/xfs_dir2.c                           |   6 +-
 fs/xfs/libxfs/xfs_dir2_block.c                     |  11 +-
 fs/xfs/libxfs/xfs_dir2_data.c                      |  14 +-
 fs/xfs/libxfs/xfs_dir2_leaf.c                      |  11 +-
 fs/xfs/libxfs/xfs_dir2_node.c                      |  10 +-
 fs/xfs/libxfs/xfs_dir2_sf.c                        |   5 +-
 fs/xfs/libxfs/xfs_dquot_buf.c                      |  10 +-
 fs/xfs/libxfs/xfs_format.h                         |   2 +-
 fs/xfs/libxfs/xfs_fs.h                             | 124 +++-
 fs/xfs/libxfs/xfs_health.h                         |   2 +-
 fs/xfs/libxfs/xfs_ialloc.c                         | 245 ++++---
 fs/xfs/libxfs/xfs_ialloc.h                         |  18 +-
 fs/xfs/libxfs/xfs_ialloc_btree.c                   |  56 +-
 fs/xfs/libxfs/xfs_ialloc_btree.h                   |   3 +
 fs/xfs/libxfs/xfs_iext_tree.c                      |   6 +-
 fs/xfs/libxfs/xfs_inode_buf.c                      |   9 +-
 fs/xfs/libxfs/xfs_inode_fork.c                     |   4 +-
 fs/xfs/libxfs/xfs_log_rlimit.c                     |   2 -
 fs/xfs/libxfs/xfs_refcount.c                       |   2 -
 fs/xfs/libxfs/xfs_refcount_btree.c                 |   4 +-
 fs/xfs/libxfs/xfs_rmap.c                           |   7 -
 fs/xfs/libxfs/xfs_rmap_btree.c                     |   6 +-
 fs/xfs/libxfs/xfs_rtbitmap.c                       |   8 -
 fs/xfs/libxfs/xfs_sb.c                             |  39 +-
 fs/xfs/libxfs/xfs_shared.h                         |  49 +-
 fs/xfs/libxfs/xfs_symlink_remote.c                 |  10 +-
 fs/xfs/libxfs/xfs_trans_resv.c                     |  17 +-
 fs/xfs/libxfs/xfs_trans_space.h                    |   7 +-
 fs/xfs/libxfs/xfs_types.c                          |  13 +-
 fs/xfs/scrub/agheader.c                            |  11 +-
 fs/xfs/scrub/agheader_repair.c                     |   5 -
 fs/xfs/scrub/alloc.c                               |   7 -
 fs/xfs/scrub/attr.c                                | 122 +++-
 fs/xfs/scrub/attr.h                                |  71 ++
 fs/xfs/scrub/bitmap.c                              |   5 -
 fs/xfs/scrub/bmap.c                                |   8 -
 fs/xfs/scrub/btree.c                               |   7 -
 fs/xfs/scrub/common.c                              |   8 -
 fs/xfs/scrub/dabtree.c                             |   8 -
 fs/xfs/scrub/dir.c                                 |  10 -
 fs/xfs/scrub/fscounters.c                          |  12 -
 fs/xfs/scrub/health.c                              |   8 -
 fs/xfs/scrub/ialloc.c                              |  28 +-
 fs/xfs/scrub/inode.c                               |  10 -
 fs/xfs/scrub/parent.c                              |   8 -
 fs/xfs/scrub/quota.c                               |  13 +-
 fs/xfs/scrub/refcount.c                            |  10 -
 fs/xfs/scrub/repair.c                              |  14 +-
 fs/xfs/scrub/rmap.c                                |   9 -
 fs/xfs/scrub/rtbitmap.c                            |   7 -
 fs/xfs/scrub/scrub.c                               |  20 -
 fs/xfs/scrub/symlink.c                             |   8 -
 fs/xfs/scrub/trace.c                               |   6 -
 fs/xfs/xfs_acl.c                                   |   4 +-
 fs/xfs/xfs_aops.c                                  | 121 ++--
 fs/xfs/xfs_aops.h                                  |   1 -
 fs/xfs/xfs_attr_inactive.c                         |   7 +-
 fs/xfs/xfs_attr_list.c                             |   7 +-
 fs/xfs/xfs_bio_io.c                                |  61 ++
 fs/xfs/xfs_bmap_item.c                             | 350 ++++++----
 fs/xfs/xfs_bmap_item.h                             |   2 -
 fs/xfs/xfs_bmap_util.c                             |  11 +-
 fs/xfs/xfs_buf.c                                   | 171 +----
 fs/xfs/xfs_buf.h                                   |  53 +-
 fs/xfs/xfs_buf_item.c                              |  40 +-
 fs/xfs/xfs_buf_item.h                              |   6 +-
 fs/xfs/xfs_dir2_readdir.c                          |   5 +-
 fs/xfs/xfs_discard.c                               |   4 +-
 fs/xfs/xfs_dquot.c                                 |   6 +-
 fs/xfs/xfs_dquot.h                                 |   1 -
 fs/xfs/xfs_dquot_item.c                            | 118 +---
 fs/xfs/xfs_dquot_item.h                            |   4 +-
 fs/xfs/xfs_error.c                                 |   3 +-
 fs/xfs/xfs_export.c                                |   4 +-
 fs/xfs/xfs_extfree_item.c                          | 410 ++++++-----
 fs/xfs/xfs_extfree_item.h                          |   6 +-
 fs/xfs/xfs_file.c                                  |  38 +-
 fs/xfs/xfs_filestream.c                            |   5 +-
 fs/xfs/xfs_fsmap.c                                 |   4 -
 fs/xfs/xfs_fsops.c                                 |   8 +-
 fs/xfs/xfs_globals.c                               |   4 +-
 fs/xfs/xfs_health.c                                |   6 +-
 fs/xfs/xfs_icache.c                                |   4 +-
 fs/xfs/xfs_icreate_item.c                          |  75 +--
 fs/xfs/xfs_inode.c                                 |  42 +-
 fs/xfs/xfs_inode_item.c                            |  16 +-
 fs/xfs/xfs_inode_item.h                            |   2 +-
 fs/xfs/xfs_ioctl.c                                 | 294 +++++++-
 fs/xfs/xfs_ioctl.h                                 |   8 +
 fs/xfs/xfs_ioctl32.c                               | 161 +++--
 fs/xfs/xfs_ioctl32.h                               |  14 +-
 fs/xfs/xfs_iomap.c                                 |   5 +-
 fs/xfs/xfs_iops.c                                  |  10 -
 fs/xfs/xfs_itable.c                                | 749 +++++++--------------
 fs/xfs/xfs_itable.h                                | 106 ++-
 fs/xfs/xfs_iwalk.c                                 | 720 ++++++++++++++++++++
 fs/xfs/xfs_iwalk.h                                 |  46 ++
 fs/xfs/xfs_linux.h                                 |   5 +-
 fs/xfs/xfs_log.c                                   | 644 ++++++++----------
 fs/xfs/xfs_log.h                                   |  17 +-
 fs/xfs/xfs_log_cil.c                               |  51 +-
 fs/xfs/xfs_log_priv.h                              |  36 +-
 fs/xfs/xfs_log_recover.c                           | 463 ++++++-------
 fs/xfs/xfs_message.c                               |   2 +-
 fs/xfs/xfs_mount.c                                 | 102 +--
 fs/xfs/xfs_mount.h                                 |  22 +-
 fs/xfs/xfs_ondisk.h                                |   5 +
 fs/xfs/xfs_pnfs.c                                  |   9 +-
 fs/xfs/xfs_pwork.c                                 | 136 ++++
 fs/xfs/xfs_pwork.h                                 |  61 ++
 fs/xfs/xfs_qm.c                                    |  68 +-
 fs/xfs/xfs_qm_bhv.c                                |   2 +-
 fs/xfs/xfs_qm_syscalls.c                           |   5 -
 fs/xfs/xfs_quotaops.c                              |   3 +-
 fs/xfs/xfs_refcount_item.c                         | 357 ++++++----
 fs/xfs/xfs_refcount_item.h                         |   2 -
 fs/xfs/xfs_reflink.c                               |  15 +-
 fs/xfs/xfs_rmap_item.c                             | 380 +++++++----
 fs/xfs/xfs_rmap_item.h                             |   2 -
 fs/xfs/xfs_rtalloc.c                               |   6 -
 fs/xfs/xfs_stats.c                                 |   1 -
 fs/xfs/xfs_super.c                                 |  32 +-
 fs/xfs/xfs_super.h                                 |  14 +
 fs/xfs/xfs_symlink.c                               |   9 -
 fs/xfs/xfs_sysctl.c                                |   3 -
 fs/xfs/xfs_sysctl.h                                |   3 +
 fs/xfs/xfs_sysfs.c                                 |  42 +-
 fs/xfs/xfs_trace.c                                 |   8 -
 fs/xfs/xfs_trace.h                                 |  61 +-
 fs/xfs/xfs_trans.c                                 |  43 +-
 fs/xfs/xfs_trans.h                                 |  70 +-
 fs/xfs/xfs_trans_ail.c                             |  53 +-
 fs/xfs/xfs_trans_bmap.c                            | 232 -------
 fs/xfs/xfs_trans_buf.c                             |  11 +-
 fs/xfs/xfs_trans_dquot.c                           |  11 -
 fs/xfs/xfs_trans_extfree.c                         | 286 --------
 fs/xfs/xfs_trans_inode.c                           |   3 -
 fs/xfs/xfs_trans_priv.h                            |   4 +-
 fs/xfs/xfs_trans_refcount.c                        | 240 -------
 fs/xfs/xfs_trans_rmap.c                            | 257 -------
 fs/xfs/xfs_xattr.c                                 |   5 +-
 include/linux/fs.h                                 |   9 +
 mm/filemap.c                                       | 110 ++-
 170 files changed, 4664 insertions(+), 4807 deletions(-)
 create mode 100644 fs/xfs/scrub/attr.h
 create mode 100644 fs/xfs/xfs_bio_io.c
 create mode 100644 fs/xfs/xfs_iwalk.c
 create mode 100644 fs/xfs/xfs_iwalk.h
 create mode 100644 fs/xfs/xfs_pwork.c
 create mode 100644 fs/xfs/xfs_pwork.h
 delete mode 100644 fs/xfs/xfs_trans_bmap.c
 delete mode 100644 fs/xfs/xfs_trans_extfree.c
 delete mode 100644 fs/xfs/xfs_trans_refcount.c
 delete mode 100644 fs/xfs/xfs_trans_rmap.c

Comments

Linus Torvalds July 13, 2019, 12:27 a.m. UTC | #1
On Fri, Jul 12, 2019 at 11:02 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> The branch merges cleanly against this morning's HEAD and survived an
> overnight run of xfstests.  The merge was completely straightforward, so
> please let me know if you run into anything weird.

Hmm. I don't know what you merged against, but it got a (fairly
trivial) conflict for me due to

  79d08f89bb1b ("block: fix .bi_size overflow")

from the block merge (from Tuesday) touching a line next to one changed by

  a24737359667 ("xfs: simplify xfs_chain_bio")

from this pull.

So it wasn't an entirely clean merge for me.

Was it a complex merge conflict? No. I'm just confused by the "merges
cleanly against this morning's HEAD", which makes me wonder what you
tried to merge against..

            Linus
pr-tracker-bot@kernel.org July 13, 2019, 12:30 a.m. UTC | #2
The pull request you sent on Fri, 12 Jul 2019 11:02:05 -0700:

> git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-5.3-merge-12

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/4ce9d181ebe53abbca5f450b8a2984b8c3a38f26

Thank you!
Darrick J. Wong July 13, 2019, 4:07 a.m. UTC | #3
On Fri, Jul 12, 2019 at 05:27:15PM -0700, Linus Torvalds wrote:
> On Fri, Jul 12, 2019 at 11:02 AM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > The branch merges cleanly against this morning's HEAD and survived an
> > overnight run of xfstests.  The merge was completely straightforward, so
> > please let me know if you run into anything weird.
> 
> Hmm. I don't know what you merged against, but it got a (fairly
> trivial) conflict for me due to
> 
>   79d08f89bb1b ("block: fix .bi_size overflow")
> 
> from the block merge (from Tuesday) touching a line next to one changed by
> 
>   a24737359667 ("xfs: simplify xfs_chain_bio")
> 
> from this pull.
> 
> So it wasn't an entirely clean merge for me.
> 
> Was it a complex merge conflict? No. I'm just confused by the "merges
> cleanly against this morning's HEAD", which makes me wonder what you
> tried to merge against..

Doh, it turns out I was merging against the same HEAD as my last two
pull requests because I forgot to re-pull.  Sorry about that.  It's been
too long of a week. :/

--D

>             Linus
Linus Torvalds July 14, 2019, 1:17 a.m. UTC | #4
On Fri, Jul 12, 2019 at 9:07 PM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> Doh, it turns out I was merging against the same HEAD as my last two
> pull requests because I forgot to re-pull.  Sorry about that.  It's been
> too long of a week. :/

Heh, no problem, I was just surprised when my merge result didn't
match expectations.

As mentioned, it wasn't like the conflict was complicated, only unexpected.

                     Linus