[v18,00/17] xfs: online repair support

Message

Darrick J. Wong Jan. 1, 2019, 2:09 a.m. UTC

Hi all,

This is the first part of the eighteenth revision of a patchset that
adds to XFS kernel support for online metadata scrubbing and repair.
There aren't any on-disk format changes.

New for v18 is a rebase against 4.20 and the creation of explicit data
structures for storing intermediate reconstructed records.  Otherwise,
there hasn't been much activity here, as we've been too busy fixing
clone/copy/deduperange ioctls.

Patch 1 moves all the inode geometry information into a separate
structure so that we can later add more fields to make the online fsck
code less messy w.r.t. inode geometry.

Patch 2 creates a new data structure that provides an abstraction of a
big memory array by using linked lists.  This is where we store records
for btree reconstruction.  This first implementation is memory
inefficient and consumes a /lot/ of kernel memory, but lays the
groundwork for the last patch in the set to convert the implementation
to use a (memfd) swap file, which enables us to use pageable memory
without pounding the slab cache.

Patches 3-9 implement reconstruction of the free space btrees, the inode
btrees, the inodes, the inode forks, the inode block maps, and symbolic
links.

Patch 10 implements a new data structure for storing arbitrary key/value
pairs, which we're going to need to reconstruct extended attribute
forks.

Patches 11-13 clean up the block unmapping code so that we will be able
to perform a mass reset of an inode's fork.  This is a key component for
salvaging extended attributes, freeing all the attr fork blocks, and
reconstructing the extended attribute data.

Patch 14 implements extended attribute salvage.

Patch 15 augments scrub to rebuild extended attributes when any of the
attr blocks are fragmented.

Patch 16 implements reconstruction of quota blocks.

Patch 17 converts both in-memory array implementations from the clunky
linked list implementation to something resembling C arrays.  The array
data are backed by a (memfd) file, which means that idle data can be
paged out to disk instead of pinning kernel memory.

If you're going to start using this mess, you probably ought to just
pull from my git trees.  The kernel patches[1] should apply against
4.20.  xfsprogs[2] and xfstests[3] can be found in their usual
places.  The git trees contain all four series' worth of changes.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

[1] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=djwong-devel
[2] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=djwong-devel
[3] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=djwong-devel