[00/13] xfs: metadata inode directories
mbox series

Message ID 154630934595.21716.17416691804044507782.stgit@magnolia
Headers show
Series
  • xfs: metadata inode directories
Related show

Message

Darrick J. Wong Jan. 1, 2019, 2:22 a.m. UTC
Hi all,

This series delivers a new feature -- metadata inode directories.  This
is a separate directory tree (rooted in the superblock) that contains
only inodes that contain filesystem metadata.  Different metadata
objects can be looked up with regular paths.

We start by creating xfs_imeta_* functions to mediate access to metadata
inode pointers.  This enables the imeta code to abstract inode pointers,
whether they're the classic five in the superblock, or the much more
complex directory tree.  All current users of metadata inodes (rt+quota)
are converted to use the boilerplate code.

Next, we define the metadir on-disk format, which consists of marking
inodes with a new iflag that says they're metadata.  This we use to
prevent bulkstat and friends from ever getting their hands on fs
metadata.

Finally, we implement metadir operations so that clients can create,
delete, zap, and look up metadata inodes by path.  Beware that much of
this code is only lightly used, because the five current users of
metadata inodes don't tend to change them very often.  This is likely to
change if and when the subvolume and multiple-rt-volume features get
written/merged/etc.

If you're going to start using this mess, you probably ought to just
pull from my git trees.  The kernel patches[1] should apply against
4.20.  xfsprogs[2] and xfstests[3] can be found in their usual
places.  The git trees contain all four series' worth of changes.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

[1] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=djwong-devel
[2] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=djwong-devel
[3] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=djwong-devel

Comments

Amir Goldstein Jan. 1, 2019, 9:29 a.m. UTC | #1
On Tue, Jan 1, 2019 at 4:22 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> Hi all,
>
> This series delivers a new feature -- metadata inode directories.  This
> is a separate directory tree (rooted in the superblock) that contains
> only inodes that contain filesystem metadata.  Different metadata
> objects can be looked up with regular paths.
>
> We start by creating xfs_imeta_* functions to mediate access to metadata
> inode pointers.  This enables the imeta code to abstract inode pointers,
> whether they're the classic five in the superblock, or the much more
> complex directory tree.  All current users of metadata inodes (rt+quota)
> are converted to use the boilerplate code.
>
> Next, we define the metadir on-disk format, which consists of marking
> inodes with a new iflag that says they're metadata.  This we use to
> prevent bulkstat and friends from ever getting their hands on fs
> metadata.
>
> Finally, we implement metadir operations so that clients can create,
> delete, zap, and look up metadata inodes by path.  Beware that much of
> this code is only lightly used, because the five current users of
> metadata inodes don't tend to change them very often.  This is likely to
> change if and when the subvolume and multiple-rt-volume features get
> written/merged/etc.
>

Finally, the motivation ;-)

How far is this feature expected to scale in the future in term of
number of entries and frequency of lookups?

It seems odd (to put it gently) for xfs to re-invent path lookup,
without cache and loosing all the maturity and vfs path lookup code.
I realize this is for xfs internal use, but is there really anything preventing
you from using internal dcache helpers to maintain an xfs internal
dcache metadata tree?
The metadata tree root dentry would be disconnected from the sb
root and cached in the in core xfs sb struct.

It may not be trivial to pull this off, but the benefits of using mature
and scalable lookup code might just out weight the initial effort in
the long run.

> If you're going to start using this mess, you probably ought to just
> pull from my git trees.  The kernel patches[1] should apply against
> 4.20.  xfsprogs[2] and xfstests[3] can be found in their usual
> places.  The git trees contain all four series' worth of changes.
>
> This is an extraordinary way to destroy everything.  Enjoy!
> Comments and questions are, as always, welcome.
>
> --D
>
> [1] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=djwong-devel
> [2] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=djwong-devel
> [3] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=djwong-devel

It's not there as far as I can tell. Forgot to push?
Many other patches you posted are on djwong-wtf and not on djwong-devel as
declared and this specific series is not even on djwong-wtf.

Happy new year!

Amir.
Christoph Hellwig Jan. 17, 2019, 2:26 p.m. UTC | #2
On Mon, Dec 31, 2018 at 06:22:26PM -0800, Darrick J. Wong wrote:
> Hi all,
> 
> This series delivers a new feature -- metadata inode directories.  This
> is a separate directory tree (rooted in the superblock) that contains
> only inodes that contain filesystem metadata.  Different metadata
> objects can be looked up with regular paths.

If we use path names we should just use regular VFS path lookup for
them.  But why do we even need names?  Can't we just work based on
inode numbers?
Darrick J. Wong Jan. 17, 2019, 11:36 p.m. UTC | #3
On Thu, Jan 17, 2019 at 06:26:36AM -0800, Christoph Hellwig wrote:
> On Mon, Dec 31, 2018 at 06:22:26PM -0800, Darrick J. Wong wrote:
> > Hi all,
> > 
> > This series delivers a new feature -- metadata inode directories.  This
> > is a separate directory tree (rooted in the superblock) that contains
> > only inodes that contain filesystem metadata.  Different metadata
> > objects can be looked up with regular paths.
> 
> If we use path names we should just use regular VFS path lookup for
> them.

I'll research how to do that, since I'm not /that/ familiar with how to
create dentries and do path lookups for a fs root that isn't really
rooted anywhere.

> But why do we even need names?  Can't we just work based on inode
> numbers?

How would that work, particularly if we start to create larger metadata
directory trees?  I might just be misreading your question, though.

The reason to take this series (metadir) is so that we can do things
like supporting multiple rt devices:

%meta%/realtime/$device.bitmap
                $device.summary
                $device.rmap
                $device.refcount

For some arbitrary number of realtime devices.  We don't need it to
support the XFS we have right now.

--D
Darrick J. Wong Jan. 17, 2019, 11:50 p.m. UTC | #4
On Thu, Jan 17, 2019 at 03:36:08PM -0800, Darrick J. Wong wrote:
> On Thu, Jan 17, 2019 at 06:26:36AM -0800, Christoph Hellwig wrote:
> > On Mon, Dec 31, 2018 at 06:22:26PM -0800, Darrick J. Wong wrote:
> > > Hi all,
> > > 
> > > This series delivers a new feature -- metadata inode directories.  This
> > > is a separate directory tree (rooted in the superblock) that contains
> > > only inodes that contain filesystem metadata.  Different metadata
> > > objects can be looked up with regular paths.
> > 
> > If we use path names we should just use regular VFS path lookup for
> > them.
> 
> I'll research how to do that, since I'm not /that/ familiar with how to
> create dentries and do path lookups for a fs root that isn't really
> rooted anywhere.

Before I run off to another doc appt -- the strongest argument I can
think of for /not/ hooking into vfs path lookup is that if xfsprogs
needs to be able to look up metadata inodes then I'd have to port that
part of the vfs to userspace, and yuck.

xfs_db kind of needs it for the fsmap command, and xfs_repair needs to
be able to rebuild the realtime rmap btree inode.

--D

> > But why do we even need names?  Can't we just work based on inode
> > numbers?
> 
> How would that work, particularly if we start to create larger metadata
> directory trees?  I might just be misreading your question, though.
> 
> The reason to take this series (metadir) is so that we can do things
> like supporting multiple rt devices:
> 
> %meta%/realtime/$device.bitmap
>                 $device.summary
>                 $device.rmap
>                 $device.refcount
> 
> For some arbitrary number of realtime devices.  We don't need it to
> support the XFS we have right now.
> 
> --D