| Message ID | 20190606195724.2975689-1-sheenobu@fb.com (mailing list archive) |
|---|---|
| State | Deferred, archived |
| Headers | show |
| Series | [v2] xfs_restore: detect rtinherit on destination | expand |
On 6/6/19 2:57 PM, Sheena Artrip wrote: > When running xfs_restore with a non-rtdev dump, > it will ignore any rtinherit flags on the destination > and send I/O to the metadata region. > > Instead, detect rtinherit on the destination XFS fileystem root inode > and use that to override the incoming inode flags. > > Original version of this patch missed some branches so multiple > invocations of xfsrestore onto the same fs caused > the rtinherit bit to get re-removed. There could be some > additional edge cases in non-realtime to realtime workflows so > the outstanding question would be: is it worth supporting? > > Changes in v2: > * Changed root inode bulkstat to just ioctl to the destdir inode Thanks for that fixup (though comment still says "root" FWIW) Thinking about this some more, I'm really kind of wondering how this should all be expected to work. There are several scenarios here, and "is this file rt?" is prescribed in different ways - either in the dump itself, or on the target fs via inheritance flags... (NB: rt is not the only inheritable flag, so would we need to handle the others?) non-rt fs dump, restored onto non-rt fs - obviously this is fine rt fs dump, restored onto rt fs - obviously this is fine as well rt fs dump, restored onto non-rt fs - this works, with errors - all rt files become non-rt - nothing else to do here other than fail outright non-rt fs dump, restored into rt fs dir/fs with "rtinherit" set - this one is your case - today it's ignored, files stay non-rt - you're suggesting it be honored and files turned into rt the one case that's not handled here is "what if I want to have my realtime dump with realtime files restored onto an rt-capable fs, but turned into regular files?" So your patch gives us one mechanism (restore non-rt files as rt files) but not the converse (restore rt files as non-rt files) - I'm not sure if that matters, but the symmetry bugs me a little. I'm trying to decide if dump/restore is truly the right way to migrate files from non-rt to rt or vice versa, TBH. Maybe dchinner or djwong will have thoughts as well... I'll worry more about the details of the patch after we decide if this is the right behavior in the first place... -Eric > Signed-off-by: Sheena Artrip <sheenobu@fb.com> > --- > restore/content.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 51 insertions(+) > > diff --git a/restore/content.c b/restore/content.c > index 6b22965..4822d1c 100644 > --- a/restore/content.c > +++ b/restore/content.c > @@ -670,6 +670,9 @@ struct tran { > /* to establish critical regions while updating pers > * inventory > */ > + bool_t t_dstisrealtime; > + /* to force the realtime flag on incoming inodes > + */ > }; > > typedef struct tran tran_t; > @@ -1803,6 +1806,37 @@ content_init(int argc, char *argv[], size64_t vmsz) > free_handle(fshanp, fshlen); > } > > + /* determine if destination root inode has rtinherit. > + * If so, we should force XFS_REALTIME on the incoming inodes. > + */ > + if (persp->a.dstdirisxfspr) { > + struct fsxattr dstxattr; > + > + int dstfd = open(persp->a.dstdir, O_RDONLY); > + if (dstfd < 0) { > + mlog(MLOG_NORMAL | MLOG_WARNING, > + _("open of %s failed: %s\n"), > + persp->a.dstdir, > + strerror(errno)); > + return BOOL_FALSE; > + } > + > + /* Get the xattr details for the destination folder */ > + if (ioctl(dstfd, XFS_IOC_FSGETXATTR, &dstxattr) < 0) { > + (void)close(dstfd); > + mlog(MLOG_ERROR, > + _("failed to get xattr information for dst inode\n")); > + return BOOL_FALSE; > + } > + > + (void)close(dstfd); > + > + /* test against rtinherit */ > + if((dstxattr.fsx_xflags & XFS_XFLAG_RTINHERIT) != 0) { > + tranp->t_dstisrealtime = true; > + } > + } > + > /* map in pers. inv. descriptors, if any. NOTE: this ptr is to be > * referenced ONLY via the macros provided; the descriptors will be > * occasionally remapped, causing the ptr to change. > @@ -7270,6 +7304,10 @@ restore_file_cb(void *cp, bool_t linkpr, char *path1, char *path2) > bool_t ahcs = contextp->cb_ahcs; > stream_context_t *strctxp = (stream_context_t *)drivep->d_strmcontextp; > > + if (tranp->t_dstisrealtime) { > + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; > + } > + > int rval; > bool_t ok; > > @@ -7480,6 +7518,10 @@ restore_reg(drive_t *drivep, > if (tranp->t_toconlypr) > return BOOL_TRUE; > > + if (tranp->t_dstisrealtime) { > + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; > + } > + > oflags = O_CREAT | O_RDWR; > if (persp->a.dstdirisxfspr && bstatp->bs_xflags & XFS_XFLAG_REALTIME) > oflags |= O_DIRECT; > @@ -8470,6 +8512,11 @@ restore_extent(filehdr_t *fhdrp, > } > assert(new_off == off); > } > + > + if (tranp->t_dstisrealtime) { > + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; > + } > + > if ((fd != -1) && (bstatp->bs_xflags & XFS_XFLAG_REALTIME)) { > if ((ioctl(fd, XFS_IOC_DIOINFO, &da) < 0)) { > mlog(MLOG_NORMAL | MLOG_WARNING, _( > @@ -8729,6 +8776,10 @@ restore_extattr(drive_t *drivep, > > assert(extattrbufp); > > + if (tranp->t_dstisrealtime) { > + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; > + } > + > if (!isdirpr) > isfilerestored = partial_check(bstatp->bs_ino, bstatp->bs_size); > >
On Thu, Jun 06, 2019 at 04:23:51PM -0500, Eric Sandeen wrote: > On 6/6/19 2:57 PM, Sheena Artrip wrote: > > When running xfs_restore with a non-rtdev dump, > > it will ignore any rtinherit flags on the destination > > and send I/O to the metadata region. > > > > Instead, detect rtinherit on the destination XFS fileystem root inode > > and use that to override the incoming inode flags. > > > > Original version of this patch missed some branches so multiple > > invocations of xfsrestore onto the same fs caused > > the rtinherit bit to get re-removed. There could be some > > additional edge cases in non-realtime to realtime workflows so > > the outstanding question would be: is it worth supporting? > > > > Changes in v2: > > * Changed root inode bulkstat to just ioctl to the destdir inode > > Thanks for that fixup (though comment still says "root" FWIW) > > Thinking about this some more, I'm really kind of wondering how this > should all be expected to work. There are several scenarios here, > and "is this file rt?" is prescribed in different ways - either in > the dump itself, or on the target fs via inheritance flags... > > (NB: rt is not the only inheritable flag, so would we need to handle > the others?) > > non-rt fs dump, restored onto non-rt fs > - obviously this is fine > > rt fs dump, restored onto rt fs > - obviously this is fine as well > > rt fs dump, restored onto non-rt fs > - this works, with errors - all rt files become non-rt > - nothing else to do here other than fail outright This should just work, without errors or warnings. > non-rt fs dump, restored into rt fs dir/fs with "rtinherit" set > - this one is your case > - today it's ignored, files stay non-rt > - you're suggesting it be honored and files turned into rt Current filesystem policy should override the policy in dump image as the dump image may contain an invalid policy.... > the one case that's not handled here is "what if I want to have my > realtime dump with realtime files restored onto an rt-capable fs, but > turned into regular files?" Which is where having the kernel policy override the dump file is necesary... > So your patch gives us one mechanism (restore non-rt files as > rt files) but not the converse (restore rt files as non-rt files) - > I'm not sure if that matters, but the symmetry bugs me a little. > > I'm trying to decide if dump/restore is truly the right way to > migrate files from non-rt to rt or vice versa, TBH. Maybe dchinner > or djwong will have thoughts as well... *nod* My take on this is that we need to decide which allocation policy to use - the kernel policy or the dump file policy - in the different situations. It's a simple, easy to document and understand solution. At minimum, if there's a mismatch between rtdev/non-rtdev between dump and restore, then restore should not try to restore or clear rt flags at all. i.e the rt flags in the dump image should be considered invalid in this situation and masked out in the restore process. This prevents errors from being reported during restore, and it does "the right thing" according to how the user has configured the destination directory. i.e. if the destdir has the rtinherit bit set and there's a rtdev present, the kernel policy will cause all file data that is restored to be allocated on the rtdev. Otherwise the kernel will place it (correctly) on the data dev. In the case where both have rtdevs, but you want to restore to ignore the dump file rtdev policy, we really only need to add a CLI option to say "ignore rt flags" and that then allows the kernel policy to dictate how the restored files are placed in the same way that having a rtdev mismatch does. This is simple, consistent, fulfils the requirements and should have no hidden surprises for users.... Cheers, Dave.
On 6/6/19 4:50 PM, Dave Chinner wrote: > On Thu, Jun 06, 2019 at 04:23:51PM -0500, Eric Sandeen wrote: >> On 6/6/19 2:57 PM, Sheena Artrip wrote: >>> When running xfs_restore with a non-rtdev dump, >>> it will ignore any rtinherit flags on the destination >>> and send I/O to the metadata region. >>> >>> Instead, detect rtinherit on the destination XFS fileystem root inode >>> and use that to override the incoming inode flags. >>> >>> Original version of this patch missed some branches so multiple >>> invocations of xfsrestore onto the same fs caused >>> the rtinherit bit to get re-removed. There could be some >>> additional edge cases in non-realtime to realtime workflows so >>> the outstanding question would be: is it worth supporting? >>> >>> Changes in v2: >>> * Changed root inode bulkstat to just ioctl to the destdir inode >> >> Thanks for that fixup (though comment still says "root" FWIW) >> >> Thinking about this some more, I'm really kind of wondering how this >> should all be expected to work. There are several scenarios here, >> and "is this file rt?" is prescribed in different ways - either in >> the dump itself, or on the target fs via inheritance flags... >> >> (NB: rt is not the only inheritable flag, so would we need to handle >> the others?) >> >> non-rt fs dump, restored onto non-rt fs >> - obviously this is fine >> >> rt fs dump, restored onto rt fs >> - obviously this is fine as well >> >> rt fs dump, restored onto non-rt fs >> - this works, with errors - all rt files become non-rt >> - nothing else to do here other than fail outright > > This should just work, without errors or warnings. I said errors but I meant warnings: xfsrestore: restoring non-directory files xfsrestore: WARNING: attempt to set extended attributes (xflags 0x80000001, extsize = 0x0, projid = 0x0) of rtdir/bar failed: Invalid argument xfsrestore: WARNING: attempt to set extended attributes (xflags 0x80000001, extsize = 0x0, projid = 0x0) of rtdir/baz failed: Invalid argument xfsrestore: restore complete: 0 seconds elapsed and yeah, probably should not be noisy there. >> non-rt fs dump, restored into rt fs dir/fs with "rtinherit" set >> - this one is your case >> - today it's ignored, files stay non-rt >> - you're suggesting it be honored and files turned into rt > > Current filesystem policy should override the policy in dump image > as the dump image may contain an invalid policy.... > >> the one case that's not handled here is "what if I want to have my >> realtime dump with realtime files restored onto an rt-capable fs, but >> turned into regular files?" > > Which is where having the kernel policy override the dump file is > necesary... The trick is that we don't have a "no-rt" flag we can set on a dir, so there is no "files in this dir ar not rt" policy to follow. >> So your patch gives us one mechanism (restore non-rt files as >> rt files) but not the converse (restore rt files as non-rt files) - >> I'm not sure if that matters, but the symmetry bugs me a little. >> >> I'm trying to decide if dump/restore is truly the right way to >> migrate files from non-rt to rt or vice versa, TBH. Maybe dchinner >> or djwong will have thoughts as well... > > *nod* > > My take on this is that we need to decide which allocation policy to > use - the kernel policy or the dump file policy - in the different > situations. It's a simple, easy to document and understand solution. > > At minimum, if there's a mismatch between rtdev/non-rtdev between > dump and restore, then restore should not try to restore or clear rt > flags at all. i.e the rt flags in the dump image should be > considered invalid in this situation and masked out in the restore > process. This prevents errors from being reported during restore, > and it does "the right thing" according to how the user has > configured the destination directory. i.e. if the destdir has the > rtinherit bit set and there's a rtdev present, the kernel policy > will cause all file data that is restored to be allocated on the > rtdev. Otherwise the kernel will place it (correctly) on the data > dev. > > In the case where both have rtdevs, but you want to restore to > ignore the dump file rtdev policy, we really only need to add a CLI > option to say "ignore rt flags" and that then allows the kernel > policy to dictate how the restored files are placed in the same way > that having a rtdev mismatch does. > > This is simple, consistent, fulfils the requirements and should have > no hidden surprises for users.... Sounds reasonable. So the CLI flag would say "ignore RT info in the dump, and write files according to the destination fs policy?" I think that makes sense. Now: do we need to do the same for all inheritable flags? projid, extsize, etc? I think we probably do. -Eric
On Thu, Jun 06, 2019 at 05:08:12PM -0500, Eric Sandeen wrote: > On 6/6/19 4:50 PM, Dave Chinner wrote: > > My take on this is that we need to decide which allocation policy to > > use - the kernel policy or the dump file policy - in the different > > situations. It's a simple, easy to document and understand solution. > > > > At minimum, if there's a mismatch between rtdev/non-rtdev between > > dump and restore, then restore should not try to restore or clear rt > > flags at all. i.e the rt flags in the dump image should be > > considered invalid in this situation and masked out in the restore > > process. This prevents errors from being reported during restore, > > and it does "the right thing" according to how the user has > > configured the destination directory. i.e. if the destdir has the > > rtinherit bit set and there's a rtdev present, the kernel policy > > will cause all file data that is restored to be allocated on the > > rtdev. Otherwise the kernel will place it (correctly) on the data > > dev. > > > > In the case where both have rtdevs, but you want to restore to > > ignore the dump file rtdev policy, we really only need to add a CLI > > option to say "ignore rt flags" and that then allows the kernel > > policy to dictate how the restored files are placed in the same way > > that having a rtdev mismatch does. > > > > This is simple, consistent, fulfils the requirements and should have > > no hidden surprises for users.... > > Sounds reasonable. So the CLI flag would say "ignore RT info in the > dump, and write files according to the destination fs policy?" > I think that makes sense. *nod* > Now: do we need to do the same for all inheritable flags? projid, > extsize, etc? I think we probably do. I disagree. These things are all supported on all destination filesystems, unlike the rtdev. They are also things that can be changed after the fact, unlike rtdev allocation policy. i.e. rtdev has to be set /before/ restore, just about everything else can be set or reset after the fact.... Cheers, Dave.
On Thu, Jun 6, 2019 at 3:37 PM Dave Chinner <david@fromorbit.com> wrote: > > On Thu, Jun 06, 2019 at 05:08:12PM -0500, Eric Sandeen wrote: > > On 6/6/19 4:50 PM, Dave Chinner wrote: > > > My take on this is that we need to decide which allocation policy to > > > use - the kernel policy or the dump file policy - in the different > > > situations. It's a simple, easy to document and understand solution. > > > > > > At minimum, if there's a mismatch between rtdev/non-rtdev between > > > dump and restore, then restore should not try to restore or clear rt > > > flags at all. i.e the rt flags in the dump image should be > > > considered invalid in this situation and masked out in the restore > > > process. This prevents errors from being reported during restore, > > > and it does "the right thing" according to how the user has > > > configured the destination directory. i.e. if the destdir has the > > > rtinherit bit set and there's a rtdev present, the kernel policy > > > will cause all file data that is restored to be allocated on the > > > rtdev. Otherwise the kernel will place it (correctly) on the data > > > dev. > > > > > > In the case where both have rtdevs, but you want to restore to > > > ignore the dump file rtdev policy, we really only need to add a CLI > > > option to say "ignore rt flags" and that then allows the kernel > > > policy to dictate how the restored files are placed in the same way > > > that having a rtdev mismatch does. > > > > > > This is simple, consistent, fulfils the requirements and should have > > > no hidden surprises for users.... > > > > Sounds reasonable. So the CLI flag would say "ignore RT info in the > > dump, and write files according to the destination fs policy?" > > I think that makes sense. Any suggested flag name/prefix for this? Last i checked all the single letters were taken up? > *nod* > > > Now: do we need to do the same for all inheritable flags? projid, > > extsize, etc? I think we probably do. > > I disagree. These things are all supported on all destination > filesystems, unlike the rtdev. They are also things that can be > changed after the fact, unlike rtdev allocation policy. i.e. rtdev > has to be set /before/ restore, just about everything else can be > set or reset after the fact.... > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com
On Mon, Jun 17, 2019 at 03:09:15PM -0700, Sheena Artrip wrote: > On Thu, Jun 6, 2019 at 3:37 PM Dave Chinner <david@fromorbit.com> wrote: > > > > On Thu, Jun 06, 2019 at 05:08:12PM -0500, Eric Sandeen wrote: > > > On 6/6/19 4:50 PM, Dave Chinner wrote: > > > > My take on this is that we need to decide which allocation policy to > > > > use - the kernel policy or the dump file policy - in the different > > > > situations. It's a simple, easy to document and understand solution. > > > > > > > > At minimum, if there's a mismatch between rtdev/non-rtdev between > > > > dump and restore, then restore should not try to restore or clear rt > > > > flags at all. i.e the rt flags in the dump image should be > > > > considered invalid in this situation and masked out in the restore > > > > process. This prevents errors from being reported during restore, > > > > and it does "the right thing" according to how the user has > > > > configured the destination directory. i.e. if the destdir has the > > > > rtinherit bit set and there's a rtdev present, the kernel policy > > > > will cause all file data that is restored to be allocated on the > > > > rtdev. Otherwise the kernel will place it (correctly) on the data > > > > dev. > > > > > > > > In the case where both have rtdevs, but you want to restore to > > > > ignore the dump file rtdev policy, we really only need to add a CLI > > > > option to say "ignore rt flags" and that then allows the kernel > > > > policy to dictate how the restored files are placed in the same way > > > > that having a rtdev mismatch does. > > > > > > > > This is simple, consistent, fulfils the requirements and should have > > > > no hidden surprises for users.... > > > > > > Sounds reasonable. So the CLI flag would say "ignore RT info in the > > > dump, and write files according to the destination fs policy?" > > > I think that makes sense. > > Any suggested flag name/prefix for this? Last i checked all the single > letters were taken up? I suggest --preserve-xflags=<same letters as xfs_io lsattr command> --D > > *nod* > > > > > Now: do we need to do the same for all inheritable flags? projid, > > > extsize, etc? I think we probably do. > > > > I disagree. These things are all supported on all destination > > filesystems, unlike the rtdev. They are also things that can be > > changed after the fact, unlike rtdev allocation policy. i.e. rtdev > > has to be set /before/ restore, just about everything else can be > > set or reset after the fact.... > > Cheers, > > > > Dave. > > -- > > Dave Chinner > > david@fromorbit.com
On Mon, Jun 17, 2019 at 3:56 PM Darrick J. Wong <darrick.wong@oracle.com> wrote: > > On Mon, Jun 17, 2019 at 03:09:15PM -0700, Sheena Artrip wrote: > > On Thu, Jun 6, 2019 at 3:37 PM Dave Chinner <david@fromorbit.com> wrote: > > > > > > On Thu, Jun 06, 2019 at 05:08:12PM -0500, Eric Sandeen wrote: > > > > On 6/6/19 4:50 PM, Dave Chinner wrote: > > > > > My take on this is that we need to decide which allocation policy to > > > > > use - the kernel policy or the dump file policy - in the different > > > > > situations. It's a simple, easy to document and understand solution. > > > > > > > > > > At minimum, if there's a mismatch between rtdev/non-rtdev between > > > > > dump and restore, then restore should not try to restore or clear rt > > > > > flags at all. i.e the rt flags in the dump image should be > > > > > considered invalid in this situation and masked out in the restore > > > > > process. This prevents errors from being reported during restore, > > > > > and it does "the right thing" according to how the user has > > > > > configured the destination directory. i.e. if the destdir has the > > > > > rtinherit bit set and there's a rtdev present, the kernel policy > > > > > will cause all file data that is restored to be allocated on the > > > > > rtdev. Otherwise the kernel will place it (correctly) on the data > > > > > dev. > > > > > > > > > > In the case where both have rtdevs, but you want to restore to > > > > > ignore the dump file rtdev policy, we really only need to add a CLI > > > > > option to say "ignore rt flags" and that then allows the kernel > > > > > policy to dictate how the restored files are placed in the same way > > > > > that having a rtdev mismatch does. > > > > > > > > > > This is simple, consistent, fulfils the requirements and should have > > > > > no hidden surprises for users.... > > > > > > > > Sounds reasonable. So the CLI flag would say "ignore RT info in the > > > > dump, and write files according to the destination fs policy?" > > > > I think that makes sense. > > > > Any suggested flag name/prefix for this? Last i checked all the single > > letters were taken up? > > I suggest --preserve-xflags=<same letters as xfs_io lsattr command> What's the implication? That we do not copy any xflags bits unless you include them in --preserve-xflags? The defaults of this would be all the available fields. That still leaves the destination needing a xflag bit like realtime and the source not having it...Maybe xfsdump needs --preserve-xflags and xfsrestore needs --apply-xflags ? That will catch all the cases and the solution is just an and/xor on the outgoing/incoming bsp_xflags field: * realtime->realtime (--preserve-xflags=all --apply-xflags=none) * non-realtime->realtime (--preserve-xflags=all --apply-xflags=t) * non-realtime->non-realtime (--preserve-xflags=all --apply-xflags=none) * realtime->non-realtime (--preserve-xflags=all-but-t) Thanks! > --D > > > > *nod* > > > > > > > Now: do we need to do the same for all inheritable flags? projid, > > > > extsize, etc? I think we probably do. > > > > > > I disagree. These things are all supported on all destination > > > filesystems, unlike the rtdev. They are also things that can be > > > changed after the fact, unlike rtdev allocation policy. i.e. rtdev > > > has to be set /before/ restore, just about everything else can be > > > set or reset after the fact.... > > > Cheers, > > > > > > Dave. > > > -- > > > Dave Chinner > > > david@fromorbit.com
On Mon, Jun 17, 2019 at 11:28:03PM -0700, Sheena Artrip wrote: > On Mon, Jun 17, 2019 at 3:56 PM Darrick J. Wong <darrick.wong@oracle.com> wrote: > > > > On Mon, Jun 17, 2019 at 03:09:15PM -0700, Sheena Artrip wrote: > > > On Thu, Jun 6, 2019 at 3:37 PM Dave Chinner <david@fromorbit.com> wrote: > > > > > > > > On Thu, Jun 06, 2019 at 05:08:12PM -0500, Eric Sandeen wrote: > > > > > On 6/6/19 4:50 PM, Dave Chinner wrote: > > > > > > My take on this is that we need to decide which allocation policy to > > > > > > use - the kernel policy or the dump file policy - in the different > > > > > > situations. It's a simple, easy to document and understand solution. > > > > > > > > > > > > At minimum, if there's a mismatch between rtdev/non-rtdev between > > > > > > dump and restore, then restore should not try to restore or clear rt > > > > > > flags at all. i.e the rt flags in the dump image should be > > > > > > considered invalid in this situation and masked out in the restore > > > > > > process. This prevents errors from being reported during restore, > > > > > > and it does "the right thing" according to how the user has > > > > > > configured the destination directory. i.e. if the destdir has the > > > > > > rtinherit bit set and there's a rtdev present, the kernel policy > > > > > > will cause all file data that is restored to be allocated on the > > > > > > rtdev. Otherwise the kernel will place it (correctly) on the data > > > > > > dev. > > > > > > > > > > > > In the case where both have rtdevs, but you want to restore to > > > > > > ignore the dump file rtdev policy, we really only need to add a CLI > > > > > > option to say "ignore rt flags" and that then allows the kernel > > > > > > policy to dictate how the restored files are placed in the same way > > > > > > that having a rtdev mismatch does. > > > > > > > > > > > > This is simple, consistent, fulfils the requirements and should have > > > > > > no hidden surprises for users.... > > > > > > > > > > Sounds reasonable. So the CLI flag would say "ignore RT info in the > > > > > dump, and write files according to the destination fs policy?" > > > > > I think that makes sense. > > > > > > Any suggested flag name/prefix for this? Last i checked all the single > > > letters were taken up? > > > > I suggest --preserve-xflags=<same letters as xfs_io lsattr command> > > What's the implication? That we do not copy any xflags bits unless you > include them in --preserve-xflags? > The defaults of this would be all the available fields. > > That still leaves the destination needing a xflag bit like realtime > and the source not having it...Maybe > xfsdump needs --preserve-xflags and xfsrestore needs --apply-xflags ? > That will catch > all the cases and the solution is just an and/xor on the > outgoing/incoming bsp_xflags field: > > * realtime->realtime (--preserve-xflags=all --apply-xflags=none) > * non-realtime->realtime (--preserve-xflags=all --apply-xflags=t) > * non-realtime->non-realtime (--preserve-xflags=all --apply-xflags=none) > * realtime->non-realtime (--preserve-xflags=all-but-t) I was clearly confusing, my apologies. :/ What I should've said more explicitly was that xfsdump continues to save all the xflags as it does now, and that we should consider adding a new cli option to xfsrestore to preserve certain xflags from the underlying fs with a new --preserve-xflags= option. Maybe it should be --preserve-dest-xflags to make it clear that the xflags of the destination are overriding the dump. For your "create dump on non-rt fs and restore to rt dev on rt fs" use case, you'd do.... # xfsdump -f /dumpfile /oldfs # mkfs.xfs -r rtdev=/dev/sdX -d rtinherit=1 /dev/sdY # mount /dev/sdY /newfs # xfsrestore -f /dumpfile --preserve-dest-xflags=t /newfs --D > Thanks! > > > --D > > > > > > *nod* > > > > > > > > > Now: do we need to do the same for all inheritable flags? projid, > > > > > extsize, etc? I think we probably do. > > > > > > > > I disagree. These things are all supported on all destination > > > > filesystems, unlike the rtdev. They are also things that can be > > > > changed after the fact, unlike rtdev allocation policy. i.e. rtdev > > > > has to be set /before/ restore, just about everything else can be > > > > set or reset after the fact.... > > > > Cheers, > > > > > > > > Dave. > > > > -- > > > > Dave Chinner > > > > david@fromorbit.com
diff --git a/restore/content.c b/restore/content.c index 6b22965..4822d1c 100644 --- a/restore/content.c +++ b/restore/content.c @@ -670,6 +670,9 @@ struct tran { /* to establish critical regions while updating pers * inventory */ + bool_t t_dstisrealtime; + /* to force the realtime flag on incoming inodes + */ }; typedef struct tran tran_t; @@ -1803,6 +1806,37 @@ content_init(int argc, char *argv[], size64_t vmsz) free_handle(fshanp, fshlen); } + /* determine if destination root inode has rtinherit. + * If so, we should force XFS_REALTIME on the incoming inodes. + */ + if (persp->a.dstdirisxfspr) { + struct fsxattr dstxattr; + + int dstfd = open(persp->a.dstdir, O_RDONLY); + if (dstfd < 0) { + mlog(MLOG_NORMAL | MLOG_WARNING, + _("open of %s failed: %s\n"), + persp->a.dstdir, + strerror(errno)); + return BOOL_FALSE; + } + + /* Get the xattr details for the destination folder */ + if (ioctl(dstfd, XFS_IOC_FSGETXATTR, &dstxattr) < 0) { + (void)close(dstfd); + mlog(MLOG_ERROR, + _("failed to get xattr information for dst inode\n")); + return BOOL_FALSE; + } + + (void)close(dstfd); + + /* test against rtinherit */ + if((dstxattr.fsx_xflags & XFS_XFLAG_RTINHERIT) != 0) { + tranp->t_dstisrealtime = true; + } + } + /* map in pers. inv. descriptors, if any. NOTE: this ptr is to be * referenced ONLY via the macros provided; the descriptors will be * occasionally remapped, causing the ptr to change. @@ -7270,6 +7304,10 @@ restore_file_cb(void *cp, bool_t linkpr, char *path1, char *path2) bool_t ahcs = contextp->cb_ahcs; stream_context_t *strctxp = (stream_context_t *)drivep->d_strmcontextp; + if (tranp->t_dstisrealtime) { + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; + } + int rval; bool_t ok; @@ -7480,6 +7518,10 @@ restore_reg(drive_t *drivep, if (tranp->t_toconlypr) return BOOL_TRUE; + if (tranp->t_dstisrealtime) { + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; + } + oflags = O_CREAT | O_RDWR; if (persp->a.dstdirisxfspr && bstatp->bs_xflags & XFS_XFLAG_REALTIME) oflags |= O_DIRECT; @@ -8470,6 +8512,11 @@ restore_extent(filehdr_t *fhdrp, } assert(new_off == off); } + + if (tranp->t_dstisrealtime) { + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; + } + if ((fd != -1) && (bstatp->bs_xflags & XFS_XFLAG_REALTIME)) { if ((ioctl(fd, XFS_IOC_DIOINFO, &da) < 0)) { mlog(MLOG_NORMAL | MLOG_WARNING, _( @@ -8729,6 +8776,10 @@ restore_extattr(drive_t *drivep, assert(extattrbufp); + if (tranp->t_dstisrealtime) { + bstatp->bs_xflags |= XFS_XFLAG_REALTIME; + } + if (!isdirpr) isfilerestored = partial_check(bstatp->bs_ino, bstatp->bs_size);
When running xfs_restore with a non-rtdev dump, it will ignore any rtinherit flags on the destination and send I/O to the metadata region. Instead, detect rtinherit on the destination XFS fileystem root inode and use that to override the incoming inode flags. Original version of this patch missed some branches so multiple invocations of xfsrestore onto the same fs caused the rtinherit bit to get re-removed. There could be some additional edge cases in non-realtime to realtime workflows so the outstanding question would be: is it worth supporting? Changes in v2: * Changed root inode bulkstat to just ioctl to the destdir inode Signed-off-by: Sheena Artrip <sheenobu@fb.com> --- restore/content.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)