| Message ID | 1484828008-27507-2-git-send-email-amir73il@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Jan 19, 2017 at 2:13 PM, Amir Goldstein <amir73il@gmail.com> wrote: > freeze/thaw of upper is all that is needed. > Miklos, Looking at it again, I believe that not even that is needed. Having fixed ovl_sync_fs() with patch #2, ovl_freeze() and ovl_unfreeze() need to be NOP. Am I right? In fact, freezing upper fs, when many overlayfs mounts share the same base fs (a-la docker) would be quite lame. WRT other use cases of freezing overlayfs, I am not sure if it is needed for correctness of docker checkpoint/restart? Pavel? > Signed-off-by: Amir Goldstein <amir73il@gmail.com> > --- > fs/overlayfs/super.c | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) > > diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c > index 6792bb7..0f8a9c8 100644 > --- a/fs/overlayfs/super.c > +++ b/fs/overlayfs/super.c > @@ -160,6 +160,20 @@ static void ovl_put_super(struct super_block *sb) > kfree(ufs); > } > > +static int ovl_freeze(struct super_block *sb) > +{ > + struct ovl_fs *ufs = sb->s_fs_info; > + > + return ufs->upper_mnt ? freeze_super(ufs->upper_mnt->mnt_sb) : 0; > +} > + > +static int ovl_unfreeze(struct super_block *sb) > +{ > + struct ovl_fs *ufs = sb->s_fs_info; > + > + return ufs->upper_mnt ? thaw_super(ufs->upper_mnt->mnt_sb) : 0; > +} > + > /** > * ovl_statfs > * @sb: The overlayfs super block > @@ -222,6 +236,8 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data) > > static const struct super_operations ovl_super_operations = { > .put_super = ovl_put_super, > + .freeze_fs = ovl_freeze, > + .unfreeze_fs = ovl_unfreeze, > .statfs = ovl_statfs, > .show_options = ovl_show_options, > .remount_fs = ovl_remount, > -- > 2.7.4 > -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 01/23/2017 11:47 AM, Amir Goldstein wrote: > On Thu, Jan 19, 2017 at 2:13 PM, Amir Goldstein <amir73il@gmail.com> wrote: >> freeze/thaw of upper is all that is needed. >> > > Miklos, > > Looking at it again, I believe that not even that is needed. > Having fixed ovl_sync_fs() with patch #2, ovl_freeze() > and ovl_unfreeze() need to be NOP. Am I right? > > In fact, freezing upper fs, when many overlayfs mounts > share the same base fs (a-la docker) would be quite lame. > > WRT other use cases of freezing overlayfs, I am not sure > if it is needed for correctness of docker checkpoint/restart? > > Pavel? Thanks for the heads-up :) Good question. Actually we haven't yet experimented with doing _full_ (with FS) snapshot of containers, but at the first glance I can't see the need for OVL freeze :( We stop all the processes in entry.S effectively, so no in-flight IO can be happening. >> Signed-off-by: Amir Goldstein <amir73il@gmail.com> >> --- >> fs/overlayfs/super.c | 16 ++++++++++++++++ >> 1 file changed, 16 insertions(+) >> >> diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c >> index 6792bb7..0f8a9c8 100644 >> --- a/fs/overlayfs/super.c >> +++ b/fs/overlayfs/super.c >> @@ -160,6 +160,20 @@ static void ovl_put_super(struct super_block *sb) >> kfree(ufs); >> } >> >> +static int ovl_freeze(struct super_block *sb) >> +{ >> + struct ovl_fs *ufs = sb->s_fs_info; >> + >> + return ufs->upper_mnt ? freeze_super(ufs->upper_mnt->mnt_sb) : 0; >> +} >> + >> +static int ovl_unfreeze(struct super_block *sb) >> +{ >> + struct ovl_fs *ufs = sb->s_fs_info; >> + >> + return ufs->upper_mnt ? thaw_super(ufs->upper_mnt->mnt_sb) : 0; >> +} >> + >> /** >> * ovl_statfs >> * @sb: The overlayfs super block >> @@ -222,6 +236,8 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data) >> >> static const struct super_operations ovl_super_operations = { >> .put_super = ovl_put_super, >> + .freeze_fs = ovl_freeze, >> + .unfreeze_fs = ovl_unfreeze, >> .statfs = ovl_statfs, >> .show_options = ovl_show_options, >> .remount_fs = ovl_remount, >> -- >> 2.7.4 >> > . > -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 23, 2017 at 3:23 PM, Pavel Emelyanov <xemul@virtuozzo.com> wrote: > On 01/23/2017 11:47 AM, Amir Goldstein wrote: >> On Thu, Jan 19, 2017 at 2:13 PM, Amir Goldstein <amir73il@gmail.com> wrote: >>> freeze/thaw of upper is all that is needed. >>> >> >> Miklos, >> >> Looking at it again, I believe that not even that is needed. >> Having fixed ovl_sync_fs() with patch #2, ovl_freeze() >> and ovl_unfreeze() need to be NOP. Am I right? >> >> In fact, freezing upper fs, when many overlayfs mounts >> share the same base fs (a-la docker) would be quite lame. >> >> WRT other use cases of freezing overlayfs, I am not sure >> if it is needed for correctness of docker checkpoint/restart? >> >> Pavel? > > Thanks for the heads-up :) > > Good question. Actually we haven't yet experimented with doing _full_ > (with FS) snapshot of containers, but at the first glance I can't > see the need for OVL freeze :( We stop all the processes in entry.S > effectively, so no in-flight IO can be happening. > Serge, Stephan, Same question. Would lxc-snapshot gain anything from the ability to fsfreeze an overlay mount? Basically, it should give you the ability to create a consistent snapshot of overlayfs upper dir from a running container, but maybe you already do that by freezing the container processes? I couldn't figure that out from lxc-snapshot documentation. The context is that I implemented overlayfs fsfreeze for my own needs, but I need other use cases to justify merging the feature. Thanks, Amir.
Quoting Amir Goldstein (amir73il@gmail.com): > On Mon, Jan 23, 2017 at 3:23 PM, Pavel Emelyanov <xemul@virtuozzo.com> wrote: > > On 01/23/2017 11:47 AM, Amir Goldstein wrote: > >> On Thu, Jan 19, 2017 at 2:13 PM, Amir Goldstein <amir73il@gmail.com> wrote: > >>> freeze/thaw of upper is all that is needed. > >>> > >> > >> Miklos, > >> > >> Looking at it again, I believe that not even that is needed. > >> Having fixed ovl_sync_fs() with patch #2, ovl_freeze() > >> and ovl_unfreeze() need to be NOP. Am I right? > >> > >> In fact, freezing upper fs, when many overlayfs mounts > >> share the same base fs (a-la docker) would be quite lame. > >> > >> WRT other use cases of freezing overlayfs, I am not sure > >> if it is needed for correctness of docker checkpoint/restart? > >> > >> Pavel? > > > > Thanks for the heads-up :) > > > > Good question. Actually we haven't yet experimented with doing _full_ > > (with FS) snapshot of containers, but at the first glance I can't > > see the need for OVL freeze :( We stop all the processes in entry.S > > effectively, so no in-flight IO can be happening. > > > > Serge, Stephan, > > Same question. > > Would lxc-snapshot gain anything from the ability to fsfreeze an overlay > mount? lxc-snapshot only works on stopped containers. 'lxc snapshot' can do live snapshots using criu. Tycho, does that do anything right now to freeze the fs? I'm not sure that freezing all the tasks is necessarily enough to settle the fs, but I assume you're doing something about that already? > Basically, it should give you the ability to create a consistent snapshot > of overlayfs upper dir from a running container, but maybe you already do > that by freezing the container processes? > I couldn't figure that out from lxc-snapshot documentation. > > The context is that I implemented overlayfs fsfreeze for my own needs, > but I need other use cases to justify merging the feature. > > Thanks, > Amir.
On Tue, Apr 04, 2017 at 12:47:52PM -0500, Serge E. Hallyn wrote: > > Would lxc-snapshot gain anything from the ability to fsfreeze an overlay > > mount? > > lxc-snapshot only works on stopped containers. 'lxc snapshot' can do live > snapshots using criu. Tycho, does that do anything right now to freeze the > fs? Not that I'm aware of (CRIU might, but we don't in liblxc). > I'm not sure that freezing all the tasks is necessarily enough to settle > the fs, but I assume you're doing something about that already? I suspect it's not, but we're not doing anything besides freezing the tasks. In fact, we freeze the tasks by using the freezer cgroup, which itself is buggy, since the freezer cgroup can race with various filesystems. So, freezing tasks is hard, and I haven't even thought about how to freeze the fs for real :) But in any case, an fs freezing primitive does sound useful for checkpoint restore, assuming that we're right and freezing the tasks is simply not enough. Tycho
On Tue, Apr 4, 2017 at 9:01 PM, Tycho Andersen <tycho@docker.com> wrote: > On Tue, Apr 04, 2017 at 12:47:52PM -0500, Serge E. Hallyn wrote: >> > Would lxc-snapshot gain anything from the ability to fsfreeze an overlay >> > mount? >> >> lxc-snapshot only works on stopped containers. 'lxc snapshot' can do live >> snapshots using criu. Tycho, does that do anything right now to freeze the >> fs? > > Not that I'm aware of (CRIU might, but we don't in liblxc). > >> I'm not sure that freezing all the tasks is necessarily enough to settle >> the fs, but I assume you're doing something about that already? > > I suspect it's not, but we're not doing anything besides freezing the > tasks. In fact, we freeze the tasks by using the freezer cgroup, > which itself is buggy, since the freezer cgroup can race with various > filesystems. So, freezing tasks is hard, and I haven't even thought > about how to freeze the fs for real :) > > But in any case, an fs freezing primitive does sound useful for > checkpoint restore, assuming that we're right and freezing the tasks > is simply not enough. > So I already asked Pavel that question and he said that freezing the tasks is enough. I am not convinced it is really enough to bring a file system image (i.e. underlying blockdev) to a quiescent state, but I think it may be enough for getting a stable view of the mounted file system, so the files could be dumped somewhere. I am guessing is what lxc snapshot does? I still didn't understand wrt lxc snapshot, is there a use case for taking live snapshots without using CRIU? (because freezer cgroup mentioned races or whatnot?). It's definitely possible with btrfs and if my overlayfs freeze patches are not terribly wrong, then it should be easy with overlayfs as well. Does lxc snapshot already support live snapshot of btrfs container? If there is interest, I can try to do a POC of live lxc snapshot of an overlayfs container. Amir.
On Tue, Apr 04, 2017 at 09:59:16PM +0300, Amir Goldstein wrote: > On Tue, Apr 4, 2017 at 9:01 PM, Tycho Andersen <tycho@docker.com> wrote: > > On Tue, Apr 04, 2017 at 12:47:52PM -0500, Serge E. Hallyn wrote: > >> > Would lxc-snapshot gain anything from the ability to fsfreeze an overlay > >> > mount? > >> > >> lxc-snapshot only works on stopped containers. 'lxc snapshot' can do live > >> snapshots using criu. Tycho, does that do anything right now to freeze the > >> fs? > > > > Not that I'm aware of (CRIU might, but we don't in liblxc). > > > >> I'm not sure that freezing all the tasks is necessarily enough to settle > >> the fs, but I assume you're doing something about that already? > > > > I suspect it's not, but we're not doing anything besides freezing the > > tasks. In fact, we freeze the tasks by using the freezer cgroup, > > which itself is buggy, since the freezer cgroup can race with various > > filesystems. So, freezing tasks is hard, and I haven't even thought > > about how to freeze the fs for real :) > > > > But in any case, an fs freezing primitive does sound useful for > > checkpoint restore, assuming that we're right and freezing the tasks > > is simply not enough. > > > > So I already asked Pavel that question and he said that freezing > the tasks is enough. I am not convinced it is really enough to bring > a file system image (i.e. underlying blockdev) to a quiescent state, > but I think it may be enough for getting a stable view of the mounted > file system, so the files could be dumped somewhere. > I am guessing is what lxc snapshot does? Yes, lxc snapshot is basically just a frontend for CRIU. > I still didn't understand wrt lxc snapshot, is there a use case for > taking live snapshots without using CRIU? (because freezer cgroup > mentioned races or whatnot?). No, I think CRIU is the only project that will ever attempt to do checkpoint restore this way ;-). CRIU supports two different ways of freezing tasks: one using the freezer cgroup and one without. The one without doesn't work against fork bombs very well, and the one with doesn't work because of some filesystems. So it's mostly a container engine implementation choice which to use. > It's definitely possible with btrfs and if my overlayfs freeze patches > are not terribly wrong, then it should be easy with overlayfs as well. > Does lxc snapshot already support live snapshot of btrfs container? Yes, it does. It freezes the tasks via the cgroup freezer and then does a btrfs snapshot of the filesystem once the tasks are frozen. Tycho
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index 6792bb7..0f8a9c8 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -160,6 +160,20 @@ static void ovl_put_super(struct super_block *sb) kfree(ufs); } +static int ovl_freeze(struct super_block *sb) +{ + struct ovl_fs *ufs = sb->s_fs_info; + + return ufs->upper_mnt ? freeze_super(ufs->upper_mnt->mnt_sb) : 0; +} + +static int ovl_unfreeze(struct super_block *sb) +{ + struct ovl_fs *ufs = sb->s_fs_info; + + return ufs->upper_mnt ? thaw_super(ufs->upper_mnt->mnt_sb) : 0; +} + /** * ovl_statfs * @sb: The overlayfs super block @@ -222,6 +236,8 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data) static const struct super_operations ovl_super_operations = { .put_super = ovl_put_super, + .freeze_fs = ovl_freeze, + .unfreeze_fs = ovl_unfreeze, .statfs = ovl_statfs, .show_options = ovl_show_options, .remount_fs = ovl_remount,
freeze/thaw of upper is all that is needed. Signed-off-by: Amir Goldstein <amir73il@gmail.com> --- fs/overlayfs/super.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)