diff mbox

[1/4] Btrfs: send, bump stream version

Message ID 1397580021-26598-1-git-send-email-fdmanana@gmail.com (mailing list archive)
State Superseded, archived
Headers show

Commit Message

Filipe Manana April 15, 2014, 4:40 p.m. UTC
This increases the send stream version from version 1 to version 2, adding
2 new commands:

1) total data size - used to tell the receiver how much file data the stream
   will add or update;

2) fallocate - used to pre-allocate space for files and to punch holes in files.

This is preparation work for subsequent changes that implement the new features
(computing total data size and use fallocate for better performance).

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
---
 fs/btrfs/send.h            | 13 ++++++++++++-
 include/uapi/linux/btrfs.h | 13 ++++++++++++-
 2 files changed, 24 insertions(+), 2 deletions(-)

Comments

Mark Fasheh April 15, 2014, 5:28 p.m. UTC | #1
On Tue, Apr 15, 2014 at 05:40:18PM +0100, Filipe David Borba Manana wrote:
> This increases the send stream version from version 1 to version 2, adding
> 2 new commands:
> 
> 1) total data size - used to tell the receiver how much file data the stream
>    will add or update;
> 
> 2) fallocate - used to pre-allocate space for files and to punch holes in files.
> 
> This is preparation work for subsequent changes that implement the new features
> (computing total data size and use fallocate for better performance).

Are these changes compatible with software using the old stream version? We
have snapshotting tools that are using send/recieve and it would be bad to
change the ABI in incompatible ways underneath them.
	--Mark

--
Mark Fasheh
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Filipe Manana April 15, 2014, 5:34 p.m. UTC | #2
On Tue, Apr 15, 2014 at 6:28 PM, Mark Fasheh <mfasheh@suse.de> wrote:
> On Tue, Apr 15, 2014 at 05:40:18PM +0100, Filipe David Borba Manana wrote:
>> This increases the send stream version from version 1 to version 2, adding
>> 2 new commands:
>>
>> 1) total data size - used to tell the receiver how much file data the stream
>>    will add or update;
>>
>> 2) fallocate - used to pre-allocate space for files and to punch holes in files.
>>
>> This is preparation work for subsequent changes that implement the new features
>> (computing total data size and use fallocate for better performance).
>
> Are these changes compatible with software using the old stream version? We
> have snapshotting tools that are using send/recieve and it would be bad to
> change the ABI in incompatible ways underneath them.
>         --Mark

New versions of btrfs-progs (send stream v2 support) will still be
able to read and process v1 streams. Older btrfs-progs (v1 only) won't
be able to process the new commands.
Does this answers your question Mark?


>
> --
> Mark Fasheh
Mark Fasheh April 15, 2014, 5:41 p.m. UTC | #3
On Tue, Apr 15, 2014 at 06:34:38PM +0100, Filipe David Manana wrote:
> On Tue, Apr 15, 2014 at 6:28 PM, Mark Fasheh <mfasheh@suse.de> wrote:
> > On Tue, Apr 15, 2014 at 05:40:18PM +0100, Filipe David Borba Manana wrote:
> >> This increases the send stream version from version 1 to version 2, adding
> >> 2 new commands:
> >>
> >> 1) total data size - used to tell the receiver how much file data the stream
> >>    will add or update;
> >>
> >> 2) fallocate - used to pre-allocate space for files and to punch holes in files.
> >>
> >> This is preparation work for subsequent changes that implement the new features
> >> (computing total data size and use fallocate for better performance).
> >
> > Are these changes compatible with software using the old stream version? We
> > have snapshotting tools that are using send/recieve and it would be bad to
> > change the ABI in incompatible ways underneath them.
> >         --Mark
> 
> New versions of btrfs-progs (send stream v2 support) will still be
> able to read and process v1 streams. Older btrfs-progs (v1 only) won't
> be able to process the new commands.
> Does this answers your question Mark?

Yes it does thanks. Unfortunately though this is unacceptable behavior -
kernel upgrades are not supposed to break existing userspace interfaces.

In particular what will happen here is that the user will grab a new kernel
and then find out that their fancy snapshotting software won't work any
more.
	--Mark

--
Mark Fasheh
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Filipe Manana April 15, 2014, 5:57 p.m. UTC | #4
On Tue, Apr 15, 2014 at 6:41 PM, Mark Fasheh <mfasheh@suse.de> wrote:
> On Tue, Apr 15, 2014 at 06:34:38PM +0100, Filipe David Manana wrote:
>> On Tue, Apr 15, 2014 at 6:28 PM, Mark Fasheh <mfasheh@suse.de> wrote:
>> > On Tue, Apr 15, 2014 at 05:40:18PM +0100, Filipe David Borba Manana wrote:
>> >> This increases the send stream version from version 1 to version 2, adding
>> >> 2 new commands:
>> >>
>> >> 1) total data size - used to tell the receiver how much file data the stream
>> >>    will add or update;
>> >>
>> >> 2) fallocate - used to pre-allocate space for files and to punch holes in files.
>> >>
>> >> This is preparation work for subsequent changes that implement the new features
>> >> (computing total data size and use fallocate for better performance).
>> >
>> > Are these changes compatible with software using the old stream version? We
>> > have snapshotting tools that are using send/recieve and it would be bad to
>> > change the ABI in incompatible ways underneath them.
>> >         --Mark
>>
>> New versions of btrfs-progs (send stream v2 support) will still be
>> able to read and process v1 streams. Older btrfs-progs (v1 only) won't
>> be able to process the new commands.
>> Does this answers your question Mark?
>
> Yes it does thanks. Unfortunately though this is unacceptable behavior -
> kernel upgrades are not supposed to break existing userspace interfaces.
>
> In particular what will happen here is that the user will grab a new kernel
> and then find out that their fancy snapshotting software won't work any
> more.

Good point. I followed this approach based on Josef's comments on a
previous rfc at http://www.spinics.net/lists/linux-btrfs/msg32999.html

The only alternative I can think of right now is to use new send ioctl
flags instead, so that new clients able to process the new commands
will pass these flags explicitly, while old clients would continue to
work without changes (and not bumping the stream version, as
btrfs-receive refuses versions higher than 1 currently). This seems to
be similar to what was done here:
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=cb95e7bf7ba481c3d35b238b1cd671b63f54238a

thanks

>         --Mark
>
> --
> Mark Fasheh
Mark Fasheh April 15, 2014, 6:04 p.m. UTC | #5
On Tue, Apr 15, 2014 at 06:57:17PM +0100, Filipe David Manana wrote:
> >> > Are these changes compatible with software using the old stream version? We
> >> > have snapshotting tools that are using send/recieve and it would be bad to
> >> > change the ABI in incompatible ways underneath them.
> >> >         --Mark
> >>
> >> New versions of btrfs-progs (send stream v2 support) will still be
> >> able to read and process v1 streams. Older btrfs-progs (v1 only) won't
> >> be able to process the new commands.
> >> Does this answers your question Mark?
> >
> > Yes it does thanks. Unfortunately though this is unacceptable behavior -
> > kernel upgrades are not supposed to break existing userspace interfaces.
> >
> > In particular what will happen here is that the user will grab a new kernel
> > and then find out that their fancy snapshotting software won't work any
> > more.
> 
> Good point. I followed this approach based on Josef's comments on a
> previous rfc at http://www.spinics.net/lists/linux-btrfs/msg32999.html

Apparently Josef doesn't get those sorts of bugs in his queue ;)


> The only alternative I can think of right now is to use new send ioctl
> flags instead, so that new clients able to process the new commands
> will pass these flags explicitly, while old clients would continue to
> work without changes (and not bumping the stream version, as
> btrfs-receive refuses versions higher than 1 currently). This seems to
> be similar to what was done here:
> https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=cb95e7bf7ba481c3d35b238b1cd671b63f54238a

Yeah that works for me - any client that understand the new features just
sends some flag that indicates it can process them. Then we know that old
clients will continue to work unaffected.

Thanks Filipe,
	--Mark

--
Mark Fasheh
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Josef Bacik April 15, 2014, 6:10 p.m. UTC | #6
Just make a SUPPORTS_V2 flag that userspace can pass in through the existing flags to make the kernel spit out V2 commands.  We don't want to break old user space, I still have to see distro guys in real life ;).  Thanks,

Josef

Mark Fasheh <mfasheh@suse.de> wrote:


On Tue, Apr 15, 2014 at 06:57:17PM +0100, Filipe David Manana wrote:
> >> > Are these changes compatible with software using the old stream version? We
> >> > have snapshotting tools that are using send/recieve and it would be bad to
> >> > change the ABI in incompatible ways underneath them.
> >> >         --Mark
> >>
> >> New versions of btrfs-progs (send stream v2 support) will still be
> >> able to read and process v1 streams. Older btrfs-progs (v1 only) won't
> >> be able to process the new commands.
> >> Does this answers your question Mark?
> >
> > Yes it does thanks. Unfortunately though this is unacceptable behavior -
> > kernel upgrades are not supposed to break existing userspace interfaces.
> >
> > In particular what will happen here is that the user will grab a new kernel
> > and then find out that their fancy snapshotting software won't work any
> > more.
>
> Good point. I followed this approach based on Josef's comments on a
> previous rfc at https://urldefense.proofpoint.com/v1/url?u=http://www.spinics.net/lists/linux-btrfs/msg32999.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=wUTU%2FIE4hdC77M9gBWyPgg1QcqkuVHjEMjOGfI79alY%3D%0A&s=4d77650ee86676b158e0f54ff93c700300f098ae6d86f36fcf61feb70fa2ac78

Apparently Josef doesn't get those sorts of bugs in his queue ;)


> The only alternative I can think of right now is to use new send ioctl
> flags instead, so that new clients able to process the new commands
> will pass these flags explicitly, while old clients would continue to
> work without changes (and not bumping the stream version, as
> btrfs-receive refuses versions higher than 1 currently). This seems to
> be similar to what was done here:
> https://urldefense.proofpoint.com/v1/url?u=https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id%3Dcb95e7bf7ba481c3d35b238b1cd671b63f54238a&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=wUTU%2FIE4hdC77M9gBWyPgg1QcqkuVHjEMjOGfI79alY%3D%0A&s=0e82acdd655c133f52a9717216f7038a09afa1bfefaa91c4d332530a5be4c3d6

Yeah that works for me - any client that understand the new features just
sends some flag that indicates it can process them. Then we know that old
clients will continue to work unaffected.

Thanks Filipe,
        --Mark

--
Mark Fasheh
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Filipe Manana April 15, 2014, 6:30 p.m. UTC | #7
On Tue, Apr 15, 2014 at 7:10 PM, Josef Bacik <jbacik@fb.com> wrote:
> Just make a SUPPORTS_V2 flag that userspace can pass in through the existing flags to make the kernel spit out V2 commands.  We don't want to break old user space, I still have to see distro guys in real life ;).  Thanks,

So would this flag named like "supports_v2" imply to send fallocate
commands and data size computation/command? Right now I made data size
optional, sent only if a new ioctl send flag is set, because for large
fs trees it can take some time to compute data size.

Also, would new btrfs-progs version send that flag (support_v2)
always, without any option to use old v1, or not really that useful?

thanks

>
> Josef
>
> Mark Fasheh <mfasheh@suse.de> wrote:
>
>
> On Tue, Apr 15, 2014 at 06:57:17PM +0100, Filipe David Manana wrote:
>> >> > Are these changes compatible with software using the old stream version? We
>> >> > have snapshotting tools that are using send/recieve and it would be bad to
>> >> > change the ABI in incompatible ways underneath them.
>> >> >         --Mark
>> >>
>> >> New versions of btrfs-progs (send stream v2 support) will still be
>> >> able to read and process v1 streams. Older btrfs-progs (v1 only) won't
>> >> be able to process the new commands.
>> >> Does this answers your question Mark?
>> >
>> > Yes it does thanks. Unfortunately though this is unacceptable behavior -
>> > kernel upgrades are not supposed to break existing userspace interfaces.
>> >
>> > In particular what will happen here is that the user will grab a new kernel
>> > and then find out that their fancy snapshotting software won't work any
>> > more.
>>
>> Good point. I followed this approach based on Josef's comments on a
>> previous rfc at https://urldefense.proofpoint.com/v1/url?u=http://www.spinics.net/lists/linux-btrfs/msg32999.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=wUTU%2FIE4hdC77M9gBWyPgg1QcqkuVHjEMjOGfI79alY%3D%0A&s=4d77650ee86676b158e0f54ff93c700300f098ae6d86f36fcf61feb70fa2ac78
>
> Apparently Josef doesn't get those sorts of bugs in his queue ;)
>
>
>> The only alternative I can think of right now is to use new send ioctl
>> flags instead, so that new clients able to process the new commands
>> will pass these flags explicitly, while old clients would continue to
>> work without changes (and not bumping the stream version, as
>> btrfs-receive refuses versions higher than 1 currently). This seems to
>> be similar to what was done here:
>> https://urldefense.proofpoint.com/v1/url?u=https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id%3Dcb95e7bf7ba481c3d35b238b1cd671b63f54238a&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=wUTU%2FIE4hdC77M9gBWyPgg1QcqkuVHjEMjOGfI79alY%3D%0A&s=0e82acdd655c133f52a9717216f7038a09afa1bfefaa91c4d332530a5be4c3d6
>
> Yeah that works for me - any client that understand the new features just
> sends some flag that indicates it can process them. Then we know that old
> clients will continue to work unaffected.
>
> Thanks Filipe,
>         --Mark
>
> --
> Mark Fasheh
Mark Fasheh April 15, 2014, 7:35 p.m. UTC | #8
On Tue, Apr 15, 2014 at 07:30:51PM +0100, Filipe David Manana wrote:
> On Tue, Apr 15, 2014 at 7:10 PM, Josef Bacik <jbacik@fb.com> wrote:
> > Just make a SUPPORTS_V2 flag that userspace can pass in through the existing flags to make the kernel spit out V2 commands.  We don't want to break old user space, I still have to see distro guys in real life ;).  Thanks,
> 
> So would this flag named like "supports_v2" imply to send fallocate
> commands and data size computation/command? Right now I made data size
> optional, sent only if a new ioctl send flag is set, because for large
> fs trees it can take some time to compute data size.

If it's already backwards-compatibilty you could keep the flag for data size computation as-is
and simply add another flag that gets passed in for the fallocate commands
(UNDERSTANDS_FALLOCATE or something lke that).



> Also, would new btrfs-progs version send that flag (support_v2)
> always, without any option to use old v1, or not really that useful?


As far as btrfs-progs go you could go with many different approaches. For
the library interface to this we might just let the callers determine the
behavior they want. For the command processing (user runs the btrfs program
directly), you could do a couple things:

- perhaps try once with the new flags, if you get an error (EINVAL right now
  for bad flags) try with the old ones and only then throw an error out to
  the caller?

- have user specify whether new flags should be tried or not (this is
  easiest)

I would generally avoid breaking compatibility inside of btrfs-progs too so
just forcing the new flag seems the most 'breaking' option.

Thanks,
	--Mark

--
Mark Fasheh
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Filipe Manana April 15, 2014, 7:49 p.m. UTC | #9
On Tue, Apr 15, 2014 at 8:35 PM, Mark Fasheh <mfasheh@suse.de> wrote:
> On Tue, Apr 15, 2014 at 07:30:51PM +0100, Filipe David Manana wrote:
>> On Tue, Apr 15, 2014 at 7:10 PM, Josef Bacik <jbacik@fb.com> wrote:
>> > Just make a SUPPORTS_V2 flag that userspace can pass in through the existing flags to make the kernel spit out V2 commands.  We don't want to break old user space, I still have to see distro guys in real life ;).  Thanks,
>>
>> So would this flag named like "supports_v2" imply to send fallocate
>> commands and data size computation/command? Right now I made data size
>> optional, sent only if a new ioctl send flag is set, because for large
>> fs trees it can take some time to compute data size.
>
> If it's already backwards-compatibilty you could keep the flag for data size computation as-is
> and simply add another flag that gets passed in for the fallocate commands
> (UNDERSTANDS_FALLOCATE or something lke that).

Yep, exactly what I had in mind.

>
>
>
>> Also, would new btrfs-progs version send that flag (support_v2)
>> always, without any option to use old v1, or not really that useful?
>
>
> As far as btrfs-progs go you could go with many different approaches. For
> the library interface to this we might just let the callers determine the
> behavior they want. For the command processing (user runs the btrfs program
> directly), you could do a couple things:
>
> - perhaps try once with the new flags, if you get an error (EINVAL right now
>   for bad flags) try with the old ones and only then throw an error out to
>   the caller?
>
> - have user specify whether new flags should be tried or not (this is
>   easiest)
>
> I would generally avoid breaking compatibility inside of btrfs-progs too so
> just forcing the new flag seems the most 'breaking' option.

Sounds good. Will probably try first to have the user explicitly pass
a --stream-version=2 for e.g to btrfs send command, with v1 (current)
being the default. I like the idea of clearing the
SUPPORT_FALLOCATE/DATA_SIZE bits from the flags and retry if the ioctl
returns -EINVAL too.

Thanks Mark

>
> Thanks,
>         --Mark
>
> --
> Mark Fasheh
David Sterba April 18, 2014, 5:18 p.m. UTC | #10
On Tue, Apr 15, 2014 at 08:49:52PM +0100, Filipe David Manana wrote:
> On Tue, Apr 15, 2014 at 8:35 PM, Mark Fasheh <mfasheh@suse.de> wrote:
> > On Tue, Apr 15, 2014 at 07:30:51PM +0100, Filipe David Manana wrote:
> >> On Tue, Apr 15, 2014 at 7:10 PM, Josef Bacik <jbacik@fb.com> wrote:
> >> > Just make a SUPPORTS_V2 flag that userspace can pass in through the existing flags to make the kernel spit out V2 commands.  We don't want to break old user space, I still have to see distro guys in real life ;).  Thanks,
> >>
> >> So would this flag named like "supports_v2" imply to send fallocate
> >> commands and data size computation/command? Right now I made data size
> >> optional, sent only if a new ioctl send flag is set, because for large
> >> fs trees it can take some time to compute data size.
> >
> > If it's already backwards-compatibilty you could keep the flag for data size computation as-is
> > and simply add another flag that gets passed in for the fallocate commands
> > (UNDERSTANDS_FALLOCATE or something lke that).
> 
> Yep, exactly what I had in mind.
> 
> > As far as btrfs-progs go you could go with many different approaches. For
> > the library interface to this we might just let the callers determine the
> > behavior they want. For the command processing (user runs the btrfs program
> > directly), you could do a couple things:
> >
> > - perhaps try once with the new flags, if you get an error (EINVAL right now
> >   for bad flags) try with the old ones and only then throw an error out to
> >   the caller?
> >
> > - have user specify whether new flags should be tried or not (this is
> >   easiest)
> >
> > I would generally avoid breaking compatibility inside of btrfs-progs too so
> > just forcing the new flag seems the most 'breaking' option.
> 
> Sounds good. Will probably try first to have the user explicitly pass
> a --stream-version=2 for e.g to btrfs send command, with v1 (current)
> being the default. I like the idea of clearing the
> SUPPORT_FALLOCATE/DATA_SIZE bits from the flags and retry if the ioctl
> returns -EINVAL too.

The fallocate and progress flags are of different nature. While
fallocate should be used by default with v2 (because it's more efficient
and the right thing to do), the progress is optional.

So if user selects protocol version=2, he gets the falloc rightaway.
Ie. no flag like VERSION_V2 or UNDERSTANDS_FALLOCATE, this should be for
free under "STREAM_VERSION=2".

(I'm not sure if the falloc should be optional at all in v2.)

The progress decreases performance and I think it should be off by
default.

The remaining v2-draft features are OTIME, file flags and single-file
clones - all of them are not optional and would be good to add them to
v2.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Filipe Manana April 18, 2014, 7:58 p.m. UTC | #11
On Fri, Apr 18, 2014 at 6:18 PM, David Sterba <dsterba@suse.cz> wrote:
> On Tue, Apr 15, 2014 at 08:49:52PM +0100, Filipe David Manana wrote:
>> On Tue, Apr 15, 2014 at 8:35 PM, Mark Fasheh <mfasheh@suse.de> wrote:
>> > On Tue, Apr 15, 2014 at 07:30:51PM +0100, Filipe David Manana wrote:
>> >> On Tue, Apr 15, 2014 at 7:10 PM, Josef Bacik <jbacik@fb.com> wrote:
>> >> > Just make a SUPPORTS_V2 flag that userspace can pass in through the existing flags to make the kernel spit out V2 commands.  We don't want to break old user space, I still have to see distro guys in real life ;).  Thanks,
>> >>
>> >> So would this flag named like "supports_v2" imply to send fallocate
>> >> commands and data size computation/command? Right now I made data size
>> >> optional, sent only if a new ioctl send flag is set, because for large
>> >> fs trees it can take some time to compute data size.
>> >
>> > If it's already backwards-compatibilty you could keep the flag for data size computation as-is
>> > and simply add another flag that gets passed in for the fallocate commands
>> > (UNDERSTANDS_FALLOCATE or something lke that).
>>
>> Yep, exactly what I had in mind.
>>
>> > As far as btrfs-progs go you could go with many different approaches. For
>> > the library interface to this we might just let the callers determine the
>> > behavior they want. For the command processing (user runs the btrfs program
>> > directly), you could do a couple things:
>> >
>> > - perhaps try once with the new flags, if you get an error (EINVAL right now
>> >   for bad flags) try with the old ones and only then throw an error out to
>> >   the caller?
>> >
>> > - have user specify whether new flags should be tried or not (this is
>> >   easiest)
>> >
>> > I would generally avoid breaking compatibility inside of btrfs-progs too so
>> > just forcing the new flag seems the most 'breaking' option.
>>
>> Sounds good. Will probably try first to have the user explicitly pass
>> a --stream-version=2 for e.g to btrfs send command, with v1 (current)
>> being the default. I like the idea of clearing the
>> SUPPORT_FALLOCATE/DATA_SIZE bits from the flags and retry if the ioctl
>> returns -EINVAL too.
>
> The fallocate and progress flags are of different nature. While
> fallocate should be used by default with v2 (because it's more efficient
> and the right thing to do), the progress is optional.
>
> So if user selects protocol version=2, he gets the falloc rightaway.
> Ie. no flag like VERSION_V2 or UNDERSTANDS_FALLOCATE, this should be for
> free under "STREAM_VERSION=2".
>
> (I'm not sure if the falloc should be optional at all in v2.)

Fine for me, the only optional one in my opinion is the total data
size calculation (and if requested, sets stream version to 2
automatically).

>
> The progress decreases performance and I think it should be off by
> default.
>
> The remaining v2-draft features are OTIME, file flags and single-file
> clones - all of them are not optional and would be good to add them to
> v2.

I'll add stubs/interfaces for them (kernel and btrfs-progs), and
implement them (kernel side and btrfs-progs) later (or perhaps someone
else might want to get his/her hands dirty too).

Thanks David
diff mbox

Patch

diff --git a/fs/btrfs/send.h b/fs/btrfs/send.h
index 48d425a..7b5df08 100644
--- a/fs/btrfs/send.h
+++ b/fs/btrfs/send.h
@@ -20,7 +20,7 @@ 
 #include "ctree.h"
 
 #define BTRFS_SEND_STREAM_MAGIC "btrfs-stream"
-#define BTRFS_SEND_STREAM_VERSION 1
+#define BTRFS_SEND_STREAM_VERSION 2
 
 #define BTRFS_SEND_BUF_SIZE (1024 * 64)
 #define BTRFS_SEND_READ_SIZE (1024 * 48)
@@ -87,6 +87,11 @@  enum btrfs_send_cmd {
 
 	BTRFS_SEND_C_END,
 	BTRFS_SEND_C_UPDATE_EXTENT,
+
+	/* added in stream version 2 */
+	BTRFS_SEND_C_TOTAL_DATA_SIZE,
+	BTRFS_SEND_C_FALLOCATE,
+
 	__BTRFS_SEND_C_MAX,
 };
 #define BTRFS_SEND_C_MAX (__BTRFS_SEND_C_MAX - 1)
@@ -125,10 +130,16 @@  enum {
 	BTRFS_SEND_A_CLONE_OFFSET,
 	BTRFS_SEND_A_CLONE_LEN,
 
+	/* added in stream version 2 */
+	BTRFS_SEND_A_FALLOCATE_FLAGS,
+
 	__BTRFS_SEND_A_MAX,
 };
 #define BTRFS_SEND_A_MAX (__BTRFS_SEND_A_MAX - 1)
 
+#define BTRFS_SEND_A_FALLOCATE_FLAG_KEEP_SIZE   (1 << 0)
+#define BTRFS_SEND_A_FALLOCATE_FLAG_PUNCH_HOLE  (1 << 1)
+
 #ifdef __KERNEL__
 long btrfs_ioctl_send(struct file *mnt_file, void __user *arg);
 #endif
diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h
index b4d6909..444893d 100644
--- a/include/uapi/linux/btrfs.h
+++ b/include/uapi/linux/btrfs.h
@@ -464,10 +464,21 @@  struct btrfs_ioctl_received_subvol_args {
  */
 #define BTRFS_SEND_FLAG_OMIT_END_CMD		0x4
 
+/*
+ * Calculate the amount (in bytes) of new file data between the send and
+ * parent snapshots, or in case of a full send, the total amount of file data
+ * we will send.
+ * This corresponds to the sum of the data lengths of each write, clone and
+ * fallocate commands that are sent through the send stream. The receiving end
+ * can use this information to compute progress.
+ */
+#define BTRFS_SEND_FLAG_CALCULATE_DATA_SIZE	0x8
+
 #define BTRFS_SEND_FLAG_MASK \
 	(BTRFS_SEND_FLAG_NO_FILE_DATA | \
 	 BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \
-	 BTRFS_SEND_FLAG_OMIT_END_CMD)
+	 BTRFS_SEND_FLAG_OMIT_END_CMD | \
+	 BTRFS_SEND_FLAG_CALCULATE_DATA_SIZE)
 
 struct btrfs_ioctl_send_args {
 	__s64 send_fd;			/* in */