[00/10] btrfs-progs: my libbtrfsutil patch queue
mbox series

Message ID cover.1542181521.git.osandov@fb.com
Headers show
Series
  • btrfs-progs: my libbtrfsutil patch queue
Related show

Message

Omar Sandoval Nov. 14, 2018, 7:46 a.m. UTC
From: Omar Sandoval <osandov@fb.com>

Hi,

This series contains my backlog of libbtrfsutil changes which I've been
collecting over the past few weeks.

Patches 1-4 are fixes. Patches 5-6 add functionality to the unit tests
which is needed for patches 7-8. Patches 7-8 add support for the
unprivileged ioctls added in Linux 4.18; more on those below. Patch 9
bumps the library version. Patch 10 adds documentation for the available
API along with examples.

Patches 7-8 are based on Misono Tomohiro's previous patch series [1],
with a few important changes.

- Both subvolume_info() and create_subvolume_iterator() now have unit
  tests for the unprivileged case.
- Both no longer explicitly check that top == 0 in the unprivileged
  case, since that will already fail with a clear permission error.
- Unprivileged iteration is much simpler: it uses openat() instead of
  fchdir() and is based more closely on the original tree search
  variant. This fixes a bug in post-order iteration in Misono's version.
- Unprivileged iteration does _not_ support passing in a non-subvolume
  path; if this behavior is desired, I'd like it to be a separate change
  with an explicit flag.

Please take a look.

Thanks!

1: https://www.spinics.net/lists/linux-btrfs/msg79285.html

Omar Sandoval (10):
  libbtrfsutil: use top=0 as default for SubvolumeIterator()
  libbtrfsutil: change async parameters to async_ in Python bindings
  libbtrfsutil: document qgroup_inherit parameter in Python bindings
  libbtrfsutil: use SubvolumeIterator as context manager in tests
  libbtrfsutil: add test helpers for dropping privileges
  libbtrfsutil: allow tests to create multiple Btrfs instances
  libbtrfsutil: relax the privileges of subvolume_info()
  libbtrfsutil: relax the privileges of subvolume iterator
  libbtrfsutil: bump version to 1.1.0
  libbtrfsutil: document API in README

 libbtrfsutil/README.md                      | 422 +++++++++++++++++++-
 libbtrfsutil/btrfsutil.h                    |  21 +-
 libbtrfsutil/errors.c                       |   8 +
 libbtrfsutil/python/module.c                |  17 +-
 libbtrfsutil/python/subvolume.c             |   6 +-
 libbtrfsutil/python/tests/__init__.py       |  66 ++-
 libbtrfsutil/python/tests/test_subvolume.py | 215 +++++++---
 libbtrfsutil/subvolume.c                    | 407 ++++++++++++++++---
 8 files changed, 1029 insertions(+), 133 deletions(-)

Comments

David Sterba Nov. 26, 2018, 4:18 p.m. UTC | #1
On Tue, Nov 13, 2018 at 11:46:55PM -0800, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
> 
> Hi,
> 
> This series contains my backlog of libbtrfsutil changes which I've been
> collecting over the past few weeks.
> 
> Patches 1-4 are fixes. Patches 5-6 add functionality to the unit tests
> which is needed for patches 7-8. Patches 7-8 add support for the
> unprivileged ioctls added in Linux 4.18; more on those below. Patch 9
> bumps the library version. Patch 10 adds documentation for the available
> API along with examples.
> 
> Patches 7-8 are based on Misono Tomohiro's previous patch series [1],
> with a few important changes.
> 
> - Both subvolume_info() and create_subvolume_iterator() now have unit
>   tests for the unprivileged case.
> - Both no longer explicitly check that top == 0 in the unprivileged
>   case, since that will already fail with a clear permission error.
> - Unprivileged iteration is much simpler: it uses openat() instead of
>   fchdir() and is based more closely on the original tree search
>   variant. This fixes a bug in post-order iteration in Misono's version.
> - Unprivileged iteration does _not_ support passing in a non-subvolume
>   path; if this behavior is desired, I'd like it to be a separate change
>   with an explicit flag.

Series merged to devel, thanks. I've added link from the main README now
that there's the API documentation.

The test-libbtrfsutil is missing from the travis CI for some reason, I
was about to add it.  So far the testing environment does not provide
'umount' that knows about '-R' so the tests fail. I'll have a look if
there's a newer base image provided, otherwise a workaround would be
necessary.

As for the unprivileged subvolume listing ioctls, the functionality in
the util library is self-contained and the interface is up to you to
design properly, so this does not depend on the 'btrfs subvolume list'
command. That one has unfortunately not bubbled high enough in my todo.
Omar Sandoval Nov. 26, 2018, 5:15 p.m. UTC | #2
On Mon, Nov 26, 2018 at 05:18:12PM +0100, David Sterba wrote:
> On Tue, Nov 13, 2018 at 11:46:55PM -0800, Omar Sandoval wrote:
> > From: Omar Sandoval <osandov@fb.com>
> > 
> > Hi,
> > 
> > This series contains my backlog of libbtrfsutil changes which I've been
> > collecting over the past few weeks.
> > 
> > Patches 1-4 are fixes. Patches 5-6 add functionality to the unit tests
> > which is needed for patches 7-8. Patches 7-8 add support for the
> > unprivileged ioctls added in Linux 4.18; more on those below. Patch 9
> > bumps the library version. Patch 10 adds documentation for the available
> > API along with examples.
> > 
> > Patches 7-8 are based on Misono Tomohiro's previous patch series [1],
> > with a few important changes.
> > 
> > - Both subvolume_info() and create_subvolume_iterator() now have unit
> >   tests for the unprivileged case.
> > - Both no longer explicitly check that top == 0 in the unprivileged
> >   case, since that will already fail with a clear permission error.
> > - Unprivileged iteration is much simpler: it uses openat() instead of
> >   fchdir() and is based more closely on the original tree search
> >   variant. This fixes a bug in post-order iteration in Misono's version.
> > - Unprivileged iteration does _not_ support passing in a non-subvolume
> >   path; if this behavior is desired, I'd like it to be a separate change
> >   with an explicit flag.
> 
> Series merged to devel, thanks.

Thanks!

> I've added link from the main README now
> that there's the API documentation.

Ah, great idea.

> The test-libbtrfsutil is missing from the travis CI for some reason, I
> was about to add it.  So far the testing environment does not provide
> 'umount' that knows about '-R' so the tests fail. I'll have a look if
> there's a newer base image provided, otherwise a workaround would be
> necessary.

It looks like it was added to util-linux in v2.23 back in 2013. Or maybe
the base image uses busybox? I believe that umount from busybox doesn't
have -R.

> As for the unprivileged subvolume listing ioctls, the functionality in
> the util library is self-contained and the interface is up to you to
> design properly, so this does not depend on the 'btrfs subvolume list'
> command. That one has unfortunately not bubbled high enough in my todo.

That comment is mostly for Misono, since the original version had that
functionality, probably for the subvolume list command.
misono.tomohiro@fujitsu.com Nov. 27, 2018, 2:51 a.m. UTC | #3
> -----Original Message-----
> From: David Sterba [mailto:dsterba@suse.cz]
> Sent: Tuesday, November 27, 2018 1:18 AM
> To: Omar Sandoval <osandov@osandov.com>
> Cc: linux-btrfs@vger.kernel.org; kernel-team@fb.com; Misono, Tomohiro
> <misono.tomohiro@fujitsu.com>
> Subject: Re: [PATCH 00/10] btrfs-progs: my libbtrfsutil patch queue
> 
> On Tue, Nov 13, 2018 at 11:46:55PM -0800, Omar Sandoval wrote:
> > From: Omar Sandoval <osandov@fb.com>
> >
> > Hi,
> >
> > This series contains my backlog of libbtrfsutil changes which I've been
> > collecting over the past few weeks.
> >
> > Patches 1-4 are fixes. Patches 5-6 add functionality to the unit tests
> > which is needed for patches 7-8. Patches 7-8 add support for the
> > unprivileged ioctls added in Linux 4.18; more on those below. Patch
> 9
> > bumps the library version. Patch 10 adds documentation for the available
> > API along with examples.
> >
> > Patches 7-8 are based on Misono Tomohiro's previous patch series [1],
> > with a few important changes.
> >
> > - Both subvolume_info() and create_subvolume_iterator() now have unit
> >   tests for the unprivileged case.
> > - Both no longer explicitly check that top == 0 in the unprivileged
> >   case, since that will already fail with a clear permission error.
> > - Unprivileged iteration is much simpler: it uses openat() instead of
> >   fchdir() and is based more closely on the original tree search
> >   variant. This fixes a bug in post-order iteration in Misono's version.
> > - Unprivileged iteration does _not_ support passing in a non-subvolume
> >   path; if this behavior is desired, I'd like it to be a separate change
> >   with an explicit flag.
> 
> Series merged to devel, thanks. I've added link from the main README now
> that there's the API documentation.
> 
> The test-libbtrfsutil is missing from the travis CI for some reason, I
> was about to add it.  So far the testing environment does not provide
> 'umount' that knows about '-R' so the tests fail. I'll have a look if
> there's a newer base image provided, otherwise a workaround would be
> necessary.
> 
> As for the unprivileged subvolume listing ioctls, the functionality in
> the util library is self-contained and the interface is up to you to
> design properly, so this does not depend on the 'btrfs subvolume list'
> command. That one has unfortunately not bubbled high enough in my todo.

Hello,
I missed the mails and am sorry for late response.

As mentioned libbtrfsuitl and other progs are mostly independent,
the patches I submitted (once in devel with your review and some modification)
can be cleanly applied to this version of libbtrfsutil.

I will resend them for easier review/comment.

Thanks,
Misono