mbox series

[v4,0/6] btrfs: simple quotas fstests

Message ID cover.1695942727.git.boris@bur.io (mailing list archive)
Headers show
Series btrfs: simple quotas fstests | expand

Message

Boris Burkov Sept. 28, 2023, 11:16 p.m. UTC
Add a new test for specific squota scenarios (btrfs/301)

Also made modifications for passing existing qgroups tests when possible
and for passing all tests with simple quota enabled via mkfs and with
squota-aware `btrfs check`. Since this required reading sysfs files of
scratch fses, did a bit of refactoring to make those checks target a
device rather than assuming TEST_DEV.

btrfs/301 depends on the kernel patchset:
https://lore.kernel.org/linux-btrfs/cover.1694563454.git.boris@bur.io/
and the btrfs-progs patchset:
https://lore.kernel.org/linux-btrfs/cover.1695836680.git.boris@bur.io/
(and config appropriate binaries to use squota-aware versions)
---
Changelog:
v4:
- fix rescan helper bug
- fix broken tab/spaces in squota helper
- cleanup comments
- improve test names, add some comments
- switch to remount commit=1 for forcing cleaner
- fix group list for 301
- use reflink helpers
- output errors to 301.out (and have expected ones there waiting)
- cleanup "/dev/ksmg" writes I missed when grepping for /dev/kmsg 
- cleanup variable names
- proper fio/btrfs/xfs_io requires
- read nodesize from dump_super
- sync before dump_tree
- documented all calls to sync
v3:
- change btrfs/400 to btrfs/301
v2:
- new sysfs helpers in common
- better gating for the new squota test
- fix various formatting issues
- get rid of noisy dmesg logging


Boris Burkov (6):
  common: refactor sysfs_attr functions
  btrfs: quota mode helpers
  btrfs/301: new test for simple quotas
  btrfs: quota rescan helpers
  btrfs: use new rescan wrapper
  btrfs: skip squota incompatible tests

 common/btrfs        |  56 ++++++
 common/rc           | 127 ++++++++-----
 tests/btrfs/017     |   1 +
 tests/btrfs/022     |   1 +
 tests/btrfs/028     |   2 +-
 tests/btrfs/057     |   1 +
 tests/btrfs/091     |   3 +-
 tests/btrfs/104     |   2 +-
 tests/btrfs/123     |   2 +-
 tests/btrfs/126     |   2 +-
 tests/btrfs/139     |   2 +-
 tests/btrfs/153     |   2 +-
 tests/btrfs/171     |   6 +-
 tests/btrfs/179     |   2 +-
 tests/btrfs/180     |   2 +-
 tests/btrfs/190     |   2 +-
 tests/btrfs/193     |   2 +-
 tests/btrfs/210     |   2 +-
 tests/btrfs/224     |   6 +-
 tests/btrfs/230     |   2 +-
 tests/btrfs/232     |   2 +-
 tests/btrfs/301     | 435 ++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/301.out |  18 ++
 23 files changed, 615 insertions(+), 65 deletions(-)
 create mode 100755 tests/btrfs/301
 create mode 100644 tests/btrfs/301.out

Comments

Anand Jain Sept. 30, 2023, 9:45 a.m. UTC | #1
On 29/09/2023 07:16, Boris Burkov wrote:
> Add a new test for specific squota scenarios (btrfs/301)
> 
> Also made modifications for passing existing qgroups tests when possible
> and for passing all tests with simple quota enabled via mkfs and with
> squota-aware `btrfs check`. Since this required reading sysfs files of
> scratch fses, did a bit of refactoring to make those checks target a
> device rather than assuming TEST_DEV.
> 
> btrfs/301 depends on the kernel patchset:
> https://lore.kernel.org/linux-btrfs/cover.1694563454.git.boris@bur.io/
> and the btrfs-progs patchset:
> https://lore.kernel.org/linux-btrfs/cover.1695836680.git.boris@bur.io/
> (and config appropriate binaries to use squota-aware versions)
> ---
> Changelog:
> v4:
> - fix rescan helper bug
> - fix broken tab/spaces in squota helper
> - cleanup comments
> - improve test names, add some comments
> - switch to remount commit=1 for forcing cleaner
> - fix group list for 301
> - use reflink helpers
> - output errors to 301.out (and have expected ones there waiting)
> - cleanup "/dev/ksmg" writes I missed when grepping for /dev/kmsg
> - cleanup variable names
> - proper fio/btrfs/xfs_io requires
> - read nodesize from dump_super
> - sync before dump_tree
> - documented all calls to sync
> v3:
> - change btrfs/400 to btrfs/301
> v2:
> - new sysfs helpers in common
> - better gating for the new squota test
> - fix various formatting issues
> - get rid of noisy dmesg logging
> 
> 
> Boris Burkov (6):
>    common: refactor sysfs_attr functions
>    btrfs: quota mode helpers
>    btrfs/301: new test for simple quotas
>    btrfs: quota rescan helpers
>    btrfs: use new rescan wrapper
>    btrfs: skip squota incompatible tests
> 
>   common/btrfs        |  56 ++++++
>   common/rc           | 127 ++++++++-----
>   tests/btrfs/017     |   1 +
>   tests/btrfs/022     |   1 +
>   tests/btrfs/028     |   2 +-
>   tests/btrfs/057     |   1 +
>   tests/btrfs/091     |   3 +-
>   tests/btrfs/104     |   2 +-
>   tests/btrfs/123     |   2 +-
>   tests/btrfs/126     |   2 +-
>   tests/btrfs/139     |   2 +-
>   tests/btrfs/153     |   2 +-
>   tests/btrfs/171     |   6 +-
>   tests/btrfs/179     |   2 +-
>   tests/btrfs/180     |   2 +-
>   tests/btrfs/190     |   2 +-
>   tests/btrfs/193     |   2 +-
>   tests/btrfs/210     |   2 +-
>   tests/btrfs/224     |   6 +-
>   tests/btrfs/230     |   2 +-
>   tests/btrfs/232     |   2 +-
>   tests/btrfs/301     | 435 ++++++++++++++++++++++++++++++++++++++++++++
>   tests/btrfs/301.out |  18 ++
>   23 files changed, 615 insertions(+), 65 deletions(-)
>   create mode 100755 tests/btrfs/301
>   create mode 100644 tests/btrfs/301.out
> 

All test cases modified here run fine on a system with PAGESIZE=64K,
except for btrfs/153 with and without MKFS_OPTIONS="-O squota".
btrfs/153 is successful on kernel v5.15 (I haven't tried other kernels)

btrfs/153 2s ... - output mismatch (see 
/xfstests-dev/results//btrfs/153.out.bad)
     --- tests/btrfs/153.out	2021-07-13 08:07:54.096971521 -0400
     +++ /xfstests-dev/results//btrfs/153.out.bad	2023-09-30 
05:33:53.560640258 -0400
     @@ -1,2 +1,3 @@
      QA output created by 153
     +pwrite: Disk quota exceeded
      Silence is golden
     ...
     (Run 'diff -u /xfstests-dev/tests/btrfs/153.out 
/xfstests-dev/results//btrfs/153.out.bad'  to see the entire diff)


Thanks, Anand
Anand Jain Oct. 5, 2023, 8:08 a.m. UTC | #2
On 9/30/23 17:45, Anand Jain wrote:
> On 29/09/2023 07:16, Boris Burkov wrote:
>> Add a new test for specific squota scenarios (btrfs/301)
>>
>> Also made modifications for passing existing qgroups tests when possible
>> and for passing all tests with simple quota enabled via mkfs and with
>> squota-aware `btrfs check`. Since this required reading sysfs files of
>> scratch fses, did a bit of refactoring to make those checks target a
>> device rather than assuming TEST_DEV.
>>
>> btrfs/301 depends on the kernel patchset:
>> https://lore.kernel.org/linux-btrfs/cover.1694563454.git.boris@bur.io/
>> and the btrfs-progs patchset:
>> https://lore.kernel.org/linux-btrfs/cover.1695836680.git.boris@bur.io/
>> (and config appropriate binaries to use squota-aware versions)
>> ---
>> Changelog:
>> v4:
>> - fix rescan helper bug
>> - fix broken tab/spaces in squota helper
>> - cleanup comments
>> - improve test names, add some comments
>> - switch to remount commit=1 for forcing cleaner
>> - fix group list for 301
>> - use reflink helpers
>> - output errors to 301.out (and have expected ones there waiting)
>> - cleanup "/dev/ksmg" writes I missed when grepping for /dev/kmsg
>> - cleanup variable names
>> - proper fio/btrfs/xfs_io requires
>> - read nodesize from dump_super
>> - sync before dump_tree
>> - documented all calls to sync
>> v3:
>> - change btrfs/400 to btrfs/301
>> v2:
>> - new sysfs helpers in common
>> - better gating for the new squota test
>> - fix various formatting issues
>> - get rid of noisy dmesg logging
>>
>>
>> Boris Burkov (6):
>>    common: refactor sysfs_attr functions
>>    btrfs: quota mode helpers
>>    btrfs/301: new test for simple quotas
>>    btrfs: quota rescan helpers
>>    btrfs: use new rescan wrapper
>>    btrfs: skip squota incompatible tests
>>
>>   common/btrfs        |  56 ++++++
>>   common/rc           | 127 ++++++++-----
>>   tests/btrfs/017     |   1 +
>>   tests/btrfs/022     |   1 +
>>   tests/btrfs/028     |   2 +-
>>   tests/btrfs/057     |   1 +
>>   tests/btrfs/091     |   3 +-
>>   tests/btrfs/104     |   2 +-
>>   tests/btrfs/123     |   2 +-
>>   tests/btrfs/126     |   2 +-
>>   tests/btrfs/139     |   2 +-
>>   tests/btrfs/153     |   2 +-
>>   tests/btrfs/171     |   6 +-
>>   tests/btrfs/179     |   2 +-
>>   tests/btrfs/180     |   2 +-
>>   tests/btrfs/190     |   2 +-
>>   tests/btrfs/193     |   2 +-
>>   tests/btrfs/210     |   2 +-
>>   tests/btrfs/224     |   6 +-
>>   tests/btrfs/230     |   2 +-
>>   tests/btrfs/232     |   2 +-
>>   tests/btrfs/301     | 435 ++++++++++++++++++++++++++++++++++++++++++++
>>   tests/btrfs/301.out |  18 ++
>>   23 files changed, 615 insertions(+), 65 deletions(-)
>>   create mode 100755 tests/btrfs/301
>>   create mode 100644 tests/btrfs/301.out
>>
> 
> All test cases modified here run fine on a system with PAGESIZE=64K,
> except for btrfs/153 with and without MKFS_OPTIONS="-O squota".
> btrfs/153 is successful on kernel v5.15 (I haven't tried other kernels)
> 
> btrfs/153 2s ... - output mismatch (see 
> /xfstests-dev/results//btrfs/153.out.bad)
>      --- tests/btrfs/153.out    2021-07-13 08:07:54.096971521 -0400
>      +++ /xfstests-dev/results//btrfs/153.out.bad    2023-09-30 
> 05:33:53.560640258 -0400
>      @@ -1,2 +1,3 @@
>       QA output created by 153
>      +pwrite: Disk quota exceeded
>       Silence is golden
>      ...
>      (Run 'diff -u /xfstests-dev/tests/btrfs/153.out 
> /xfstests-dev/results//btrfs/153.out.bad'  to see the entire diff)
> 

The test case works with the standard 4k page size. It's already in
the Staged branch. Please send any required fix patches when they're
ready to be applied on top of this set.

Thanks, Anand