[v2,1/3] xfstests/btrfs: add qgroup rescan stress test
diff mbox

Message ID 1394423051-17356-1-git-send-email-wangsl.fnst@cn.fujitsu.com
State Not Applicable
Headers show

Commit Message

Wang Shilong March 10, 2014, 3:44 a.m. UTC
Test flow is to run fsstress after triggering quota rescan.
the ruler is simple, we just remove all files and directories,
sync filesystem and see if qgroup's ref and excl are nodesize.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
---
v1->v2:
	switch into new helper _run_btrfs_util_prog()
---
 tests/btrfs/041     | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/041.out |  3 +++
 tests/btrfs/group   |  1 +
 3 files changed, 80 insertions(+)
 create mode 100644 tests/btrfs/041
 create mode 100644 tests/btrfs/041.out

Comments

Josef Bacik March 10, 2014, 7:43 p.m. UTC | #1
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/09/2014 11:44 PM, Wang Shilong wrote:
> Test flow is to run fsstress after triggering quota rescan. the
> ruler is simple, we just remove all files and directories, sync
> filesystem and see if qgroup's ref and excl are nodesize.
> 
> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com> --- 
> v1->v2: switch into new helper _run_btrfs_util_prog() --- 
> tests/btrfs/041     | 76
> +++++++++++++++++++++++++++++++++++++++++++++++++++++ 
> tests/btrfs/041.out |  3 +++ tests/btrfs/group   |  1 + 3 files
> changed, 80 insertions(+) create mode 100644 tests/btrfs/041 create
> mode 100644 tests/btrfs/041.out
> 

So this doesn't pass for me but it's not because we're broken, it's
because the test is broken.  What you should do is run sync after you
do the rm, and then capture the output of btrfs qgroup show, and then
do the rescan and redo qgroup show and make sure the numbers match.  I
did this in my qgroup test, you can look there as an example.  Thanks,

Josef

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTHhXkAAoJEANb+wAKly3BQI8QAIEFXMPrFEJxj9uKuwnBZSSC
r4g7pHyac82EJA3T/jx+KIzH3aeUz0r1mEg8sYXXACAW5BZguyhe1LR0rU2ZBDaP
9ilgWwCVm3d3EhiDmZJDCZueB7IXqE4/GX6D1zyF9OxVNk3Ng9gOY0/3G7Rb52FI
UTT/UhA8f3mFHQq4DbOebfacu8wbMH9rjHucxgin63UQLx9J9k+3lv5Y7AHihSrX
3ly+B9+wqu9zl8gQGhxyvdayWeka6Hr43lkniaOehuW118WjjwIivE/CbTkla/3f
P3hoJSbUiqlCPONKkfGExYsIny0y9I3igMnLqs58RLeyd4+iSF1O4wFyFGn6hnDy
sazS8mz4V3PqnfHAj0D3scSYVyYvcrHcUU5WGs+0JiyExfQPFSrbybfwncIgbBg6
+63iqd7qHwwjClSVNckFurU4B+lj1aVyyk/S1/Z2jN/9s3nrM3/WAR3cqJGIuNyT
RPT9mzmkc1OICO0+dvwHMol894CCVh9b0iQ6e2NBWYmEfv3BIj9tZNjUZ/ySJdQC
h2f00uLovRPazI/X/gKVraHj13Go3giL5LCIbnUso/B18iA2V3qyyXGPMBTB1Rws
wzaxlXsC8cZsMUgFkg7z6Q1Kc9hyUj4AF3Vg/qZze4F216LLAOa9Bni5CxKOVytk
MPlXItpXYAEX8orapQdO
=/ZGH
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Wang Shilong March 11, 2014, 1:19 a.m. UTC | #2
On 03/11/2014 03:43 AM, Josef Bacik wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 03/09/2014 11:44 PM, Wang Shilong wrote:
>> Test flow is to run fsstress after triggering quota rescan. the
>> ruler is simple, we just remove all files and directories, sync
>> filesystem and see if qgroup's ref and excl are nodesize.
>>
>> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com> ---
>> v1->v2: switch into new helper _run_btrfs_util_prog() ---
>> tests/btrfs/041     | 76
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/041.out |  3 +++ tests/btrfs/group   |  1 + 3 files
>> changed, 80 insertions(+) create mode 100644 tests/btrfs/041 create
>> mode 100644 tests/btrfs/041.out
>>
> So this doesn't pass for me but it's not because we're broken, it's
> because the test is broken.  What you should do is run sync after you
> do the rm, and then capture the output of btrfs qgroup show, and then
> do the rescan and redo qgroup show and make sure the numbers match.  I
> did this in my qgroup test, you can look there as an example.  Thanks,
Let's take a look at my test flows:

step1: write some random data to fs tree and then create a snapshot of 
fs tree.

step2: also try to write some random data to newly created snapshot.

step3: enable quota and wait quota rescan finished.(btrfs quota rescan -w)

step4: remove all files and sync btrfs filesystem.

So i did a simple test that if we don't create snapshot here, i get 
qgroup accounting
right here, so i think it is realted to qgroup accounting that differs 
'refer' and 'excl'.

Thanks,
Wang
>
> Josef
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBAgAGBQJTHhXkAAoJEANb+wAKly3BQI8QAIEFXMPrFEJxj9uKuwnBZSSC
> r4g7pHyac82EJA3T/jx+KIzH3aeUz0r1mEg8sYXXACAW5BZguyhe1LR0rU2ZBDaP
> 9ilgWwCVm3d3EhiDmZJDCZueB7IXqE4/GX6D1zyF9OxVNk3Ng9gOY0/3G7Rb52FI
> UTT/UhA8f3mFHQq4DbOebfacu8wbMH9rjHucxgin63UQLx9J9k+3lv5Y7AHihSrX
> 3ly+B9+wqu9zl8gQGhxyvdayWeka6Hr43lkniaOehuW118WjjwIivE/CbTkla/3f
> P3hoJSbUiqlCPONKkfGExYsIny0y9I3igMnLqs58RLeyd4+iSF1O4wFyFGn6hnDy
> sazS8mz4V3PqnfHAj0D3scSYVyYvcrHcUU5WGs+0JiyExfQPFSrbybfwncIgbBg6
> +63iqd7qHwwjClSVNckFurU4B+lj1aVyyk/S1/Z2jN/9s3nrM3/WAR3cqJGIuNyT
> RPT9mzmkc1OICO0+dvwHMol894CCVh9b0iQ6e2NBWYmEfv3BIj9tZNjUZ/ySJdQC
> h2f00uLovRPazI/X/gKVraHj13Go3giL5LCIbnUso/B18iA2V3qyyXGPMBTB1Rws
> wzaxlXsC8cZsMUgFkg7z6Q1Kc9hyUj4AF3Vg/qZze4F216LLAOa9Bni5CxKOVytk
> MPlXItpXYAEX8orapQdO
> =/ZGH
> -----END PGP SIGNATURE-----
>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Josef Bacik May 7, 2014, 8:58 p.m. UTC | #3
On 03/09/2014 11:44 PM, Wang Shilong wrote:
> Test flow is to run fsstress after triggering quota rescan.
> the ruler is simple, we just remove all files and directories,
> sync filesystem and see if qgroup's ref and excl are nodesize.
>
> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
> ---
> v1->v2:
> 	switch into new helper _run_btrfs_util_prog()
> ---
>   tests/btrfs/041     | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>   tests/btrfs/041.out |  3 +++
>   tests/btrfs/group   |  1 +
>   3 files changed, 80 insertions(+)
>   create mode 100644 tests/btrfs/041
>   create mode 100644 tests/btrfs/041.out
>
> diff --git a/tests/btrfs/041 b/tests/btrfs/041
> new file mode 100644
> index 0000000..92bd080
> --- /dev/null
> +++ b/tests/btrfs/041
> @@ -0,0 +1,76 @@
> +#! /bin/bash
> +# FSQA Test No. btrfs/041
> +#
> +# Quota rescan stress test, we run fsstress and quota rescan concurrently
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (C) 2014 Fujitsu.  All rights reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_need_to_be_root
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +run_check _scratch_mkfs "-b 1g --nodesize 4096"
> +run_check _scratch_mount
> +

Add "-o nospace_cache" here please, otherwise I don't get the same output.

> +# -w ensures that the only ops are ones which cause write I/O
> +run_check $FSSTRESS_PROG -d $SCRATCH_MNT -w -p 5 -n 1000 \
> +		$FSSTRESS_AVOID >&/dev/null
> +
> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT \
> +       $SCRATCH_MNT/snap1 >>$seqres.full 2>&1

_run_btrfs_util_prog will already redirect to $seqres.full, you don't 
need this part.

> +
> +run_check $FSSTRESS_PROG -d $SCRATCH_MNT/snap1 -w -p 5 -n 1000 \
> +       $FSSTRESS_AVOID >&/dev/null
> +
> +_run_btrfs_util_prog quota enable $SCRATCH_MNT
> +_run_btrfs_util_prog quota rescan -w $SCRATCH_MNT
> +
> +#ignore removing subvolume errors
> +rm -rf $SCRATCH_MNT/* >& /dev/null
> +
> +_run_btrfs_util_prog filesystem sync $SCRATCH_MNT >> $seqres.full 2>&1

Same here.

> +_run_btrfs_util_prog qgroup show $SCRATCH_MNT | $SED_PROG -n '/[0-9]/p' \
> +	| $AWK_PROG '{print $1" "$2" "$3 }'
> +

You can't use _run_btrfs_util_prog here, it will eat the output.  You 
need to use $BTRFS_UTIL_PROG instead.  Fix these up and resend, this is 
a really important test and I needed it to make sure my qgroups patch 
was right (which it is now.)  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Wang Shilong May 8, 2014, 3:38 a.m. UTC | #4
On 05/08/2014 04:58 AM, Josef Bacik wrote:
> On 03/09/2014 11:44 PM, Wang Shilong wrote:
>> Test flow is to run fsstress after triggering quota rescan.
>> the ruler is simple, we just remove all files and directories,
>> sync filesystem and see if qgroup's ref and excl are nodesize.
>>
>> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
>> ---
>> v1->v2:
>>     switch into new helper _run_btrfs_util_prog()
>> ---
>>   tests/btrfs/041     | 76 
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   tests/btrfs/041.out |  3 +++
>>   tests/btrfs/group   |  1 +
>>   3 files changed, 80 insertions(+)
>>   create mode 100644 tests/btrfs/041
>>   create mode 100644 tests/btrfs/041.out
>>
>> diff --git a/tests/btrfs/041 b/tests/btrfs/041
>> new file mode 100644
>> index 0000000..92bd080
>> --- /dev/null
>> +++ b/tests/btrfs/041
>> @@ -0,0 +1,76 @@
>> +#! /bin/bash
>> +# FSQA Test No. btrfs/041
>> +#
>> +# Quota rescan stress test, we run fsstress and quota rescan 
>> concurrently
>> +#
>> +#----------------------------------------------------------------------- 
>>
>> +# Copyright (C) 2014 Fujitsu.  All rights reserved.
>> +#
>> +# This program is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU General Public License as
>> +# published by the Free Software Foundation.
>> +#
>> +# This program is distributed in the hope that it would be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program; if not, write the Free Software Foundation,
>> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
>> +#
>> +#----------------------------------------------------------------------- 
>>
>> +#
>> +
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1
>> +
>> +_cleanup()
>> +{
>> +    cd /
>> +    rm -f $tmp.*
>> +}
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +
>> +# real QA test starts here
>> +_need_to_be_root
>> +_supported_fs btrfs
>> +_supported_os Linux
>> +_require_scratch
>> +
>> +rm -f $seqres.full
>> +
>> +run_check _scratch_mkfs "-b 1g --nodesize 4096"
>> +run_check _scratch_mount
>> +
>
> Add "-o nospace_cache" here please, otherwise I don't get the same 
> output.

I am little confused why we need specify this mount option explicitly?
As far as i know, space cache is not included into qgroup accounting space.

Thanks,
Wang
>
>> +# -w ensures that the only ops are ones which cause write I/O
>> +run_check $FSSTRESS_PROG -d $SCRATCH_MNT -w -p 5 -n 1000 \
>> +        $FSSTRESS_AVOID >&/dev/null
>> +
>> +_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT \
>> +       $SCRATCH_MNT/snap1 >>$seqres.full 2>&1
>
> _run_btrfs_util_prog will already redirect to $seqres.full, you don't 
> need this part.
>
>> +
>> +run_check $FSSTRESS_PROG -d $SCRATCH_MNT/snap1 -w -p 5 -n 1000 \
>> +       $FSSTRESS_AVOID >&/dev/null
>> +
>> +_run_btrfs_util_prog quota enable $SCRATCH_MNT
>> +_run_btrfs_util_prog quota rescan -w $SCRATCH_MNT
>> +
>> +#ignore removing subvolume errors
>> +rm -rf $SCRATCH_MNT/* >& /dev/null
>> +
>> +_run_btrfs_util_prog filesystem sync $SCRATCH_MNT >> $seqres.full 2>&1
>
> Same here.
>
>> +_run_btrfs_util_prog qgroup show $SCRATCH_MNT | $SED_PROG -n 
>> '/[0-9]/p' \
>> +    | $AWK_PROG '{print $1" "$2" "$3 }'
>> +
>
> You can't use _run_btrfs_util_prog here, it will eat the output. You 
> need to use $BTRFS_UTIL_PROG instead.  Fix these up and resend, this 
> is a really important test and I needed it to make sure my qgroups 
> patch was right (which it is now.)  Thanks,
>
> Josef
> .
>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Josef Bacik May 8, 2014, 6:33 p.m. UTC | #5
On 05/07/2014 11:38 PM, Wang Shilong wrote:
> On 05/08/2014 04:58 AM, Josef Bacik wrote:
>> On 03/09/2014 11:44 PM, Wang Shilong wrote:
>>> Test flow is to run fsstress after triggering quota rescan.
>>> the ruler is simple, we just remove all files and directories,
>>> sync filesystem and see if qgroup's ref and excl are nodesize.
>>>
>>> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
>>> ---
>>> v1->v2:
>>>     switch into new helper _run_btrfs_util_prog()
>>> ---
>>>   tests/btrfs/041     | 76
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>   tests/btrfs/041.out |  3 +++
>>>   tests/btrfs/group   |  1 +
>>>   3 files changed, 80 insertions(+)
>>>   create mode 100644 tests/btrfs/041
>>>   create mode 100644 tests/btrfs/041.out
>>>
>>> diff --git a/tests/btrfs/041 b/tests/btrfs/041
>>> new file mode 100644
>>> index 0000000..92bd080
>>> --- /dev/null
>>> +++ b/tests/btrfs/041
>>> @@ -0,0 +1,76 @@
>>> +#! /bin/bash
>>> +# FSQA Test No. btrfs/041
>>> +#
>>> +# Quota rescan stress test, we run fsstress and quota rescan
>>> concurrently
>>> +#
>>> +#-----------------------------------------------------------------------
>>>
>>> +# Copyright (C) 2014 Fujitsu.  All rights reserved.
>>> +#
>>> +# This program is free software; you can redistribute it and/or
>>> +# modify it under the terms of the GNU General Public License as
>>> +# published by the Free Software Foundation.
>>> +#
>>> +# This program is distributed in the hope that it would be useful,
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> +# GNU General Public License for more details.
>>> +#
>>> +# You should have received a copy of the GNU General Public License
>>> +# along with this program; if not, write the Free Software Foundation,
>>> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
>>> +#
>>> +#-----------------------------------------------------------------------
>>>
>>> +#
>>> +
>>> +seq=`basename $0`
>>> +seqres=$RESULT_DIR/$seq
>>> +echo "QA output created by $seq"
>>> +
>>> +here=`pwd`
>>> +tmp=/tmp/$$
>>> +status=1
>>> +
>>> +_cleanup()
>>> +{
>>> +    cd /
>>> +    rm -f $tmp.*
>>> +}
>>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>>> +
>>> +# get standard environment, filters and checks
>>> +. ./common/rc
>>> +. ./common/filter
>>> +
>>> +# real QA test starts here
>>> +_need_to_be_root
>>> +_supported_fs btrfs
>>> +_supported_os Linux
>>> +_require_scratch
>>> +
>>> +rm -f $seqres.full
>>> +
>>> +run_check _scratch_mkfs "-b 1g --nodesize 4096"
>>> +run_check _scratch_mount
>>> +
>>
>> Add "-o nospace_cache" here please, otherwise I don't get the same
>> output.
>
> I am little confused why we need specify this mount option explicitly?
> As far as i know, space cache is not included into qgroup accounting space.
>

It is.  Thanks,

Josef

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Wang Shilong May 9, 2014, 1:17 a.m. UTC | #6
On 05/09/2014 02:33 AM, Josef Bacik wrote:
> On 05/07/2014 11:38 PM, Wang Shilong wrote:
>> On 05/08/2014 04:58 AM, Josef Bacik wrote:
>>> On 03/09/2014 11:44 PM, Wang Shilong wrote:
>>>> Test flow is to run fsstress after triggering quota rescan.
>>>> the ruler is simple, we just remove all files and directories,
>>>> sync filesystem and see if qgroup's ref and excl are nodesize.
>>>>
>>>> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
>>>> ---
>>>> v1->v2:
>>>>     switch into new helper _run_btrfs_util_prog()
>>>> ---
>>>>   tests/btrfs/041     | 76
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>   tests/btrfs/041.out |  3 +++
>>>>   tests/btrfs/group   |  1 +
>>>>   3 files changed, 80 insertions(+)
>>>>   create mode 100644 tests/btrfs/041
>>>>   create mode 100644 tests/btrfs/041.out
>>>>
>>>> diff --git a/tests/btrfs/041 b/tests/btrfs/041
>>>> new file mode 100644
>>>> index 0000000..92bd080
>>>> --- /dev/null
>>>> +++ b/tests/btrfs/041
>>>> @@ -0,0 +1,76 @@
>>>> +#! /bin/bash
>>>> +# FSQA Test No. btrfs/041
>>>> +#
>>>> +# Quota rescan stress test, we run fsstress and quota rescan
>>>> concurrently
>>>> +#
>>>> +#----------------------------------------------------------------------- 
>>>>
>>>>
>>>> +# Copyright (C) 2014 Fujitsu.  All rights reserved.
>>>> +#
>>>> +# This program is free software; you can redistribute it and/or
>>>> +# modify it under the terms of the GNU General Public License as
>>>> +# published by the Free Software Foundation.
>>>> +#
>>>> +# This program is distributed in the hope that it would be useful,
>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>> +# GNU General Public License for more details.
>>>> +#
>>>> +# You should have received a copy of the GNU General Public License
>>>> +# along with this program; if not, write the Free Software 
>>>> Foundation,
>>>> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>>>> +#
>>>> +#----------------------------------------------------------------------- 
>>>>
>>>>
>>>> +#
>>>> +
>>>> +seq=`basename $0`
>>>> +seqres=$RESULT_DIR/$seq
>>>> +echo "QA output created by $seq"
>>>> +
>>>> +here=`pwd`
>>>> +tmp=/tmp/$$
>>>> +status=1
>>>> +
>>>> +_cleanup()
>>>> +{
>>>> +    cd /
>>>> +    rm -f $tmp.*
>>>> +}
>>>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>>>> +
>>>> +# get standard environment, filters and checks
>>>> +. ./common/rc
>>>> +. ./common/filter
>>>> +
>>>> +# real QA test starts here
>>>> +_need_to_be_root
>>>> +_supported_fs btrfs
>>>> +_supported_os Linux
>>>> +_require_scratch
>>>> +
>>>> +rm -f $seqres.full
>>>> +
>>>> +run_check _scratch_mkfs "-b 1g --nodesize 4096"
>>>> +run_check _scratch_mount
>>>> +
>>>
>>> Add "-o nospace_cache" here please, otherwise I don't get the same
>>> output.
>>
>> I am little confused why we need specify this mount option explicitly?
>> As far as i know, space cache is not included into qgroup accounting 
>> space.
>>
>
> It is.  Thanks,
Free space cache are inserted in tree root, they can not be shared with 
other trees(fs/file tree).
So normal fs/file tree's qgroup accounting is independeng of free space 
cache.

With or without the option, subvolume qgroup's value should not have any 
differences.
Did i miss something here?:-)

Thanks,
Wang
>
> Josef
>
> .
>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Josef Bacik May 9, 2014, 2:13 a.m. UTC | #7
The inode cache is saved in the FS tree itself for every individual FS tree, that affects the sizes reported by qgroup show so we need to explicitly turn it off to get consistent values.  Thanks,

Josef

Wang Shilong <wangsl.fnst@cn.fujitsu.com> wrote:


On 05/09/2014 02:33 AM, Josef Bacik wrote:
> On 05/07/2014 11:38 PM, Wang Shilong wrote:
>> On 05/08/2014 04:58 AM, Josef Bacik wrote:
>>> On 03/09/2014 11:44 PM, Wang Shilong wrote:
>>>> Test flow is to run fsstress after triggering quota rescan.
>>>> the ruler is simple, we just remove all files and directories,
>>>> sync filesystem and see if qgroup's ref and excl are nodesize.
>>>>
>>>> Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
>>>> ---
>>>> v1->v2:
>>>>     switch into new helper _run_btrfs_util_prog()
>>>> ---
>>>>   tests/btrfs/041     | 76
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>   tests/btrfs/041.out |  3 +++
>>>>   tests/btrfs/group   |  1 +
>>>>   3 files changed, 80 insertions(+)
>>>>   create mode 100644 tests/btrfs/041
>>>>   create mode 100644 tests/btrfs/041.out
>>>>
>>>> diff --git a/tests/btrfs/041 b/tests/btrfs/041
>>>> new file mode 100644
>>>> index 0000000..92bd080
>>>> --- /dev/null
>>>> +++ b/tests/btrfs/041
>>>> @@ -0,0 +1,76 @@
>>>> +#! /bin/bash
>>>> +# FSQA Test No. btrfs/041
>>>> +#
>>>> +# Quota rescan stress test, we run fsstress and quota rescan
>>>> concurrently
>>>> +#
>>>> +#-----------------------------------------------------------------------
>>>>
>>>>
>>>> +# Copyright (C) 2014 Fujitsu.  All rights reserved.
>>>> +#
>>>> +# This program is free software; you can redistribute it and/or
>>>> +# modify it under the terms of the GNU General Public License as
>>>> +# published by the Free Software Foundation.
>>>> +#
>>>> +# This program is distributed in the hope that it would be useful,
>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>> +# GNU General Public License for more details.
>>>> +#
>>>> +# You should have received a copy of the GNU General Public License
>>>> +# along with this program; if not, write the Free Software
>>>> Foundation,
>>>> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>>>> +#
>>>> +#-----------------------------------------------------------------------
>>>>
>>>>
>>>> +#
>>>> +
>>>> +seq=`basename $0`
>>>> +seqres=$RESULT_DIR/$seq
>>>> +echo "QA output created by $seq"
>>>> +
>>>> +here=`pwd`
>>>> +tmp=/tmp/$$
>>>> +status=1
>>>> +
>>>> +_cleanup()
>>>> +{
>>>> +    cd /
>>>> +    rm -f $tmp.*
>>>> +}
>>>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>>>> +
>>>> +# get standard environment, filters and checks
>>>> +. ./common/rc
>>>> +. ./common/filter
>>>> +
>>>> +# real QA test starts here
>>>> +_need_to_be_root
>>>> +_supported_fs btrfs
>>>> +_supported_os Linux
>>>> +_require_scratch
>>>> +
>>>> +rm -f $seqres.full
>>>> +
>>>> +run_check _scratch_mkfs "-b 1g --nodesize 4096"
>>>> +run_check _scratch_mount
>>>> +
>>>
>>> Add "-o nospace_cache" here please, otherwise I don't get the same
>>> output.
>>
>> I am little confused why we need specify this mount option explicitly?
>> As far as i know, space cache is not included into qgroup accounting
>> space.
>>
>
> It is.  Thanks,
Free space cache are inserted in tree root, they can not be shared with
other trees(fs/file tree).
So normal fs/file tree's qgroup accounting is independeng of free space
cache.

With or without the option, subvolume qgroup's value should not have any
differences.
Did i miss something here?:-)

Thanks,
Wang
>
> Josef
>
> .
>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  https://urldefense.proofpoint.com/v1/url?u=http://vger.kernel.org/majordomo-info.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=G4HqImi%2Fy7KWWbAvCZf%2Bl8CSV3XzU9ODKrgtdQCTPYQ%3D%0A&s=e27422f119365ab04dddfb06e08c152eec84c4de29e19b6811310607e6a15c99
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Wang Shilong May 9, 2014, 2:43 a.m. UTC | #8
On 05/09/2014 10:13 AM, Josef Bacik wrote:
> The inode cache is saved in the FS tree itself for every individual FS tree, that affects the sizes reported by qgroup show so we need to explicitly turn it off to get consistent values.  Thanks,
Right, so we need turn off inode_cache explicitly rather than free_space 
option.
I will fix all your addressed comments and resend the patch.

Thanks,
Wang
> Josef
>
> Wang Shilong<wangsl.fnst@cn.fujitsu.com>  wrote:
>
>
> On 05/09/2014 02:33 AM, Josef Bacik wrote:
>> On 05/07/2014 11:38 PM, Wang Shilong wrote:
>>> On 05/08/2014 04:58 AM, Josef Bacik wrote:
>>>> On 03/09/2014 11:44 PM, Wang Shilong wrote:
>>>>> Test flow is to run fsstress after triggering quota rescan.
>>>>> the ruler is simple, we just remove all files and directories,
>>>>> sync filesystem and see if qgroup's ref and excl are nodesize.
>>>>>
>>>>> Signed-off-by: Wang Shilong<wangsl.fnst@cn.fujitsu.com>
>>>>> ---
>>>>> v1->v2:
>>>>>      switch into new helper _run_btrfs_util_prog()
>>>>> ---
>>>>>    tests/btrfs/041     | 76
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>    tests/btrfs/041.out |  3 +++
>>>>>    tests/btrfs/group   |  1 +
>>>>>    3 files changed, 80 insertions(+)
>>>>>    create mode 100644 tests/btrfs/041
>>>>>    create mode 100644 tests/btrfs/041.out
>>>>>
>>>>> diff --git a/tests/btrfs/041 b/tests/btrfs/041
>>>>> new file mode 100644
>>>>> index 0000000..92bd080
>>>>> --- /dev/null
>>>>> +++ b/tests/btrfs/041
>>>>> @@ -0,0 +1,76 @@
>>>>> +#! /bin/bash
>>>>> +# FSQA Test No. btrfs/041
>>>>> +#
>>>>> +# Quota rescan stress test, we run fsstress and quota rescan
>>>>> concurrently
>>>>> +#
>>>>> +#-----------------------------------------------------------------------
>>>>>
>>>>>
>>>>> +# Copyright (C) 2014 Fujitsu.  All rights reserved.
>>>>> +#
>>>>> +# This program is free software; you can redistribute it and/or
>>>>> +# modify it under the terms of the GNU General Public License as
>>>>> +# published by the Free Software Foundation.
>>>>> +#
>>>>> +# This program is distributed in the hope that it would be useful,
>>>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>>>> +# GNU General Public License for more details.
>>>>> +#
>>>>> +# You should have received a copy of the GNU General Public License
>>>>> +# along with this program; if not, write the Free Software
>>>>> Foundation,
>>>>> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>>>>> +#
>>>>> +#-----------------------------------------------------------------------
>>>>>
>>>>>
>>>>> +#
>>>>> +
>>>>> +seq=`basename $0`
>>>>> +seqres=$RESULT_DIR/$seq
>>>>> +echo "QA output created by $seq"
>>>>> +
>>>>> +here=`pwd`
>>>>> +tmp=/tmp/$$
>>>>> +status=1
>>>>> +
>>>>> +_cleanup()
>>>>> +{
>>>>> +    cd /
>>>>> +    rm -f $tmp.*
>>>>> +}
>>>>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>>>>> +
>>>>> +# get standard environment, filters and checks
>>>>> +. ./common/rc
>>>>> +. ./common/filter
>>>>> +
>>>>> +# real QA test starts here
>>>>> +_need_to_be_root
>>>>> +_supported_fs btrfs
>>>>> +_supported_os Linux
>>>>> +_require_scratch
>>>>> +
>>>>> +rm -f $seqres.full
>>>>> +
>>>>> +run_check _scratch_mkfs "-b 1g --nodesize 4096"
>>>>> +run_check _scratch_mount
>>>>> +
>>>> Add "-o nospace_cache" here please, otherwise I don't get the same
>>>> output.
>>> I am little confused why we need specify this mount option explicitly?
>>> As far as i know, space cache is not included into qgroup accounting
>>> space.
>>>
>> It is.  Thanks,
> Free space cache are inserted in tree root, they can not be shared with
> other trees(fs/file tree).
> So normal fs/file tree's qgroup accounting is independeng of free space
> cache.
>
> With or without the option, subvolume qgroup's value should not have any
> differences.
> Did i miss something here?:-)
>
> Thanks,
> Wang
>> Josef
>>
>> .
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message tomajordomo@vger.kernel.org
> More majordomo info athttps://urldefense.proofpoint.com/v1/url?u=http://vger.kernel.org/majordomo-info.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=cKCbChRKsMpTX8ybrSkonQ%3D%3D%0A&m=G4HqImi%2Fy7KWWbAvCZf%2Bl8CSV3XzU9ODKrgtdQCTPYQ%3D%0A&s=e27422f119365ab04dddfb06e08c152eec84c4de29e19b6811310607e6a15c99
> .
>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Josef Bacik May 9, 2014, 5:24 p.m. UTC | #9
On 05/08/2014 10:43 PM, Wang Shilong wrote:
> On 05/09/2014 10:13 AM, Josef Bacik wrote:
>> The inode cache is saved in the FS tree itself for every individual FS
>> tree, that affects the sizes reported by qgroup show so we need to
>> explicitly turn it off to get consistent values.  Thanks,
> Right, so we need turn off inode_cache explicitly rather than free_space
> option.
> I will fix all your addressed comments and resend the patch.
>

Sigh I totally said nospace_cache didn't I?  I really meant 
noinode_cache, sorry,

Josef

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch
diff mbox

diff --git a/tests/btrfs/041 b/tests/btrfs/041
new file mode 100644
index 0000000..92bd080
--- /dev/null
+++ b/tests/btrfs/041
@@ -0,0 +1,76 @@ 
+#! /bin/bash
+# FSQA Test No. btrfs/041
+#
+# Quota rescan stress test, we run fsstress and quota rescan concurrently
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2014 Fujitsu.  All rights reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+run_check _scratch_mkfs "-b 1g --nodesize 4096"
+run_check _scratch_mount
+
+# -w ensures that the only ops are ones which cause write I/O
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -w -p 5 -n 1000 \
+		$FSSTRESS_AVOID >&/dev/null
+
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT \
+       $SCRATCH_MNT/snap1 >>$seqres.full 2>&1
+
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT/snap1 -w -p 5 -n 1000 \
+       $FSSTRESS_AVOID >&/dev/null
+
+_run_btrfs_util_prog quota enable $SCRATCH_MNT
+_run_btrfs_util_prog quota rescan -w $SCRATCH_MNT
+
+#ignore removing subvolume errors
+rm -rf $SCRATCH_MNT/* >& /dev/null
+
+_run_btrfs_util_prog filesystem sync $SCRATCH_MNT >> $seqres.full 2>&1
+_run_btrfs_util_prog qgroup show $SCRATCH_MNT | $SED_PROG -n '/[0-9]/p' \
+	| $AWK_PROG '{print $1" "$2" "$3 }'
+
+status=0
+exit
diff --git a/tests/btrfs/041.out b/tests/btrfs/041.out
new file mode 100644
index 0000000..fb59896
--- /dev/null
+++ b/tests/btrfs/041.out
@@ -0,0 +1,3 @@ 
+QA output created by 041
+0/5 4096 4096
+0/258 4096 4096
diff --git a/tests/btrfs/group b/tests/btrfs/group
index f5452b5..fd0193c 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -43,3 +43,4 @@ 
 038 auto quick
 039 auto quick
 040 auto quick
+041 auto quick