| Message ID | 20160720073450.8952-1-wangxg.fnst@cn.fujitsu.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Wed, Jul 20, 2016 at 8:34 AM, Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com> wrote: > Currently in btrfs, there is something wrong with data space reservation. > For example, if we try to preallocate more than haf of whole fs space, > ENOSPC will occur, but indeed fs still has free space to satisfy this > request. > > To easily reproduce this bug, this test case needs fs is mixed mode(btrfs > specific), so put this test case in btrfs group, not generic group. Why does it need to be in mixed mode (why is it easier)? As far as I can see from the corresponding btrfs patch (and its description), the problem affects non-mixed mode, therefore it could be a generic test. > > Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com> > --- > tests/btrfs/127 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > tests/btrfs/127.out | 2 ++ > tests/btrfs/group | 1 + > 3 files changed, 64 insertions(+) > create mode 100755 tests/btrfs/127 > create mode 100644 tests/btrfs/127.out > > diff --git a/tests/btrfs/127 b/tests/btrfs/127 > new file mode 100755 > index 0000000..f95c72f > --- /dev/null > +++ b/tests/btrfs/127 > @@ -0,0 +1,61 @@ > +#! /bin/bash > +# FS QA Test 127 > +# > +# Test whether fallocate(2) can preallocate half of the whole fs space. > +# > +#----------------------------------------------------------------------- > +# Copyright (c) 2016 Fujitsu. All Rights Reserved. > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/filter > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > + > +# Modify as appropriate. > +_supported_fs btrfs > +_supported_os Linux > +_require_scratch > + > +MKFS_OPTIONS="-M" This now overrides any specific mkfs options specified in the command line, doesn't it? > +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1 > +_scratch_mount > + > +$XFS_IO_PROG -f -c "falloc 0 128M" $SCRATCH_MNT/testfile | _filter_xfs_io No need for the filter here. > + > +echo "Silence is golden" > +status=0 > +exit > diff --git a/tests/btrfs/127.out b/tests/btrfs/127.out > new file mode 100644 > index 0000000..0af84c0 > --- /dev/null > +++ b/tests/btrfs/127.out > @@ -0,0 +1,2 @@ > +QA output created by 127 > +Silence is golden > diff --git a/tests/btrfs/group b/tests/btrfs/group > index a21a80a..4c3ac00 100644 > --- a/tests/btrfs/group > +++ b/tests/btrfs/group > @@ -129,3 +129,4 @@ > 124 auto replace > 125 auto replace > 126 auto quick qgroup > +127 auto quick metadata enospc > -- > 2.9.0 > > > > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jul 20, 2016 at 03:34:50PM +0800, Wang Xiaoguang wrote: > Currently in btrfs, there is something wrong with data space reservation. > For example, if we try to preallocate more than haf of whole fs space, > ENOSPC will occur, but indeed fs still has free space to satisfy this > request. > > To easily reproduce this bug, this test case needs fs is mixed mode(btrfs > specific), so put this test case in btrfs group, not generic group. But the actual test isn't btrfs specific, and other file systems might have issues in this area as well. Please add it to the generic group instead. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
hello, On 07/20/2016 05:25 PM, Filipe Manana wrote: > On Wed, Jul 20, 2016 at 8:34 AM, Wang Xiaoguang > <wangxg.fnst@cn.fujitsu.com> wrote: >> Currently in btrfs, there is something wrong with data space reservation. >> For example, if we try to preallocate more than haf of whole fs space, >> ENOSPC will occur, but indeed fs still has free space to satisfy this >> request. >> >> To easily reproduce this bug, this test case needs fs is mixed mode(btrfs >> specific), so put this test case in btrfs group, not generic group. > Why does it need to be in mixed mode (why is it easier)? As far as I > can see from the corresponding btrfs patch (and its description), > the problem affects non-mixed mode, therefore it could be a generic test. In btrfs mixed mode, as you know, data and metadata share the same btrfs_space_info. In btrfs_fallocate(), if we try to pre-allocate 64M data, it first makes bytes_may_use += 64M, later btrfs_reserve_extent() only makes bytes_reserved += 64M, but does not decrease bytes_may_use. Assume our fs is 128M, now bytes_may_use + bytes_reserved is already greater than 128M, so any later transaction operations in btrfs_fallocate() will fail for ENOSPC reason. But note, byte_may_use will only be decreased in end of btrfs_fallocate(). So I think this bug is easy to reproduce in mixed mode :) For non-mixed mode, still assume fs is 128M. For data's btrfs_space_info, before we call btrfs_free_reserved_data_space() in end of btrfs_fallocate(), though bytes_may_use +=64M, bytes_reserved += 64M, and bytes_may_use + bytes_reserved > 128M, there is no other process to allocating data space, so there will still be no ENOSPC error. I have rewritten a test case which do write(2) and fallocate(2) in parallel and put this test case to generic group, thanks. Regards, Xiaoguang Wang > >> Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com> >> --- >> tests/btrfs/127 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++ >> tests/btrfs/127.out | 2 ++ >> tests/btrfs/group | 1 + >> 3 files changed, 64 insertions(+) >> create mode 100755 tests/btrfs/127 >> create mode 100644 tests/btrfs/127.out >> >> diff --git a/tests/btrfs/127 b/tests/btrfs/127 >> new file mode 100755 >> index 0000000..f95c72f >> --- /dev/null >> +++ b/tests/btrfs/127 >> @@ -0,0 +1,61 @@ >> +#! /bin/bash >> +# FS QA Test 127 >> +# >> +# Test whether fallocate(2) can preallocate half of the whole fs space. >> +# >> +#----------------------------------------------------------------------- >> +# Copyright (c) 2016 Fujitsu. All Rights Reserved. >> +# >> +# This program is free software; you can redistribute it and/or >> +# modify it under the terms of the GNU General Public License as >> +# published by the Free Software Foundation. >> +# >> +# This program is distributed in the hope that it would be useful, >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> +# GNU General Public License for more details. >> +# >> +# You should have received a copy of the GNU General Public License >> +# along with this program; if not, write the Free Software Foundation, >> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA >> +#----------------------------------------------------------------------- >> +# >> + >> +seq=`basename $0` >> +seqres=$RESULT_DIR/$seq >> +echo "QA output created by $seq" >> + >> +here=`pwd` >> +tmp=/tmp/$$ >> +status=1 # failure is the default! >> +trap "_cleanup; exit \$status" 0 1 2 3 15 >> + >> +_cleanup() >> +{ >> + cd / >> + rm -f $tmp.* >> +} >> + >> +# get standard environment, filters and checks >> +. ./common/rc >> +. ./common/filter >> + >> +# remove previous $seqres.full before test >> +rm -f $seqres.full >> + >> +# real QA test starts here >> + >> +# Modify as appropriate. >> +_supported_fs btrfs >> +_supported_os Linux >> +_require_scratch >> + >> +MKFS_OPTIONS="-M" > This now overrides any specific mkfs options specified in the command > line, doesn't it? > >> +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1 >> +_scratch_mount >> + >> +$XFS_IO_PROG -f -c "falloc 0 128M" $SCRATCH_MNT/testfile | _filter_xfs_io > No need for the filter here. > >> + >> +echo "Silence is golden" >> +status=0 >> +exit >> diff --git a/tests/btrfs/127.out b/tests/btrfs/127.out >> new file mode 100644 >> index 0000000..0af84c0 >> --- /dev/null >> +++ b/tests/btrfs/127.out >> @@ -0,0 +1,2 @@ >> +QA output created by 127 >> +Silence is golden >> diff --git a/tests/btrfs/group b/tests/btrfs/group >> index a21a80a..4c3ac00 100644 >> --- a/tests/btrfs/group >> +++ b/tests/btrfs/group >> @@ -129,3 +129,4 @@ >> 124 auto replace >> 125 auto replace >> 126 auto quick qgroup >> +127 auto quick metadata enospc >> -- >> 2.9.0 >> >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe fstests" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
hello, On 07/21/2016 03:00 AM, Christoph Hellwig wrote: > On Wed, Jul 20, 2016 at 03:34:50PM +0800, Wang Xiaoguang wrote: >> Currently in btrfs, there is something wrong with data space reservation. >> For example, if we try to preallocate more than haf of whole fs space, >> ENOSPC will occur, but indeed fs still has free space to satisfy this >> request. >> >> To easily reproduce this bug, this test case needs fs is mixed mode(btrfs >> specific), so put this test case in btrfs group, not generic group. > But the actual test isn't btrfs specific, and other file systems might > have issues in this area as well. Please add it to the generic group > instead. OK, new version will be sent soon. ext4 & xfs pass the test, btrfs fails. Regards, Xiaoguang Wang > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/tests/btrfs/127 b/tests/btrfs/127 new file mode 100755 index 0000000..f95c72f --- /dev/null +++ b/tests/btrfs/127 @@ -0,0 +1,61 @@ +#! /bin/bash +# FS QA Test 127 +# +# Test whether fallocate(2) can preallocate half of the whole fs space. +# +#----------------------------------------------------------------------- +# Copyright (c) 2016 Fujitsu. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs btrfs +_supported_os Linux +_require_scratch + +MKFS_OPTIONS="-M" +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1 +_scratch_mount + +$XFS_IO_PROG -f -c "falloc 0 128M" $SCRATCH_MNT/testfile | _filter_xfs_io + +echo "Silence is golden" +status=0 +exit diff --git a/tests/btrfs/127.out b/tests/btrfs/127.out new file mode 100644 index 0000000..0af84c0 --- /dev/null +++ b/tests/btrfs/127.out @@ -0,0 +1,2 @@ +QA output created by 127 +Silence is golden diff --git a/tests/btrfs/group b/tests/btrfs/group index a21a80a..4c3ac00 100644 --- a/tests/btrfs/group +++ b/tests/btrfs/group @@ -129,3 +129,4 @@ 124 auto replace 125 auto replace 126 auto quick qgroup +127 auto quick metadata enospc
Currently in btrfs, there is something wrong with data space reservation. For example, if we try to preallocate more than haf of whole fs space, ENOSPC will occur, but indeed fs still has free space to satisfy this request. To easily reproduce this bug, this test case needs fs is mixed mode(btrfs specific), so put this test case in btrfs group, not generic group. Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com> --- tests/btrfs/127 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/127.out | 2 ++ tests/btrfs/group | 1 + 3 files changed, 64 insertions(+) create mode 100755 tests/btrfs/127 create mode 100644 tests/btrfs/127.out