| Message ID | 1455069001-17846-3-git-send-email-tytso@mit.edu (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Tue, Feb 09, 2016 at 08:49:51PM -0500, Theodore Ts'o wrote: > From: Hugh Dickins <hughd@google.com> > > _scratch_mkfs_sized() avoid blockdev and update MOUNT_OPTIONS with > required size on tmpfs, so those tests using it can now run. > > Signed-off-by: Hugh Dickins <hughd@google.com> > Signed-off-by: Junho Ryu <jayr@google.com> > Signed-off-by: Theodore Ts'o <tytso@mit.edu> > --- > common/rc | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/common/rc b/common/rc > index 76e02e4..aca723f 100644 > --- a/common/rc > +++ b/common/rc > @@ -813,7 +813,7 @@ _scratch_mkfs_sized() > > blocks=`expr $fssize / $blocksize` > > - if [ "$HOSTOS" == "Linux" ]; then > + if [ "$HOSTOS" == "Linux" -a -b "$SCRATCH_DEV" ]; then > devsize=`blockdev --getsize64 $SCRATCH_DEV` > [ "$fssize" -gt "$devsize" ] && _notrun "Scratch device too small" > fi > @@ -849,6 +849,9 @@ _scratch_mkfs_sized() > sector_size=`blockdev --getss $SCRATCH_DEV` > $MKFS_F2FS_PROG $MKFS_OPTIONS $SCRATCH_DEV `expr $fssize / $sector_size` > ;; > + tmpfs) > + export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS" So what happens when the test asks for 10GB of device space, and you only have 4GB of RAM? Cheers, Dave.
On Wed, Feb 10, 2016 at 05:00:58PM +1100, Dave Chinner wrote: > > + tmpfs) > > + export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS" > > So what happens when the test asks for 10GB of device space, and you > only have 4GB of RAM? The largest amount of storage space requested by shared/generic tests via _scratch_mkfs_sized is 2G, and that's from generic/27[345]. We currently could run into problems today if the storage device only had 3G of space, and some test, such as f2fs/001, called _scratch_mkfs_sized requesting 4G of space. We currently aren't checking error returns from the mkfs command; we just assume that it will succeed. Because of the someone interesting behavior of tmpfs assuming inode space is free, we already have problems where even with 8G of memory, generic/269 and generic/273 will result in the test getting OOM killed, and if you use less than 4G, there are a more tests that end up getting OOM killed as a result. So I think it would be fair to simply document that if you are testing tmpfs, you need a VM or or a memory container with at least 8G of memory. If we start limiting the number of inodes in a tmpfs mount, we could probably bring that requirement down to 4G, which is certainly fair. We do document minimum requirements for the size of the TEST_DEV and SCRATCH_DEV, don't we? (Minimum requirements certainly do exist in practice, in any case.) Cheers, - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Feb 10, 2016 at 10:58:37AM -0500, Theodore Ts'o wrote: > On Wed, Feb 10, 2016 at 05:00:58PM +1100, Dave Chinner wrote: > > > + tmpfs) > > > + export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS" > > > > So what happens when the test asks for 10GB of device space, and you > > only have 4GB of RAM? > > The largest amount of storage space requested by shared/generic tests > via _scratch_mkfs_sized is 2G, and that's from generic/27[345]. Sure, but that doesn't doesn't invalidate my question. I run xfstests on machines with 1GB RAM, so those tests you list above are going to have problems with tmpfs, yes? > We currently could run into problems today if the storage device only > had 3G of space, and some test, such as f2fs/001, called > _scratch_mkfs_sized requesting 4G of space. We currently aren't > checking error returns from the mkfs command; we just assume that it > will succeed. That would be a test bug. We have a general rule of thumb that test/scratch devices are typically a couple of GB in size, so if the test needs less space than that it doesn't need to check. If the test needs more than 1-2GB of space, then it needs to call _require_fs_space to abort the test if the device is too small. Hence we end up with tests being not run with: generic/312 [not run] this test requires $SCRATCH_DEV has 5368709120B space This needs to work for tmpfs, too, so that we don't run tests that will overcommit RAM on machines with limited memory. > Because of the someone interesting behavior of tmpfs assuming inode > space is free, we already have problems where even with 8G of memory, > generic/269 and generic/273 will result in the test getting OOM > killed, and if you use less than 4G, there are a more tests that end > up getting OOM killed as a result. As I said, this is a tmpfs bug that needs fixing, not working around in xfstests. If tmpfs is limited to 512MB or 1GB of space, then having it be able to consume multiple times it's requested size in memory is simply wrong. > So I think it would be fair to simply document that if you are > testing tmpfs, you need a VM or or a memory container with at > least 8G of memory. If we start limiting the number of inodes in a > tmpfs mount, we could probably bring that requirement down to 4G, > which is certainly fair. We do document minimum requirements for > the size of the TEST_DEV and SCRATCH_DEV, don't we? No, not that I know of. > (Minimum > requirements certainly do exist in practice, in any case.) See above. Cheers, Dave.
diff --git a/common/rc b/common/rc index 76e02e4..aca723f 100644 --- a/common/rc +++ b/common/rc @@ -813,7 +813,7 @@ _scratch_mkfs_sized() blocks=`expr $fssize / $blocksize` - if [ "$HOSTOS" == "Linux" ]; then + if [ "$HOSTOS" == "Linux" -a -b "$SCRATCH_DEV" ]; then devsize=`blockdev --getsize64 $SCRATCH_DEV` [ "$fssize" -gt "$devsize" ] && _notrun "Scratch device too small" fi @@ -849,6 +849,9 @@ _scratch_mkfs_sized() sector_size=`blockdev --getss $SCRATCH_DEV` $MKFS_F2FS_PROG $MKFS_OPTIONS $SCRATCH_DEV `expr $fssize / $sector_size` ;; + tmpfs) + export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS" + ;; *) _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized" ;;