| Message ID | 1479412353-35154-1-git-send-email-ebiggers@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hi Eric, On Thu, Nov 17, 2016 at 11:52:33AM -0800, Eric Biggers wrote: > Add basic f2fs support to xfstests-bld. I am using this to verify that > the encryption tests I am adding to xfstests pass on both ext4 and f2fs. > It may be useful to f2fs developers too. For now, only one > configuration, "f2fs/default", is offered. > > For the f2fs tests to work, it's required to build a kernel with f2fs > support and build a rootfs with f2fs-tools installed. For testing > encryption, f2fs-tools must be v1.5.0 or later. > > It was necessary to remove the /etc/fstab lines for /dev/vd[b-f] because > they interfered with non-ext4 fsck. They are not needed for the tests. > > Signed-off-by: Eric Biggers <ebiggers@google.com> > --- > kvm-xfstests/test-appliance/files/etc/fstab | 5 -- > .../test-appliance/files/root/fs/f2fs/cfg/all.list | 1 + > .../test-appliance/files/root/fs/f2fs/cfg/default | 4 ++ > .../test-appliance/files/root/fs/f2fs/config | 62 ++++++++++++++++++++++ > kvm-xfstests/util/parse_cli | 2 +- > 5 files changed, 68 insertions(+), 6 deletions(-) > create mode 100644 kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list > create mode 100644 kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default > create mode 100644 kvm-xfstests/test-appliance/files/root/fs/f2fs/config > > diff --git a/kvm-xfstests/test-appliance/files/etc/fstab b/kvm-xfstests/test-appliance/files/etc/fstab > index 864478b..27fb1df 100644 > --- a/kvm-xfstests/test-appliance/files/etc/fstab > +++ b/kvm-xfstests/test-appliance/files/etc/fstab > @@ -5,9 +5,4 @@ proc /proc proc defaults 0 0 > tmpfs /tmp tmpfs mode=1777 0 0 > debugfs /sys/kernel/debug debugfs defaults 0 0 > /dev/rootfs / ext4 noatime 0 1 > -/dev/vdb /vdb ext4 defaults,noauto 0 0 > -/dev/vdc /vdc ext4 defaults,noauto 0 0 > -/dev/vdd /vdd ext4 defaults,noauto 0 0 > -/dev/vde /vde ext4 defaults,noauto 0 0 > -/dev/vdf /vdf ext4 defaults,noauto 0 0 > /dev/vdg /results ext4 defaults 0 2 > diff --git a/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list > new file mode 100644 > index 0000000..4ad96d5 > --- /dev/null > +++ b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list > @@ -0,0 +1 @@ > +default > diff --git a/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default > new file mode 100644 > index 0000000..616a070 > --- /dev/null > +++ b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default > @@ -0,0 +1,4 @@ > +SIZE=small > +export MKFS_OPTIONS="" > +export F2FS_MOUNT_OPTIONS="" > +TESTNAME="f2fs" > diff --git a/kvm-xfstests/test-appliance/files/root/fs/f2fs/config b/kvm-xfstests/test-appliance/files/root/fs/f2fs/config > new file mode 100644 > index 0000000..3cab963 > --- /dev/null > +++ b/kvm-xfstests/test-appliance/files/root/fs/f2fs/config > @@ -0,0 +1,62 @@ > +# > +# Configuration file for f2fs > +# > + > +DEFAULT_MKFS_OPTIONS="" > + > +function check_filesystem() > +{ > + local dev="$1" > + local ret > + > + /sbin/fsck.f2fs -f "$dev" Could you change to call fsck.f2fs instead of /sbin/xx? I'm almost finished to import f2fs-tools in xfstests-bld for kvm-xfstests. Given f2fs-tools from git, it seems /sbin/xx doesn't work. > + ret="$?" > + echo fsck.f2fs exited with status "$ret" > + return "$ret" > +} > + > +function format_filesystem() > +{ > + local dev="$1" > + local opts="$2" > + > + /sbin/mkfs.f2fs -q $opts "$dev" Ditto, mkfs.f2fs. Thanks, > + return $? > +} > + > +function setup_mount_opts() > +{ > + if test -n "$MNTOPTS" ; then > + if test -n "$F2FS_MOUNT_OPTIONS" ; then > + F2FS_MOUNT_OPTIONS="$F2FS_MOUNT_OPTIONS,$MNTOPTS" > + else > + F2FS_MOUNT_OPTIONS="-o $MNTOPTS" > + fi > + fi > +} > + > +function get_mkfs_opts() > +{ > + echo "$MKFS_OPTIONS" > +} > + > +function show_mkfs_opts() > +{ > + echo MKFS_OPTIONS: "$MKFS_OPTIONS" > +} > + > +function show_mount_opts() > +{ > + echo F2FS_MOUNT_OPTIONS: "$F2FS_MOUNT_OPTIONS" > +} > + > +function test_name_alias() > +{ > + echo "$1" > +} > + > +function reset_vars() > +{ > + unset F2FS_MOUNT_OPTIONS > + unset MKFS_OPTIONS > +} > diff --git a/kvm-xfstests/util/parse_cli b/kvm-xfstests/util/parse_cli > index ea747be..b42c8ba 100644 > --- a/kvm-xfstests/util/parse_cli > +++ b/kvm-xfstests/util/parse_cli > @@ -59,7 +59,7 @@ print_help () > validate_test_name() > { > case "$1" in > - btrfs*|cifs*|ext4*|generic*|shared*|udf*|xfs*|overlay*) ;; > + btrfs*|cifs*|ext4*|f2fs*|generic*|shared*|udf*|xfs*|overlay*) ;; > *) > echo -e "Invalid test name: $1\n" > print_help > -- > 2.8.0.rc3.226.g39d4020 > > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 17, 2016 at 08:38:35PM -0800, Jaegeuk Kim wrote: > > + /sbin/fsck.f2fs -f "$dev" > > Could you change to call fsck.f2fs instead of /sbin/xx? > I'm almost finished to import f2fs-tools in xfstests-bld for kvm-xfstests. > Given f2fs-tools from git, it seems /sbin/xx doesn't work. So why won't /sbin/fsck.f2fs not work? The /sbin/fsck driver does assume that /sbin/fsck.<type> will work, and other file systems do have hard links or put their actual file system checker there: % ls /sbin/fsck.* 36 /sbin/fsck.cramfs* 0 /sbin/fsck.ext3@ 60 /sbin/fsck.fat* 4 /sbin/fsck.nfs* 0 /sbin/fsck.exfat@ 0 /sbin/fsck.ext4@ 76 /sbin/fsck.minix* 0 /sbin/fsck.vfat@ 0 /sbin/fsck.ext2@ 0 /sbin/fsck.ext4dev@ 0 /sbin/fsck.msdos@ 4 /sbin/fsck.xfs* > > + /sbin/mkfs.f2fs -q $opts "$dev" > > Ditto, mkfs.f2fs. Ditto: % ls /sbin/mkfs.* 28 /sbin/mkfs.bfs* 0 /sbin/mkfs.ext3@ 76 /sbin/mkfs.minix* 352 /sbin/mkfs.xfs* 36 /sbin/mkfs.cramfs* 0 /sbin/mkfs.ext4@ 0 /sbin/mkfs.msdos@ 0 /sbin/mkfs.exfat@ 0 /sbin/mkfs.ext4dev@ 0 /sbin/mkfs.ntfs@ 0 /sbin/mkfs.ext2@ 36 /sbin/mkfs.fat* 0 /sbin/mkfs.vfat@ - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 17, 2016 at 11:51:57PM -0500, Theodore Ts'o wrote: > On Thu, Nov 17, 2016 at 08:38:35PM -0800, Jaegeuk Kim wrote: > > > + /sbin/fsck.f2fs -f "$dev" > > > > Could you change to call fsck.f2fs instead of /sbin/xx? > > I'm almost finished to import f2fs-tools in xfstests-bld for kvm-xfstests. > > Given f2fs-tools from git, it seems /sbin/xx doesn't work. > > So why won't /sbin/fsck.f2fs not work? The /sbin/fsck driver does > assume that /sbin/fsck.<type> will work, and other file systems do > have hard links or put their actual file system checker there: > > % ls /sbin/fsck.* > 36 /sbin/fsck.cramfs* 0 /sbin/fsck.ext3@ 60 /sbin/fsck.fat* 4 /sbin/fsck.nfs* > 0 /sbin/fsck.exfat@ 0 /sbin/fsck.ext4@ 76 /sbin/fsck.minix* 0 /sbin/fsck.vfat@ > 0 /sbin/fsck.ext2@ 0 /sbin/fsck.ext4dev@ 0 /sbin/fsck.msdos@ 4 /sbin/fsck.xfs* > IIUC, I guess it'd be okay if VM installed that as a package. I tried to add f2fs-tools repo likewise fio, I could't find /sbin/mkfs.f2fs, even if it was installed into bld/sbin/. Instead, bld/sbin/* were copied into xfstests/bin/ in VM, which enables to execute mkfs.f2fs globally through $PATH. I set up an almost default kvm-xfstests, and I could see: # ls /sbin/mkfs* /sbin/mkfs /sbin/mkfs.cramfs /sbin/mkfs.ext3 /sbin/mkfs.ext4dev /sbin/mkfs.bfs /sbin/mkfs.ext2 /sbin/mkfs.ext4 /sbin/mkfs.minix # ls /root/xfstests/bin/mkfs.* /root/xfstests/bin/mkfs.f2fs /root/xfstests/bin/mkfs.xfs Let me know, if I'm missing something. Thanks, > > > > + /sbin/mkfs.f2fs -q $opts "$dev" > > > > Ditto, mkfs.f2fs. > > Ditto: > > % ls /sbin/mkfs.* > 28 /sbin/mkfs.bfs* 0 /sbin/mkfs.ext3@ 76 /sbin/mkfs.minix* 352 /sbin/mkfs.xfs* > 36 /sbin/mkfs.cramfs* 0 /sbin/mkfs.ext4@ 0 /sbin/mkfs.msdos@ > 0 /sbin/mkfs.exfat@ 0 /sbin/mkfs.ext4dev@ 0 /sbin/mkfs.ntfs@ > 0 /sbin/mkfs.ext2@ 36 /sbin/mkfs.fat* 0 /sbin/mkfs.vfat@ > > - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 17, 2016 at 09:11:13PM -0800, Jaegeuk Kim wrote: > > IIUC, I guess it'd be okay if VM installed that as a package. I tried to add > f2fs-tools repo likewise fio, I could't find /sbin/mkfs.f2fs, even if it was > installed into bld/sbin/. Instead, bld/sbin/* were copied into xfstests/bin/ > in VM, which enables to execute mkfs.f2fs globally through $PATH. Ah, I see what you mean; I thought you were talking about your local f2fstools git repo, and not changes you've made to your local xfstests-bld repo. The reason why xfsprogs is installed in xfstests/bin is that historically, xfstests needed some of the xfsprogs tools (no matter what file system type you were testing), and the xfsprogs installed in Debian and Ubuntu was too old or didn't install all of the utilities that xfstests required. However, I have not tried to build the file system utilities for other file systems in xfstests-bld --- and this includes e2fsprogs. My goal is to allow xfstests-bld to support a large number of file systems, and it doesn't scale to build all file system utilities in xfstests-bld. Also, most of the time, the file system utilities tend to move at a slower pace, and the requirements for needing a newer file system utility than what is shipped in the distribution is rarer. I have sometimes built a newer version of e2fsprogs as a Debian package, and installed it into the VM. These days, what I tend to do when that's been necessary is to build a new Debian package for jessie-backports, and upload it to the jessie-backports repository, so everyone can use the same binary package. I guess I don't have any objections to allowing people to put locally compiled versions of the utilities in /root/xfstests/sbin and rely on the PATH to use that version in preference to the /sbin/mkfs.f2fs version of the utilities. As far as adding the ability to download and build local versions of f2fstools in the upstream xfstests-bld, I guess I could accept that so long as it's optional, and not turned on by default. The main issue here is that if i have a half dozen file system-specific utilities included in xfstests-bld, which were all enabled for download by default, it would increase the build time of xfstests-bld significantly, so it doesn't scale. I'd much prefer to see newer versions of the utilities (if necessary) in debian-backports. Cheers, - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Nov 18, 2016 at 11:22:49AM -0500, Theodore Ts'o wrote: > On Thu, Nov 17, 2016 at 09:11:13PM -0800, Jaegeuk Kim wrote: > > > > IIUC, I guess it'd be okay if VM installed that as a package. I tried to add > > f2fs-tools repo likewise fio, I could't find /sbin/mkfs.f2fs, even if it was > > installed into bld/sbin/. Instead, bld/sbin/* were copied into xfstests/bin/ > > in VM, which enables to execute mkfs.f2fs globally through $PATH. > > Ah, I see what you mean; I thought you were talking about your local > f2fstools git repo, and not changes you've made to your local > xfstests-bld repo. > > The reason why xfsprogs is installed in xfstests/bin is that > historically, xfstests needed some of the xfsprogs tools (no matter > what file system type you were testing), and the xfsprogs installed in > Debian and Ubuntu was too old or didn't install all of the utilities > that xfstests required. > > However, I have not tried to build the file system utilities for other > file systems in xfstests-bld --- and this includes e2fsprogs. My goal > is to allow xfstests-bld to support a large number of file systems, > and it doesn't scale to build all file system utilities in > xfstests-bld. Also, most of the time, the file system utilities tend > to move at a slower pace, and the requirements for needing a newer > file system utility than what is shipped in the distribution is rarer. > > I have sometimes built a newer version of e2fsprogs as a Debian > package, and installed it into the VM. These days, what I tend to do > when that's been necessary is to build a new Debian package for > jessie-backports, and upload it to the jessie-backports repository, so > everyone can use the same binary package. > > I guess I don't have any objections to allowing people to put locally > compiled versions of the utilities in /root/xfstests/sbin and rely on > the PATH to use that version in preference to the /sbin/mkfs.f2fs > version of the utilities. As far as adding the ability to download > and build local versions of f2fstools in the upstream xfstests-bld, I > guess I could accept that so long as it's optional, and not turned on > by default. The main issue here is that if i have a half dozen file > system-specific utilities included in xfstests-bld, which were all > enabled for download by default, it would increase the build time of > xfstests-bld significantly, so it doesn't scale. I'd much prefer to > see newer versions of the utilities (if necessary) in > debian-backports. Agreed to all of them. What I tried to set up actually was to test the latest f2fs-tools in prior to release out. Anyway, I agree it'd be good to add an option to select local fs utilities through config.custom to avoid the scaling problem. I realize that xfstests-bld is really great to test multiple kernel versions. I guess it can be quite doable to test arm-based kernel as well, as I can see cross compile environment. Thank you for sharing this tool. > > Cheers, > > - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Nov 18, 2016 at 10:04:21AM -0800, Jaegeuk Kim wrote: > > I realize that xfstests-bld is really great to test multiple kernel versions. > I guess it can be quite doable to test arm-based kernel as well, as I can see > cross compile environment. I have had reports that some folks have run into problems using the armhf chroot on an Android device. I'm not sure what problems they ran into, since their report was quite vague. I'm aware of one namespace-related test that will fail when you run xfstests in a chroot, and I would expect that SELinux would have to be disabled, but other than that I would have expected it to work. If you (or anyone else) have managed to use the armhf chroot successfully on an Android device, please let me know, and let me know if there are any updates we can make to Documentation/android-xfstests.md. Unfortunately I haven't had time to investigate those reports myself. If someone has time to try it out and send me a more detailed report, that would be great. Thanks!! - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 17, 2016 at 11:52:33AM -0800, Eric Biggers wrote: > Add basic f2fs support to xfstests-bld. I am using this to verify that > the encryption tests I am adding to xfstests pass on both ext4 and f2fs. > It may be useful to f2fs developers too. For now, only one > configuration, "f2fs/default", is offered. > > For the f2fs tests to work, it's required to build a kernel with f2fs > support and build a rootfs with f2fs-tools installed. For testing > encryption, f2fs-tools must be v1.5.0 or later. > > It was necessary to remove the /etc/fstab lines for /dev/vd[b-f] because > they interfered with non-ext4 fsck. They are not needed for the tests. > > Signed-off-by: Eric Biggers <ebiggers@google.com> Thanks, applied. - Ted -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/kvm-xfstests/test-appliance/files/etc/fstab b/kvm-xfstests/test-appliance/files/etc/fstab index 864478b..27fb1df 100644 --- a/kvm-xfstests/test-appliance/files/etc/fstab +++ b/kvm-xfstests/test-appliance/files/etc/fstab @@ -5,9 +5,4 @@ proc /proc proc defaults 0 0 tmpfs /tmp tmpfs mode=1777 0 0 debugfs /sys/kernel/debug debugfs defaults 0 0 /dev/rootfs / ext4 noatime 0 1 -/dev/vdb /vdb ext4 defaults,noauto 0 0 -/dev/vdc /vdc ext4 defaults,noauto 0 0 -/dev/vdd /vdd ext4 defaults,noauto 0 0 -/dev/vde /vde ext4 defaults,noauto 0 0 -/dev/vdf /vdf ext4 defaults,noauto 0 0 /dev/vdg /results ext4 defaults 0 2 diff --git a/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list new file mode 100644 index 0000000..4ad96d5 --- /dev/null +++ b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list @@ -0,0 +1 @@ +default diff --git a/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default new file mode 100644 index 0000000..616a070 --- /dev/null +++ b/kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default @@ -0,0 +1,4 @@ +SIZE=small +export MKFS_OPTIONS="" +export F2FS_MOUNT_OPTIONS="" +TESTNAME="f2fs" diff --git a/kvm-xfstests/test-appliance/files/root/fs/f2fs/config b/kvm-xfstests/test-appliance/files/root/fs/f2fs/config new file mode 100644 index 0000000..3cab963 --- /dev/null +++ b/kvm-xfstests/test-appliance/files/root/fs/f2fs/config @@ -0,0 +1,62 @@ +# +# Configuration file for f2fs +# + +DEFAULT_MKFS_OPTIONS="" + +function check_filesystem() +{ + local dev="$1" + local ret + + /sbin/fsck.f2fs -f "$dev" + ret="$?" + echo fsck.f2fs exited with status "$ret" + return "$ret" +} + +function format_filesystem() +{ + local dev="$1" + local opts="$2" + + /sbin/mkfs.f2fs -q $opts "$dev" + return $? +} + +function setup_mount_opts() +{ + if test -n "$MNTOPTS" ; then + if test -n "$F2FS_MOUNT_OPTIONS" ; then + F2FS_MOUNT_OPTIONS="$F2FS_MOUNT_OPTIONS,$MNTOPTS" + else + F2FS_MOUNT_OPTIONS="-o $MNTOPTS" + fi + fi +} + +function get_mkfs_opts() +{ + echo "$MKFS_OPTIONS" +} + +function show_mkfs_opts() +{ + echo MKFS_OPTIONS: "$MKFS_OPTIONS" +} + +function show_mount_opts() +{ + echo F2FS_MOUNT_OPTIONS: "$F2FS_MOUNT_OPTIONS" +} + +function test_name_alias() +{ + echo "$1" +} + +function reset_vars() +{ + unset F2FS_MOUNT_OPTIONS + unset MKFS_OPTIONS +} diff --git a/kvm-xfstests/util/parse_cli b/kvm-xfstests/util/parse_cli index ea747be..b42c8ba 100644 --- a/kvm-xfstests/util/parse_cli +++ b/kvm-xfstests/util/parse_cli @@ -59,7 +59,7 @@ print_help () validate_test_name() { case "$1" in - btrfs*|cifs*|ext4*|generic*|shared*|udf*|xfs*|overlay*) ;; + btrfs*|cifs*|ext4*|f2fs*|generic*|shared*|udf*|xfs*|overlay*) ;; *) echo -e "Invalid test name: $1\n" print_help
Add basic f2fs support to xfstests-bld. I am using this to verify that the encryption tests I am adding to xfstests pass on both ext4 and f2fs. It may be useful to f2fs developers too. For now, only one configuration, "f2fs/default", is offered. For the f2fs tests to work, it's required to build a kernel with f2fs support and build a rootfs with f2fs-tools installed. For testing encryption, f2fs-tools must be v1.5.0 or later. It was necessary to remove the /etc/fstab lines for /dev/vd[b-f] because they interfered with non-ext4 fsck. They are not needed for the tests. Signed-off-by: Eric Biggers <ebiggers@google.com> --- kvm-xfstests/test-appliance/files/etc/fstab | 5 -- .../test-appliance/files/root/fs/f2fs/cfg/all.list | 1 + .../test-appliance/files/root/fs/f2fs/cfg/default | 4 ++ .../test-appliance/files/root/fs/f2fs/config | 62 ++++++++++++++++++++++ kvm-xfstests/util/parse_cli | 2 +- 5 files changed, 68 insertions(+), 6 deletions(-) create mode 100644 kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/all.list create mode 100644 kvm-xfstests/test-appliance/files/root/fs/f2fs/cfg/default create mode 100644 kvm-xfstests/test-appliance/files/root/fs/f2fs/config