| Message ID | 162199360248.3744214.17042613373014687643.stgit@locust (mailing list archive) |
|---|---|
| Headers | show |
| Series | fstests: move test group lists into test files | expand |
On Wed, May 26, 2021 at 5:50 AM Darrick J. Wong <djwong@kernel.org> wrote: > > Hi all, > > Test group files (e.g. tests/generic/group) are a pain to keep up. > Every week I rebase on Eryu's latest upstream, and every week I have to > slog through dozens of trivial merge conflicts because of the > groupfiles. Moving tests is annoying because we have to maintain all > this code to move the group associations from one /group file to > another. > > It doesn't need to be this way -- we could move each test's group > information into the test itself, and automatically generate the group > files as part of the make process. This series does exactly that. > This looks very nice :) I do have one concern. If the auto-generated group files keep the same path as the existing source controlled group files, checkout of pre/post this change is going to be challenging for developers running make in the source directory. Also .gitignore entries are needed for the auto-generated group files. I wonder if it wouldn't be easier for everyone if the auto-generated groups had a different name. Thanks, Amir.
On Thu, Jun 03, 2021 at 09:32:16AM +0300, Amir Goldstein wrote: > On Wed, May 26, 2021 at 5:50 AM Darrick J. Wong <djwong@kernel.org> wrote: > > > > Hi all, > > > > Test group files (e.g. tests/generic/group) are a pain to keep up. > > Every week I rebase on Eryu's latest upstream, and every week I have to > > slog through dozens of trivial merge conflicts because of the > > groupfiles. Moving tests is annoying because we have to maintain all > > this code to move the group associations from one /group file to > > another. > > > > It doesn't need to be this way -- we could move each test's group > > information into the test itself, and automatically generate the group > > files as part of the make process. This series does exactly that. > > > > This looks very nice :) > > I do have one concern. > If the auto-generated group files keep the same path as the existing > source controlled group files, checkout of pre/post this change is > going to be challenging for developers running make in the source > directory. <nod> A temporary pain until everyone rebases, but you're right, that's going to annoy people unnecessarily. Admittedly I left it as tests/*/group to avoid making any other changes. All three lines' worth in check. :P > Also .gitignore entries are needed for the auto-generated group files. Heh, oops. Will fix. > I wonder if it wouldn't be easier for everyone if the auto-generated > groups had a different name. Probably. We could blast fstests 25 years into the future and change the name to "hashtag" :P How about "group.map" ? --D > > Thanks, > Amir.
On Thu, Jun 3, 2021 at 6:34 PM Darrick J. Wong <djwong@kernel.org> wrote: > > On Thu, Jun 03, 2021 at 09:32:16AM +0300, Amir Goldstein wrote: > > On Wed, May 26, 2021 at 5:50 AM Darrick J. Wong <djwong@kernel.org> wrote: > > > > > > Hi all, > > > > > > Test group files (e.g. tests/generic/group) are a pain to keep up. > > > Every week I rebase on Eryu's latest upstream, and every week I have to > > > slog through dozens of trivial merge conflicts because of the > > > groupfiles. Moving tests is annoying because we have to maintain all > > > this code to move the group associations from one /group file to > > > another. > > > > > > It doesn't need to be this way -- we could move each test's group > > > information into the test itself, and automatically generate the group > > > files as part of the make process. This series does exactly that. > > > > > > > This looks very nice :) > > > > I do have one concern. > > If the auto-generated group files keep the same path as the existing > > source controlled group files, checkout of pre/post this change is > > going to be challenging for developers running make in the source > > directory. > > <nod> A temporary pain until everyone rebases, but you're right, that's > going to annoy people unnecessarily. > > Admittedly I left it as tests/*/group to avoid making any other changes. > All three lines' worth in check. :P > > > Also .gitignore entries are needed for the auto-generated group files. > > Heh, oops. Will fix. > > > I wonder if it wouldn't be easier for everyone if the auto-generated > > groups had a different name. > > Probably. We could blast fstests 25 years into the future and change > the name to "hashtag" :P > > How about "group.map" ? > <shrug> that's the hardest part ;-) I'm fine with group.map group.list group.autogen or whatnot. I am used to typing the prefix of the group file path in shell commands, but auto-complete should take care of any suffix. Thanks, Amir.
On Thu, Jun 03, 2021 at 07:56:49PM +0300, Amir Goldstein wrote: > On Thu, Jun 3, 2021 at 6:34 PM Darrick J. Wong <djwong@kernel.org> wrote: > > > > On Thu, Jun 03, 2021 at 09:32:16AM +0300, Amir Goldstein wrote: > > > On Wed, May 26, 2021 at 5:50 AM Darrick J. Wong <djwong@kernel.org> wrote: > > > > > > > > Hi all, > > > > > > > > Test group files (e.g. tests/generic/group) are a pain to keep up. > > > > Every week I rebase on Eryu's latest upstream, and every week I have to > > > > slog through dozens of trivial merge conflicts because of the > > > > groupfiles. Moving tests is annoying because we have to maintain all > > > > this code to move the group associations from one /group file to > > > > another. > > > > > > > > It doesn't need to be this way -- we could move each test's group > > > > information into the test itself, and automatically generate the group > > > > files as part of the make process. This series does exactly that. > > > > > > > > > > This looks very nice :) > > > > > > I do have one concern. > > > If the auto-generated group files keep the same path as the existing > > > source controlled group files, checkout of pre/post this change is > > > going to be challenging for developers running make in the source > > > directory. > > > > <nod> A temporary pain until everyone rebases, but you're right, that's > > going to annoy people unnecessarily. > > > > Admittedly I left it as tests/*/group to avoid making any other changes. > > All three lines' worth in check. :P > > > > > Also .gitignore entries are needed for the auto-generated group files. > > > > Heh, oops. Will fix. > > > > > I wonder if it wouldn't be easier for everyone if the auto-generated > > > groups had a different name. > > > > Probably. We could blast fstests 25 years into the future and change > > the name to "hashtag" :P > > > > How about "group.map" ? > > > > <shrug> that's the hardest part ;-) > > I'm fine with group.map group.list group.autogen or whatnot. > > I am used to typing the prefix of the group file path in shell commands, > but auto-complete should take care of any suffix. I went with group.list. --D > Thanks, > Amir.
On Tue, May 25, 2021 at 06:46:42PM -0700, Darrick J. Wong wrote: > Hi all, > > Test group files (e.g. tests/generic/group) are a pain to keep up. > Every week I rebase on Eryu's latest upstream, and every week I have to > slog through dozens of trivial merge conflicts because of the > groupfiles. Moving tests is annoying because we have to maintain all > this code to move the group associations from one /group file to > another. Thanks for doing this! It looks find overall from a quick look. Mind adding some words in README file as well? > > It doesn't need to be this way -- we could move each test's group > information into the test itself, and automatically generate the group > files as part of the make process. This series does exactly that. > > The first few patches add some convenient anchors for the new > per-testfile group tagging and a conversion script to migrate existing > test files. Next there's a huge patch that is the results of running > the conversion script, followed by cleanup of the golden outputs. After > that comes the build infrastructure to generate group files and other The group files are auto-generated correctly upon "make", but "make group" printed tons of errors like: [root@fedoravm xfstests]# make group [GROUP] /root/workspace/xfstests/group /root/workspace/xfstests/tests/btrfs/001: line 9: ./common/test_names: No such file or directory /root/workspace/xfstests/tests/btrfs/001: line 10: _set_seq_and_groups: command not found /root/workspace/xfstests/tests/btrfs/001: line 21: ./common/rc: No such file or directory /root/workspace/xfstests/tests/btrfs/001: line 22: ./common/filter: No such file or directory /root/workspace/xfstests/tests/btrfs/001: line 23: ./common/filter.btrfs: No such file or directory /root/workspace/xfstests/tests/btrfs/001: line 26: _supported_fs: command not found /root/workspace/xfstests/tests/btrfs/001: line 27: _require_scratch: command not found /root/workspace/xfstests/tests/btrfs/001: line 30: _scratch_mount: command not found /root/workspace/xfstests/tests/btrfs/001: line 38: _filter_scratch: command not found /root/workspace/xfstests/tests/btrfs/001: line 38: subvolume: command not found ls: cannot access '/snap': No such file or directory ls: cannot access '/snap': No such file or directory /root/workspace/xfstests/tests/btrfs/001: line 50: _filter_scratch: command not found /root/workspace/xfstests/tests/btrfs/001: line 50: subvolume: command not found ls: cannot access '/subvol': No such file or directory /root/workspace/xfstests/tests/btrfs/001: line 60: _btrfs_get_subvolid: command not found /root/workspace/xfstests/tests/btrfs/001: line 61: _filter_scratch: command not found /root/workspace/xfstests/tests/btrfs/001: line 61: subvolume: command not found /root/workspace/xfstests/tests/btrfs/001: line 62: _scratch_cycle_mount: command not found /root/workspace/xfstests/tests/btrfs/001: line 65: _scratch_unmount: command not found /root/workspace/xfstests/tests/btrfs/001: line 67: _scratch_mount: command not found /root/workspace/xfstests/tests/btrfs/001: line 71: _filter_scratch: command not found /root/workspace/xfstests/tests/btrfs/001: line 71: subvolume: command not found /root/workspace/xfstests/tests/btrfs/001: line 72: _scratch_cycle_mount: command not found /root/workspace/xfstests/tests/btrfs/001: line 78: subvolume: command not found /root/workspace/xfstests/tests/btrfs/001: line 81: _filter_btrfs_subvol_delete: command not found /root/workspace/xfstests/tests/btrfs/001: line 81: subvolume: command not found /root/workspace/xfstests/tests/btrfs/001: line 84: _scratch_cycle_mount: command not found I think it's better to let 'make group' work as well, it might be easier to re-generated just group files. > tweaks to the existing maintainer scripts to use the new infrastructure. > Finally, remove the group files themselves and the (now unnecessary) > code that maintained them. > > If you're going to start using this mess, you probably ought to just > pull from my git trees, which are linked below. > > This is an extraordinary way to destroy everything. Enjoy! > Comments and questions are, as always, welcome. > > --D > > fstests git tree: > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=autogenerate-groupfiles I noticed that your patches are based on your internal base, and this branch couldn't be pulled into current master branch. But this is fine for RFC patches I think :) Thanks, Eryu
On Sun, Jun 06, 2021 at 10:44:39PM +0800, Eryu Guan wrote: > On Tue, May 25, 2021 at 06:46:42PM -0700, Darrick J. Wong wrote: > > Hi all, > > > > Test group files (e.g. tests/generic/group) are a pain to keep up. > > Every week I rebase on Eryu's latest upstream, and every week I have to > > slog through dozens of trivial merge conflicts because of the > > groupfiles. Moving tests is annoying because we have to maintain all > > this code to move the group associations from one /group file to > > another. > > Thanks for doing this! It looks find overall from a quick look. Mind > adding some words in README file as well? Ok. > > It doesn't need to be this way -- we could move each test's group > > information into the test itself, and automatically generate the group > > files as part of the make process. This series does exactly that. > > > > The first few patches add some convenient anchors for the new > > per-testfile group tagging and a conversion script to migrate existing > > test files. Next there's a huge patch that is the results of running > > the conversion script, followed by cleanup of the golden outputs. After > > that comes the build infrastructure to generate group files and other > > The group files are auto-generated correctly upon "make", but "make > group" printed tons of errors like: > > [root@fedoravm xfstests]# make group > [GROUP] /root/workspace/xfstests/group > /root/workspace/xfstests/tests/btrfs/001: line 9: ./common/test_names: No such file or directory > /root/workspace/xfstests/tests/btrfs/001: line 10: _set_seq_and_groups: command not found > /root/workspace/xfstests/tests/btrfs/001: line 21: ./common/rc: No such file or directory > /root/workspace/xfstests/tests/btrfs/001: line 22: ./common/filter: No such file or directory > /root/workspace/xfstests/tests/btrfs/001: line 23: ./common/filter.btrfs: No such file or directory > /root/workspace/xfstests/tests/btrfs/001: line 26: _supported_fs: command not found > /root/workspace/xfstests/tests/btrfs/001: line 27: _require_scratch: command not found > /root/workspace/xfstests/tests/btrfs/001: line 30: _scratch_mount: command not found > /root/workspace/xfstests/tests/btrfs/001: line 38: _filter_scratch: command not found > /root/workspace/xfstests/tests/btrfs/001: line 38: subvolume: command not found > ls: cannot access '/snap': No such file or directory > ls: cannot access '/snap': No such file or directory > /root/workspace/xfstests/tests/btrfs/001: line 50: _filter_scratch: command not found > /root/workspace/xfstests/tests/btrfs/001: line 50: subvolume: command not found > ls: cannot access '/subvol': No such file or directory > /root/workspace/xfstests/tests/btrfs/001: line 60: _btrfs_get_subvolid: command not found > /root/workspace/xfstests/tests/btrfs/001: line 61: _filter_scratch: command not found > /root/workspace/xfstests/tests/btrfs/001: line 61: subvolume: command not found > /root/workspace/xfstests/tests/btrfs/001: line 62: _scratch_cycle_mount: command not found > /root/workspace/xfstests/tests/btrfs/001: line 65: _scratch_unmount: command not found > /root/workspace/xfstests/tests/btrfs/001: line 67: _scratch_mount: command not found > /root/workspace/xfstests/tests/btrfs/001: line 71: _filter_scratch: command not found > /root/workspace/xfstests/tests/btrfs/001: line 71: subvolume: command not found > /root/workspace/xfstests/tests/btrfs/001: line 72: _scratch_cycle_mount: command not found > /root/workspace/xfstests/tests/btrfs/001: line 78: subvolume: command not found > /root/workspace/xfstests/tests/btrfs/001: line 81: _filter_btrfs_subvol_delete: command not found > /root/workspace/xfstests/tests/btrfs/001: line 81: subvolume: command not found > /root/workspace/xfstests/tests/btrfs/001: line 84: _scratch_cycle_mount: command not found > > I think it's better to let 'make group' work as well, it might be easier > to re-generated just group files. Yikes. I will investigate this before reposting. > > tweaks to the existing maintainer scripts to use the new infrastructure. > > Finally, remove the group files themselves and the (now unnecessary) > > code that maintained them. > > > > If you're going to start using this mess, you probably ought to just > > pull from my git trees, which are linked below. > > > > This is an extraordinary way to destroy everything. Enjoy! > > Comments and questions are, as always, welcome. > > > > --D > > > > fstests git tree: > > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=autogenerate-groupfiles > > I noticed that your patches are based on your internal base, and this > branch couldn't be pulled into current master branch. But this is fine > for RFC patches I think :) Yep. Since this is a treewide change I'll put this in a special branch and rebase my dev tree after it lands. --D > > Thanks, > Eryu
On Sun, Jun 06, 2021 at 11:05:09AM -0700, Darrick J. Wong wrote: > On Sun, Jun 06, 2021 at 10:44:39PM +0800, Eryu Guan wrote: > > On Tue, May 25, 2021 at 06:46:42PM -0700, Darrick J. Wong wrote: > > > Hi all, > > > > > > Test group files (e.g. tests/generic/group) are a pain to keep up. > > > Every week I rebase on Eryu's latest upstream, and every week I have to > > > slog through dozens of trivial merge conflicts because of the > > > groupfiles. Moving tests is annoying because we have to maintain all > > > this code to move the group associations from one /group file to > > > another. > > > > Thanks for doing this! It looks find overall from a quick look. Mind > > adding some words in README file as well? > > Ok. > > > > It doesn't need to be this way -- we could move each test's group > > > information into the test itself, and automatically generate the group > > > files as part of the make process. This series does exactly that. > > > > > > The first few patches add some convenient anchors for the new > > > per-testfile group tagging and a conversion script to migrate existing > > > test files. Next there's a huge patch that is the results of running > > > the conversion script, followed by cleanup of the golden outputs. After > > > that comes the build infrastructure to generate group files and other > > > > The group files are auto-generated correctly upon "make", but "make > > group" printed tons of errors like: > > > > [root@fedoravm xfstests]# make group > > [GROUP] /root/workspace/xfstests/group > > /root/workspace/xfstests/tests/btrfs/001: line 9: ./common/test_names: No such file or directory > > /root/workspace/xfstests/tests/btrfs/001: line 10: _set_seq_and_groups: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 21: ./common/rc: No such file or directory > > /root/workspace/xfstests/tests/btrfs/001: line 22: ./common/filter: No such file or directory > > /root/workspace/xfstests/tests/btrfs/001: line 23: ./common/filter.btrfs: No such file or directory > > /root/workspace/xfstests/tests/btrfs/001: line 26: _supported_fs: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 27: _require_scratch: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 30: _scratch_mount: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 38: _filter_scratch: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 38: subvolume: command not found > > ls: cannot access '/snap': No such file or directory > > ls: cannot access '/snap': No such file or directory > > /root/workspace/xfstests/tests/btrfs/001: line 50: _filter_scratch: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 50: subvolume: command not found > > ls: cannot access '/subvol': No such file or directory > > /root/workspace/xfstests/tests/btrfs/001: line 60: _btrfs_get_subvolid: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 61: _filter_scratch: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 61: subvolume: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 62: _scratch_cycle_mount: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 65: _scratch_unmount: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 67: _scratch_mount: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 71: _filter_scratch: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 71: subvolume: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 72: _scratch_cycle_mount: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 78: subvolume: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 81: _filter_btrfs_subvol_delete: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 81: subvolume: command not found > > /root/workspace/xfstests/tests/btrfs/001: line 84: _scratch_cycle_mount: command not found > > > > I think it's better to let 'make group' work as well, it might be easier > > to re-generated just group files. > > Yikes. I will investigate this before reposting. Ah, that's because I put a recipe for group.list in buildrules, even though it is only the tests/xfs/ directories that should ever know about that build rule. I think I can fix it by putting the group.list rule in a separate include file and then only include the new file from tests/*/Makefile. --D > > > tweaks to the existing maintainer scripts to use the new infrastructure. > > > Finally, remove the group files themselves and the (now unnecessary) > > > code that maintained them. > > > > > > If you're going to start using this mess, you probably ought to just > > > pull from my git trees, which are linked below. > > > > > > This is an extraordinary way to destroy everything. Enjoy! > > > Comments and questions are, as always, welcome. > > > > > > --D > > > > > > fstests git tree: > > > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=autogenerate-groupfiles > > > > I noticed that your patches are based on your internal base, and this > > branch couldn't be pulled into current master branch. But this is fine > > for RFC patches I think :) > > Yep. Since this is a treewide change I'll put this in a special branch > and rebase my dev tree after it lands. > > --D > > > > > Thanks, > > Eryu