mbox series

[0/3] Make test selection easier by specifying description substrings instead of just numeric counters

Message ID pull.878.git.git.1602545164.gitgitgadget@gmail.com (mailing list archive)
Headers show
Series Make test selection easier by specifying description substrings instead of just numeric counters | expand

Message

Nicolas Guichard via GitGitGadget Oct. 12, 2020, 11:26 p.m. UTC
This patch series enables me to change

 ./t9999-my-test.sh --run=1-3,5,17,19

into

 ./t9999-my-test.sh --run=setup,rename,symlink

and have it pick out tests with "setup", "rename", or "symlink" in their
description and run those. Saves me a lot of time, especially since numbers
for tests aren't readily accessible. The easiest way for me to get the
numbers corresponding to the tests I want to run, is to run all the tests
and look at the output to match up the descriptions with their numbers --
thus defeating the point of selecting just a subset of the tests to run in
the first place.

Elijah Newren (3):
  test-lib: allow selecting tests by substring/regex with --run
  t6006, t6012: adjust tests to use 'setup' instead of synonyms
  test-lib: reduce verbosity of skipped tests

 t/README                     | 26 +++++++++++++++++------
 t/t0000-basic.sh             | 41 ++++++++++++++++++++++--------------
 t/t6006-rev-list-format.sh   |  2 +-
 t/t6012-rev-list-simplify.sh |  2 +-
 t/test-lib.sh                | 15 +++++++------
 5 files changed, 54 insertions(+), 32 deletions(-)


base-commit: d98273ba77e1ab9ec755576bc86c716a97bf59d7
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-878%2Fnewren%2Ftest-selection-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-878/newren/test-selection-v1
Pull-Request: https://github.com/git/git/pull/878

Comments

Phillip Wood Oct. 13, 2020, 1:08 p.m. UTC | #1
Hi Elijah

I'm worryingly excited by this series

On 13/10/2020 00:26, Elijah Newren via GitGitGadget wrote:
> This patch series enables me to change
> 
>   ./t9999-my-test.sh --run=1-3,5,17,19
> 
> into
> 
>   ./t9999-my-test.sh --run=setup,rename,symlink
> 
> and have it pick out tests with "setup", "rename", or "symlink" in their
> description and run those. Saves me a lot of time, especially since numbers
> for tests aren't readily accessible. The easiest way for me to get the
> numbers corresponding to the tests I want to run, is to run all the tests
> and look at the output to match up the descriptions with their numbers --
> thus defeating the point of selecting just a subset of the tests to run in
> the first place.

Yes it's a real pain to have to run all the tests to find the numbers of 
the ones you actually wanted to run. I like patch 3 as well, without it 
if you're only running a couple of tests it can be hard to find then 
amongst all the skipped tests. I had a quick read of the patches and 
they looked fine to me.

Best Wishes

Phillip

> Elijah Newren (3):
>    test-lib: allow selecting tests by substring/regex with --run
>    t6006, t6012: adjust tests to use 'setup' instead of synonyms
>    test-lib: reduce verbosity of skipped tests
> 
>   t/README                     | 26 +++++++++++++++++------
>   t/t0000-basic.sh             | 41 ++++++++++++++++++++++--------------
>   t/t6006-rev-list-format.sh   |  2 +-
>   t/t6012-rev-list-simplify.sh |  2 +-
>   t/test-lib.sh                | 15 +++++++------
>   5 files changed, 54 insertions(+), 32 deletions(-)
> 
> 
> base-commit: d98273ba77e1ab9ec755576bc86c716a97bf59d7
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-878%2Fnewren%2Ftest-selection-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-878/newren/test-selection-v1
> Pull-Request: https://github.com/git/git/pull/878
>
Johannes Schindelin Oct. 15, 2020, 11:48 a.m. UTC | #2
Hi,

On Tue, 13 Oct 2020, Phillip Wood wrote:

> I'm worryingly excited by this series

FWIW I am in the same boat.

> On 13/10/2020 00:26, Elijah Newren via GitGitGadget wrote:
> > This patch series enables me to change
> >
> >   ./t9999-my-test.sh --run=1-3,5,17,19
> >
> > into
> >
> >   ./t9999-my-test.sh --run=setup,rename,symlink
> >
> > and have it pick out tests with "setup", "rename", or "symlink" in their
> > description and run those. Saves me a lot of time, especially since numbers
> > for tests aren't readily accessible. The easiest way for me to get the
> > numbers corresponding to the tests I want to run, is to run all the tests
> > and look at the output to match up the descriptions with their numbers --
> > thus defeating the point of selecting just a subset of the tests to run in
> > the first place.
>
> Yes it's a real pain to have to run all the tests to find the numbers of the
> ones you actually wanted to run. I like patch 3 as well, without it if you're
> only running a couple of tests it can be hard to find then amongst all the
> skipped tests. I had a quick read of the patches and they looked fine to me.

I learned the trick to run with `--run=0` just to figure out the numbers
in a recent Outreachy project, I think it was Slavica who taught me.

But yes, without patch 3, the output looks awfully verbose and you have to
scroll back an insane amount of lines for long test scripts.

Giddy,
Dscho
Phillip Wood Oct. 15, 2020, 1:42 p.m. UTC | #3
On 15/10/2020 12:48, Johannes Schindelin wrote:
> Hi,
> 
> On Tue, 13 Oct 2020, Phillip Wood wrote:
> 
>> I'm worryingly excited by this series
> 
> FWIW I am in the same boat.

I'm glad it's not just me!

>> On 13/10/2020 00:26, Elijah Newren via GitGitGadget wrote:
>>> This patch series enables me to change
>>>
>>>    ./t9999-my-test.sh --run=1-3,5,17,19
>>>
>>> into
>>>
>>>    ./t9999-my-test.sh --run=setup,rename,symlink
>>>
>>> and have it pick out tests with "setup", "rename", or "symlink" in their
>>> description and run those. Saves me a lot of time, especially since numbers
>>> for tests aren't readily accessible. The easiest way for me to get the
>>> numbers corresponding to the tests I want to run, is to run all the tests
>>> and look at the output to match up the descriptions with their numbers --
>>> thus defeating the point of selecting just a subset of the tests to run in
>>> the first place.
>>
>> Yes it's a real pain to have to run all the tests to find the numbers of the
>> ones you actually wanted to run. I like patch 3 as well, without it if you're
>> only running a couple of tests it can be hard to find then amongst all the
>> skipped tests. I had a quick read of the patches and they looked fine to me.
> 
> I learned the trick to run with `--run=0` just to figure out the numbers
> in a recent Outreachy project, I think it was Slavica who taught me.

That's a handy tip, thanks (though hopefully I soon wont need to use it)

Phillip

> But yes, without patch 3, the output looks awfully verbose and you have to
> scroll back an insane amount of lines for long test scripts.
> 
> Giddy,
> Dscho
>