[v3,0/6] kselftest: add fixture parameters

Message

Jakub Kicinski March 16, 2020, 10:56 p.m. UTC

Hi!

Shuah please consider applying to the kselftest tree.

This set is an attempt to make running tests for different
sets of data easier. The direct motivation is the tls
test which we'd like to run for TLS 1.2 and TLS 1.3,
but currently there is no easy way to invoke the same
tests with different parameters.

Tested all users of kselftest_harness.h.

v2:
 - don't run tests by fixture
 - don't pass params as an explicit argument

v3:
 - go back to the orginal implementation with an extra
   parameter, and running by fixture (Kees);
 - add LIST_APPEND helper (Kees);
 - add a dot between fixture and param name (Kees);
 - rename the params to variants (Tim);

v1: https://lore.kernel.org/netdev/[e-mail]/
v2: https://lore.kernel.org/netdev/[e-mail]/

Jakub Kicinski (6):
  selftests/seccomp: use correct FIXTURE macro
  kselftest: factor out list manipulation to a helper
  kselftest: create fixture objects
  kselftest: run tests by fixture
  kselftest: add fixture variants
  selftests: tls: run all tests for TLS 1.2 and TLS 1.3

 Documentation/dev-tools/kselftest.rst         |   3 +-
 tools/testing/selftests/kselftest_harness.h   | 233 ++++++++++++++----
 tools/testing/selftests/net/tls.c             |  93 ++-----
 tools/testing/selftests/seccomp/seccomp_bpf.c |  10 +-
 4 files changed, 206 insertions(+), 133 deletions(-)

Comments

Jakub Kicinski March 16, 2020, 10:59 p.m. UTC | #1

On Mon, 16 Mar 2020 15:56:40 -0700 Jakub Kicinski wrote:
> Hi!
> 
> Shuah please consider applying to the kselftest tree.
> 
> This set is an attempt to make running tests for different
> sets of data easier. The direct motivation is the tls
> test which we'd like to run for TLS 1.2 and TLS 1.3,
> but currently there is no easy way to invoke the same
> tests with different parameters.
> 
> Tested all users of kselftest_harness.h.
> 
> v2:
>  - don't run tests by fixture
>  - don't pass params as an explicit argument
> 
> v3:
>  - go back to the orginal implementation with an extra
>    parameter, and running by fixture (Kees);
>  - add LIST_APPEND helper (Kees);
>  - add a dot between fixture and param name (Kees);
>  - rename the params to variants (Tim);
> 
> v1: https://lore.kernel.org/netdev/[e-mail]/
> v2: https://lore.kernel.org/netdev/[e-mail]/

Ugh, sorry I forgot to realign things after the rename :S

I'll send a whitespace-only v4 in a hour, allowing a little bit 
of time in case there are some comments already.

Kees Cook March 17, 2020, 8:47 p.m. UTC | #2

On Mon, Mar 16, 2020 at 03:59:17PM -0700, Jakub Kicinski wrote:
> On Mon, 16 Mar 2020 15:56:40 -0700 Jakub Kicinski wrote:
> > Hi!
> > 
> > Shuah please consider applying to the kselftest tree.
> > 
> > This set is an attempt to make running tests for different
> > sets of data easier. The direct motivation is the tls
> > test which we'd like to run for TLS 1.2 and TLS 1.3,
> > but currently there is no easy way to invoke the same
> > tests with different parameters.
> > 
> > Tested all users of kselftest_harness.h.
> > 
> > v2:
> >  - don't run tests by fixture
> >  - don't pass params as an explicit argument
> > 
> > v3:
> >  - go back to the orginal implementation with an extra
> >    parameter, and running by fixture (Kees);
> >  - add LIST_APPEND helper (Kees);
> >  - add a dot between fixture and param name (Kees);
> >  - rename the params to variants (Tim);
> > 
> > v1: https://lore.kernel.org/netdev/[e-mail]/
> > v2: https://lore.kernel.org/netdev/[e-mail]/
> 
> Ugh, sorry I forgot to realign things after the rename :S
> 
> I'll send a whitespace-only v4 in a hour, allowing a little bit 
> of time in case there are some comments already.

No worries! I think a few small changes are needed for a v5 (please
carry my Acked-bys with for v5). Thanks for working on this; I love it!
:)

-Kees