| Message ID | 20240220192235.2953484-1-kuba@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | selftests: kselftest_harness: support using xfail | expand |
On Tue, 2024-02-20 at 11:22 -0800, Jakub Kicinski wrote: > When running selftests for our subsystem in our CI we'd like all > tests to pass. Currently some tests use SKIP for cases they > expect to fail, because the kselftest_harness limits the return > codes to pass/fail/skip. > > Clean up and support the use of the full range of ksft exit codes > under kselftest_harness. > > Merge plan is to put it on top of -rc4 and merge into net-next. > That way others should be able to pull the patches without > any networking changes. > > v2: https://lore.kernel.org/all/20240216002619.1999225-1-kuba@kernel.org/ > - fix alignment > follow up RFC: https://lore.kernel.org/all/20240216004122.2004689-1-kuba@kernel.org/ > v1: https://lore.kernel.org/all/20240213154416.422739-1-kuba@kernel.org/ @Shuah: it's not clear to me if you prefer to take this series via the kselftests tree or we can take it via the net-next tree. Could you please advise? thanks! Paolo p.s. if this was already clarified in the past, I'm sorry: I lost track of it.
On Wed, Feb 21, 2024 at 01:03:26PM +0100, Paolo Abeni wrote: > On Tue, 2024-02-20 at 11:22 -0800, Jakub Kicinski wrote: > > When running selftests for our subsystem in our CI we'd like all > > tests to pass. Currently some tests use SKIP for cases they > > expect to fail, because the kselftest_harness limits the return > > codes to pass/fail/skip. > > > > Clean up and support the use of the full range of ksft exit codes > > under kselftest_harness. > > > > Merge plan is to put it on top of -rc4 and merge into net-next. > > That way others should be able to pull the patches without > > any networking changes. > > > > v2: https://lore.kernel.org/all/20240216002619.1999225-1-kuba@kernel.org/ > > - fix alignment > > follow up RFC: https://lore.kernel.org/all/20240216004122.2004689-1-kuba@kernel.org/ > > v1: https://lore.kernel.org/all/20240213154416.422739-1-kuba@kernel.org/ > > @Shuah: it's not clear to me if you prefer to take this series via the > kselftests tree or we can take it via the net-next tree. Could you > please advise? > > thanks! > > Paolo > > p.s. if this was already clarified in the past, I'm sorry: I lost track > of it. Given the urgency for net-dev and the lack of conflicts with other kselftest changes (AFAICT), I would assume it would be fine to carry this in net-dev (especially since the merge window fast approaches). Shuah, any objection? -Kees
On Tue, 20 Feb 2024 11:22:24 -0800 Jakub Kicinski wrote: > When running selftests for our subsystem in our CI we'd like all > tests to pass. Currently some tests use SKIP for cases they > expect to fail, because the kselftest_harness limits the return > codes to pass/fail/skip. > > Clean up and support the use of the full range of ksft exit codes > under kselftest_harness. > > Merge plan is to put it on top of -rc4 and merge into net-next. > That way others should be able to pull the patches without > any networking changes. Hi Mickaël, would you be able to take a look at those changes? landlock seems to be the sole user of the "no_print" functionality in the selftest harness. If the patches look good I'll create a branch based on Linus's tree so that anyone interested can pull the changes in..
Hi, This is a long due cleanup to merge TEST_F_FORK() into TEST_F(). This should simplify Jakub's patches by removing the step counter: https://lore.kernel.org/r/20240220192235.2953484-1-kuba@kernel.org Regards, Mickaël Salaün (2): selftests/landlock: Redefine TEST_F() as TEST_F_FORK() selftests/harness: Merge TEST_F_FORK() into TEST_F() tools/testing/selftests/kselftest_harness.h | 56 +++++++++----------- tools/testing/selftests/landlock/common.h | 58 +-------------------- 2 files changed, 27 insertions(+), 87 deletions(-)
On Fri, Feb 23, 2024 at 04:02:59PM -0800, Jakub Kicinski wrote: > On Tue, 20 Feb 2024 11:22:24 -0800 Jakub Kicinski wrote: > > When running selftests for our subsystem in our CI we'd like all > > tests to pass. Currently some tests use SKIP for cases they > > expect to fail, because the kselftest_harness limits the return > > codes to pass/fail/skip. > > > > Clean up and support the use of the full range of ksft exit codes > > under kselftest_harness. > > > > Merge plan is to put it on top of -rc4 and merge into net-next. > > That way others should be able to pull the patches without > > any networking changes. > > Hi Mickaël, > > would you be able to take a look at those changes? landlock seems to be > the sole user of the "no_print" functionality in the selftest harness. > If the patches look good I'll create a branch based on Linus's tree > so that anyone interested can pull the changes in.. Hi Jakub, I missed your patches before this series. I just sent two patches to clean things up before you change them. This should simplify your patches and improve the overall maintenance. I'd appreciate if you rebase on top of them. Regards, Mickaël
On Mon, 26 Feb 2024 17:27:34 +0100 Mickaël Salaün wrote: > > would you be able to take a look at those changes? landlock seems to be > > the sole user of the "no_print" functionality in the selftest harness. > > If the patches look good I'll create a branch based on Linus's tree > > so that anyone interested can pull the changes in.. > > Hi Jakub, > > I missed your patches before this series. I just sent two patches to > clean things up before you change them. This should simplify your > patches and improve the overall maintenance. I'd appreciate if you > rebase on top of them. Nice! vfork() is a bit scary so I'll give people a couple of days to review and then post a combined+rebased series.
Hi! When running selftests for our subsystem in our CI we'd like all tests to pass. Currently some tests use SKIP for cases they expect to fail, because the kselftest_harness limits the return codes to pass/fail/skip. Clean up and support the use of the full range of ksft exit codes under kselftest_harness. Merge plan is to put it on top of -rc4 and merge into net-next. That way others should be able to pull the patches without any networking changes. v2: https://lore.kernel.org/all/20240216002619.1999225-1-kuba@kernel.org/ - fix alignment follow up RFC: https://lore.kernel.org/all/20240216004122.2004689-1-kuba@kernel.org/ v1: https://lore.kernel.org/all/20240213154416.422739-1-kuba@kernel.org/ Jakub Kicinski (11): selftests: kselftest_harness: pass step via shared memory selftests: kselftest_harness: use KSFT_* exit codes selftests: kselftest_harness: generate test name once selftests: kselftest_harness: save full exit code in metadata selftests: kselftest_harness: use exit code to store skip selftests: kselftest: add ksft_test_result_code(), handling all exit codes selftests: kselftest_harness: print test name for SKIP selftests: kselftest_harness: separate diagnostic message with # in ksft_test_result_code() selftests: kselftest_harness: let PASS / FAIL provide diagnostic selftests: kselftest_harness: support using xfail selftests: ip_local_port_range: use XFAIL instead of SKIP tools/testing/selftests/kselftest.h | 45 ++++++ tools/testing/selftests/kselftest_harness.h | 148 ++++++++++++------ tools/testing/selftests/landlock/base_test.c | 2 +- tools/testing/selftests/landlock/common.h | 22 +-- tools/testing/selftests/landlock/fs_test.c | 4 +- tools/testing/selftests/landlock/net_test.c | 4 +- .../testing/selftests/landlock/ptrace_test.c | 7 +- .../selftests/net/ip_local_port_range.c | 6 +- tools/testing/selftests/net/tls.c | 2 +- tools/testing/selftests/seccomp/seccomp_bpf.c | 9 +- 10 files changed, 168 insertions(+), 81 deletions(-)