[net-next,v2,0/4] selftests: kselftest_harness: support using xfail

Message

Jakub Kicinski Feb. 16, 2024, 12:26 a.m. UTC

Hi!

When running selftests for our subsystem in our CI we'd like all
tests to pass. Currently some tests use SKIP for cases they
expect to fail, because the kselftest_harness limits the return
codes to pass/fail/skip.

Clean up and support the use of the full range of ksft exit codes
under kselftest_harness.

Merge plan is to put it on top of -rc4 and merge into net-next.
That way others should be able to pull the patches without
any networking changes.

v2:
 - fix alignment
v1: https://lore.kernel.org/all/[e-mail]/

Jakub Kicinski (4):
  selftests: kselftest_harness: pass step via shared memory
  selftests: kselftest_harness: use KSFT_* exit codes
  selftests: kselftest_harness: support using xfail
  selftests: ip_local_port_range: use XFAIL instead of SKIP

 tools/testing/selftests/kselftest_harness.h   | 67 ++++++++++++++-----
 .../selftests/net/ip_local_port_range.c       |  2 +-
 2 files changed, 52 insertions(+), 17 deletions(-)

Comments

Jakub Kicinski Feb. 17, 2024, 1:40 a.m. UTC | #1

On Thu, 15 Feb 2024 16:26:15 -0800 Jakub Kicinski wrote:
> Hi!
> 
> When running selftests for our subsystem in our CI we'd like all
> tests to pass. Currently some tests use SKIP for cases they
> expect to fail, because the kselftest_harness limits the return
> codes to pass/fail/skip.
> 
> Clean up and support the use of the full range of ksft exit codes
> under kselftest_harness.
> 
> Merge plan is to put it on top of -rc4 and merge into net-next.
> That way others should be able to pull the patches without
> any networking changes.

I need to rejig these to follow Kees's suggestion from:

https://lore.kernel.org/all/[e-mail]/