From patchwork Thu Feb 29 00:59:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576248 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79D4C381AA; Thu, 29 Feb 2024 00:59:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168368; cv=none; b=fPlS0eqmWxW5oI9zjV3Vd/UEPEHcKM/7KOciXjvB5MYeQMSyryJQT1ItzU29BsnjQFnskVCX2S87/a9ZMxgit29+ORGmm2vZE+b/5X0cPRxgMrPBFchDl0v0822ljHqfzpf7DBeBrdQuIe3FwLvdXXq9V/Uujue44eN8ejGv15s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168368; c=relaxed/simple; bh=8TDlLakfXfH1TgFjYOG2kAdbF+RRLERm0lTrtPVp+PQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ceA2bvV5i+/JPz0WtMbhAfMmnI6orxZh17r5pgwVfaqjhK6thNLwvs2+jS4vfoc2WYfOR+v8JgYk6DVUW+BYloEvuY7/OQFK6OjFyZIX5/c5KLfrQHZ0A3vvBiD5Gy7+V8PtSIcDXO0j+4wiH0nEUnaZ2GOKDF25+VFWptfKhsM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MBVYyakL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MBVYyakL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BEBE1C433F1; Thu, 29 Feb 2024 00:59:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168368; bh=8TDlLakfXfH1TgFjYOG2kAdbF+RRLERm0lTrtPVp+PQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MBVYyakLuTRcHP6qxqUxu4OiJD4DHGQwWjKnMK9Xct0TUe5cB27D4kPMkduVF6buX N4jx1VcsGRQsY64GSy1TvGqU909NT4hIEDkvQ/h1GFgEd1jXCIji3FBL2pSkx6/IgR 3PEKZmV64X/bIjrbZivzgnI/i7mqYPVJpIMY8Job4Xj+7tZJJ2E9qubVd2ny5txZgh PtGOWrhNwSGBwChLSFfzoNiAVIKPu729WRj8yMbzvR71EevtyQ23JUWh3dHLq/JytY uCxtQKIZO6uzb4AO+NBa63dCE34NJfmaGLmfoWIAuGsPnDRDY3vuG7/Qlrv0tyuduw oIiAYA5U4h0fQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, =?utf-8?q?G=C3=BCnther_Noack?= , Will Drewry , Jakub Kicinski Subject: [PATCH v4 01/12] selftests/landlock: Redefine TEST_F() as TEST_F_FORK() Date: Wed, 28 Feb 2024 16:59:08 -0800 Message-ID: <20240229005920.2407409-2-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Mickaël Salaün This has the effect of creating a new test process for either TEST_F() or TEST_F_FORK(), which doesn't change tests but will ease potential backports. See next commit for the TEST_F_FORK() merge into TEST_F(). Cc: Günther Noack Cc: Kees Cook Cc: Shuah Khan Cc: Will Drewry Signed-off-by: Mickaël Salaün Signed-off-by: Jakub Kicinski --- tools/testing/selftests/landlock/common.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h index e64bbdf0e86e..f40146d40763 100644 --- a/tools/testing/selftests/landlock/common.h +++ b/tools/testing/selftests/landlock/common.h @@ -37,7 +37,7 @@ struct __test_metadata *_metadata, \ FIXTURE_DATA(fixture_name) *self, \ const FIXTURE_VARIANT(fixture_name) *variant); \ - TEST_F(fixture_name, test_name) \ + __TEST_F_IMPL(fixture_name, test_name, -1, TEST_TIMEOUT_DEFAULT) \ { \ int status; \ const pid_t child = fork(); \ @@ -80,6 +80,10 @@ __attribute__((unused)) *variant) /* clang-format on */ +/* Makes backporting easier. */ +#undef TEST_F +#define TEST_F(fixture_name, test_name) TEST_F_FORK(fixture_name, test_name) + #ifndef landlock_create_ruleset static inline int landlock_create_ruleset(const struct landlock_ruleset_attr *const attr, From patchwork Thu Feb 29 00:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576249 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0ED8838384; Thu, 29 Feb 2024 00:59:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168369; cv=none; b=nTn6lKIq3zPAhQU5/nJNZSpzBMUv/81a5A2ZlEKmuM8tcNlo/TfP/nJBXUlTGb0H8LA5LS3sAguhSlCrKK1BrgF5rkc9lXO2UWkkQSV/U9ypfbeSKDmIeoMu8VGyhF24OcVzAipcrfU7KQMNOLJ92xXR+OJCbRdI+gmq4B9KafY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168369; c=relaxed/simple; bh=3W/w3sB6s+o2mYNxB+grh0G+Rt5J8Jq2BZKr1wCas0E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QCRc4qBlTrYC1ggCHRS8TBvDEokWvaqe3Ni14LRC/9kEK9KjFdtUIrtzsuPGFO/B33tybxar0DUVNvciCvD3j2BXcFPZ0f4Ry1Vq3xWLqNfvcIT+D19YwRMV34kyGdiYNjbB8DL+Lk/vhorzexlpFj/VjZBfKPrO0zZbMOHM+bc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=e7ZVFJ2G; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="e7ZVFJ2G" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4F604C43399; Thu, 29 Feb 2024 00:59:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168368; bh=3W/w3sB6s+o2mYNxB+grh0G+Rt5J8Jq2BZKr1wCas0E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e7ZVFJ2G5wgoBU+ebNkm80uoUs1rwxlJMWpgTcbZHSvrSDuI/yLhWEJ45T7nAV1pO kxJD/bL+9y9Ufmzx+vh734OpI+cF99sMJOeNDoxpANeBOoVl8+vdHslAzll+4ehtRK To7Zs5jVq9SibUcGo697DeUPSeRUvNj9ErTLxqNdsdzSNncfyXrINwgtzcSNiI7qdZ 69px+qk67tNFRRjapnJupa5tgE3EQU8I+imQUExpKtCZaYABuGHWocFEjyotm7u9+8 3I0FvJSbCL1J8R/V2+JX5OIz+plUhy7sxwnGfv0n0xCTA5oO+2nXtXi7qWkmgmlryW GNID53GlzD7Pg== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, =?utf-8?q?G=C3=BCnther_Noack?= , Will Drewry , Jakub Kicinski Subject: [PATCH v4 02/12] selftests/harness: Merge TEST_F_FORK() into TEST_F() Date: Wed, 28 Feb 2024 16:59:09 -0800 Message-ID: <20240229005920.2407409-3-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Mickaël Salaün Replace Landlock-specific TEST_F_FORK() with an improved TEST_F() which brings four related changes: Run TEST_F()'s tests in a grandchild process to make it possible to drop privileges and delegate teardown to the parent. Compared to TEST_F_FORK(), simplify handling of the test grandchild process thanks to vfork(2), and makes it generic (e.g. no explicit conversion between exit code and _metadata). Compared to TEST_F_FORK(), run teardown even when tests failed with an assert thanks to commit 63e6b2a42342 ("selftests/harness: Run TEARDOWN for ASSERT failures"). Simplify the test harness code by removing the no_print and step fields which are not used. I added this feature just after I made kselftest_harness.h more broadly available but this step counter remained even though it wasn't needed after all. See commit 369130b63178 ("selftests: Enhance kselftest_harness.h to print which assert failed"). Replace spaces with tabs in one line of __TEST_F_IMPL(). Cc: Günther Noack Cc: Shuah Khan Cc: Will Drewry Signed-off-by: Mickaël Salaün Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- v4: - GAND -> GRAND - init child to 1, otherwise assert in setup triggers a longjmp which in turn reads child without it ever getting initialized (or being 0, i.e. we mistakenly assume we're in the grandchild) --- tools/testing/selftests/kselftest_harness.h | 56 +++++++++---------- tools/testing/selftests/landlock/common.h | 62 +-------------------- 2 files changed, 27 insertions(+), 91 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index e05ac8261046..ad49832457af 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -95,14 +95,6 @@ * E.g., #define TH_LOG_ENABLED 1 * * If no definition is provided, logging is enabled by default. - * - * If there is no way to print an error message for the process running the - * test (e.g. not allowed to write to stderr), it is still possible to get the - * ASSERT_* number for which the test failed. This behavior can be enabled by - * writing `_metadata->no_print = true;` before the check sequence that is - * unable to print. When an error occur, instead of printing an error message - * and calling `abort(3)`, the test process call `_exit(2)` with the assert - * number as argument, which is then printed by the parent process. */ #define TH_LOG(fmt, ...) do { \ if (TH_LOG_ENABLED) \ @@ -363,6 +355,11 @@ * Defines a test that depends on a fixture (e.g., is part of a test case). * Very similar to TEST() except that *self* is the setup instance of fixture's * datatype exposed for use by the implementation. + * + * The @test_name code is run in a separate process sharing the same memory + * (i.e. vfork), which means that the test process can update its privileges + * without impacting the related FIXTURE_TEARDOWN() (e.g. to remove files from + * a directory where write access was dropped). */ #define TEST_F(fixture_name, test_name) \ __TEST_F_IMPL(fixture_name, test_name, -1, TEST_TIMEOUT_DEFAULT) @@ -384,15 +381,28 @@ { \ /* fixture data is alloced, setup, and torn down per call. */ \ FIXTURE_DATA(fixture_name) self; \ + pid_t child = 1; \ memset(&self, 0, sizeof(FIXTURE_DATA(fixture_name))); \ if (setjmp(_metadata->env) == 0) { \ fixture_name##_setup(_metadata, &self, variant->data); \ /* Let setup failure terminate early. */ \ - if (!_metadata->passed || _metadata->skip) \ + if (!_metadata->passed || _metadata->skip) \ return; \ _metadata->setup_completed = true; \ - fixture_name##_##test_name(_metadata, &self, variant->data); \ + /* Use the same _metadata. */ \ + child = vfork(); \ + if (child == 0) { \ + fixture_name##_##test_name(_metadata, &self, variant->data); \ + _exit(0); \ + } \ + if (child < 0) { \ + ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \ + _metadata->passed = 0; \ + } \ } \ + if (child == 0) \ + /* Child failed and updated the shared _metadata. */ \ + _exit(0); \ if (_metadata->setup_completed) \ fixture_name##_teardown(_metadata, &self, variant->data); \ __test_check_assert(_metadata); \ @@ -694,18 +704,12 @@ for (; _metadata->trigger; _metadata->trigger = \ __bail(_assert, _metadata)) -#define __INC_STEP(_metadata) \ - /* Keep "step" below 255 (which is used for "SKIP" reporting). */ \ - if (_metadata->passed && _metadata->step < 253) \ - _metadata->step++; - #define is_signed_type(var) (!!(((__typeof__(var))(-1)) < (__typeof__(var))1)) #define __EXPECT(_expected, _expected_str, _seen, _seen_str, _t, _assert) do { \ /* Avoid multiple evaluation of the cases */ \ __typeof__(_expected) __exp = (_expected); \ __typeof__(_seen) __seen = (_seen); \ - if (_assert) __INC_STEP(_metadata); \ if (!(__exp _t __seen)) { \ /* Report with actual signedness to avoid weird output. */ \ switch (is_signed_type(__exp) * 2 + is_signed_type(__seen)) { \ @@ -751,7 +755,6 @@ #define __EXPECT_STR(_expected, _seen, _t, _assert) do { \ const char *__exp = (_expected); \ const char *__seen = (_seen); \ - if (_assert) __INC_STEP(_metadata); \ if (!(strcmp(__exp, __seen) _t 0)) { \ __TH_LOG("Expected '%s' %s '%s'.", __exp, #_t, __seen); \ _metadata->passed = 0; \ @@ -837,8 +840,6 @@ struct __test_metadata { int trigger; /* extra handler after the evaluation */ int timeout; /* seconds to wait for test timeout */ bool timed_out; /* did this test timeout instead of exiting? */ - __u8 step; - bool no_print; /* manual trigger when TH_LOG_STREAM is not available */ bool aborted; /* stopped test due to failed ASSERT */ bool setup_completed; /* did setup finish? */ jmp_buf env; /* for exiting out of test early */ @@ -873,11 +874,8 @@ static inline int __bail(int for_realz, struct __test_metadata *t) static inline void __test_check_assert(struct __test_metadata *t) { - if (t->aborted) { - if (t->no_print) - _exit(t->step); + if (t->aborted) abort(); - } } struct __test_metadata *__active_test; @@ -954,13 +952,12 @@ void __wait_for_test(struct __test_metadata *t) case 0: t->passed = 1; break; - /* Other failure, assume step report. */ + /* Failure */ default: t->passed = 0; fprintf(TH_LOG_STREAM, - "# %s: Test failed at step #%d\n", - t->name, - WEXITSTATUS(status)); + "# %s: Test failed\n", + t->name); } } } else if (WIFSIGNALED(status)) { @@ -1114,8 +1111,6 @@ void __run_test(struct __fixture_metadata *f, t->passed = 1; t->skip = 0; t->trigger = 0; - t->step = 1; - t->no_print = 0; memset(t->results->reason, 0, sizeof(t->results->reason)); ksft_print_msg(" RUN %s%s%s.%s ...\n", @@ -1137,8 +1132,7 @@ void __run_test(struct __fixture_metadata *f, /* Pass is exit 0 */ if (t->passed) _exit(0); - /* Something else happened, report the step. */ - _exit(t->step); + _exit(1); } else { __wait_for_test(t); } diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h index f40146d40763..401e2eb092a3 100644 --- a/tools/testing/selftests/landlock/common.h +++ b/tools/testing/selftests/landlock/common.h @@ -23,66 +23,8 @@ #define __maybe_unused __attribute__((__unused__)) #endif -/* - * TEST_F_FORK() is useful when a test drop privileges but the corresponding - * FIXTURE_TEARDOWN() requires them (e.g. to remove files from a directory - * where write actions are denied). For convenience, FIXTURE_TEARDOWN() is - * also called when the test failed, but not when FIXTURE_SETUP() failed. For - * this to be possible, we must not call abort() but instead exit smoothly - * (hence the step print). - */ -/* clang-format off */ -#define TEST_F_FORK(fixture_name, test_name) \ - static void fixture_name##_##test_name##_child( \ - struct __test_metadata *_metadata, \ - FIXTURE_DATA(fixture_name) *self, \ - const FIXTURE_VARIANT(fixture_name) *variant); \ - __TEST_F_IMPL(fixture_name, test_name, -1, TEST_TIMEOUT_DEFAULT) \ - { \ - int status; \ - const pid_t child = fork(); \ - if (child < 0) \ - abort(); \ - if (child == 0) { \ - _metadata->no_print = 1; \ - fixture_name##_##test_name##_child(_metadata, self, variant); \ - if (_metadata->skip) \ - _exit(255); \ - if (_metadata->passed) \ - _exit(0); \ - _exit(_metadata->step); \ - } \ - if (child != waitpid(child, &status, 0)) \ - abort(); \ - if (WIFSIGNALED(status) || !WIFEXITED(status)) { \ - _metadata->passed = 0; \ - _metadata->step = 1; \ - return; \ - } \ - switch (WEXITSTATUS(status)) { \ - case 0: \ - _metadata->passed = 1; \ - break; \ - case 255: \ - _metadata->passed = 1; \ - _metadata->skip = 1; \ - break; \ - default: \ - _metadata->passed = 0; \ - _metadata->step = WEXITSTATUS(status); \ - break; \ - } \ - } \ - static void fixture_name##_##test_name##_child( \ - struct __test_metadata __attribute__((unused)) *_metadata, \ - FIXTURE_DATA(fixture_name) __attribute__((unused)) *self, \ - const FIXTURE_VARIANT(fixture_name) \ - __attribute__((unused)) *variant) -/* clang-format on */ - -/* Makes backporting easier. */ -#undef TEST_F -#define TEST_F(fixture_name, test_name) TEST_F_FORK(fixture_name, test_name) +/* TEST_F_FORK() should not be used for new tests. */ +#define TEST_F_FORK(fixture_name, test_name) TEST_F(fixture_name, test_name) #ifndef landlock_create_ruleset static inline int From patchwork Thu Feb 29 00:59:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576250 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69CBB383A6; Thu, 29 Feb 2024 00:59:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168369; cv=none; b=D992hQ5S5DMnM+7JeL8D0n9fyu6NuX67O38nWb/nag2vWHejBwas3lVS2MQtmtomDXNCBR5UEN0KtJyxy7wMd/bji63qlkyLnuOhOywodiMYrxHINIVaB6c6U9JwIloT/RQvPLqX7SwoEaUfPBtnNmowtymazmvlPT0EQ6tlfJ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168369; c=relaxed/simple; bh=bjj80Pj8TBirff2iGjilAsXR8Ov3uSNGQxuJFTzd6G8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JoxuZzQ8v5x/tihCrQPhxbAG3UCwKJ3uV5c5/zqXhAot1SApWH2NNmKNXY1BB92CGUBSRaLQLrqZ0/OairISCL8nuqvb5nt4W/w69aJf4LuTkNkCirK5wV9a1lTndavFku3wYnwnNz4Pa5gEYxnWGTyeGmUPsKfD9GImSVZvzkg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uGyglxSn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uGyglxSn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D953AC433C7; Thu, 29 Feb 2024 00:59:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168369; bh=bjj80Pj8TBirff2iGjilAsXR8Ov3uSNGQxuJFTzd6G8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uGyglxSn9kY+W8+CIhSmYrcJTHl6+H0ODObTgT1NlTXfFgmXqlfMo2gz/2CoCaCIo 7MA6b/k9fO1Svk1e74kekY8FpBIDPWPe53wuwGCfUNI31/usk2XJCd9h4M0CGB8JKZ ezm7Mu8T/jn2oVMjH6MubYJMF6R70ZSvzyQ0+1pfoSc2O1fa/ue1zMM6kJtQMFbXM8 HsuqtMvYg4fTLYi47g1N8Ca6LXh3i7wCRrp+ylZOMsVoVpGe8juc/fxrxvTil+PGt5 XDY8LAz/9TXbTBtWjiZE/c7ju1fY8wh7OsHMtBp4j1gNJSOyccnOnq3hHAG19y5JH4 5QrXOIs4sLupQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski , luto@amacapital.net, wad@chromium.org Subject: [PATCH v4 03/12] selftests: kselftest_harness: use KSFT_* exit codes Date: Wed, 28 Feb 2024 16:59:10 -0800 Message-ID: <20240229005920.2407409-4-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now that we no longer need low exit codes to communicate assertion steps - use normal KSFT exit codes. Acked-by: Kees Cook Tested-by: Jakub Sitnicki Signed-off-by: Jakub Kicinski --- CC: keescook@chromium.org CC: luto@amacapital.net CC: wad@chromium.org CC: shuah@kernel.org CC: mic@digikod.net CC: linux-kselftest@vger.kernel.org CC: linux-security-module@vger.kernel.org --- tools/testing/selftests/kselftest_harness.h | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index ad49832457af..62ce258b0853 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -936,7 +936,7 @@ void __wait_for_test(struct __test_metadata *t) fprintf(TH_LOG_STREAM, "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { - if (WEXITSTATUS(status) == 255) { + if (WEXITSTATUS(status) == KSFT_SKIP) { /* SKIP */ t->passed = 1; t->skip = 1; @@ -949,7 +949,7 @@ void __wait_for_test(struct __test_metadata *t) } else { switch (WEXITSTATUS(status)) { /* Success */ - case 0: + case KSFT_PASS: t->passed = 1; break; /* Failure */ @@ -1128,11 +1128,10 @@ void __run_test(struct __fixture_metadata *f, setpgrp(); t->fn(t, variant); if (t->skip) - _exit(255); - /* Pass is exit 0 */ + _exit(KSFT_SKIP); if (t->passed) - _exit(0); - _exit(1); + _exit(KSFT_PASS); + _exit(KSFT_FAIL); } else { __wait_for_test(t); } From patchwork Thu Feb 29 00:59:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576251 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BD6338DFB; Thu, 29 Feb 2024 00:59:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168370; cv=none; b=sM2AyJXG/64njxTIA+BvKoKUDVdWdlVdO3npEgNLQY7AyfLn5njSghHSw/prfs0p8GDWybzeqxXuZ5REfY/nUlHqk/jBWU802xGBtkeTtzGJcBn/hmyEZBwnciFzrIv7w9dBRjns60t0xtaMrfUV4FW+NLpew3gROuqY1uhP5+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168370; c=relaxed/simple; bh=MVloGXNlog3iP7Ml6839weCrZDYXR4a68tqdF53teYE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m1CoeV7x6dvB8DUeXJZvSe7OcRIXhuXNITKBxVM2EppRUSOL+zaLMxT5OOTT1gszf/0YU/a+kS7+tlcc8KNS8MY7dSoJ9ZikDsuQ5nc1vPhYtnSh3dhTASXP62AXsOaRDa5FovkYMQmLZP2dW9A0IHk+ce7LQYrwo7AAAA8Lxsc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hUgVMPJY; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hUgVMPJY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70608C43390; Thu, 29 Feb 2024 00:59:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168369; bh=MVloGXNlog3iP7Ml6839weCrZDYXR4a68tqdF53teYE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hUgVMPJYIF7T9z4DdE1EavlhiXX/RzU8rxJFwNBbLUQG+AuSqsjUXLkSngAULuajl Fkupd3eSe1pQmFTimelUzRbfjAPtf1HhA7vUUSe0TVVXIKwPopk4Eof7RYqjGKGCE1 6zW/FbEP37ZgYbP9mfsKM1wi4qJTK1sYRMIxLox2zpG4/DucxWSyJYDNN44gZGaXsa N3L00YqWc/BUK9jYNPweFNXnlymtqMGIxtfvGCGOY2+/csuR4BS/cbvgQzlng2kMyd +6IGZ5CQtAe9kgkXhGyn3gsw8ZFBrByZdjgLfp+oQXodAsrTR+Pjz4yFbO6RGKM23A ZcPLQ7/wubwRQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 04/12] selftests: kselftest_harness: generate test name once Date: Wed, 28 Feb 2024 16:59:11 -0800 Message-ID: <20240229005920.2407409-5-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since we added variant support generating full test case name takes 4 string arguments. We're about to need it in another two places. Stop the duplication and print once into a temporary buffer. Suggested-by: Jakub Sitnicki Acked-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest_harness.h | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 62ce258b0853..4a2bda6a67ed 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -56,6 +56,7 @@ #include #include #include +#include #include #include #include @@ -1107,14 +1108,18 @@ void __run_test(struct __fixture_metadata *f, struct __fixture_variant_metadata *variant, struct __test_metadata *t) { + char test_name[LINE_MAX]; + /* reset test struct */ t->passed = 1; t->skip = 0; t->trigger = 0; memset(t->results->reason, 0, sizeof(t->results->reason)); - ksft_print_msg(" RUN %s%s%s.%s ...\n", - f->name, variant->name[0] ? "." : "", variant->name, t->name); + snprintf(test_name, sizeof(test_name), "%s%s%s.%s", + f->name, variant->name[0] ? "." : "", variant->name, t->name); + + ksft_print_msg(" RUN %s ...\n", test_name); /* Make sure output buffers are flushed before fork */ fflush(stdout); @@ -1135,15 +1140,14 @@ void __run_test(struct __fixture_metadata *f, } else { __wait_for_test(t); } - ksft_print_msg(" %4s %s%s%s.%s\n", t->passed ? "OK" : "FAIL", - f->name, variant->name[0] ? "." : "", variant->name, t->name); + ksft_print_msg(" %4s %s\n", + t->passed ? "OK" : "FAIL", test_name); if (t->skip) ksft_test_result_skip("%s\n", t->results->reason[0] ? t->results->reason : "unknown"); else - ksft_test_result(t->passed, "%s%s%s.%s\n", - f->name, variant->name[0] ? "." : "", variant->name, t->name); + ksft_test_result(t->passed, "%s\n", test_name); } static int test_harness_run(int argc, char **argv) From patchwork Thu Feb 29 00:59:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576252 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C352D38384; Thu, 29 Feb 2024 00:59:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168370; cv=none; b=DbEI3loKZuCDuzdrPHbgWrURPjW+VzZAOHsaRWdR6o0f202XOYqTWm00uDaIerSHUdRjKVdkWvTy8FPoImTa85+kH4gsvdzS7GzvKbqoez91DVx3hI4rHk9D3SMb3kUXutwaCJDgOXq+J0aJ6uNdQb0RVQEauweEHH7bgvooMa8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168370; c=relaxed/simple; bh=hXaWz/me8BWakiJe20Wvd6tHJti2IGMhZfsrdezOZSY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V85qDFK5id+v4HTALhL0rcXKXhzREYCSwAK61fpmI9WpD5wGs9jjHdMMafnjU/7g4RtvOyFbe+4putv43pkt9QfoZIRSqUJf2uVbjD0k25AdaUhcXY7eIhpXtSraSybr1sIxKrctDEfIAm+XWoeh10Kp8/+30jvgvdJOODLQeq4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BACuFpFb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BACuFpFb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC83AC43399; Thu, 29 Feb 2024 00:59:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168370; bh=hXaWz/me8BWakiJe20Wvd6tHJti2IGMhZfsrdezOZSY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BACuFpFb/PGmn1HvbZ26qenOTqPmJzL6fWwmKEi+ZB1qmWncXlJSZWMYpV+289T95 5cz097RCRKLFvKIm6UJXKaEbmMtOlJuo61gOVL1hknsFUU51oqigrdUw5ZDocBX+kv O4xVUKkeUX10Bn+uTbSHfLWiee2HeQpw9vdiWokl/CTG44BbcLrSvM9Zd+p5zTriaI RpmMuvSqjPT9fn24ojFaFySI4wE8E5U1SUwgsa/sR5IJ+wJSWyHh3PfflkytBCPNzV xyS8MeQIDoAOVP2myQrhswo9d5Z/LBdTPJItCM+BKpgKBnaNAgp2iRIYrXIEaXMEcU vr/dKPKKbM7eQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 05/12] selftests: kselftest_harness: save full exit code in metadata Date: Wed, 28 Feb 2024 16:59:12 -0800 Message-ID: <20240229005920.2407409-6-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Instead of tracking passed = 0/1 rename the field to exit_code and invert the values so that they match the KSFT_* exit codes. This will allow us to fold SKIP / XFAIL into the same value. Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest_harness.h | 48 +++++++++++-------- tools/testing/selftests/landlock/base_test.c | 2 +- tools/testing/selftests/landlock/fs_test.c | 4 +- tools/testing/selftests/landlock/net_test.c | 4 +- .../testing/selftests/landlock/ptrace_test.c | 7 +-- tools/testing/selftests/net/tls.c | 2 +- tools/testing/selftests/seccomp/seccomp_bpf.c | 9 ++-- 7 files changed, 41 insertions(+), 35 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 4a2bda6a67ed..d90d4a9039ee 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -128,7 +128,7 @@ fprintf(TH_LOG_STREAM, "# SKIP %s\n", \ _metadata->results->reason); \ } \ - _metadata->passed = 1; \ + _metadata->exit_code = KSFT_PASS; \ _metadata->skip = 1; \ _metadata->trigger = 0; \ statement; \ @@ -387,7 +387,7 @@ if (setjmp(_metadata->env) == 0) { \ fixture_name##_setup(_metadata, &self, variant->data); \ /* Let setup failure terminate early. */ \ - if (!_metadata->passed || _metadata->skip) \ + if (!__test_passed(_metadata) || _metadata->skip) \ return; \ _metadata->setup_completed = true; \ /* Use the same _metadata. */ \ @@ -398,7 +398,7 @@ } \ if (child < 0) { \ ksft_print_msg("ERROR SPAWNING TEST GRANDCHILD\n"); \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ } \ } \ if (child == 0) \ @@ -747,7 +747,7 @@ break; \ } \ } \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ /* Ensure the optional handler is triggered */ \ _metadata->trigger = 1; \ } \ @@ -758,7 +758,7 @@ const char *__seen = (_seen); \ if (!(strcmp(__exp, __seen) _t 0)) { \ __TH_LOG("Expected '%s' %s '%s'.", __exp, #_t, __seen); \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ _metadata->trigger = 1; \ } \ } while (0); OPTIONAL_HANDLER(_assert) @@ -836,7 +836,7 @@ struct __test_metadata { pid_t pid; /* pid of test when being run */ struct __fixture_metadata *fixture; int termsig; - int passed; + int exit_code; int skip; /* did SKIP get used? */ int trigger; /* extra handler after the evaluation */ int timeout; /* seconds to wait for test timeout */ @@ -848,6 +848,12 @@ struct __test_metadata { struct __test_metadata *prev, *next; }; +static inline bool __test_passed(struct __test_metadata *metadata) +{ + return metadata->exit_code != KSFT_FAIL && + metadata->exit_code <= KSFT_SKIP; +} + /* * Since constructors are called in reverse order, reverse the test * list so tests are run in source declaration order. @@ -912,7 +918,7 @@ void __wait_for_test(struct __test_metadata *t) int status; if (sigaction(SIGALRM, &action, &saved_action)) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: unable to install SIGALRM handler\n", t->name); @@ -924,7 +930,7 @@ void __wait_for_test(struct __test_metadata *t) waitpid(t->pid, &status, 0); alarm(0); if (sigaction(SIGALRM, &saved_action, NULL)) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: unable to uninstall SIGALRM handler\n", t->name); @@ -933,16 +939,16 @@ void __wait_for_test(struct __test_metadata *t) __active_test = NULL; if (t->timed_out) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { if (WEXITSTATUS(status) == KSFT_SKIP) { /* SKIP */ - t->passed = 1; + t->exit_code = KSFT_PASS; t->skip = 1; } else if (t->termsig != -1) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: Test exited normally instead of by signal (code: %d)\n", t->name, @@ -951,24 +957,24 @@ void __wait_for_test(struct __test_metadata *t) switch (WEXITSTATUS(status)) { /* Success */ case KSFT_PASS: - t->passed = 1; + t->exit_code = KSFT_PASS; break; /* Failure */ default: - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: Test failed\n", t->name); } } } else if (WIFSIGNALED(status)) { - t->passed = 0; + t->exit_code = KSFT_FAIL; if (WTERMSIG(status) == SIGABRT) { fprintf(TH_LOG_STREAM, "# %s: Test terminated by assertion\n", t->name); } else if (WTERMSIG(status) == t->termsig) { - t->passed = 1; + t->exit_code = KSFT_PASS; } else { fprintf(TH_LOG_STREAM, "# %s: Test terminated unexpectedly by signal %d\n", @@ -1111,7 +1117,7 @@ void __run_test(struct __fixture_metadata *f, char test_name[LINE_MAX]; /* reset test struct */ - t->passed = 1; + t->exit_code = KSFT_PASS; t->skip = 0; t->trigger = 0; memset(t->results->reason, 0, sizeof(t->results->reason)); @@ -1128,26 +1134,26 @@ void __run_test(struct __fixture_metadata *f, t->pid = fork(); if (t->pid < 0) { ksft_print_msg("ERROR SPAWNING TEST CHILD\n"); - t->passed = 0; + t->exit_code = KSFT_FAIL; } else if (t->pid == 0) { setpgrp(); t->fn(t, variant); if (t->skip) _exit(KSFT_SKIP); - if (t->passed) + if (__test_passed(t)) _exit(KSFT_PASS); _exit(KSFT_FAIL); } else { __wait_for_test(t); } ksft_print_msg(" %4s %s\n", - t->passed ? "OK" : "FAIL", test_name); + __test_passed(t) ? "OK" : "FAIL", test_name); if (t->skip) ksft_test_result_skip("%s\n", t->results->reason[0] ? t->results->reason : "unknown"); else - ksft_test_result(t->passed, "%s\n", test_name); + ksft_test_result(__test_passed(t), "%s\n", test_name); } static int test_harness_run(int argc, char **argv) @@ -1195,7 +1201,7 @@ static int test_harness_run(int argc, char **argv) t->results = results; __run_test(f, v, t); t->results = NULL; - if (t->passed) + if (__test_passed(t)) pass_count++; else ret = 1; diff --git a/tools/testing/selftests/landlock/base_test.c b/tools/testing/selftests/landlock/base_test.c index 646f778dfb1e..a6f89aaea77d 100644 --- a/tools/testing/selftests/landlock/base_test.c +++ b/tools/testing/selftests/landlock/base_test.c @@ -307,7 +307,7 @@ TEST(ruleset_fd_transfer) dir_fd = open("/tmp", O_RDONLY | O_DIRECTORY | O_CLOEXEC); ASSERT_LE(0, dir_fd); ASSERT_EQ(0, close(dir_fd)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } diff --git a/tools/testing/selftests/landlock/fs_test.c b/tools/testing/selftests/landlock/fs_test.c index 2d6d9b43d958..98817a14c91b 100644 --- a/tools/testing/selftests/landlock/fs_test.c +++ b/tools/testing/selftests/landlock/fs_test.c @@ -1964,7 +1964,7 @@ static void test_execute(struct __test_metadata *const _metadata, const int err, strerror(errno)); }; ASSERT_EQ(err, errno); - _exit(_metadata->passed ? 2 : 1); + _exit(__test_passed(_metadata) ? 2 : 1); return; } ASSERT_EQ(child, waitpid(child, &status, 0)); @@ -3807,7 +3807,7 @@ TEST_F_FORK(ftruncate, open_and_ftruncate_in_different_processes) ASSERT_EQ(0, close(socket_fds[0])); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } diff --git a/tools/testing/selftests/landlock/net_test.c b/tools/testing/selftests/landlock/net_test.c index 936cfc879f1d..f21cfbbc3638 100644 --- a/tools/testing/selftests/landlock/net_test.c +++ b/tools/testing/selftests/landlock/net_test.c @@ -539,7 +539,7 @@ static void test_bind_and_connect(struct __test_metadata *const _metadata, } EXPECT_EQ(0, close(connect_fd)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } @@ -834,7 +834,7 @@ TEST_F(protocol, connect_unspec) } EXPECT_EQ(0, close(connect_fd)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } diff --git a/tools/testing/selftests/landlock/ptrace_test.c b/tools/testing/selftests/landlock/ptrace_test.c index 55e7871631a1..a19db4d0b3bd 100644 --- a/tools/testing/selftests/landlock/ptrace_test.c +++ b/tools/testing/selftests/landlock/ptrace_test.c @@ -314,7 +314,7 @@ TEST_F(hierarchy, trace) ASSERT_EQ(0, pipe2(pipe_parent, O_CLOEXEC)); if (variant->domain_both) { create_domain(_metadata); - if (!_metadata->passed) + if (!__test_passed(_metadata)) /* Aborts before forking. */ return; } @@ -375,7 +375,7 @@ TEST_F(hierarchy, trace) /* Waits for the parent PTRACE_ATTACH test. */ ASSERT_EQ(1, read(pipe_parent[0], &buf_child, 1)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } @@ -430,9 +430,10 @@ TEST_F(hierarchy, trace) /* Signals that the parent PTRACE_ATTACH test is done. */ ASSERT_EQ(1, write(pipe_parent[1], ".", 1)); ASSERT_EQ(child, waitpid(child, &status, 0)); + if (WIFSIGNALED(status) || !WIFEXITED(status) || WEXITSTATUS(status) != EXIT_SUCCESS) - _metadata->passed = 0; + _metadata->exit_code = KSFT_FAIL; } TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c index b95c249f81c2..c6eda21cefb6 100644 --- a/tools/testing/selftests/net/tls.c +++ b/tools/testing/selftests/net/tls.c @@ -1927,7 +1927,7 @@ TEST_F(tls_err, poll_partial_rec_async) pfd.events = POLLIN; EXPECT_EQ(poll(&pfd, 1, 20), 1); - exit(!_metadata->passed); + exit(!__test_passed(_metadata)); } } diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c index 38f651469968..1027e6170186 100644 --- a/tools/testing/selftests/seccomp/seccomp_bpf.c +++ b/tools/testing/selftests/seccomp/seccomp_bpf.c @@ -1576,7 +1576,7 @@ void start_tracer(struct __test_metadata *_metadata, int fd, pid_t tracee, ASSERT_EQ(0, ret); } /* Directly report the status of our test harness results. */ - syscall(__NR_exit, _metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + syscall(__NR_exit, _metadata->exit_code); } /* Common tracer setup/teardown functions. */ @@ -1623,7 +1623,7 @@ void teardown_trace_fixture(struct __test_metadata *_metadata, ASSERT_EQ(0, kill(tracer, SIGUSR1)); ASSERT_EQ(tracer, waitpid(tracer, &status, 0)); if (WEXITSTATUS(status)) - _metadata->passed = 0; + _metadata->exit_code = KSFT_FAIL; } } @@ -3088,8 +3088,7 @@ TEST(syscall_restart) } /* Directly report the status of our test harness results. */ - syscall(__NR_exit, _metadata->passed ? EXIT_SUCCESS - : EXIT_FAILURE); + syscall(__NR_exit, _metadata->exit_code); } EXPECT_EQ(0, close(pipefd[0])); @@ -3174,7 +3173,7 @@ TEST(syscall_restart) ASSERT_EQ(child_pid, waitpid(child_pid, &status, 0)); if (WIFSIGNALED(status) || WEXITSTATUS(status)) - _metadata->passed = 0; + _metadata->exit_code = KSFT_FAIL; } TEST_SIGNAL(filter_flag_log, SIGSYS) From patchwork Thu Feb 29 00:59:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576253 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 548D83A264; Thu, 29 Feb 2024 00:59:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168371; cv=none; b=R39PqiRxDRSUF72vNUPAY1P31FpSblccHyyJTkek5I/Tm07taiB298RVvXo7xTMeyHdmrP1S+T7/0NNA/M7jrRydrS8TCdEJTFhutIjESkv9NKi1192ZSOS5W0A2CFi+3h1+DDrN9u708Jq7TX4rtQkImeQMXD4ed52+r4aLF8U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168371; c=relaxed/simple; bh=1uzNaoOrtVsgyeQyEd2SsqCkXw4FwnFerhQ5aFrZi6Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i5gY1zy8X0n6exy8TlLI3Aff3WQifY/CvqB08nd6P6dsjqGKCUBPVHFtchElC+Fu+My9ldNjso5MCQ3uhRP2JyjBy+9WAQEV3GiFDVpINGoXqsmeVtYsAMIwK8NjtE1yaD8TnazB9EpvxVTxIW3IVQwcRRZUnj73MG1Bc/f8b3w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LgAQ9l80; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LgAQ9l80" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7D655C433F1; Thu, 29 Feb 2024 00:59:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168370; bh=1uzNaoOrtVsgyeQyEd2SsqCkXw4FwnFerhQ5aFrZi6Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LgAQ9l80B7H03f3KMAw+5/WzafKQAa844T7WyfFYxq1kx/fATm8iAtQaiqPjEExrz 2tc660aGzeZIaV4BSsdxjdrQxS/QnlquLivqJ5tW9+n4zWeGOIcpoGw5a1Er94RpXk NtU3qHknYeLvvq6Ba7Os8r9cEGCdi3U+oHz6MDl4kiQ1chbUyOKWmddYfm4aHNoOez DVWQ1UgWgI277apLF02gpnRCDV3TTsxKaQ/B+PJZn0Bu542iw+ivD9wWw8EXs/9dav O9WZmtwdnFD8n56m4RHlU0CGCbUUokZz5wzQAaXrrKxmv3pQKORy6u5XlCGpp3j3V1 u8XtOUkCJeWvA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 06/12] selftests: kselftest_harness: use exit code to store skip Date: Wed, 28 Feb 2024 16:59:13 -0800 Message-ID: <20240229005920.2407409-7-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 We always use skip in combination with exit_code being 0 (KSFT_PASS). This are basic KSFT / KTAP semantics. Store the right KSFT_* code in exit_code directly. This makes it easier to support tests reporting other extended KSFT_* codes like XFAIL / XPASS. Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest_harness.h | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index d90d4a9039ee..5a48177c8c00 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -128,8 +128,7 @@ fprintf(TH_LOG_STREAM, "# SKIP %s\n", \ _metadata->results->reason); \ } \ - _metadata->exit_code = KSFT_PASS; \ - _metadata->skip = 1; \ + _metadata->exit_code = KSFT_SKIP; \ _metadata->trigger = 0; \ statement; \ } while (0) @@ -387,7 +386,7 @@ if (setjmp(_metadata->env) == 0) { \ fixture_name##_setup(_metadata, &self, variant->data); \ /* Let setup failure terminate early. */ \ - if (!__test_passed(_metadata) || _metadata->skip) \ + if (_metadata->exit_code) \ return; \ _metadata->setup_completed = true; \ /* Use the same _metadata. */ \ @@ -837,7 +836,6 @@ struct __test_metadata { struct __fixture_metadata *fixture; int termsig; int exit_code; - int skip; /* did SKIP get used? */ int trigger; /* extra handler after the evaluation */ int timeout; /* seconds to wait for test timeout */ bool timed_out; /* did this test timeout instead of exiting? */ @@ -944,9 +942,7 @@ void __wait_for_test(struct __test_metadata *t) "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { if (WEXITSTATUS(status) == KSFT_SKIP) { - /* SKIP */ - t->exit_code = KSFT_PASS; - t->skip = 1; + t->exit_code = WEXITSTATUS(status); } else if (t->termsig != -1) { t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, @@ -1118,7 +1114,6 @@ void __run_test(struct __fixture_metadata *f, /* reset test struct */ t->exit_code = KSFT_PASS; - t->skip = 0; t->trigger = 0; memset(t->results->reason, 0, sizeof(t->results->reason)); @@ -1138,18 +1133,14 @@ void __run_test(struct __fixture_metadata *f, } else if (t->pid == 0) { setpgrp(); t->fn(t, variant); - if (t->skip) - _exit(KSFT_SKIP); - if (__test_passed(t)) - _exit(KSFT_PASS); - _exit(KSFT_FAIL); + _exit(t->exit_code); } else { __wait_for_test(t); } ksft_print_msg(" %4s %s\n", __test_passed(t) ? "OK" : "FAIL", test_name); - if (t->skip) + if (t->exit_code == KSFT_SKIP) ksft_test_result_skip("%s\n", t->results->reason[0] ? t->results->reason : "unknown"); else From patchwork Thu Feb 29 00:59:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576254 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 808403A8F9; Thu, 29 Feb 2024 00:59:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168371; cv=none; b=WOLB6XAKBaaicOGm/eOa3TgkUGSrYDZgMIt4y+lC6lWevSpgoacAWAGyfuQe8U/gojxHQXsDNKBxTxpZTIBff3KvSrWu3hmpCez7bchFHQILVbcOqO/FBrHhMjKUpZaSIrQe850OziuatfHwsJLKkU40jONPkRhYlmUttK9iQAI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168371; c=relaxed/simple; bh=EFt0aBxDIBf/bWE08iMZghTHjUVAl3MKx8jWmqCjZsY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RfC1asc0OK4AH+/52Zfw6QMUmmomTwEv2oZc0uRd5o6tEfpWvWCotM49bza492bEPDD/UdkfUExZkZTUyU42mXucUUuq71Fo96eaeFX2EdBSZn5LT20rkBfhaHLS/0j8xIpmFTJnmfEsijRTI9owKDyiOz+3E6wmFmJDhlJwfHs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cf+2pKiz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cf+2pKiz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 09ED8C433B1; Thu, 29 Feb 2024 00:59:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168371; bh=EFt0aBxDIBf/bWE08iMZghTHjUVAl3MKx8jWmqCjZsY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cf+2pKizrwbkG3cr0ZCWseXEGscV9O0MuU/aQ1TUB/F95v5PyHmzOytK3KXdugXDZ x2jQ7r//KNtWD4Uj9tn2xeD50uArOeLZND7gIMBpVuWKRq7COz1qfDbAXyQNjMpnNW r8Z8WAP35MvitAxLeanpE7wgH8yDfp80a/Ci54Sz7Q7KqqHJO4s/ceQSj4hVBQEwMW IgeiCyv5z+kx07tY7YVaizEENIFIXu787FYqMz3YYzYYKV9dBIFbugaVd/ryCl55M2 7L2rSkfhWSQBqGr9IcL9NgRN5N0Y2SX0qaenUX6pT8jPvRlzFCKXsVegbPHR8p036o +1WIStHGF+CNw== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 07/12] selftests: kselftest: add ksft_test_result_code(), handling all exit codes Date: Wed, 28 Feb 2024 16:59:14 -0800 Message-ID: <20240229005920.2407409-8-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For generic test harness code it's more useful to deal with exit codes directly, rather than having to switch on them and call the right ksft_test_result_*() helper. Add such function to kselftest.h. Note that "directive" and "diagnostic" are what ktap docs call those parts of the message. Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest.h | 39 +++++++++++++++++++++ tools/testing/selftests/kselftest_harness.h | 9 +++-- 2 files changed, 46 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index a781e6311810..12ad7f8dfe3a 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -25,6 +25,7 @@ * ksft_test_result_skip(fmt, ...); * ksft_test_result_xfail(fmt, ...); * ksft_test_result_error(fmt, ...); + * ksft_test_result_code(exit_code, test_name, fmt, ...); * * When all tests are finished, clean up and exit the program with one of: * @@ -254,6 +255,44 @@ static inline __printf(1, 2) void ksft_test_result_error(const char *msg, ...) va_end(args); } +static inline __printf(2, 3) +void ksft_test_result_code(int exit_code, const char *msg, ...) +{ + const char *tap_code = "ok"; + const char *directive = ""; + int saved_errno = errno; + va_list args; + + switch (exit_code) { + case KSFT_PASS: + ksft_cnt.ksft_pass++; + break; + case KSFT_XFAIL: + directive = " # XFAIL "; + ksft_cnt.ksft_xfail++; + break; + case KSFT_XPASS: + directive = " # XPASS "; + ksft_cnt.ksft_xpass++; + break; + case KSFT_SKIP: + directive = " # SKIP "; + ksft_cnt.ksft_xskip++; + break; + case KSFT_FAIL: + default: + tap_code = "not ok"; + ksft_cnt.ksft_fail++; + break; + } + + va_start(args, msg); + printf("%s %u%s", tap_code, ksft_test_num(), directive); + errno = saved_errno; + vprintf(msg, args); + va_end(args); +} + static inline int ksft_exit_pass(void) { ksft_print_cnts(); diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 5a48177c8c00..4fb30fcc7774 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1111,6 +1111,7 @@ void __run_test(struct __fixture_metadata *f, struct __test_metadata *t) { char test_name[LINE_MAX]; + const char *diagnostic; /* reset test struct */ t->exit_code = KSFT_PASS; @@ -1140,9 +1141,13 @@ void __run_test(struct __fixture_metadata *f, ksft_print_msg(" %4s %s\n", __test_passed(t) ? "OK" : "FAIL", test_name); + if (t->results->reason[0]) + diagnostic = t->results->reason; + else + diagnostic = "unknown"; + if (t->exit_code == KSFT_SKIP) - ksft_test_result_skip("%s\n", t->results->reason[0] ? - t->results->reason : "unknown"); + ksft_test_result_code(t->exit_code, "%s\n", diagnostic); else ksft_test_result(__test_passed(t), "%s\n", test_name); } From patchwork Thu Feb 29 00:59:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576255 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A6DC3AC1E; Thu, 29 Feb 2024 00:59:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168372; cv=none; b=M9rgnGIEq8o/K5nFYyfC3+Y6/57IgEpxK7rGX5g1CLYZ6vpe7cQZGE/T13fulnqXKkuzwqKkbYRCJ8k7mfT/G7WQ35QSXfc82xX2imdmNtHvpN1sGLeNeaUChedsT2k69UMclo36KUc42F2+7r9lCvaDTafVxwM7fz897mboPEQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168372; c=relaxed/simple; bh=kUXNicxEFPS6Tw9TLyUiMTsBYRZKhgDBnAsNtFKri38=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dSOuBLqt8BFOrCArQy4haTBdW8RGU1+oo+mzdwRYMzM5g39o3dSEq0WnZqVG8ZqTnPuQMbf2zWiNc/wvu76G0MwC5Ef6mOi//pLJM4AVuzK/zv6NCmjdLceTxTZcki0yplrXw3XIh2h9KOHdkHw88LnuQFwndP8IUE8BhZDihYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cR1xXo+Y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cR1xXo+Y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 877FDC43390; Thu, 29 Feb 2024 00:59:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168371; bh=kUXNicxEFPS6Tw9TLyUiMTsBYRZKhgDBnAsNtFKri38=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cR1xXo+YAKlwYOKBuI6WmDZOK4s4h5daoh/Pgj+ToXvARbbPkuY7GjM2wGMiDv88w ap3z/GH7Le7eF2JnlJzxmnfKYQZA+tZc0JjJfNx2vFY2ja8nCqLekfxsVZ7iB5Vo3b r3d7p9oryxbX/fKdqwIfEHWacwlnfO+kbdM/REySPTUJAexYcARgI+RyL2cScS2wbE nhMtlUWzyg5rx6KzA7VFwRcerj4BZnsVxe7uprYeu5oOogvi3Ymiz03Pc66TA2lTYf BDpNSX+Zuf31LjrUEkiqHibSQc4Fl9LpZ7A0x2821000BNQhmO55mfgmQCvk+x8/mA QGiDunw8CjoXQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 08/12] selftests: kselftest_harness: print test name for SKIP Date: Wed, 28 Feb 2024 16:59:15 -0800 Message-ID: <20240229005920.2407409-9-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Jakub points out that for parsers it's rather useful to always have the test name on the result line. Currently if we SKIP (or soon XFAIL or XPASS), we will print: ok 17 # SKIP SCTP doesn't support IP_BIND_ADDRESS_NO_PORT ^ no test name Always print the test name. KTAP format seems to allow or even call for it, per: https://docs.kernel.org/dev-tools/ktap.html Suggested-by: Jakub Sitnicki Link: https://lore.kernel.org/all/87jzn6lnou.fsf@cloudflare.com/ Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest.h | 7 ++++--- tools/testing/selftests/kselftest_harness.h | 3 ++- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 12ad7f8dfe3a..25e29626566e 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -255,8 +255,9 @@ static inline __printf(1, 2) void ksft_test_result_error(const char *msg, ...) va_end(args); } -static inline __printf(2, 3) -void ksft_test_result_code(int exit_code, const char *msg, ...) +static inline __printf(3, 4) +void ksft_test_result_code(int exit_code, const char *test_name, + const char *msg, ...) { const char *tap_code = "ok"; const char *directive = ""; @@ -287,7 +288,7 @@ void ksft_test_result_code(int exit_code, const char *msg, ...) } va_start(args, msg); - printf("%s %u%s", tap_code, ksft_test_num(), directive); + printf("%s %u %s%s", tap_code, ksft_test_num(), test_name, directive); errno = saved_errno; vprintf(msg, args); va_end(args); diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 4fb30fcc7774..82377051aa54 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1147,7 +1147,8 @@ void __run_test(struct __fixture_metadata *f, diagnostic = "unknown"; if (t->exit_code == KSFT_SKIP) - ksft_test_result_code(t->exit_code, "%s\n", diagnostic); + ksft_test_result_code(t->exit_code, test_name, + "%s\n", diagnostic); else ksft_test_result(__test_passed(t), "%s\n", test_name); } From patchwork Thu Feb 29 00:59:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576256 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD0B03D0BE; Thu, 29 Feb 2024 00:59:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168372; cv=none; b=DjEKXjUUVJH9Mg9keoK2khM62cSzMHtWccgB0cYzwRJgD3HATVDQBE6fRHXlgbd1WYyubUbP96yFKBJGMcavm4Gehx7tdvNPRtC4KGUDZZAIjG104W4OskCHoSUjbu1KFom3YoZBW6aSyDMGy9u3rHqDh8FCR/mQ9U+UMWKOoPk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168372; c=relaxed/simple; bh=NmuieFZoScnkuFnA/13Z3/d13a37ja5rWHwFFbZTMHQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MfThPJc/IUjBzlXO7jPnmffaibe6OwiwF+39XmgHbJXypx9Bx6Bpa8AJwUrx/kh+f6LmKveDgxoXR2S2ot9jPL2IHLE7beH9AKuWXL9/jUcT5qWgriglKORDjBVceywmV84lidsOHeaMpQ1EjYcMi+HkvdzEEO2q79njnX/i5WY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=d9OMXeV6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="d9OMXeV6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13BC9C433F1; Thu, 29 Feb 2024 00:59:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168372; bh=NmuieFZoScnkuFnA/13Z3/d13a37ja5rWHwFFbZTMHQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d9OMXeV6MiN8yIbDvSozoyam2NDP9HF5K2VSaFn/BPrCP+qdNXaEBRSK2qW+KdJPB 7IJJ9JSDaR40NpRlE+iyq9tk8HT6CIW/Q6fXj0vDNk1w8DuuxG+AdDDse+gZe6I7DA OzDbeuWb0DGjvLX87VORVTfSGPvAuqa4I7SU0AjfPIl2Jxz57BvfIbqmI1FwCDEHSa 2nf4AJwPv7GgdDDqdRxdc7m3wTFsKjkgekZuMKkxfkbKba7DfT8wv/2WRTQ55D0SUa Hp5Nsbcn0G/iwAn31DCmezwf0UpVAH2Oc/vdgkyr3eU3YSamUzcjuSTBrM3SnBXvQu ZZqVJqwIT/IUw== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 09/12] selftests: kselftest_harness: separate diagnostic message with # in ksft_test_result_code() Date: Wed, 28 Feb 2024 16:59:16 -0800 Message-ID: <20240229005920.2407409-10-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 According to the spec we should always print a # if we add a diagnostic message. Having the caller pass in the new line as part of diagnostic message makes handling this a bit counter-intuitive, so append the new line in the helper. Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest.h | 5 +++++ tools/testing/selftests/kselftest_harness.h | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 25e29626566e..541bf192e30e 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -287,10 +287,15 @@ void ksft_test_result_code(int exit_code, const char *test_name, break; } + /* Docs seem to call for double space if directive is absent */ + if (!directive[0] && msg[0]) + directive = " # "; + va_start(args, msg); printf("%s %u %s%s", tap_code, ksft_test_num(), test_name, directive); errno = saved_errno; vprintf(msg, args); + printf("\n"); va_end(args); } diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 82377051aa54..5b0592e4b7a4 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1148,7 +1148,7 @@ void __run_test(struct __fixture_metadata *f, if (t->exit_code == KSFT_SKIP) ksft_test_result_code(t->exit_code, test_name, - "%s\n", diagnostic); + "%s", diagnostic); else ksft_test_result(__test_passed(t), "%s\n", test_name); } From patchwork Thu Feb 29 00:59:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576257 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CFBE3E48C; Thu, 29 Feb 2024 00:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168373; cv=none; b=QL1zrknVjHPrnk2wbxSf5VoCje7JthFW361uSVPbvlWigMn8XPntCAluaafhkRlEzOZ/zOktfLika5iti1HYfCzeB6Qg1H9S8xPcrQfW6xdLYKSbLy/bFbu9g9AyF5DqTG29kSZfVYrl3p5xC9Oi2HubBjq1fzTQ5ODc84rhlMM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168373; c=relaxed/simple; bh=eo0XA91MK0x1+sBfoHYjHZ+5G044GpNBTQJ956u/WAU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oZBZQiAN63AwLFMciTN6fA7h6I/VEDvCcfup+NiuiZU03XUQSPFfH02K5pnSXkGo0jPtal/LvV3ioK5t+6kHNs4MHHDjrWP+/rpsm7HkAKZs0U1JdNhktsJmU9x0fT9un+C/n23yDfb/rOULWCzlrEQVe1LRIgsBW3tTvd66U3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WWwYWriX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WWwYWriX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 958B1C433A6; Thu, 29 Feb 2024 00:59:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168373; bh=eo0XA91MK0x1+sBfoHYjHZ+5G044GpNBTQJ956u/WAU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WWwYWriXWf2Xoj76wTXPfvsaejmC9Ra4EfPbmFwpSSU4Zz6ga0iSNHXsADMa0Ge7K gVQFDT/mrBWQI2JSXJCRmnYsP2BAvutMBTdsCOb2TQTAHatGR7m8n9IAyOcpGeJKfB qhj4SqdHR0C8aG/u4LjdZw1rRAGa9NyOm/LtJcGBLuKVTC0aAIq6GxNIbSAAlXOuGu q03Oow1urWT8pn6wlimQ4joBiGzv7YZ2h2Hpu8Qe13wmltzFC0qabZCkkMlQ3RUv8Q n//QmKkEwXl5QhAt+emTeUKLIgAkUVjbzbHE4f/oPWVK6xzug7CznupXS1UjsT/Crd Axvk9ig28OIcQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 10/12] selftests: kselftest_harness: let PASS / FAIL provide diagnostic Date: Wed, 28 Feb 2024 16:59:17 -0800 Message-ID: <20240229005920.2407409-11-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Switch to printing KTAP line for PASS / FAIL with ksft_test_result_code(), this gives us the ability to report diagnostic messages. Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest_harness.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 5b0592e4b7a4..b643a577f9e1 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1143,14 +1143,13 @@ void __run_test(struct __fixture_metadata *f, if (t->results->reason[0]) diagnostic = t->results->reason; + else if (t->exit_code == KSFT_PASS || t->exit_code == KSFT_FAIL) + diagnostic = NULL; else diagnostic = "unknown"; - if (t->exit_code == KSFT_SKIP) - ksft_test_result_code(t->exit_code, test_name, - "%s", diagnostic); - else - ksft_test_result(__test_passed(t), "%s\n", test_name); + ksft_test_result_code(t->exit_code, test_name, + diagnostic ? "%s" : "", diagnostic); } static int test_harness_run(int argc, char **argv) From patchwork Thu Feb 29 00:59:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576258 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3A013EA88; Thu, 29 Feb 2024 00:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168374; cv=none; b=oQkRg8vtAKJwvcRrOP0OCU9b8cOdd5nTa0DxR2QBd2/SbQ8rh9aApVO1yV+DcCGI7ccXU3vrPMYWthNs8ETCNn+VWCxkqK2k58Rq1RxH4dF271IZgwRDtxEmDO5faS53FluPcSrWShr1zjlMn2rOw7m2lCtMxNZQECc/O2DGVhw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168374; c=relaxed/simple; bh=jwhW1TkA3MbOALZnEwQYVt60KQ7UyC35+HlF5r2Rhao=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XPy+uFOzFwcGua0Al2gJ3WVh88LKc2bZ+D2cPowGexBTvtvHhGaScJ6EDkW+UuXsp2xYlUXt5J9MvHE8PEJiWY1iwMefriXN7E1LQMBcLUFybsrYL9bKdCMDcPOBglDrRGYWelDN5XlP6B44vXROCOYq+TCSGJ4J7962zzW4GDQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Fy8WOSgp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Fy8WOSgp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 210EFC43390; Thu, 29 Feb 2024 00:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168373; bh=jwhW1TkA3MbOALZnEwQYVt60KQ7UyC35+HlF5r2Rhao=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fy8WOSgpTofaV0Ijy0uCQVHKP1ygyaNp5Z5DIkD6YYtiXzLbaDaW5e8y8OsNJB0cF 9RGmQ3wLkqN22pbdIIkQ9Q+JZ6mtFC46juyLY84G1L552VRSUJwnbaJVbHuBEFssWJ aiVJw61NktxzfWBj4DepkAXPAcd6mg2zFz8Fg9+qwCPf8wDsJRMAoI4RotPsrmmLRj OUa78ve1vjpsmr0T2EeU2Zz07BjkVKJD6YwS8l5jN1RPAgZ4f/JsTA20EizXv5WKyd HvY0/jt+6mh8peb7w0F4HsRCUbVtB4DxZtHCqvdq4dLcXfVxMWIg9GImW4BqqLtxf4 mhVvmTR3ZAB0w== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 11/12] selftests: kselftest_harness: support using xfail Date: Wed, 28 Feb 2024 16:59:18 -0800 Message-ID: <20240229005920.2407409-12-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Currently some tests report skip for things they expect to fail e.g. when given combination of parameters is known to be unsupported. This is confusing because in an ideal test environment and fully featured kernel no tests should be skipped. Selftest summary line already includes xfail and xpass counters, e.g.: Totals: pass:725 fail:0 xfail:0 xpass:0 skip:0 error:0 but there's no way to use it from within the harness. Add a new per-fixture+variant combination list of test cases we expect to fail. Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- v3: change the paradigm to declarative --- tools/testing/selftests/kselftest_harness.h | 49 ++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index b643a577f9e1..634be793ad58 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -803,6 +803,37 @@ struct __fixture_metadata { .prev = &_fixture_global, }; +struct __test_xfail { + struct __fixture_metadata *fixture; + struct __fixture_variant_metadata *variant; + struct __test_metadata *test; + struct __test_xfail *prev, *next; +}; + +/** + * XFAIL_ADD() - mark variant + test case combination as expected to fail + * @fixture_name: name of the fixture + * @variant_name: name of the variant + * @test_name: name of the test case + * + * Mark a combination of variant + test case for a given fixture as expected + * to fail. Tests marked this way will report XPASS / XFAIL return codes, + * instead of PASS / FAIL,and use respective counters. + */ +#define XFAIL_ADD(fixture_name, variant_name, test_name) \ + static struct __test_xfail \ + _##fixture_name##_##variant_name##_##test_name##_xfail = \ + { \ + .fixture = &_##fixture_name##_fixture_object, \ + .variant = &_##fixture_name##_##variant_name##_object, \ + .test = &_##fixture_name##_##test_name##_object, \ + }; \ + static void __attribute__((constructor)) \ + _register_##fixture_name##_##variant_name##_##test_name##_xfail(void) \ + { \ + __register_xfail(&_##fixture_name##_##variant_name##_##test_name##_xfail); \ + } + static struct __fixture_metadata *__fixture_list = &_fixture_global; static int __constructor_order; @@ -817,6 +848,7 @@ static inline void __register_fixture(struct __fixture_metadata *f) struct __fixture_variant_metadata { const char *name; const void *data; + struct __test_xfail *xfails; struct __fixture_variant_metadata *prev, *next; }; @@ -866,6 +898,11 @@ static inline void __register_test(struct __test_metadata *t) __LIST_APPEND(t->fixture->tests, t); } +static inline void __register_xfail(struct __test_xfail *xf) +{ + __LIST_APPEND(xf->variant->xfails, xf); +} + static inline int __bail(int for_realz, struct __test_metadata *t) { /* if this is ASSERT, return immediately. */ @@ -941,7 +978,9 @@ void __wait_for_test(struct __test_metadata *t) fprintf(TH_LOG_STREAM, "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { - if (WEXITSTATUS(status) == KSFT_SKIP) { + if (WEXITSTATUS(status) == KSFT_SKIP || + WEXITSTATUS(status) == KSFT_XPASS || + WEXITSTATUS(status) == KSFT_XFAIL) { t->exit_code = WEXITSTATUS(status); } else if (t->termsig != -1) { t->exit_code = KSFT_FAIL; @@ -1110,6 +1149,7 @@ void __run_test(struct __fixture_metadata *f, struct __fixture_variant_metadata *variant, struct __test_metadata *t) { + struct __test_xfail *xfail; char test_name[LINE_MAX]; const char *diagnostic; @@ -1141,6 +1181,13 @@ void __run_test(struct __fixture_metadata *f, ksft_print_msg(" %4s %s\n", __test_passed(t) ? "OK" : "FAIL", test_name); + /* Check if we're expecting this test to fail */ + for (xfail = variant->xfails; xfail; xfail = xfail->next) + if (xfail->test == t) + break; + if (xfail) + t->exit_code = __test_passed(t) ? KSFT_XPASS : KSFT_XFAIL; + if (t->results->reason[0]) diagnostic = t->results->reason; else if (t->exit_code == KSFT_PASS || t->exit_code == KSFT_FAIL) From patchwork Thu Feb 29 00:59:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13576259 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B72741A80; Thu, 29 Feb 2024 00:59:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168374; cv=none; b=V3zGpOvd/jKmNYBiV0BPvrRYCaPxMkSEEQWhzq+zKUB57uqRGTTYptqXjI5+lnaMgQkumIOQevq5wOtZWPPJ8vGEupu6nT0xBDwUsSz/nCOFXCBC1/Mi4sc/+7e285Yp/HKbsqr0sfdYAu6njb83AFZirCujtaKJQjwy4B9qqy4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709168374; c=relaxed/simple; bh=dZrum7wy2AkWih698zz9wzfDSh+S33eFRkVxQiIUlOo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RBwTCeY3PnkyaStM/WGcn3Urv2YLvgOZz70e4WyKRHS9FtGvtFzxCRFQYspCigVoT8URlEhrFDOGLGB9/wOgFm7rf2OYhKI8s5TJvFluOVNXga/McJRGtyJicujPWXvI85ZuMMsE88yr81tAXXilY8UJXxxQy0j7ZxeF9uw3wII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QZFWBXCZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QZFWBXCZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E88CC433C7; Thu, 29 Feb 2024 00:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1709168374; bh=dZrum7wy2AkWih698zz9wzfDSh+S33eFRkVxQiIUlOo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QZFWBXCZE/9dSdiE26uw5BHMfgRw2W7yRhsnaN0Py6t42X4ZOy05sPOUu0faQMOjW +gZxzBcpcMWbhtwjbNWCxejtB1Q6hKjSYSOELfRb+qPm2ExvpzOZ5cSKMq4N8Q3lqu mI8G2/YzYYwGmHM8m3fKcud4T7cHWOxHy3z3t7cm6kPpk601p3FYI2KndnhKifj0Fy qv8tshXhaAPkvemEfcHkolU1vrAfqP9fXlQS6i5zNz/uGNMhHLjPlxOqk9tQK3JqsR idPi6SX+tKQAGLtNCRUU/wrpF6us2H0UMLt+V1BQMO1nz9ZUKU1gZiLJgOliK6zsgN esV+XySVF4RtA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH v4 12/12] selftests: ip_local_port_range: use XFAIL instead of SKIP Date: Wed, 28 Feb 2024 16:59:19 -0800 Message-ID: <20240229005920.2407409-13-kuba@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240229005920.2407409-1-kuba@kernel.org> References: <20240229005920.2407409-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org SCTP does not support IP_LOCAL_PORT_RANGE and we know it, so use XFAIL instead of SKIP. Reviewed-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/net/ip_local_port_range.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/net/ip_local_port_range.c b/tools/testing/selftests/net/ip_local_port_range.c index 6ebd58869a63..193b82745fd8 100644 --- a/tools/testing/selftests/net/ip_local_port_range.c +++ b/tools/testing/selftests/net/ip_local_port_range.c @@ -365,9 +365,6 @@ TEST_F(ip_local_port_range, late_bind) __u32 range; __u16 port; - if (variant->so_protocol == IPPROTO_SCTP) - SKIP(return, "SCTP doesn't support IP_BIND_ADDRESS_NO_PORT"); - fd = socket(variant->so_domain, variant->so_type, 0); ASSERT_GE(fd, 0) TH_LOG("socket failed"); @@ -414,6 +411,9 @@ TEST_F(ip_local_port_range, late_bind) ASSERT_TRUE(!err) TH_LOG("close failed"); } +XFAIL_ADD(ip_local_port_range, ip4_stcp, late_bind); +XFAIL_ADD(ip_local_port_range, ip6_stcp, late_bind); + TEST_F(ip_local_port_range, get_port_range) { __u16 lo, hi;