From patchwork Tue Feb 20 19:22:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564410 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 582BA14A4EA; Tue, 20 Feb 2024 19:22:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456967; cv=none; b=WMtASy8T+SEOYTKg3l58XU0qCpGsRCmWpftlNbB5AB1/h1Gj2UXNATvS7fNfTHqlSGQeCaw+UY049TgwHK5sbVg3Q5M3HpqphfA+gAERay1uebYqpmbPKIkM1r1BBU2cxcafL3kluALEXTAzNclCQzu8Rl6MiW9ObQL8SZGzrvE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456967; c=relaxed/simple; bh=QFefN2pMJk3wF8CJAX8FXQUW7G2GGo/zUOy3/fKo+Jk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Uo7Rf5YReXukZC3kz+UfcD28orZ+sMbmrmDuIasdKUI5syU/Mth0bKPJ/nsT02DjppJv4SJc6jFGV8VU+W94kFBp4BYVHkCaWo2HV4ba8TIKEjG/28r5bgq65WBQCDliR2aCBH/WvBWJ4nA65/PRCZioDMfNjVMmzNIMrgclvXM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kSf2H14R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kSf2H14R" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 63FF2C433F1; Tue, 20 Feb 2024 19:22:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456966; bh=QFefN2pMJk3wF8CJAX8FXQUW7G2GGo/zUOy3/fKo+Jk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kSf2H14R17b9wRAMRCwaQoK+jbbx2y3e+w5Tbf9VJWWb01idSvCdAmZhRrVcNjNVP 53XUW4IwJajjIVl+Br3A2gD0lDYB7UkDvloq31KYtO+qC4W192NjM0N58eOBpcXLLu GkPCW1ahPTAcMsu1Dm9JChB9GfJL2r3O2Z5RJ1ak03jaCQW/1rVed7MvjYW/+8f9Xn J4Rgj6mm3tWdXG1gc58dsZujxzgqxIr5mWnysBjwStQH34MK6Xr7M6ZafOzH/si971 RT1gpFcQ4tjPOpFBgXFyF1oTjPAAzkNVp9oYVC7586gIa3YNvVGk+8uxlijXo0s65V n16xlKAX/LeCg== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski , luto@amacapital.net, wad@chromium.org Subject: [PATCH net-next v3 01/11] selftests: kselftest_harness: pass step via shared memory Date: Tue, 20 Feb 2024 11:22:25 -0800 Message-ID: <20240220192235.2953484-2-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Commit 0ef67a888375 ("selftests/harness: Report skip reason") added shared memory to communicate between harness and test. Use that instead of exit codes to send the failing step back to the harness. The exit codes are limited and because of the step passing we can't use the full range of KSFT_* exit codes. Acked-by: Kees Cook Tested-by: Jakub Sitnicki Signed-off-by: Jakub Kicinski --- CC: mic@digikod.net CC: luto@amacapital.net CC: wad@chromium.org CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org CC: linux-security-module@vger.kernel.org --- tools/testing/selftests/kselftest_harness.h | 21 ++++++++++----------- tools/testing/selftests/landlock/common.h | 5 ++--- 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index e05ac8261046..98bdedf9a53a 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -101,8 +101,8 @@ * ASSERT_* number for which the test failed. This behavior can be enabled by * writing `_metadata->no_print = true;` before the check sequence that is * unable to print. When an error occur, instead of printing an error message - * and calling `abort(3)`, the test process call `_exit(2)` with the assert - * number as argument, which is then printed by the parent process. + * and calling `abort(3)`, the test process call `_exit(2)` and pass the error + * to be printed to the parent process via shared memory. */ #define TH_LOG(fmt, ...) do { \ if (TH_LOG_ENABLED) \ @@ -695,9 +695,8 @@ __bail(_assert, _metadata)) #define __INC_STEP(_metadata) \ - /* Keep "step" below 255 (which is used for "SKIP" reporting). */ \ - if (_metadata->passed && _metadata->step < 253) \ - _metadata->step++; + if (_metadata->passed) \ + _metadata->results->step++; #define is_signed_type(var) (!!(((__typeof__(var))(-1)) < (__typeof__(var))1)) @@ -784,6 +783,7 @@ struct __test_results { char reason[1024]; /* Reason for test result */ + unsigned int step; /* Test step reached without failure */ }; struct __test_metadata; @@ -837,7 +837,6 @@ struct __test_metadata { int trigger; /* extra handler after the evaluation */ int timeout; /* seconds to wait for test timeout */ bool timed_out; /* did this test timeout instead of exiting? */ - __u8 step; bool no_print; /* manual trigger when TH_LOG_STREAM is not available */ bool aborted; /* stopped test due to failed ASSERT */ bool setup_completed; /* did setup finish? */ @@ -875,7 +874,7 @@ static inline void __test_check_assert(struct __test_metadata *t) { if (t->aborted) { if (t->no_print) - _exit(t->step); + _exit(1); abort(); } } @@ -960,7 +959,7 @@ void __wait_for_test(struct __test_metadata *t) fprintf(TH_LOG_STREAM, "# %s: Test failed at step #%d\n", t->name, - WEXITSTATUS(status)); + t->results->step); } } } else if (WIFSIGNALED(status)) { @@ -1114,9 +1113,9 @@ void __run_test(struct __fixture_metadata *f, t->passed = 1; t->skip = 0; t->trigger = 0; - t->step = 1; t->no_print = 0; memset(t->results->reason, 0, sizeof(t->results->reason)); + t->results->step = 1; ksft_print_msg(" RUN %s%s%s.%s ...\n", f->name, variant->name[0] ? "." : "", variant->name, t->name); @@ -1137,8 +1136,8 @@ void __run_test(struct __fixture_metadata *f, /* Pass is exit 0 */ if (t->passed) _exit(0); - /* Something else happened, report the step. */ - _exit(t->step); + /* Something else happened. */ + _exit(1); } else { __wait_for_test(t); } diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h index e64bbdf0e86e..c15e22abdca6 100644 --- a/tools/testing/selftests/landlock/common.h +++ b/tools/testing/selftests/landlock/common.h @@ -50,13 +50,13 @@ _exit(255); \ if (_metadata->passed) \ _exit(0); \ - _exit(_metadata->step); \ + _exit(1); \ } \ if (child != waitpid(child, &status, 0)) \ abort(); \ if (WIFSIGNALED(status) || !WIFEXITED(status)) { \ _metadata->passed = 0; \ - _metadata->step = 1; \ + _metadata->results->step = 1; \ return; \ } \ switch (WEXITSTATUS(status)) { \ @@ -69,7 +69,6 @@ break; \ default: \ _metadata->passed = 0; \ - _metadata->step = WEXITSTATUS(status); \ break; \ } \ } \ From patchwork Tue Feb 20 19:22:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564411 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1223F135A4D; Tue, 20 Feb 2024 19:22:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456968; cv=none; b=IgsVLWtL0EyJBoR4xWjha3Tm7vXbbklAdlzpCPmaaq/X21fKRo81pauzIxgIuq+fHhTTehwLZPXW0WH3u8m7vJGUae4fy+4C6rwEZ3fXKLjtQWMzhWd5nMQ/yEabot2VDZCnMjj6YT4KpxrEhW9XZIeUKgExBlii37wRkQe2FDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456968; c=relaxed/simple; bh=y1jR1SPhv1Na8mdweF3oZSOv0tk/hgB4rcZ8s6PB+T4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AqwqXQ4RhtaYtvVR494wx5m83U6/Mi4dNPeSupegnrtTZbg0AEgOuwFERtjm+VoqgpiSExLqnt2mxGpS3iZaPMykbOjAEFA4z5/f3rkLu3aflBxY7fwbhhTagUc3n/FdtfyKUcI0mGCONlQTfpabycaUQ6ywnAbNK8Zw7besOTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q90ixN/f; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q90ixN/f" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B11DC43394; Tue, 20 Feb 2024 19:22:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456967; bh=y1jR1SPhv1Na8mdweF3oZSOv0tk/hgB4rcZ8s6PB+T4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q90ixN/ftQd5E433nAU54KmV61A34UqBg+LV+Itk6jz2YGj5L4GDZSyMCbuNO4Xkd nw41HW+hVfYjJZHkm28GH/PI8Qst6z4xfoaMBDoZGFucZiKwZW7gwcWWkAnpgiImX+ +CsNU4du5h8k7IeHKP+3eDrOER5XZiruRy87q2ZOgprVA8AN9mxACIdAR4PH8MV5Xq J3OBn8tm4nv7FMU8uzSPFI/mVQ6QjShFo03ST880rCH1l2x3ax/21L3ohzzdzrnI+2 5BHia/i6VTN3zb74Ro1dwwmBc5wv+jRNJnuKKQpDzhFbgnkfUuIlRbWHjibJKj/Cif fImUkwBTHE77w== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski , luto@amacapital.net, wad@chromium.org Subject: [PATCH net-next v3 02/11] selftests: kselftest_harness: use KSFT_* exit codes Date: Tue, 20 Feb 2024 11:22:26 -0800 Message-ID: <20240220192235.2953484-3-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now that we no longer need low exit codes to communicate assertion steps - use normal KSFT exit codes. Acked-by: Kees Cook Tested-by: Jakub Sitnicki Signed-off-by: Jakub Kicinski --- CC: keescook@chromium.org CC: luto@amacapital.net CC: wad@chromium.org CC: shuah@kernel.org CC: mic@digikod.net CC: linux-kselftest@vger.kernel.org CC: linux-security-module@vger.kernel.org --- tools/testing/selftests/kselftest_harness.h | 13 ++++++------- tools/testing/selftests/landlock/common.h | 10 +++++----- 2 files changed, 11 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 98bdedf9a53a..618b41eac749 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -874,7 +874,7 @@ static inline void __test_check_assert(struct __test_metadata *t) { if (t->aborted) { if (t->no_print) - _exit(1); + _exit(KSFT_FAIL); abort(); } } @@ -937,7 +937,7 @@ void __wait_for_test(struct __test_metadata *t) fprintf(TH_LOG_STREAM, "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { - if (WEXITSTATUS(status) == 255) { + if (WEXITSTATUS(status) == KSFT_SKIP) { /* SKIP */ t->passed = 1; t->skip = 1; @@ -950,7 +950,7 @@ void __wait_for_test(struct __test_metadata *t) } else { switch (WEXITSTATUS(status)) { /* Success */ - case 0: + case KSFT_PASS: t->passed = 1; break; /* Other failure, assume step report. */ @@ -1132,12 +1132,11 @@ void __run_test(struct __fixture_metadata *f, setpgrp(); t->fn(t, variant); if (t->skip) - _exit(255); - /* Pass is exit 0 */ + _exit(KSFT_SKIP); if (t->passed) - _exit(0); + _exit(KSFT_PASS); /* Something else happened. */ - _exit(1); + _exit(KSFT_FAIL); } else { __wait_for_test(t); } diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h index c15e22abdca6..f5b40399a538 100644 --- a/tools/testing/selftests/landlock/common.h +++ b/tools/testing/selftests/landlock/common.h @@ -47,10 +47,10 @@ _metadata->no_print = 1; \ fixture_name##_##test_name##_child(_metadata, self, variant); \ if (_metadata->skip) \ - _exit(255); \ + _exit(KSFT_SKIP); \ if (_metadata->passed) \ - _exit(0); \ - _exit(1); \ + _exit(KSFT_PASS); \ + _exit(KSFT_FAIL); \ } \ if (child != waitpid(child, &status, 0)) \ abort(); \ @@ -60,10 +60,10 @@ return; \ } \ switch (WEXITSTATUS(status)) { \ - case 0: \ + case KSFT_PASS: \ _metadata->passed = 1; \ break; \ - case 255: \ + case KSFT_SKIP: \ _metadata->passed = 1; \ _metadata->skip = 1; \ break; \ From patchwork Tue Feb 20 19:22:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564412 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4EF014A4E2; Tue, 20 Feb 2024 19:22:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456968; cv=none; b=KD2ELl8PHHPx0J+WciF9FQb3qJgBD3v2XTuFslcCnhFmIUr0HooejL5ubDip61hWTuBbqBv9zLjF9pBd6Xp7peMQwFM9lK11xkPkh16Gz1zN0D6g1SHun0l/+U+XUZ7Muq+coA0uwH/CdO7qyS1GMy8o2ZEEJMWMoXsdNfjiHso= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456968; c=relaxed/simple; bh=i7OQ0CBPUvcZuB1M/xw7S8wpiJMS1dvOsrgRmQy3mEw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TehmCCE+8Vqw92y7I0s7j+XZzcK/GUPFuBC04x9V3iBmnBZ0dlgym3tddORzb7azNB6QJUcaFIR4BTpK7Kamvrhk0JHujpJcCaxV9hOeyB2NlxKGK15IVz8mrEp89yTDguPON/ta6k1Z6nyzvyH4lXRk/oTUjRLOALtBh5D9mOc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XYfzcEMD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XYfzcEMD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A88ACC43390; Tue, 20 Feb 2024 19:22:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456968; bh=i7OQ0CBPUvcZuB1M/xw7S8wpiJMS1dvOsrgRmQy3mEw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XYfzcEMDSxpyolsXWnaoC/5SujI3JSi38UCTNU5oQRkSW3yEEe/x5uoW9lpvadSAN mnnFwC+uQwDPQuyNV+FcdLMZqyEli6BVFXfSFud8Tdy1TjVAaFxgmysKkUj6xRKDhR 7LAHYhD+ULh7CMcQFjQK+PkBBXDLsvm69TVf0n+V7wZGbr+SOkfuxu7pQCd7I2il03 NVAZ9dbdp0eUZMQGyBkyctd5bF65dkBQc6PjERuN8avXCGoPKS7jIM/Q4JaT8JK8e2 c6N301DF6Sv62OJvBhf0XEzZu3IAi95O2qjuVyla/WOCgML40ZaekDXfcLgPcEwoS6 mYlQ+HX7YpZ5A== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 03/11] selftests: kselftest_harness: generate test name once Date: Tue, 20 Feb 2024 11:22:27 -0800 Message-ID: <20240220192235.2953484-4-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since we added variant support generating full test case name takes 4 string arguments. We're about to need it in another two places. Stop the duplication and print once into a temporary buffer. Suggested-by: Jakub Sitnicki Acked-by: Kees Cook Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest_harness.h | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 618b41eac749..8fc24a728240 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -56,6 +56,7 @@ #include #include #include +#include #include #include #include @@ -1109,6 +1110,8 @@ void __run_test(struct __fixture_metadata *f, struct __fixture_variant_metadata *variant, struct __test_metadata *t) { + char test_name[LINE_MAX]; + /* reset test struct */ t->passed = 1; t->skip = 0; @@ -1117,8 +1120,10 @@ void __run_test(struct __fixture_metadata *f, memset(t->results->reason, 0, sizeof(t->results->reason)); t->results->step = 1; - ksft_print_msg(" RUN %s%s%s.%s ...\n", - f->name, variant->name[0] ? "." : "", variant->name, t->name); + snprintf(test_name, sizeof(test_name), "%s%s%s.%s", + f->name, variant->name[0] ? "." : "", variant->name, t->name); + + ksft_print_msg(" RUN %s ...\n", test_name); /* Make sure output buffers are flushed before fork */ fflush(stdout); @@ -1140,15 +1145,14 @@ void __run_test(struct __fixture_metadata *f, } else { __wait_for_test(t); } - ksft_print_msg(" %4s %s%s%s.%s\n", t->passed ? "OK" : "FAIL", - f->name, variant->name[0] ? "." : "", variant->name, t->name); + ksft_print_msg(" %4s %s\n", + t->passed ? "OK" : "FAIL", test_name); if (t->skip) ksft_test_result_skip("%s\n", t->results->reason[0] ? t->results->reason : "unknown"); else - ksft_test_result(t->passed, "%s%s%s.%s\n", - f->name, variant->name[0] ? "." : "", variant->name, t->name); + ksft_test_result(t->passed, "%s\n", test_name); } static int test_harness_run(int argc, char **argv) From patchwork Tue Feb 20 19:22:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564413 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5353C14C5BB; Tue, 20 Feb 2024 19:22:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456969; cv=none; b=BLW4u1t7w7tcPB3uVxukheTRNX4Aw2JJJv1fT6O7O516cADqIf2U/i4NU/jQC6XKQKlYYqf1dmuhqmtf0zUAIMK7dKweJuGpPtKDkx8aISLpXzWOvvjmdAyzTBFWfLT9HRYMSNEE+pxCt2oANaKS6hA3azR1VK4bjf9YKoar4RA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456969; c=relaxed/simple; bh=sSNbyi4vVMgE8gCscNKhi1u1ZHd/Rxrt/+4HOyAonzM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NO5GDyF4GiP8Jf3G44RouW+eiYG0M3pVOciBOuC74aLVnbcUJt/LVuyWPR0Z7gNiRoBS/apRzuIbNLUjpzAqch2pOEO+sJlmf4aux2fyCOriOYCY1FsPjU9q02HKvHaXpiK+zAPaRRKMRe9pYW6084IGzcyN3HPq6z/FQMVVYLE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HYf+tgXH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HYf+tgXH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45A61C433F1; Tue, 20 Feb 2024 19:22:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456968; bh=sSNbyi4vVMgE8gCscNKhi1u1ZHd/Rxrt/+4HOyAonzM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HYf+tgXHQir3ByUdTyEObjAfzBqhsP1aI4UifIcriyBjndKzcuztQ1PNHQC3LTvRO Z49VPPtt5o99Uec3fpxyce1xLFd7qz+pjtV9Qd3k4XOdVY4UWcP1OZK2jFOvZQuFaH M6qvC0wxIXh8Pzb8FW46XN1tPE9SBTIG2vNwjDpg7QILrJcFw1+2q99MaR4S6BlqQP 8aRIk5+Zbl2ti9VL6UusSobEZgBIngNwboMfUbjdFU542/h1XZtTh/AOMUk+3gL9Or 4pwy4fCS+gn+veHjnAJ4Y/Cmm4HtnVWHSzed8kKxWy6XYYYjp9ZbUoMmgywJj9AdmP vpNb+0JpEm30A== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 04/11] selftests: kselftest_harness: save full exit code in metadata Date: Tue, 20 Feb 2024 11:22:28 -0800 Message-ID: <20240220192235.2953484-5-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Instead of tracking passed = 0/1 rename the field to exit_code and invert the values so that they match the KSFT_* exit codes. This will allow us to fold SKIP / XFAIL into the same value. Signed-off-by: Jakub Kicinski --- tools/testing/selftests/kselftest_harness.h | 48 +++++++++++-------- tools/testing/selftests/landlock/base_test.c | 2 +- tools/testing/selftests/landlock/common.h | 14 ++---- tools/testing/selftests/landlock/fs_test.c | 4 +- tools/testing/selftests/landlock/net_test.c | 4 +- .../testing/selftests/landlock/ptrace_test.c | 7 +-- tools/testing/selftests/net/tls.c | 2 +- tools/testing/selftests/seccomp/seccomp_bpf.c | 9 ++-- 8 files changed, 46 insertions(+), 44 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 8fc24a728240..bbb339642a72 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -136,7 +136,7 @@ fprintf(TH_LOG_STREAM, "# SKIP %s\n", \ _metadata->results->reason); \ } \ - _metadata->passed = 1; \ + _metadata->exit_code = KSFT_PASS; \ _metadata->skip = 1; \ _metadata->trigger = 0; \ statement; \ @@ -389,7 +389,7 @@ if (setjmp(_metadata->env) == 0) { \ fixture_name##_setup(_metadata, &self, variant->data); \ /* Let setup failure terminate early. */ \ - if (!_metadata->passed || _metadata->skip) \ + if (!__test_passed(_metadata) || _metadata->skip) \ return; \ _metadata->setup_completed = true; \ fixture_name##_##test_name(_metadata, &self, variant->data); \ @@ -696,7 +696,7 @@ __bail(_assert, _metadata)) #define __INC_STEP(_metadata) \ - if (_metadata->passed) \ + if (__test_passed(_metadata)) \ _metadata->results->step++; #define is_signed_type(var) (!!(((__typeof__(var))(-1)) < (__typeof__(var))1)) @@ -742,7 +742,7 @@ break; \ } \ } \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ /* Ensure the optional handler is triggered */ \ _metadata->trigger = 1; \ } \ @@ -754,7 +754,7 @@ if (_assert) __INC_STEP(_metadata); \ if (!(strcmp(__exp, __seen) _t 0)) { \ __TH_LOG("Expected '%s' %s '%s'.", __exp, #_t, __seen); \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ _metadata->trigger = 1; \ } \ } while (0); OPTIONAL_HANDLER(_assert) @@ -833,7 +833,7 @@ struct __test_metadata { pid_t pid; /* pid of test when being run */ struct __fixture_metadata *fixture; int termsig; - int passed; + int exit_code; int skip; /* did SKIP get used? */ int trigger; /* extra handler after the evaluation */ int timeout; /* seconds to wait for test timeout */ @@ -846,6 +846,12 @@ struct __test_metadata { struct __test_metadata *prev, *next; }; +static inline bool __test_passed(struct __test_metadata *metadata) +{ + return metadata->exit_code != KSFT_FAIL && + metadata->exit_code <= KSFT_SKIP; +} + /* * Since constructors are called in reverse order, reverse the test * list so tests are run in source declaration order. @@ -913,7 +919,7 @@ void __wait_for_test(struct __test_metadata *t) int status; if (sigaction(SIGALRM, &action, &saved_action)) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: unable to install SIGALRM handler\n", t->name); @@ -925,7 +931,7 @@ void __wait_for_test(struct __test_metadata *t) waitpid(t->pid, &status, 0); alarm(0); if (sigaction(SIGALRM, &saved_action, NULL)) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: unable to uninstall SIGALRM handler\n", t->name); @@ -934,16 +940,16 @@ void __wait_for_test(struct __test_metadata *t) __active_test = NULL; if (t->timed_out) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { if (WEXITSTATUS(status) == KSFT_SKIP) { /* SKIP */ - t->passed = 1; + t->exit_code = KSFT_PASS; t->skip = 1; } else if (t->termsig != -1) { - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: Test exited normally instead of by signal (code: %d)\n", t->name, @@ -952,11 +958,11 @@ void __wait_for_test(struct __test_metadata *t) switch (WEXITSTATUS(status)) { /* Success */ case KSFT_PASS: - t->passed = 1; + t->exit_code = KSFT_PASS; break; /* Other failure, assume step report. */ default: - t->passed = 0; + t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, "# %s: Test failed at step #%d\n", t->name, @@ -964,13 +970,13 @@ void __wait_for_test(struct __test_metadata *t) } } } else if (WIFSIGNALED(status)) { - t->passed = 0; + t->exit_code = KSFT_FAIL; if (WTERMSIG(status) == SIGABRT) { fprintf(TH_LOG_STREAM, "# %s: Test terminated by assertion\n", t->name); } else if (WTERMSIG(status) == t->termsig) { - t->passed = 1; + t->exit_code = KSFT_PASS; } else { fprintf(TH_LOG_STREAM, "# %s: Test terminated unexpectedly by signal %d\n", @@ -1113,7 +1119,7 @@ void __run_test(struct __fixture_metadata *f, char test_name[LINE_MAX]; /* reset test struct */ - t->passed = 1; + t->exit_code = KSFT_PASS; t->skip = 0; t->trigger = 0; t->no_print = 0; @@ -1132,13 +1138,13 @@ void __run_test(struct __fixture_metadata *f, t->pid = fork(); if (t->pid < 0) { ksft_print_msg("ERROR SPAWNING TEST CHILD\n"); - t->passed = 0; + t->exit_code = KSFT_FAIL; } else if (t->pid == 0) { setpgrp(); t->fn(t, variant); if (t->skip) _exit(KSFT_SKIP); - if (t->passed) + if (__test_passed(t)) _exit(KSFT_PASS); /* Something else happened. */ _exit(KSFT_FAIL); @@ -1146,13 +1152,13 @@ void __run_test(struct __fixture_metadata *f, __wait_for_test(t); } ksft_print_msg(" %4s %s\n", - t->passed ? "OK" : "FAIL", test_name); + __test_passed(t) ? "OK" : "FAIL", test_name); if (t->skip) ksft_test_result_skip("%s\n", t->results->reason[0] ? t->results->reason : "unknown"); else - ksft_test_result(t->passed, "%s\n", test_name); + ksft_test_result(__test_passed(t), "%s\n", test_name); } static int test_harness_run(int argc, char **argv) @@ -1200,7 +1206,7 @@ static int test_harness_run(int argc, char **argv) t->results = results; __run_test(f, v, t); t->results = NULL; - if (t->passed) + if (__test_passed(t)) pass_count++; else ret = 1; diff --git a/tools/testing/selftests/landlock/base_test.c b/tools/testing/selftests/landlock/base_test.c index 646f778dfb1e..a6f89aaea77d 100644 --- a/tools/testing/selftests/landlock/base_test.c +++ b/tools/testing/selftests/landlock/base_test.c @@ -307,7 +307,7 @@ TEST(ruleset_fd_transfer) dir_fd = open("/tmp", O_RDONLY | O_DIRECTORY | O_CLOEXEC); ASSERT_LE(0, dir_fd); ASSERT_EQ(0, close(dir_fd)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h index f5b40399a538..89f8183bd8f5 100644 --- a/tools/testing/selftests/landlock/common.h +++ b/tools/testing/selftests/landlock/common.h @@ -46,29 +46,25 @@ if (child == 0) { \ _metadata->no_print = 1; \ fixture_name##_##test_name##_child(_metadata, self, variant); \ - if (_metadata->skip) \ - _exit(KSFT_SKIP); \ - if (_metadata->passed) \ - _exit(KSFT_PASS); \ - _exit(KSFT_FAIL); \ + _exit(_metadata->exit_code); \ } \ if (child != waitpid(child, &status, 0)) \ abort(); \ if (WIFSIGNALED(status) || !WIFEXITED(status)) { \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ _metadata->results->step = 1; \ return; \ } \ switch (WEXITSTATUS(status)) { \ case KSFT_PASS: \ - _metadata->passed = 1; \ + _metadata->exit_code = KSFT_PASS; \ break; \ case KSFT_SKIP: \ - _metadata->passed = 1; \ + _metadata->exit_code = KSFT_PASS; \ _metadata->skip = 1; \ break; \ default: \ - _metadata->passed = 0; \ + _metadata->exit_code = KSFT_FAIL; \ break; \ } \ } \ diff --git a/tools/testing/selftests/landlock/fs_test.c b/tools/testing/selftests/landlock/fs_test.c index 2d6d9b43d958..98817a14c91b 100644 --- a/tools/testing/selftests/landlock/fs_test.c +++ b/tools/testing/selftests/landlock/fs_test.c @@ -1964,7 +1964,7 @@ static void test_execute(struct __test_metadata *const _metadata, const int err, strerror(errno)); }; ASSERT_EQ(err, errno); - _exit(_metadata->passed ? 2 : 1); + _exit(__test_passed(_metadata) ? 2 : 1); return; } ASSERT_EQ(child, waitpid(child, &status, 0)); @@ -3807,7 +3807,7 @@ TEST_F_FORK(ftruncate, open_and_ftruncate_in_different_processes) ASSERT_EQ(0, close(socket_fds[0])); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } diff --git a/tools/testing/selftests/landlock/net_test.c b/tools/testing/selftests/landlock/net_test.c index 936cfc879f1d..f21cfbbc3638 100644 --- a/tools/testing/selftests/landlock/net_test.c +++ b/tools/testing/selftests/landlock/net_test.c @@ -539,7 +539,7 @@ static void test_bind_and_connect(struct __test_metadata *const _metadata, } EXPECT_EQ(0, close(connect_fd)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } @@ -834,7 +834,7 @@ TEST_F(protocol, connect_unspec) } EXPECT_EQ(0, close(connect_fd)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } diff --git a/tools/testing/selftests/landlock/ptrace_test.c b/tools/testing/selftests/landlock/ptrace_test.c index 55e7871631a1..a19db4d0b3bd 100644 --- a/tools/testing/selftests/landlock/ptrace_test.c +++ b/tools/testing/selftests/landlock/ptrace_test.c @@ -314,7 +314,7 @@ TEST_F(hierarchy, trace) ASSERT_EQ(0, pipe2(pipe_parent, O_CLOEXEC)); if (variant->domain_both) { create_domain(_metadata); - if (!_metadata->passed) + if (!__test_passed(_metadata)) /* Aborts before forking. */ return; } @@ -375,7 +375,7 @@ TEST_F(hierarchy, trace) /* Waits for the parent PTRACE_ATTACH test. */ ASSERT_EQ(1, read(pipe_parent[0], &buf_child, 1)); - _exit(_metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + _exit(_metadata->exit_code); return; } @@ -430,9 +430,10 @@ TEST_F(hierarchy, trace) /* Signals that the parent PTRACE_ATTACH test is done. */ ASSERT_EQ(1, write(pipe_parent[1], ".", 1)); ASSERT_EQ(child, waitpid(child, &status, 0)); + if (WIFSIGNALED(status) || !WIFEXITED(status) || WEXITSTATUS(status) != EXIT_SUCCESS) - _metadata->passed = 0; + _metadata->exit_code = KSFT_FAIL; } TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c index 49c84602707f..046d1ccedcf3 100644 --- a/tools/testing/selftests/net/tls.c +++ b/tools/testing/selftests/net/tls.c @@ -1882,7 +1882,7 @@ TEST_F(tls_err, poll_partial_rec_async) pfd.events = POLLIN; EXPECT_EQ(poll(&pfd, 1, 20), 1); - exit(!_metadata->passed); + exit(!__test_passed(_metadata)); } } diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c index 38f651469968..1027e6170186 100644 --- a/tools/testing/selftests/seccomp/seccomp_bpf.c +++ b/tools/testing/selftests/seccomp/seccomp_bpf.c @@ -1576,7 +1576,7 @@ void start_tracer(struct __test_metadata *_metadata, int fd, pid_t tracee, ASSERT_EQ(0, ret); } /* Directly report the status of our test harness results. */ - syscall(__NR_exit, _metadata->passed ? EXIT_SUCCESS : EXIT_FAILURE); + syscall(__NR_exit, _metadata->exit_code); } /* Common tracer setup/teardown functions. */ @@ -1623,7 +1623,7 @@ void teardown_trace_fixture(struct __test_metadata *_metadata, ASSERT_EQ(0, kill(tracer, SIGUSR1)); ASSERT_EQ(tracer, waitpid(tracer, &status, 0)); if (WEXITSTATUS(status)) - _metadata->passed = 0; + _metadata->exit_code = KSFT_FAIL; } } @@ -3088,8 +3088,7 @@ TEST(syscall_restart) } /* Directly report the status of our test harness results. */ - syscall(__NR_exit, _metadata->passed ? EXIT_SUCCESS - : EXIT_FAILURE); + syscall(__NR_exit, _metadata->exit_code); } EXPECT_EQ(0, close(pipefd[0])); @@ -3174,7 +3173,7 @@ TEST(syscall_restart) ASSERT_EQ(child_pid, waitpid(child_pid, &status, 0)); if (WIFSIGNALED(status) || WEXITSTATUS(status)) - _metadata->passed = 0; + _metadata->exit_code = KSFT_FAIL; } TEST_SIGNAL(filter_flag_log, SIGSYS) From patchwork Tue Feb 20 19:22:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564414 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B98CC14D435; Tue, 20 Feb 2024 19:22:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456969; cv=none; b=ivhyFpuw5Ykbu27TH3skF2hZJOesyjE73YtZR/lya6adFqsAf7oOSxCSFvhjpNO/QVnjg/itDz+1ZWV/Jz6CpLfMVeljvp2TaxcbfaN6tyYJnAm69CgtCkWAB5cEIGkhPB7PNAdtFSVDRmNxxu3ozaMTYPN3Syr1kclFuJPq+q8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456969; c=relaxed/simple; bh=yQO677NdufTEtYEpl+ewTAnUbcerKSpLBt1eERj1/SQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q+zMWLokMk713Or0j2H63sIgQqU76yRbLaOPtTL5xuxSKSRWJKtQW3++qxGhd3EXEuxsfqxPxIufjbA5N4tSxvp1ktPsllKolkoD0e0YE9kdYNpUDyau1BT7awlb5KyFwTHotNMxVOVwE4Fro4R62KAGtkR7r8w2sh02cO6+VUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=F43OqDMN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="F43OqDMN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D41B2C43399; Tue, 20 Feb 2024 19:22:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456969; bh=yQO677NdufTEtYEpl+ewTAnUbcerKSpLBt1eERj1/SQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F43OqDMNXUy78B8/sXZBH+lBUw2CKY4zXUZL3Mvu1zbeFsYzB8jU84XpxKdKThfJI gAni/U3xQPSMXVXbBvXeYwcus4/eDsW8BDRMZvUPUTZV7DH+T/VsYhpy4opEMh0N/P l9LLdV1DS+BBEfYGu77fM7OKpCPy02jut2qLV3fbNcb2mkvzLyELcBF2SAtQGRFG65 kOou316kcnD06WlQs3wcH450VTkIfK+wqD9vYNWnK3Cc9aAAwzW8UApjH2DHL3E4jF QGPKU9RdKVdr8QFuZLG3oc27sGOSPOD5qgzLiKYcsRJ+XPswKkCb2nMZ4iPu3dN8An 2P/ylWITiYEOw== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 05/11] selftests: kselftest_harness: use exit code to store skip Date: Tue, 20 Feb 2024 11:22:29 -0800 Message-ID: <20240220192235.2953484-6-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 We always use skip in combination with exit_code being 0 (KSFT_PASS). This are basic KSFT / KTAP semantics. Store the right KSFT_* code in exit_code directly. This makes it easier to support tests reporting other extended KSFT_* codes like XFAIL / XPASS. Signed-off-by: Jakub Kicinski Reviewed-by: Kees Cook --- tools/testing/selftests/kselftest_harness.h | 20 +++++--------------- tools/testing/selftests/landlock/common.h | 5 +---- 2 files changed, 6 insertions(+), 19 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index bbb339642a72..42ec3ca29c4b 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -136,8 +136,7 @@ fprintf(TH_LOG_STREAM, "# SKIP %s\n", \ _metadata->results->reason); \ } \ - _metadata->exit_code = KSFT_PASS; \ - _metadata->skip = 1; \ + _metadata->exit_code = KSFT_SKIP; \ _metadata->trigger = 0; \ statement; \ } while (0) @@ -389,7 +388,7 @@ if (setjmp(_metadata->env) == 0) { \ fixture_name##_setup(_metadata, &self, variant->data); \ /* Let setup failure terminate early. */ \ - if (!__test_passed(_metadata) || _metadata->skip) \ + if (_metadata->exit_code) \ return; \ _metadata->setup_completed = true; \ fixture_name##_##test_name(_metadata, &self, variant->data); \ @@ -834,7 +833,6 @@ struct __test_metadata { struct __fixture_metadata *fixture; int termsig; int exit_code; - int skip; /* did SKIP get used? */ int trigger; /* extra handler after the evaluation */ int timeout; /* seconds to wait for test timeout */ bool timed_out; /* did this test timeout instead of exiting? */ @@ -945,9 +943,7 @@ void __wait_for_test(struct __test_metadata *t) "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { if (WEXITSTATUS(status) == KSFT_SKIP) { - /* SKIP */ - t->exit_code = KSFT_PASS; - t->skip = 1; + t->exit_code = WEXITSTATUS(status); } else if (t->termsig != -1) { t->exit_code = KSFT_FAIL; fprintf(TH_LOG_STREAM, @@ -1120,7 +1116,6 @@ void __run_test(struct __fixture_metadata *f, /* reset test struct */ t->exit_code = KSFT_PASS; - t->skip = 0; t->trigger = 0; t->no_print = 0; memset(t->results->reason, 0, sizeof(t->results->reason)); @@ -1142,19 +1137,14 @@ void __run_test(struct __fixture_metadata *f, } else if (t->pid == 0) { setpgrp(); t->fn(t, variant); - if (t->skip) - _exit(KSFT_SKIP); - if (__test_passed(t)) - _exit(KSFT_PASS); - /* Something else happened. */ - _exit(KSFT_FAIL); + _exit(t->exit_code); } else { __wait_for_test(t); } ksft_print_msg(" %4s %s\n", __test_passed(t) ? "OK" : "FAIL", test_name); - if (t->skip) + if (t->exit_code == KSFT_SKIP) ksft_test_result_skip("%s\n", t->results->reason[0] ? t->results->reason : "unknown"); else diff --git a/tools/testing/selftests/landlock/common.h b/tools/testing/selftests/landlock/common.h index 89f8183bd8f5..56c5b5c5676a 100644 --- a/tools/testing/selftests/landlock/common.h +++ b/tools/testing/selftests/landlock/common.h @@ -57,11 +57,8 @@ } \ switch (WEXITSTATUS(status)) { \ case KSFT_PASS: \ - _metadata->exit_code = KSFT_PASS; \ - break; \ case KSFT_SKIP: \ - _metadata->exit_code = KSFT_PASS; \ - _metadata->skip = 1; \ + _metadata->exit_code = WEXITSTATUS(status); \ break; \ default: \ _metadata->exit_code = KSFT_FAIL; \ From patchwork Tue Feb 20 19:22:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564415 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C6BF14D449; Tue, 20 Feb 2024 19:22:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456970; cv=none; b=ee5GzdqAEigsiVvzFMXHdjJ2Zb+h1I/gb7qDh5tRO1AL+7hPgjzbd7Y+Gf9G3Z4Km1NqiDFo8IrK2lRvbLxIPbeumh2StyQi0sz9HnvLFrpWjoDzs0QycIOj5x4TlT/CxvCrNJgaZoFM/sUXCyquosC87IVkgWSXwfI+QsDIHX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456970; c=relaxed/simple; bh=m+npEDlZdX3gPR2VtCHypCxYq1e2U2q87S4XOIU1I7k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UmodaRR+duNWqgke+U5N8YqbGAzqo/uGOviAbSHeYYuaLT09jqJHWl/TE+BY20LP6D8HWX1+W+SeOiNCRdzN/L4r8lFuOedO1Q9AunwJuVv3pNturwWTKOiHzSFwFbxGofupHO/gsz1O7VI91OK03OjcwcNeIIPvRrAI4hJwsvg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ANAVcN3k; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ANAVcN3k" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D9E2C433C7; Tue, 20 Feb 2024 19:22:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456969; bh=m+npEDlZdX3gPR2VtCHypCxYq1e2U2q87S4XOIU1I7k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ANAVcN3kCS6MxK4xQ8kDLdT49yoJvqXx87ctE5vZ1bUU4d34BhOUSUU4+IVFNitSD 1bv08ML2PM/TNQo+mDA3cGZlMLYSFgpqghSgnNr6UXuiITiCZ4DhPpHoNx+2YzHyuB tC3V+8sfac8DPJ6kK+bEr0DIxLXiQQvjYQKOm8x3rlkOK/efQ4NN2AXDjTIfCAPoDv rJuRcG5RxqfJCsGElUMEkUbY8lablp7bJLoVSMnAipRe+resYqd+vS3WeyvjehvNdM bj0JIwFa6jDm0usU2eyAxyqaKaPkxJBKuoBTU4/tLutF3cjmtxddsa0uCq+ECShtEK MIEVCd5wqCjGg== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 06/11] selftests: kselftest: add ksft_test_result_code(), handling all exit codes Date: Tue, 20 Feb 2024 11:22:30 -0800 Message-ID: <20240220192235.2953484-7-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For generic test harness code it's more useful to deal with exit codes directly, rather than having to switch on them and call the right ksft_test_result_*() helper. Add such function to kselftest.h. Note that "directive" and "diagnostic" are what ktap docs call those parts of the message. Signed-off-by: Jakub Kicinski Reviewed-by: Kees Cook --- tools/testing/selftests/kselftest.h | 39 +++++++++++++++++++++ tools/testing/selftests/kselftest_harness.h | 9 +++-- 2 files changed, 46 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index a781e6311810..12ad7f8dfe3a 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -25,6 +25,7 @@ * ksft_test_result_skip(fmt, ...); * ksft_test_result_xfail(fmt, ...); * ksft_test_result_error(fmt, ...); + * ksft_test_result_code(exit_code, test_name, fmt, ...); * * When all tests are finished, clean up and exit the program with one of: * @@ -254,6 +255,44 @@ static inline __printf(1, 2) void ksft_test_result_error(const char *msg, ...) va_end(args); } +static inline __printf(2, 3) +void ksft_test_result_code(int exit_code, const char *msg, ...) +{ + const char *tap_code = "ok"; + const char *directive = ""; + int saved_errno = errno; + va_list args; + + switch (exit_code) { + case KSFT_PASS: + ksft_cnt.ksft_pass++; + break; + case KSFT_XFAIL: + directive = " # XFAIL "; + ksft_cnt.ksft_xfail++; + break; + case KSFT_XPASS: + directive = " # XPASS "; + ksft_cnt.ksft_xpass++; + break; + case KSFT_SKIP: + directive = " # SKIP "; + ksft_cnt.ksft_xskip++; + break; + case KSFT_FAIL: + default: + tap_code = "not ok"; + ksft_cnt.ksft_fail++; + break; + } + + va_start(args, msg); + printf("%s %u%s", tap_code, ksft_test_num(), directive); + errno = saved_errno; + vprintf(msg, args); + va_end(args); +} + static inline int ksft_exit_pass(void) { ksft_print_cnts(); diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 42ec3ca29c4b..684c5676dfcb 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1113,6 +1113,7 @@ void __run_test(struct __fixture_metadata *f, struct __test_metadata *t) { char test_name[LINE_MAX]; + const char *diagnostic; /* reset test struct */ t->exit_code = KSFT_PASS; @@ -1144,9 +1145,13 @@ void __run_test(struct __fixture_metadata *f, ksft_print_msg(" %4s %s\n", __test_passed(t) ? "OK" : "FAIL", test_name); + if (t->results->reason[0]) + diagnostic = t->results->reason; + else + diagnostic = "unknown"; + if (t->exit_code == KSFT_SKIP) - ksft_test_result_skip("%s\n", t->results->reason[0] ? - t->results->reason : "unknown"); + ksft_test_result_code(t->exit_code, "%s\n", diagnostic); else ksft_test_result(__test_passed(t), "%s\n", test_name); } From patchwork Tue Feb 20 19:22:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564416 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC11E14E2C3; Tue, 20 Feb 2024 19:22:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456970; cv=none; b=a5otnL88jEKQdLYIRGNXcWo2xCel333irgmrKlgcqJHPg2/RjVzBjb2/H2XC2h2PeBX/A5u0pZg86BV4BHLm3UaVItdke/F6k/zYXKBAl04/WYhMS4p5YVQzn1EA8qDsNKV5Y3LXIRNOJz63e3JaRLvo0Mv3ogt1fC/AJSe9yfs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456970; c=relaxed/simple; bh=dOHmxdg3EP760oxxQuwhluOWp5ADJEFGi+z4Te1Ix98=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fcmJQuW7TPfHduMWrPtt+55YIUAptn9uc0m4PQtIa3BKjoXE9grPSqZRzzpp0Fa6GSD5N8asSSkzIQgwoJR091xsRwDLEW5iGlBtr6uXvqvd1nxReBWSfIvXbaxYWwbbr8tOlz34bEkgYBgG/YKOtZAkgOZVvI737TpsIctzCEk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EzhjzG7l; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EzhjzG7l" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02831C433F1; Tue, 20 Feb 2024 19:22:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456970; bh=dOHmxdg3EP760oxxQuwhluOWp5ADJEFGi+z4Te1Ix98=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EzhjzG7lxvTipRQSfDA59QNrA/VebCwHM591nOKvV+80vxub9BgqGaWauDmDFayow 7SPiWjoxu5wcLupYoezmoL+Dj2bX3AmW2afJEVc4MV1zZC1LRdBP5HLnBm9SR5ve0q b8wnGd6SKVWLaY6dLpiVRvBrEhWBHsFF6DBhmQkZ73Ix9BuyQP05Cs0MNn1byVv/rt HSf1ZxDib0fNy7+Av4lQhUaU31e1kKWMDMdW5x2TGCFG46uxyVULJmzRsSlZGb6237 I/QygWWXJfVRQzvn8+y3YWKK5PuvyLGvlmtRgElcVNekk0F/iC9fULWX+Xf+HB4gRc LojDXgWvSG9wQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 07/11] selftests: kselftest_harness: print test name for SKIP Date: Tue, 20 Feb 2024 11:22:31 -0800 Message-ID: <20240220192235.2953484-8-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Jakub points out that for parsers it's rather useful to always have the test name on the result line. Currently if we SKIP (or soon XFAIL or XPASS), we will print: ok 17 # SKIP SCTP doesn't support IP_BIND_ADDRESS_NO_PORT ^ no test name Always print the test name. KTAP format seems to allow or even call for it, per: https://docs.kernel.org/dev-tools/ktap.html Suggested-by: Jakub Sitnicki Link: https://lore.kernel.org/all/87jzn6lnou.fsf@cloudflare.com/ Signed-off-by: Jakub Kicinski Reviewed-by: Kees Cook --- tools/testing/selftests/kselftest.h | 7 ++++--- tools/testing/selftests/kselftest_harness.h | 3 ++- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 12ad7f8dfe3a..25e29626566e 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -255,8 +255,9 @@ static inline __printf(1, 2) void ksft_test_result_error(const char *msg, ...) va_end(args); } -static inline __printf(2, 3) -void ksft_test_result_code(int exit_code, const char *msg, ...) +static inline __printf(3, 4) +void ksft_test_result_code(int exit_code, const char *test_name, + const char *msg, ...) { const char *tap_code = "ok"; const char *directive = ""; @@ -287,7 +288,7 @@ void ksft_test_result_code(int exit_code, const char *msg, ...) } va_start(args, msg); - printf("%s %u%s", tap_code, ksft_test_num(), directive); + printf("%s %u %s%s", tap_code, ksft_test_num(), test_name, directive); errno = saved_errno; vprintf(msg, args); va_end(args); diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 684c5676dfcb..a1a794e4b8f6 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1151,7 +1151,8 @@ void __run_test(struct __fixture_metadata *f, diagnostic = "unknown"; if (t->exit_code == KSFT_SKIP) - ksft_test_result_code(t->exit_code, "%s\n", diagnostic); + ksft_test_result_code(t->exit_code, test_name, + "%s\n", diagnostic); else ksft_test_result(__test_passed(t), "%s\n", test_name); } From patchwork Tue Feb 20 19:22:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564417 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F1AE14E2DC; Tue, 20 Feb 2024 19:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456971; cv=none; b=RWYvB8ran9tp3u5Hql4vd95pjAFSqHEvIE22395uyIEWrTEKOJS04/x7bwtHPEpU1/eXLPfPKSSXkxhjBrG+/AjmHqkHYBmg8DlIQnrOO4i99FNE46RRZH1F+YUA3L2fUCODApaAawD1twbDsIbnbBw41StEgFUpNhII37Qq4ig= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456971; c=relaxed/simple; bh=cYocYM4a4DJFgijvA8HgeBwLvdbc/wQhimkAwiVCBwM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fnveUABhgVhmWjXSbNamiXPzKfvdOjLNCVedvMZAj8OP/UTHoj48iuun0OFrdd9I6pE0vbetRzOZs9Zh6GRpI+YJBbvDWT0tEg5+3Mcr6w0KIOUkkkncEYYpgTjsu9qVCqN9oKeJibEdue7V2bJZcteGmvUf3pplN36StZfIjSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=djnSW9lC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="djnSW9lC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90D67C43399; Tue, 20 Feb 2024 19:22:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456971; bh=cYocYM4a4DJFgijvA8HgeBwLvdbc/wQhimkAwiVCBwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=djnSW9lCiqYnAdp7+cy6FTQO9RrJxRa9z+5+WC4sBozKgWg4jzggnOitloUELOZNp Z6Pqcli+5Biig4ssTcW5T28+fSIaoonq3HupSmHIUdLFqacXeN6d8W7ZJpbH3r7w5j qQg857qmWm2xbalk1Mc9+ad63Mbd1KmJriqmgxeEzp+9sLoAs/ANS9slvMH/VtRYxT 8feHXbLMitgvzevMmQMTXD/XaqwQeI/kpCtBWnJk5+zjGBqNGJRRLj4RAmuTZmS7g6 DQ02Pe+jfc3PE2JupAku7cEjHhz2m6re0ygkEdOjcMBCp5qMXsuvd0ZyPuA3aRlET4 8/9GVymjZLRQQ== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 08/11] selftests: kselftest_harness: separate diagnostic message with # in ksft_test_result_code() Date: Tue, 20 Feb 2024 11:22:32 -0800 Message-ID: <20240220192235.2953484-9-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 According to the spec we should always print a # if we add a diagnostic message. Having the caller pass in the new line as part of diagnostic message makes handling this a bit counter-intuitive, so append the new line in the helper. Signed-off-by: Jakub Kicinski Reviewed-by: Kees Cook --- tools/testing/selftests/kselftest.h | 5 +++++ tools/testing/selftests/kselftest_harness.h | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 25e29626566e..541bf192e30e 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -287,10 +287,15 @@ void ksft_test_result_code(int exit_code, const char *test_name, break; } + /* Docs seem to call for double space if directive is absent */ + if (!directive[0] && msg[0]) + directive = " # "; + va_start(args, msg); printf("%s %u %s%s", tap_code, ksft_test_num(), test_name, directive); errno = saved_errno; vprintf(msg, args); + printf("\n"); va_end(args); } diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index a1a794e4b8f6..6157a7b059c6 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1152,7 +1152,7 @@ void __run_test(struct __fixture_metadata *f, if (t->exit_code == KSFT_SKIP) ksft_test_result_code(t->exit_code, test_name, - "%s\n", diagnostic); + "%s", diagnostic); else ksft_test_result(__test_passed(t), "%s\n", test_name); } From patchwork Tue Feb 20 19:22:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564418 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E680714E2FC; Tue, 20 Feb 2024 19:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456972; cv=none; b=SVeFj7rsIHX+Viyrhm3iP3coNYrr11bcXMLSVIV/k0DafY5S2lAP7hIUUbOREiq5GRSuN9X1A3IgacsE+cxvhpJyfmbaYA8xFlPtzB2Ck9gNBbeZ9kukYg4hnMBgL/q6HOvudhRgykBW/h/vG1X121KOqazLJD9wVCEffQ/ASCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456972; c=relaxed/simple; bh=9Wj3EGg1bJZZ78CqquR3OBcBaDowmip0RKN2lao/f3o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qQ5KU3K/gO0HvVrCfrRO7q8+JNHbX62GDnj8MW2y3+/BlfAsOciw1GbQE1plYAiFiQ4JLiLSi2GXv9nOr0QwhkapBxTSxzZCaO0OIY0ozZLTgxwbjTrmtfdbLto0Du8MO2uFexgwCepRny0NhYuu5IJDlWp4wAcHpY8+6LepcrU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VzKOQD48; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VzKOQD48" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29AEBC433A6; Tue, 20 Feb 2024 19:22:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456971; bh=9Wj3EGg1bJZZ78CqquR3OBcBaDowmip0RKN2lao/f3o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VzKOQD48qaw1EA/D+lYIcRBeFLcCHN/vIkeocRaQzEcokFnYb1vvT+zomxN1NJxAB UezFBNUtGDmIqvkeVA1pm+MlKlEEg2vJBg7VhFrW1rRLVz26SAwnu+Jm+kQf1UnGL9 e6N8/RUAKx1n1nit8qir7VOyTt32mLfR+JLgotDCaAmgcR5N9GG5YQrGkZvLH24k7u LMgQZ/eK7LRLLID/iqQ869Gktb2gtECkSWVXvfAGs0kiVTzkKetGpNzj8tlYVkMddj ynaxvcQR7Zbp24jfghoA1BMOwSEC9NOfAfym5dH7BEnUgLm6PWBFiH/oQ0SXO2JUlb Fo2mLaDlHxbAw== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 09/11] selftests: kselftest_harness: let PASS / FAIL provide diagnostic Date: Tue, 20 Feb 2024 11:22:33 -0800 Message-ID: <20240220192235.2953484-10-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Switch to printing KTAP line for PASS / FAIL with ksft_test_result_code(), this gives us the ability to report diagnostic messages. Signed-off-by: Jakub Kicinski Reviewed-by: Kees Cook --- tools/testing/selftests/kselftest_harness.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 6157a7b059c6..7c3d19bf4220 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -1147,14 +1147,13 @@ void __run_test(struct __fixture_metadata *f, if (t->results->reason[0]) diagnostic = t->results->reason; + else if (t->exit_code == KSFT_PASS || t->exit_code == KSFT_FAIL) + diagnostic = NULL; else diagnostic = "unknown"; - if (t->exit_code == KSFT_SKIP) - ksft_test_result_code(t->exit_code, test_name, - "%s", diagnostic); - else - ksft_test_result(__test_passed(t), "%s\n", test_name); + ksft_test_result_code(t->exit_code, test_name, + diagnostic ? "%s" : "", diagnostic); } static int test_harness_run(int argc, char **argv) From patchwork Tue Feb 20 19:22:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564419 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92FB614F9D1; Tue, 20 Feb 2024 19:22:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456972; cv=none; b=u56Q1bOGPhtdrwQRWzLZ3M3jW8sXbSvzudCW86yaf4aGNsOt0M7S9wsa914b20AqdvaXMixRBvcEMoK6jgKkAg09o05fGtvyCZwAqtvpTkA9LSWV0arTx74ahPzZr7bfCnOdmsN781rF8zUHkhr5RUBAJKjSph9H/1/9kGUzAzU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456972; c=relaxed/simple; bh=iaTH+1eL2/uaHAafS7u7tUseHm4TKZG3irvM72NEE70=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=L0CqWHLvuVJMpOzcujPg2fgQp9bziy1cLacZBP+p9qT/8k5QsrqNymXNx3R/XDNNoazJUPMp0p/pwVOZCNMRjvoxFQUyLHCGxKc/EpsK6xJgtCx38edzQmRS7tiXOtXm+a6CjZc/Byy8XKCEJlImMWlFxXXUX3f3GzG87iFbX0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tnSZVadW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tnSZVadW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4F99C433F1; Tue, 20 Feb 2024 19:22:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456972; bh=iaTH+1eL2/uaHAafS7u7tUseHm4TKZG3irvM72NEE70=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tnSZVadWji5OkYzZQTP7VCnbRbccg5d67G7P3fM9hYE3evZ1Cmq63VmEfOnRpBQyh 8okX26jifCJfo0OVWrKliWzXZGLufAc989+0wwBc0llm0pE7Nh4Ifo0F+lhlqA7zfb 3lNlh765Veli1etyB2PLx5CcNOtmmbM+xkO/IQXpR7FesoPmmMo5hAROsF3GhdgxaU DDU0aPKh8lIfx7TH1IrCjE+aF4qkPuxtsZFhmzA37EvHruu+dE7jtKd5uZ1S+L0/c4 h9cwWX3oBtGSRzQhYh1BTB1ZPO9cxH30qkKWZMmcTTDQ0XS3kAbiQPMgErW/LgWnQc 2IhkwHPf3jqYA== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 10/11] selftests: kselftest_harness: support using xfail Date: Tue, 20 Feb 2024 11:22:34 -0800 Message-ID: <20240220192235.2953484-11-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Currently some tests report skip for things they expect to fail e.g. when given combination of parameters is known to be unsupported. This is confusing because in an ideal test environment and fully featured kernel no tests should be skipped. Selftest summary line already includes xfail and xpass counters, e.g.: but there's no way to use it from within the harness. Add a new per-fixture+variant combination list of test cases we expect to fail. Signed-off-by: Jakub Kicinski Reviewed-by: Kees Cook --- v3: change the paradigm to declarative --- tools/testing/selftests/kselftest_harness.h | 49 ++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 7c3d19bf4220..1ade9fac0b5f 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -800,6 +800,37 @@ struct __fixture_metadata { .prev = &_fixture_global, }; +struct __test_xfail { + struct __fixture_metadata *fixture; + struct __fixture_variant_metadata *variant; + struct __test_metadata *test; + struct __test_xfail *prev, *next; +}; + +/** + * XFAIL_ADD() - mark variant + test case combination as expected to fail + * @fixture_name: name of the fixture + * @variant_name: name of the variant + * @test_name: name of the test case + * + * Mark a combination of variant + test case for a given fixture as expected + * to fail. Tests marked this way will report XPASS / XFAIL return codes, + * instead of PASS / FAIL,and use respective counters. + */ +#define XFAIL_ADD(fixture_name, variant_name, test_name) \ + static struct __test_xfail \ + _##fixture_name##_##variant_name##_##test_name##_xfail = \ + { \ + .fixture = &_##fixture_name##_fixture_object, \ + .variant = &_##fixture_name##_##variant_name##_object, \ + .test = &_##fixture_name##_##test_name##_object, \ + }; \ + static void __attribute__((constructor)) \ + _register_##fixture_name##_##variant_name##_##test_name##_xfail(void) \ + { \ + __register_xfail(&_##fixture_name##_##variant_name##_##test_name##_xfail); \ + } + static struct __fixture_metadata *__fixture_list = &_fixture_global; static int __constructor_order; @@ -814,6 +845,7 @@ static inline void __register_fixture(struct __fixture_metadata *f) struct __fixture_variant_metadata { const char *name; const void *data; + struct __test_xfail *xfails; struct __fixture_variant_metadata *prev, *next; }; @@ -864,6 +896,11 @@ static inline void __register_test(struct __test_metadata *t) __LIST_APPEND(t->fixture->tests, t); } +static inline void __register_xfail(struct __test_xfail *xf) +{ + __LIST_APPEND(xf->variant->xfails, xf); +} + static inline int __bail(int for_realz, struct __test_metadata *t) { /* if this is ASSERT, return immediately. */ @@ -942,7 +979,9 @@ void __wait_for_test(struct __test_metadata *t) fprintf(TH_LOG_STREAM, "# %s: Test terminated by timeout\n", t->name); } else if (WIFEXITED(status)) { - if (WEXITSTATUS(status) == KSFT_SKIP) { + if (WEXITSTATUS(status) == KSFT_SKIP || + WEXITSTATUS(status) == KSFT_XPASS || + WEXITSTATUS(status) == KSFT_XFAIL) { t->exit_code = WEXITSTATUS(status); } else if (t->termsig != -1) { t->exit_code = KSFT_FAIL; @@ -1112,6 +1151,7 @@ void __run_test(struct __fixture_metadata *f, struct __fixture_variant_metadata *variant, struct __test_metadata *t) { + struct __test_xfail *xfail; char test_name[LINE_MAX]; const char *diagnostic; @@ -1145,6 +1185,13 @@ void __run_test(struct __fixture_metadata *f, ksft_print_msg(" %4s %s\n", __test_passed(t) ? "OK" : "FAIL", test_name); + /* Check if we're expecting this test to fail */ + for (xfail = variant->xfails; xfail; xfail = xfail->next) + if (xfail->test == t) + break; + if (xfail) + t->exit_code = __test_passed(t) ? KSFT_XPASS : KSFT_XFAIL; + if (t->results->reason[0]) diagnostic = t->results->reason; else if (t->exit_code == KSFT_PASS || t->exit_code == KSFT_FAIL) From patchwork Tue Feb 20 19:22:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 13564420 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A14514F9E4; Tue, 20 Feb 2024 19:22:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456973; cv=none; b=owV77To9qknZN9WDySc+QmVrVteKS5Eph0u4LzbFPJYxnWAeRQwG6d+zoH1aI/QU4Xy2Q+RGiDKYofj4uCkMpppb2XpHILgQTSEOUXIZW3JCKDj+4lTvW7k2f6EEmNGfIdbC3yvZEUJ3UOZ8mQBctiugK5PtVOXC85LFV4/cQoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708456973; c=relaxed/simple; bh=lOTPndp0IMsasoyW4JuCFzuHcrAfQnrv/Lkla2nksNI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DNF7Tna3zyE4/whvt9YlTUaB9xFtFZnqWrxFTypMPgSem4IZlTIe/z0pGTAf5d7o/7WQvPEa6tiY7XuJltGn7aOu1hK3qqv7FCFsduXeOamfXErqlO8uzm1RscII1tinzPossiUb6WyIQSv7F5stVcuMf0nYrJypSxCzPijTp94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uKEf/j0a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uKEf/j0a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4BBFFC43143; Tue, 20 Feb 2024 19:22:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708456972; bh=lOTPndp0IMsasoyW4JuCFzuHcrAfQnrv/Lkla2nksNI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uKEf/j0aeJoxPfj9bSOxbHe+O+B02D2jn5fLRP7aUssAYd6ZijPIeBYKU0rJ8/iQ7 +O6v1QcgmigQpGUEldp0Ss1vELEGERn+Lq8HIJTyUu76HNqDPCA76uB9/NUdbMIenr /6YnMqq2lV8xaaeqECAkwdVAP4nxjrtnXv9GkDxLAa8QRvJmi6FIggf8xZQGAUXjfZ zY6IVY2b0EBMOQLoFg7ZcX/w7ZCSgJXSL5+YmAU54A0d2iUWjaRtjNbgdoZw39A+eu sBnjHkxGoz0awabFldSnuxVmsdXnBQlxJB63jerpj+N6PBa1xWnfavGxutY2AzhmFZ zOsltWyUVXE9g== From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, shuah@kernel.org, linux-kselftest@vger.kernel.org, mic@digikod.net, linux-security-module@vger.kernel.org, keescook@chromium.org, jakub@cloudflare.com, Jakub Kicinski Subject: [PATCH net-next v3 11/11] selftests: ip_local_port_range: use XFAIL instead of SKIP Date: Tue, 20 Feb 2024 11:22:35 -0800 Message-ID: <20240220192235.2953484-12-kuba@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240220192235.2953484-1-kuba@kernel.org> References: <20240220192235.2953484-1-kuba@kernel.org> Precedence: bulk X-Mailing-List: linux-security-module@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 SCTP does not support IP_LOCAL_PORT_RANGE and we know it, so use XFAIL instead of SKIP. Reviewed-by: Kees Cook Tested-by: Jakub Sitnicki Signed-off-by: Jakub Kicinski --- tools/testing/selftests/net/ip_local_port_range.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/net/ip_local_port_range.c b/tools/testing/selftests/net/ip_local_port_range.c index 6ebd58869a63..193b82745fd8 100644 --- a/tools/testing/selftests/net/ip_local_port_range.c +++ b/tools/testing/selftests/net/ip_local_port_range.c @@ -365,9 +365,6 @@ TEST_F(ip_local_port_range, late_bind) __u32 range; __u16 port; - if (variant->so_protocol == IPPROTO_SCTP) - SKIP(return, "SCTP doesn't support IP_BIND_ADDRESS_NO_PORT"); - fd = socket(variant->so_domain, variant->so_type, 0); ASSERT_GE(fd, 0) TH_LOG("socket failed"); @@ -414,6 +411,9 @@ TEST_F(ip_local_port_range, late_bind) ASSERT_TRUE(!err) TH_LOG("close failed"); } +XFAIL_ADD(ip_local_port_range, ip4_stcp, late_bind); +XFAIL_ADD(ip_local_port_range, ip6_stcp, late_bind); + TEST_F(ip_local_port_range, get_port_range) { __u16 lo, hi;