diff mbox series

[net-next] selftests: avoid using SKIP(exit()) in harness fixure setup

Message ID 20240304233621.646054-1-kuba@kernel.org (mailing list archive)
State Accepted
Commit e3350ba4a5b7573d4e14ee1dff8c414646435a04
Headers show
Series [net-next] selftests: avoid using SKIP(exit()) in harness fixure setup | expand

Commit Message

Jakub Kicinski March 4, 2024, 11:36 p.m. UTC
selftest harness uses various exit codes to signal test
results. Avoid calling exit() directly, otherwise tests
may get broken by harness refactoring (like the commit
under Fixes). SKIP() will instruct the harness that the
test shouldn't run, it used to not be the case, but that
has been fixed. So just return, no need to exit.

Note that for hmm-tests this actually changes the result
from pass to skip. Which seems fair, the test is skipped,
after all.

Reported-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/all/05f7bf89-04a5-4b65-bf59-c19456aeb1f0@sirena.org.uk
Fixes: a724707976b0 ("selftests: kselftest_harness: use KSFT_* exit codes")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
This needs to go to net-next because that's where the breaking
patch was (mis?)-applied.

CC: ivan.orlov0322@gmail.com
CC: perex@perex.cz
CC: tiwai@suse.com
CC: broonie@kernel.org
CC: shuah@kernel.org
CC: jglisse@redhat.com
CC: akpm@linux-foundation.org
CC: keescook@chromium.org
CC: linux-sound@vger.kernel.org
CC: linux-kselftest@vger.kernel.org
CC: linux-mm@kvack.org
---
 tools/testing/selftests/alsa/test-pcmtest-driver.c | 4 ++--
 tools/testing/selftests/mm/hmm-tests.c             | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

Comments

Kees Cook March 5, 2024, 12:49 a.m. UTC | #1
On Mon, Mar 04, 2024 at 03:36:20PM -0800, Jakub Kicinski wrote:
> selftest harness uses various exit codes to signal test
> results. Avoid calling exit() directly, otherwise tests
> may get broken by harness refactoring (like the commit
> under Fixes). SKIP() will instruct the harness that the
> test shouldn't run, it used to not be the case, but that
> has been fixed. So just return, no need to exit.
> 
> Note that for hmm-tests this actually changes the result
> from pass to skip. Which seems fair, the test is skipped,
> after all.
> 
> Reported-by: Mark Brown <broonie@kernel.org>
> Link: https://lore.kernel.org/all/05f7bf89-04a5-4b65-bf59-c19456aeb1f0@sirena.org.uk
> Fixes: a724707976b0 ("selftests: kselftest_harness: use KSFT_* exit codes")
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Reviewed-by: Kees Cook <keescook@chromium.org>
Mark Brown March 5, 2024, 3:32 p.m. UTC | #2
On Mon, Mar 04, 2024 at 03:36:20PM -0800, Jakub Kicinski wrote:
> selftest harness uses various exit codes to signal test
> results. Avoid calling exit() directly, otherwise tests
> may get broken by harness refactoring (like the commit
> under Fixes). SKIP() will instruct the harness that the
> test shouldn't run, it used to not be the case, but that
> has been fixed. So just return, no need to exit.
> 
> Note that for hmm-tests this actually changes the result
> from pass to skip. Which seems fair, the test is skipped,
> after all.

Reviewed-by: Mark Brown <broonie@kernel.org>
Tested-by: Mark Brown <broonie@kernel.org>

This fixes at least the pcmtest-driver suite.
Przemek Kitszel March 5, 2024, 3:54 p.m. UTC | #3
On 3/5/24 00:36, Jakub Kicinski wrote:
> selftest harness uses various exit codes to signal test
> results. Avoid calling exit() directly, otherwise tests
> may get broken by harness refactoring (like the commit
> under Fixes). SKIP() will instruct the harness that the
> test shouldn't run, it used to not be the case, but that
> has been fixed. So just return, no need to exit.
> 
> Note that for hmm-tests this actually changes the result
> from pass to skip. Which seems fair, the test is skipped,
> after all.
> 
> Reported-by: Mark Brown <broonie@kernel.org>
> Link: https://lore.kernel.org/all/05f7bf89-04a5-4b65-bf59-c19456aeb1f0@sirena.org.uk
> Fixes: a724707976b0 ("selftests: kselftest_harness: use KSFT_* exit codes")

I believe that the next patch of the linked series is a culprit,
but that does not mandate a next revision in my eyes

> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> This needs to go to net-next because that's where the breaking
> patch was (mis?)-applied.
> 
> CC: ivan.orlov0322@gmail.com
> CC: perex@perex.cz
> CC: tiwai@suse.com
> CC: broonie@kernel.org
> CC: shuah@kernel.org
> CC: jglisse@redhat.com
> CC: akpm@linux-foundation.org
> CC: keescook@chromium.org
> CC: linux-sound@vger.kernel.org
> CC: linux-kselftest@vger.kernel.org
> CC: linux-mm@kvack.org
> ---
>   tools/testing/selftests/alsa/test-pcmtest-driver.c | 4 ++--
>   tools/testing/selftests/mm/hmm-tests.c             | 4 ++--
>   2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/alsa/test-pcmtest-driver.c b/tools/testing/selftests/alsa/test-pcmtest-driver.c
> index a52ecd43dbe3..ca81afa4ee90 100644
> --- a/tools/testing/selftests/alsa/test-pcmtest-driver.c
> +++ b/tools/testing/selftests/alsa/test-pcmtest-driver.c
> @@ -127,11 +127,11 @@ FIXTURE_SETUP(pcmtest) {
>   	int err;
>   
>   	if (geteuid())
> -		SKIP(exit(-1), "This test needs root to run!");
> +		SKIP(return, "This test needs root to run!");
>   
>   	err = read_patterns();
>   	if (err)
> -		SKIP(exit(-1), "Can't read patterns. Probably, module isn't loaded");
> +		SKIP(return, "Can't read patterns. Probably, module isn't loaded");
>   
>   	card_name = malloc(127);
>   	ASSERT_NE(card_name, NULL);
> diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c
> index 20294553a5dd..d2cfc9b494a0 100644
> --- a/tools/testing/selftests/mm/hmm-tests.c
> +++ b/tools/testing/selftests/mm/hmm-tests.c
> @@ -138,7 +138,7 @@ FIXTURE_SETUP(hmm)
>   
>   	self->fd = hmm_open(variant->device_number);
>   	if (self->fd < 0 && hmm_is_coherent_type(variant->device_number))
> -		SKIP(exit(0), "DEVICE_COHERENT not available");
> +		SKIP(return, "DEVICE_COHERENT not available");
>   	ASSERT_GE(self->fd, 0);
>   }
>   
> @@ -149,7 +149,7 @@ FIXTURE_SETUP(hmm2)
>   
>   	self->fd0 = hmm_open(variant->device_number0);
>   	if (self->fd0 < 0 && hmm_is_coherent_type(variant->device_number0))
> -		SKIP(exit(0), "DEVICE_COHERENT not available");
> +		SKIP(return, "DEVICE_COHERENT not available");
>   	ASSERT_GE(self->fd0, 0);
>   	self->fd1 = hmm_open(variant->device_number1);
>   	ASSERT_GE(self->fd1, 0);

Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
- that's totally what mandates vfork() introduced in recent refactor

(Sorry for pointing the same in the Link:-ed thread, it was just
newer/higher in my email client)
patchwork-bot+netdevbpf@kernel.org March 6, 2024, 3:40 a.m. UTC | #4
Hello:

This patch was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Mon,  4 Mar 2024 15:36:20 -0800 you wrote:
> selftest harness uses various exit codes to signal test
> results. Avoid calling exit() directly, otherwise tests
> may get broken by harness refactoring (like the commit
> under Fixes). SKIP() will instruct the harness that the
> test shouldn't run, it used to not be the case, but that
> has been fixed. So just return, no need to exit.
> 
> [...]

Here is the summary with links:
  - [net-next] selftests: avoid using SKIP(exit()) in harness fixure setup
    https://git.kernel.org/netdev/net-next/c/e3350ba4a5b7

You are awesome, thank you!
diff mbox series

Patch

diff --git a/tools/testing/selftests/alsa/test-pcmtest-driver.c b/tools/testing/selftests/alsa/test-pcmtest-driver.c
index a52ecd43dbe3..ca81afa4ee90 100644
--- a/tools/testing/selftests/alsa/test-pcmtest-driver.c
+++ b/tools/testing/selftests/alsa/test-pcmtest-driver.c
@@ -127,11 +127,11 @@  FIXTURE_SETUP(pcmtest) {
 	int err;
 
 	if (geteuid())
-		SKIP(exit(-1), "This test needs root to run!");
+		SKIP(return, "This test needs root to run!");
 
 	err = read_patterns();
 	if (err)
-		SKIP(exit(-1), "Can't read patterns. Probably, module isn't loaded");
+		SKIP(return, "Can't read patterns. Probably, module isn't loaded");
 
 	card_name = malloc(127);
 	ASSERT_NE(card_name, NULL);
diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c
index 20294553a5dd..d2cfc9b494a0 100644
--- a/tools/testing/selftests/mm/hmm-tests.c
+++ b/tools/testing/selftests/mm/hmm-tests.c
@@ -138,7 +138,7 @@  FIXTURE_SETUP(hmm)
 
 	self->fd = hmm_open(variant->device_number);
 	if (self->fd < 0 && hmm_is_coherent_type(variant->device_number))
-		SKIP(exit(0), "DEVICE_COHERENT not available");
+		SKIP(return, "DEVICE_COHERENT not available");
 	ASSERT_GE(self->fd, 0);
 }
 
@@ -149,7 +149,7 @@  FIXTURE_SETUP(hmm2)
 
 	self->fd0 = hmm_open(variant->device_number0);
 	if (self->fd0 < 0 && hmm_is_coherent_type(variant->device_number0))
-		SKIP(exit(0), "DEVICE_COHERENT not available");
+		SKIP(return, "DEVICE_COHERENT not available");
 	ASSERT_GE(self->fd0, 0);
 	self->fd1 = hmm_open(variant->device_number1);
 	ASSERT_GE(self->fd1, 0);