From patchwork Mon Sep 25 17:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13398170 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65938CE7AB1 for ; Mon, 25 Sep 2023 17:58:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232326AbjIYR6I (ORCPT ); Mon, 25 Sep 2023 13:58:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231360AbjIYR6G (ORCPT ); Mon, 25 Sep 2023 13:58:06 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A3FD1A5 for ; Mon, 25 Sep 2023 10:57:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695664678; x=1727200678; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8kO+KrNTR8kWxuX/LyKXcB+Gb7mODQeojOqMSeENlHM=; b=MEKKYlgIloyLK/Hwmho6D2XOiIMTPXRt2KKhx3Gez4KbHwjLuE0Pth2q 9Xm24if/rt+PAx6+ZSq6SXFN96Wuyhi96OwAGfXoJB76BV7gYI5bf3l+x tHZELoA9iRwb/NbbtoF2be8lI7B4ZT+YO3lTwgDvlkLxs/uLZwB10eU8A NNlQox2XGIBr2iM/6Pw6gYPsYnzT2b26uyouBiTVf10lk9ojawMpnY87h 6+IsWtrK0hZhkd1uZpcWzvznxuZTDVYUzeNfquWE2h5HSJJwA+eJsWbZS Auf4FAGGA+Pkz0wXBG8RrSllxe+xwcXrWzQbr5hA4qENQZJ+ilV+ueNk4 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="371643649" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="371643649" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:57:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="995489266" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="995489266" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.131.28]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:57:54 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Cc: Michal Wajdeczko , David Gow , Rae Moar Subject: [PATCH 1/4] kunit: Drop redundant text from suite init failure message Date: Mon, 25 Sep 2023 19:57:30 +0200 Message-Id: <20230925175733.1379-2-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230925175733.1379-1-michal.wajdeczko@intel.com> References: <20230925175733.1379-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org If a suite initialization fails, then our diagnostic message will include redundant indent and hash sign as all this was already added by the kunit_printk() used by kunit_err() macro. This could be easily seen if we force some error in our example test by modyfing example_test_init_suite() and running: $ ./tools/testing/kunit/kunit.py run --raw_output \ --kunitconfig ./lib/kunit/.kunitconfig "example.*" KTAP version 1 1..1 # example: initializing suite # example: # failed to initialize (-19) not ok 1 example Fix that and while around improve error code reporting by using error pointer format %pe that gives more user friendly output: KTAP version 1 1..1 # example: initializing suite # example: failed to initialize (-ENODEV) not ok 1 example Signed-off-by: Michal Wajdeczko Cc: David Gow Cc: Rae Moar Reviewed-by: Rae Moar Reviewed-by: David Gow --- lib/kunit/test.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index f2eb71f1a66c..fb5981ce578d 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -568,8 +568,8 @@ int kunit_run_tests(struct kunit_suite *suite) if (suite->suite_init) { suite->suite_init_err = suite->suite_init(suite); if (suite->suite_init_err) { - kunit_err(suite, KUNIT_SUBTEST_INDENT - "# failed to initialize (%d)", suite->suite_init_err); + kunit_err(suite, "failed to initialize (%pe)", + ERR_PTR(suite->suite_init_err)); goto suite_end; } } From patchwork Mon Sep 25 17:57:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13398174 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DEF3CE7AB4 for ; Mon, 25 Sep 2023 17:58:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232462AbjIYR6K (ORCPT ); Mon, 25 Sep 2023 13:58:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232554AbjIYR6G (ORCPT ); Mon, 25 Sep 2023 13:58:06 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7157B1A7 for ; Mon, 25 Sep 2023 10:57:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695664678; x=1727200678; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GZYYgPqw7ohm87675/5XKWP5wf2DjbWIEkF5V01oH7M=; b=EZo3ekO5VuaUIbr+pkkdafBPYJx0mX4nlqgGRPpycyFltI98qOBkZTV6 Yy3pCOhp9KTTYF06y6y9dLxCSM+wfOtRjFOjqG6yY1aPrHaeaDKhTB46p VWt/gVde1n9fjcaD4+PHx//chmKvET2jneEy1GMWDSBcw6nT0KRkTGs0M RudbFv5kpGHJMytMEscZMOrNDMr416aFU00jipgVB50HvmaUBQcwShrWW hPULOktK3E0gvlKRHE5kY5kJ0j0H3H1OiposCJ4vk0KGH1Fp8iL/geLZl HLtnwfgWVX5SeRN9k/RrhzYAlJcin5QEgn0eQJHoOEFoTt1nkfVbw89tx Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="371643653" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="371643653" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:57:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="995489273" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="995489273" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.131.28]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:57:56 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Cc: Michal Wajdeczko , David Gow , Rae Moar Subject: [PATCH 2/4] kunit: Fix indentation level of suite messages Date: Mon, 25 Sep 2023 19:57:31 +0200 Message-Id: <20230925175733.1379-3-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230925175733.1379-1-michal.wajdeczko@intel.com> References: <20230925175733.1379-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org A kunit suite is a top level test from the KTAP point of view but all suite diagnostic messages are printed at the subtest level: $ ./tools/testing/kunit/kunit.py run --raw_output \ --kunitconfig ./lib/kunit/.kunitconfig "example.*simple*" KTAP version 1 1..1 # example: initializing suite # example: failed to initialize (-ENODEV) not ok 1 example KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example # module: kunit_example_test 1..1 # example_simple_test: initializing # example_simple_test: cleaning up ok 1 example_simple_test # example: exiting suite ok 1 example Replace hardcoded indent in kunit_printk() with flexible indentation based on the argument type (test or suite): KTAP version 1 1..1 # example: initializing suite # example: failed to initialize (-ENODEV) not ok 1 example KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example # module: kunit_example_test 1..1 # example_simple_test: initializing # example_simple_test: cleaning up ok 1 example_simple_test # example: exiting suite ok 1 example Signed-off-by: Michal Wajdeczko Cc: David Gow Cc: Rae Moar --- include/kunit/test.h | 24 ++++++++++++++++++++++-- lib/kunit/test.c | 7 ------- 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 20ed9f9275c9..158876c4cb43 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -509,6 +509,21 @@ void __printf(2, 3) kunit_log_append(struct string_stream *log, const char *fmt, kunit_try_catch_throw(&((test_or_suite)->try_catch)); \ } while (0) +/* Currently supported test levels */ +enum { + KUNIT_LEVEL_SUITE = 0, + KUNIT_LEVEL_CASE, + KUNIT_LEVEL_CASE_PARAM, +}; + +#define kunit_level(test_or_suite) \ + _Generic((test_or_suite), \ + struct kunit_suite * : KUNIT_LEVEL_SUITE, \ + struct kunit * : KUNIT_LEVEL_CASE) + +#define kunit_indent_level(test_or_suite) \ + (KUNIT_INDENT_LEN * kunit_level(test_or_suite)) + /* * printk and log to per-test or per-suite log buffer. Logging only done * if CONFIG_KUNIT_DEBUGFS is 'y'; if it is 'n', no log is allocated/used. @@ -520,9 +535,14 @@ void __printf(2, 3) kunit_log_append(struct string_stream *log, const char *fmt, ##__VA_ARGS__); \ } while (0) +#define kunit_log_indent(lvl, test_or_suite, fmt, ...) \ + kunit_log(lvl, test_or_suite, "%*s" fmt, \ + kunit_indent_level(test_or_suite), "", \ + ##__VA_ARGS__) + #define kunit_printk(lvl, test, fmt, ...) \ - kunit_log(lvl, test, KUNIT_SUBTEST_INDENT "# %s: " fmt, \ - (test)->name, ##__VA_ARGS__) + kunit_log_indent(lvl, test, "# %s: " fmt, \ + (test)->name, ##__VA_ARGS__) /** * kunit_info() - Prints an INFO level message associated with @test. diff --git a/lib/kunit/test.c b/lib/kunit/test.c index fb5981ce578d..d10e6d610e20 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -135,13 +135,6 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite) } EXPORT_SYMBOL_GPL(kunit_suite_num_test_cases); -/* Currently supported test levels */ -enum { - KUNIT_LEVEL_SUITE = 0, - KUNIT_LEVEL_CASE, - KUNIT_LEVEL_CASE_PARAM, -}; - static void kunit_print_suite_start(struct kunit_suite *suite) { /* From patchwork Mon Sep 25 17:57:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13398173 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 224C7CE7AB6 for ; Mon, 25 Sep 2023 17:58:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232165AbjIYR6L (ORCPT ); Mon, 25 Sep 2023 13:58:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232102AbjIYR6H (ORCPT ); Mon, 25 Sep 2023 13:58:07 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 866BE1B0 for ; Mon, 25 Sep 2023 10:58:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695664680; x=1727200680; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xGMeTT1k3Boz0PoAgFBIhfApJ6Bl/sQXlmU26NPL9aE=; b=RWMVVv9zX9Hd5GDJajtnFWkUJ0TwS+cchN0pdPRGYjmZKE4zHkcsuma8 AVCmHJHRmXvwkA6YZ2q+HRfnE2eL/qugZMwp2lWXqJvPXQJy18ktF7yWL G+5PSWKBDFEpH5Xuvgg7wG1Y0A6YoRbWZ20gyfoFiwDwjg0dj1E5q3H9v zDLKbxyAbIiWkHj4TNfM4hxLKtU0goONCGuFc87mIstFOJ+1ZleaENJCW huI2NonPQQVJCLKGrLSlpU2ProKgjHGGggePB1gk5lVZQkGtcpSh9w4Z8 ZUcRDUQZhoe6YCQNTSnUvLpvZfNIaySZsPnoCBFAs4kTmESdMdvFiV7O2 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="371643657" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="371643657" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:57:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="995489286" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="995489286" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.131.28]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:57:58 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Cc: Michal Wajdeczko , David Gow , Rae Moar Subject: [PATCH 3/4] kunit: Fix indentation of parameterized tests messages Date: Mon, 25 Sep 2023 19:57:32 +0200 Message-Id: <20230925175733.1379-4-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230925175733.1379-1-michal.wajdeczko@intel.com> References: <20230925175733.1379-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org When running parametrized test cases, diagnostic messages are not properly aligned with the test result lines: $ ./tools/testing/kunit/kunit.py run --raw_output \ --kunitconfig ./lib/kunit/.kunitconfig *.example_params* KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example # module: kunit_example_test 1..1 KTAP version 1 # Subtest: example_params_test # example_params_test: initializing # example_params_test: cleaning up ok 1 example value 3 # SKIP unsupported param value 3 # example_params_test: initializing # example_params_test: cleaning up ok 2 example value 2 # example_params_test: initializing # example_params_test: cleaning up ok 3 example value 1 # example_params_test: initializing # example_params_test: cleaning up ok 4 example value 0 # SKIP unsupported param value 0 # example_params_test: pass:2 fail:0 skip:2 total:4 ok 1 example_params_test # example: exiting suite # Totals: pass:2 fail:0 skip:2 total:4 ok 1 example Add test level attribute and use it to generate proper indent at the runtime: KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example # module: kunit_example_test 1..1 KTAP version 1 # Subtest: example_params_test # example_params_test: initializing # example_params_test: cleaning up ok 1 example value 3 # SKIP unsupported param value 3 # example_params_test: initializing # example_params_test: cleaning up ok 2 example value 2 # example_params_test: initializing # example_params_test: cleaning up ok 3 example value 1 # example_params_test: initializing # example_params_test: cleaning up ok 4 example value 0 # SKIP unsupported param value 0 # example_params_test: pass:2 fail:0 skip:2 total:4 ok 1 example_params_test # example: exiting suite # Totals: pass:2 fail:0 skip:2 total:4 ok 1 example Signed-off-by: Michal Wajdeczko Cc: David Gow Cc: Rae Moar --- include/kunit/test.h | 3 ++- lib/kunit/test.c | 52 ++++++++++++++++++++------------------------ 2 files changed, 26 insertions(+), 29 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 158876c4cb43..4804d539e10f 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -276,6 +276,7 @@ struct kunit { void *priv; /* private: internal use only. */ + unsigned int level; /* Helps in proper log indent */ const char *name; /* Read only after initialization! */ struct string_stream *log; /* Points at case log after initialization */ struct kunit_try_catch try_catch; @@ -519,7 +520,7 @@ enum { #define kunit_level(test_or_suite) \ _Generic((test_or_suite), \ struct kunit_suite * : KUNIT_LEVEL_SUITE, \ - struct kunit * : KUNIT_LEVEL_CASE) + struct kunit * : ((struct kunit *)(test_or_suite))->level) #define kunit_indent_level(test_or_suite) \ (KUNIT_INDENT_LEN * kunit_level(test_or_suite)) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index d10e6d610e20..43c3efc286e4 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -99,14 +99,13 @@ static void kunit_print_test_stats(struct kunit *test, if (!kunit_should_print_stats(stats)) return; - kunit_log(KERN_INFO, test, - KUNIT_SUBTEST_INDENT - "# %s: pass:%lu fail:%lu skip:%lu total:%lu", - test->name, - stats.passed, - stats.failed, - stats.skipped, - stats.total); + kunit_log_indent(KERN_INFO, test, + "# %s: pass:%lu fail:%lu skip:%lu total:%lu", + test->name, + stats.passed, + stats.failed, + stats.skipped, + stats.total); } /* Append formatted message to log. */ @@ -154,7 +153,6 @@ static void kunit_print_suite_start(struct kunit_suite *suite) } static void kunit_print_ok_not_ok(struct kunit *test, - unsigned int test_level, enum kunit_status status, size_t test_number, const char *description, @@ -163,12 +161,6 @@ static void kunit_print_ok_not_ok(struct kunit *test, const char *directive_header = (status == KUNIT_SKIPPED) ? " # SKIP " : ""; const char *directive_body = (status == KUNIT_SKIPPED) ? directive : ""; - /* - * When test is NULL assume that results are from the suite - * and today suite results are expected at level 0 only. - */ - WARN(!test && test_level, "suite test level can't be %u!\n", test_level); - /* * We do not log the test suite results as doing so would * mean debugfs display would consist of an incorrect test @@ -182,12 +174,11 @@ static void kunit_print_ok_not_ok(struct kunit *test, test_number, description, directive_header, directive_body); else - kunit_log(KERN_INFO, test, - "%*s%s %zd %s%s%s", - KUNIT_INDENT_LEN * test_level, "", - kunit_status_to_ok_not_ok(status), - test_number, description, directive_header, - directive_body); + kunit_log_indent(KERN_INFO, test, + "%s %zd %s%s%s", + kunit_status_to_ok_not_ok(status), + test_number, description, directive_header, + directive_body); } enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite) @@ -213,7 +204,7 @@ static size_t kunit_suite_counter = 1; static void kunit_print_suite_end(struct kunit_suite *suite) { - kunit_print_ok_not_ok(NULL, KUNIT_LEVEL_SUITE, + kunit_print_ok_not_ok(NULL, kunit_suite_has_succeeded(suite), kunit_suite_counter++, suite->name, @@ -322,6 +313,7 @@ void kunit_init_test(struct kunit *test, const char *name, struct string_stream { spin_lock_init(&test->lock); INIT_LIST_HEAD(&test->resources); + test->level = KUNIT_LEVEL_CASE; test->name = name; test->log = log; if (test->log) @@ -584,14 +576,15 @@ int kunit_run_tests(struct kunit_suite *suite) kunit_run_case_catch_errors(suite, test_case, &test); kunit_update_stats(¶m_stats, test.status); } else { + /* Parameterized test is one level up from simple test-case. */ + test.level++; + /* Get initial param. */ param_desc[0] = '\0'; test.param_value = test_case->generate_params(NULL, param_desc); test_case->status = KUNIT_SKIPPED; - kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT - "KTAP version 1\n"); - kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT - "# Subtest: %s", test_case->name); + kunit_log_indent(KERN_INFO, &test, "KTAP version 1\n"); + kunit_log_indent(KERN_INFO, &test, "# Subtest: %s", test_case->name); while (test.param_value) { kunit_run_case_catch_errors(suite, test_case, &test); @@ -601,7 +594,7 @@ int kunit_run_tests(struct kunit_suite *suite) "param-%d", test.param_index); } - kunit_print_ok_not_ok(&test, KUNIT_LEVEL_CASE_PARAM, + kunit_print_ok_not_ok(&test, test.status, test.param_index + 1, param_desc, @@ -616,13 +609,16 @@ int kunit_run_tests(struct kunit_suite *suite) test.status = KUNIT_SUCCESS; test.status_comment[0] = '\0'; } + + /* Return to parent (test-case) level. */ + test.level--; } kunit_print_attr((void *)test_case, true, KUNIT_LEVEL_CASE); kunit_print_test_stats(&test, param_stats); - kunit_print_ok_not_ok(&test, KUNIT_LEVEL_CASE, test_case->status, + kunit_print_ok_not_ok(&test, test_case->status, kunit_test_case_num(suite, test_case), test_case->name, test.status_comment); From patchwork Mon Sep 25 17:57:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13398172 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1103CE7AB5 for ; Mon, 25 Sep 2023 17:58:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230319AbjIYR6M (ORCPT ); Mon, 25 Sep 2023 13:58:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232248AbjIYR6I (ORCPT ); Mon, 25 Sep 2023 13:58:08 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A9DD11D for ; Mon, 25 Sep 2023 10:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695664682; x=1727200682; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=03C41OMffArkupQnMzMOXyeB8LKSiHhumuP6kOtlVCU=; b=LJfZ338Hm/tGUc6esvgMEHhLY11KRNpCEY7G4OtGKUVttmxwZy53jbak nhrzw3cJV70mAWdfUcAwN1zqmA5JEpBx/DYIn9i5gkI/Je5TEagIRNGfl gSjndp0mehIteLTBt2pmPU6j5fnfbI/PHkwf65tc85gtToP8qsG8rlYLr sRFfDngxXgde8J6cmjnqmyFzYlNX3iUT5BMV8NF2LStufVDYXetI2F+Aq +EMdsWbuQyUXzyLAAqbAC9F4Uj1TEycbpgjQ7M6j2mXN3bbdt5mlfDtlV am+VcjVYNeRfBy/DK8vBpLIDSkzzJHM+6z/MerIJaa6/aQPddlw6QSK0r A==; X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="371643663" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="371643663" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:58:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="995489289" X-IronPort-AV: E=Sophos;i="6.03,175,1694761200"; d="scan'208";a="995489289" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.131.28]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 10:58:00 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Cc: Michal Wajdeczko , David Gow , Rae Moar Subject: [PATCH 4/4] kunit: Prepare test plan for parameterized subtests Date: Mon, 25 Sep 2023 19:57:33 +0200 Message-Id: <20230925175733.1379-5-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230925175733.1379-1-michal.wajdeczko@intel.com> References: <20230925175733.1379-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org In case of parameterized tests we are not providing a test plan so we can't detect if any result is missing. Count available params using the same generator as during a test execution Signed-off-by: Michal Wajdeczko Cc: David Gow Cc: Rae Moar --- lib/kunit/test.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 43c3efc286e4..55eabb324f39 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -540,6 +540,20 @@ static void kunit_accumulate_stats(struct kunit_result_stats *total, total->total += add.total; } +static size_t count_test_case_params(struct kunit_case *test_case) +{ + char param_desc[KUNIT_PARAM_DESC_SIZE]; + const void *param_value = NULL; + size_t num = 0; + + if (test_case->generate_params) + while ((param_value = test_case->generate_params(param_value, + param_desc))) + num++; + + return num; +} + int kunit_run_tests(struct kunit_suite *suite) { char param_desc[KUNIT_PARAM_DESC_SIZE]; @@ -585,6 +599,8 @@ int kunit_run_tests(struct kunit_suite *suite) test_case->status = KUNIT_SKIPPED; kunit_log_indent(KERN_INFO, &test, "KTAP version 1\n"); kunit_log_indent(KERN_INFO, &test, "# Subtest: %s", test_case->name); + kunit_log_indent(KERN_INFO, &test, "1..%zd\n", + count_test_case_params(test_case)); while (test.param_value) { kunit_run_case_catch_errors(suite, test_case, &test);