From patchwork Wed May 17 11:18:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13244736 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 842C9C77B75 for ; Wed, 17 May 2023 11:20:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230064AbjEQLUe (ORCPT ); Wed, 17 May 2023 07:20:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbjEQLUd (ORCPT ); Wed, 17 May 2023 07:20:33 -0400 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09F41103 for ; Wed, 17 May 2023 04:20:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684322433; x=1715858433; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=FzGWov9BBJvwViTRy8xFbr/fsg+jM+ytcaLpUaA0EIo=; b=UXQZqFLpqHTUO2FjCYyv+6j3dmyZZ1ht2Uh407xtUWPT+1PI/CYoloBX qi2RWwXK9lKcwM7xHgfoyiMqikVTUNu0wzz0e4fd0ZkKkOlsKCeXWDbFY +mWihr13yFrrfhwfp97z1GamcPgBmrIJooN9URP2GndfrTmVJhJOZh3YX OUBBDmjUF9PMIGuMJhTNI9dgYFCkFq8Lnt0Tv3g2xu3A/VX5s05Jv5NdI yAXEh7wAiSik5KwtVzp9MD1TWaglDILjbP/gFZs7K6hrSTC7D/PEGSTIl +ZY3lFWMxrcJLXwcilSkJqCL6zTJcE2GcZXVJ+N1F6yIAKe5PWroAgdYr w==; X-IronPort-AV: E=McAfee;i="6600,9927,10712"; a="341115904" X-IronPort-AV: E=Sophos;i="5.99,281,1677571200"; d="scan'208";a="341115904" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 04:20:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10712"; a="766738735" X-IronPort-AV: E=Sophos;i="5.99,281,1677571200"; d="scan'208";a="766738735" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.142.110]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 04:20:30 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Subject: [PATCH v3 1/3] kunit/test: Add example test showing parameterized testing Date: Wed, 17 May 2023 13:18:14 +0200 Message-Id: <20230517111816.984-2-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230517111816.984-1-michal.wajdeczko@intel.com> References: <20230517111816.984-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Use of parameterized testing is documented [1] but such use case is not present in demo kunit test. Add small subtest for that. [1] https://kernel.org/doc/html/latest/dev-tools/kunit/usage.html#parameterized-testing Signed-off-by: Michal Wajdeczko Cc: David Gow Reviewed-by: David Gow --- lib/kunit/kunit-example-test.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c index 24315c882b31..b69b689ea850 100644 --- a/lib/kunit/kunit-example-test.c +++ b/lib/kunit/kunit-example-test.c @@ -187,6 +187,39 @@ static void example_static_stub_test(struct kunit *test) KUNIT_EXPECT_EQ(test, add_one(1), 2); } +static const struct example_param { + int value; +} example_params_array[] = { + { .value = 2, }, + { .value = 1, }, + { .value = 0, }, +}; + +static void example_param_get_desc(const struct example_param *p, char *desc) +{ + snprintf(desc, KUNIT_PARAM_DESC_SIZE, "example value %d", p->value); +} + +KUNIT_ARRAY_PARAM(example, example_params_array, example_param_get_desc); + +/* + * This test shows the use of params. + */ +static void example_params_test(struct kunit *test) +{ + const struct example_param *param = test->param_value; + + /* By design, param pointer will not be NULL */ + KUNIT_ASSERT_NOT_NULL(test, param); + + /* Test can be skipped on unsupported param values */ + if (!param->value) + kunit_skip(test, "unsupported param value"); + + /* You can use param values for parameterized testing */ + KUNIT_EXPECT_EQ(test, param->value % param->value, 0); +} + /* * Here we make a list of all the test cases we want to add to the test suite * below. @@ -203,6 +236,7 @@ static struct kunit_case example_test_cases[] = { KUNIT_CASE(example_mark_skipped_test), KUNIT_CASE(example_all_expect_macros_test), KUNIT_CASE(example_static_stub_test), + KUNIT_CASE_PARAM(example_params_test, example_gen_params), {} }; From patchwork Wed May 17 11:18:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13244738 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F928C77B7A for ; Wed, 17 May 2023 11:20:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230258AbjEQLUg (ORCPT ); Wed, 17 May 2023 07:20:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230093AbjEQLUe (ORCPT ); Wed, 17 May 2023 07:20:34 -0400 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FD8C103 for ; Wed, 17 May 2023 04:20:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684322434; x=1715858434; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=8SpyRf+yeQrxbHtQ75fj/0ADY3ykc7S3MEaXhzRIDVw=; b=JvxVhVktXJbclCFUolhyymiK55D2E7BR2+QEMkEGrZINThwl1LcOO34d 39Sy72Wl7koFkhHJIXpGArCiK6l4mGdhENDOWnDeFnhAXLSvyT3SGIq1H UfIlQ+OP8fWfqFgGmQrooXZpFkO1Rk58nm+kMPHPu5hEazAAdkUKjeRXk XFQXZJftM7zF5TmESCKjEAQt4vhzpTQu7a8v02BO3XW7wA5IyW+GAvP5X 64Dlb6x4lKSsdoOOza2vDHOuJgTcAjqzgYFQXuDWobeqFBLGnKYsDaFrG 6g2F6sya2qd95nuBazRydLLpCzMRjrbFrNcZ72gdrV9tsKd8vyicSYd6P A==; X-IronPort-AV: E=McAfee;i="6600,9927,10712"; a="341115909" X-IronPort-AV: E=Sophos;i="5.99,281,1677571200"; d="scan'208";a="341115909" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 04:20:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10712"; a="766738744" X-IronPort-AV: E=Sophos;i="5.99,281,1677571200"; d="scan'208";a="766738744" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.142.110]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 04:20:31 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Subject: [PATCH v3 2/3] kunit: Fix reporting of the skipped parameterized tests Date: Wed, 17 May 2023 13:18:15 +0200 Message-Id: <20230517111816.984-3-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230517111816.984-1-michal.wajdeczko@intel.com> References: <20230517111816.984-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Logs from the parameterized tests that were skipped don't include SKIP directive thus they are displayed as PASSED. Fix that. Signed-off-by: Michal Wajdeczko Cc: David Gow Reviewed-by: Rae Moar Reviewed-by: David Gow --- lib/kunit/test.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index f5e4ceffd282..af48d0761d26 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -627,9 +627,11 @@ int kunit_run_tests(struct kunit_suite *suite) kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT - "%s %d %s", + "%s %d %s%s%s", kunit_status_to_ok_not_ok(test.status), - test.param_index + 1, param_desc); + test.param_index + 1, param_desc, + test.status == KUNIT_SKIPPED ? " # SKIP " : "", + test.status == KUNIT_SKIPPED ? test.status_comment : ""); /* Get next param. */ param_desc[0] = '\0'; From patchwork Wed May 17 11:18:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Wajdeczko X-Patchwork-Id: 13244739 X-Patchwork-Delegate: brendanhiggins@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D261FC7EE2A for ; Wed, 17 May 2023 11:20:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229623AbjEQLUh (ORCPT ); Wed, 17 May 2023 07:20:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbjEQLUg (ORCPT ); Wed, 17 May 2023 07:20:36 -0400 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E130B212B for ; Wed, 17 May 2023 04:20:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684322434; x=1715858434; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=GQucUMzI6Nkp4iPVZaB2quqORjlxJoX3GnCD93qSWnU=; b=hRAb3/+wasmXdExEuEvz2ENDtfqeOsu528vU83Z4PSJmgLZlkxOKrQKu N1PjMnoGqv6TB+gVFNLhmZHEVMOQlzTdmN+Bfv4LXhkw+8uvENh9glni+ 5TycKl95BgYoizq+4vVrylNh7efZijvqJulZOO2qEk4QH/a8RHQRsndeI M5aMspItoEnwNYVUPnQd8J4dZ+waXdFHM901rnFcqxgORoLGvQrwX1+lu uO6ig/L3t33niKWrgWepQBnqeBmCfqc+szjJmlbS6Ofsd/7kmJD/hfV05 OCDisgSO0LIV/TdmUuwM+bsRo/3iJ3QikWO6V43f8E1PEsLhpg2PN6aGR g==; X-IronPort-AV: E=McAfee;i="6600,9927,10712"; a="341115918" X-IronPort-AV: E=Sophos;i="5.99,281,1677571200"; d="scan'208";a="341115918" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 04:20:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10712"; a="766738752" X-IronPort-AV: E=Sophos;i="5.99,281,1677571200"; d="scan'208";a="766738752" Received: from mwajdecz-mobl.ger.corp.intel.com ([10.249.142.110]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2023 04:20:32 -0700 From: Michal Wajdeczko To: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com Subject: [PATCH v3 3/3] kunit: Update kunit_print_ok_not_ok function Date: Wed, 17 May 2023 13:18:16 +0200 Message-Id: <20230517111816.984-4-michal.wajdeczko@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230517111816.984-1-michal.wajdeczko@intel.com> References: <20230517111816.984-1-michal.wajdeczko@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org There is no need use opaque test_or_suite pointer and is_test flag as we don't use anything from the suite struct. Always expect test pointer and use NULL as indication that provided results are from the suite so we can treat them differently. Since results could be from nested tests, like parameterized tests, add explicit level parameter to properly indent output messages and thus allow to reuse this function from other places. While around, remove small code duplication near skip directive. Signed-off-by: Michal Wajdeczko Cc: David Gow Cc: Rae Moar Reviewed-by: David Gow --- include/kunit/test.h | 1 + lib/kunit/test.c | 45 +++++++++++++++++++++++++++----------------- 2 files changed, 29 insertions(+), 17 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 3028a1a3fcad..d717fc055e06 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -47,6 +47,7 @@ struct kunit; * sub-subtest. See the "Subtests" section in * https://node-tap.org/tap-protocol/ */ +#define KUNIT_INDENT_LEN 4 #define KUNIT_SUBTEST_INDENT " " #define KUNIT_SUBSUBTEST_INDENT " " diff --git a/lib/kunit/test.c b/lib/kunit/test.c index af48d0761d26..6bc92ced82ee 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -185,16 +185,28 @@ static void kunit_print_suite_start(struct kunit_suite *suite) kunit_suite_num_test_cases(suite)); } -static void kunit_print_ok_not_ok(void *test_or_suite, - bool is_test, +/* Currently supported test levels */ +enum { + KUNIT_LEVEL_SUITE = 0, + KUNIT_LEVEL_CASE, + KUNIT_LEVEL_CASE_PARAM, +}; + +static void kunit_print_ok_not_ok(struct kunit *test, + unsigned int test_level, enum kunit_status status, size_t test_number, const char *description, const char *directive) { - struct kunit_suite *suite = is_test ? NULL : test_or_suite; - struct kunit *test = is_test ? test_or_suite : NULL; const char *directive_header = (status == KUNIT_SKIPPED) ? " # SKIP " : ""; + const char *directive_body = (status == KUNIT_SKIPPED) ? directive : ""; + + /* + * When test is NULL assume that results are from the suite + * and today suite results are expected at level 0 only. + */ + WARN(!test && test_level, "suite test level can't be %u!\n", test_level); /* * We do not log the test suite results as doing so would @@ -203,17 +215,18 @@ static void kunit_print_ok_not_ok(void *test_or_suite, * separately seq_printf() the suite results for the debugfs * representation. */ - if (suite) + if (!test) pr_info("%s %zd %s%s%s\n", kunit_status_to_ok_not_ok(status), test_number, description, directive_header, - (status == KUNIT_SKIPPED) ? directive : ""); + directive_body); else kunit_log(KERN_INFO, test, - KUNIT_SUBTEST_INDENT "%s %zd %s%s%s", + "%*s%s %zd %s%s%s", + KUNIT_INDENT_LEN * test_level, "", kunit_status_to_ok_not_ok(status), test_number, description, directive_header, - (status == KUNIT_SKIPPED) ? directive : ""); + directive_body); } enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite) @@ -239,7 +252,7 @@ static size_t kunit_suite_counter = 1; static void kunit_print_suite_end(struct kunit_suite *suite) { - kunit_print_ok_not_ok((void *)suite, false, + kunit_print_ok_not_ok(NULL, KUNIT_LEVEL_SUITE, kunit_suite_has_succeeded(suite), kunit_suite_counter++, suite->name, @@ -625,13 +638,11 @@ int kunit_run_tests(struct kunit_suite *suite) "param-%d", test.param_index); } - kunit_log(KERN_INFO, &test, - KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT - "%s %d %s%s%s", - kunit_status_to_ok_not_ok(test.status), - test.param_index + 1, param_desc, - test.status == KUNIT_SKIPPED ? " # SKIP " : "", - test.status == KUNIT_SKIPPED ? test.status_comment : ""); + kunit_print_ok_not_ok(&test, KUNIT_LEVEL_CASE_PARAM, + test.status, + test.param_index + 1, + param_desc, + test.status_comment); /* Get next param. */ param_desc[0] = '\0'; @@ -645,7 +656,7 @@ int kunit_run_tests(struct kunit_suite *suite) kunit_print_test_stats(&test, param_stats); - kunit_print_ok_not_ok(&test, true, test_case->status, + kunit_print_ok_not_ok(&test, KUNIT_LEVEL_CASE, test_case->status, kunit_test_case_num(suite, test_case), test_case->name, test.status_comment);