| Message ID | 20210610163959.71634-1-andrealmeid@collabora.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | lib: Convert UUID runtime test to KUnit | expand |
On Thu, Jun 10, 2021 at 01:39:58PM -0300, André Almeida wrote: > Hi, > > This patch converts existing UUID runtime test to use KUnit framework. > > Below, there's a comparison between the old output format and the new > one. Keep in mind that even if KUnit seems very verbose, this is the > corner case where _every_ test has failed. Btw, do we have test coverage statistics? I mean since we reduced 18 test cases to 12, do we still have the same / better test coverage?
Hi Andy, Às 06:55 de 11/06/21, Andy Shevchenko escreveu: > On Thu, Jun 10, 2021 at 01:39:58PM -0300, André Almeida wrote: >> Hi, >> >> This patch converts existing UUID runtime test to use KUnit framework. >> >> Below, there's a comparison between the old output format and the new >> one. Keep in mind that even if KUnit seems very verbose, this is the >> corner case where _every_ test has failed. > > Btw, do we have test coverage statistics? > > I mean since we reduced 18 test cases to 12, do we still have the same / better > test coverage? > I don't think we have automated statistics, but I can assure you that the coverage it's exactly the same. We are testing two correlated functions with the same input, in a single test case, instead of having a single case for each one, so that's why the number of cases is reduced. For example, instead of: total_tests++; if (guid_parse(data->uuid, &le)) total_tests++; if (!guid_equal(&data->le, &le)) We now have: KUNIT_ASSERT_EQ(guid_parse(data->guid, &le), 0) KUNIT_EXPECT_TRUE(guid_equal(&data->le, &le)) That will count as a single test.
Hi, This patch converts existing UUID runtime test to use KUnit framework. Below, there's a comparison between the old output format and the new one. Keep in mind that even if KUnit seems very verbose, this is the corner case where _every_ test has failed. * This is how the current output looks like in success: test_uuid: all 18 tests passed * And when it fails: test_uuid: conversion test #1 failed on LE data: 'c33f4995-3701-450e-9fbf-206a2e98e576' test_uuid: cmp test #2 failed on LE data: 'c33f4995-3701-450e-9fbf-206a2e98e576' test_uuid: cmp test #2 actual data: 'c33f4995-3701-450e-9fbf-206a2e98e576' test_uuid: conversion test #3 failed on BE data: 'c33f4995-3701-450e-9fbf-206a2e98e576' test_uuid: cmp test #4 failed on BE data: 'c33f4995-3701-450e-9fbf-206a2e98e576' test_uuid: cmp test #4 actual data: 'c33f4995-3701-450e-9fbf-206a2e98e576' test_uuid: conversion test #5 failed on LE data: '64b4371c-77c1-48f9-8221-29f054fc023b' test_uuid: cmp test #6 failed on LE data: '64b4371c-77c1-48f9-8221-29f054fc023b' test_uuid: cmp test #6 actual data: '64b4371c-77c1-48f9-8221-29f054fc023b' test_uuid: conversion test #7 failed on BE data: '64b4371c-77c1-48f9-8221-29f054fc023b' test_uuid: cmp test #8 failed on BE data: '64b4371c-77c1-48f9-8221-29f054fc023b' test_uuid: cmp test #8 actual data: '64b4371c-77c1-48f9-8221-29f054fc023b' test_uuid: conversion test #9 failed on LE data: '0cb4ddff-a545-4401-9d06-688af53e7f84' test_uuid: cmp test #10 failed on LE data: '0cb4ddff-a545-4401-9d06-688af53e7f84' test_uuid: cmp test #10 actual data: '0cb4ddff-a545-4401-9d06-688af53e7f84' test_uuid: conversion test #11 failed on BE data: '0cb4ddff-a545-4401-9d06-688af53e7f84' test_uuid: cmp test #12 failed on BE data: '0cb4ddff-a545-4401-9d06-688af53e7f84' test_uuid: cmp test #12 actual data: '0cb4ddff-a545-4401-9d06-688af53e7f84' test_uuid: negative test #13 passed on wrong LE data: 'c33f4995-3701-450e-9fbf206a2e98e576 ' test_uuid: negative test #14 passed on wrong BE data: 'c33f4995-3701-450e-9fbf206a2e98e576 ' test_uuid: negative test #15 passed on wrong LE data: '64b4371c-77c1-48f9-8221-29f054XX023b' test_uuid: negative test #16 passed on wrong BE data: '64b4371c-77c1-48f9-8221-29f054XX023b' test_uuid: negative test #17 passed on wrong LE data: '0cb4ddff-a545-4401-9d06-688af53e' test_uuid: negative test #18 passed on wrong BE data: '0cb4ddff-a545-4401-9d06-688af53e' test_uuid: failed 18 out of 18 tests * Now, here's how it looks like with KUnit: ======== [PASSED] uuid ======== [PASSED] uuid_correct_be [PASSED] uuid_correct_le [PASSED] uuid_wrong_be [PASSED] uuid_wrong_le * And if every test fail with KUnit: ======== [FAILED] uuid ======== [FAILED] uuid_correct_be # uuid_correct_be: ASSERTION FAILED at lib/test_uuid.c:57 Expected uuid_parse(data->uuid, &be) == 1, but uuid_parse(data->uuid, &be) == 0 failed to parse 'c33f4995-3701-450e-9fbf-206a2e98e576' # uuid_correct_be: not ok 1 - c33f4995-3701-450e-9fbf-206a2e98e576 # uuid_correct_be: ASSERTION FAILED at lib/test_uuid.c:57 Expected uuid_parse(data->uuid, &be) == 1, but uuid_parse(data->uuid, &be) == 0 failed to parse '64b4371c-77c1-48f9-8221-29f054fc023b' # uuid_correct_be: not ok 2 - 64b4371c-77c1-48f9-8221-29f054fc023b # uuid_correct_be: ASSERTION FAILED at lib/test_uuid.c:57 Expected uuid_parse(data->uuid, &be) == 1, but uuid_parse(data->uuid, &be) == 0 failed to parse '0cb4ddff-a545-4401-9d06-688af53e7f84' # uuid_correct_be: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e7f84 not ok 1 - uuid_correct_be [FAILED] uuid_correct_le # uuid_correct_le: ASSERTION FAILED at lib/test_uuid.c:46 Expected guid_parse(data->uuid, &le) == 1, but guid_parse(data->uuid, &le) == 0 failed to parse 'c33f4995-3701-450e-9fbf-206a2e98e576' # uuid_correct_le: not ok 1 - c33f4995-3701-450e-9fbf-206a2e98e576 # uuid_correct_le: ASSERTION FAILED at lib/test_uuid.c:46 Expected guid_parse(data->uuid, &le) == 1, but guid_parse(data->uuid, &le) == 0 failed to parse '64b4371c-77c1-48f9-8221-29f054fc023b' # uuid_correct_le: not ok 2 - 64b4371c-77c1-48f9-8221-29f054fc023b # uuid_correct_le: ASSERTION FAILED at lib/test_uuid.c:46 Expected guid_parse(data->uuid, &le) == 1, but guid_parse(data->uuid, &le) == 0 failed to parse '0cb4ddff-a545-4401-9d06-688af53e7f84' # uuid_correct_le: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e7f84 not ok 2 - uuid_correct_le [FAILED] uuid_wrong_be # uuid_wrong_be: ASSERTION FAILED at lib/test_uuid.c:77 Expected uuid_parse(*data, &be) == 0, but uuid_parse(*data, &be) == -22 parsing of 'c33f4995-3701-450e-9fbf206a2e98e576 ' should've failed # uuid_wrong_be: not ok 1 - c33f4995-3701-450e-9fbf206a2e98e576 # uuid_wrong_be: ASSERTION FAILED at lib/test_uuid.c:77 Expected uuid_parse(*data, &be) == 0, but uuid_parse(*data, &be) == -22 parsing of '64b4371c-77c1-48f9-8221-29f054XX023b' should've failed # uuid_wrong_be: not ok 2 - 64b4371c-77c1-48f9-8221-29f054XX023b # uuid_wrong_be: ASSERTION FAILED at lib/test_uuid.c:77 Expected uuid_parse(*data, &be) == 0, but uuid_parse(*data, &be) == -22 parsing of '0cb4ddff-a545-4401-9d06-688af53e' should've failed # uuid_wrong_be: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e not ok 3 - uuid_wrong_be [FAILED] uuid_wrong_le # uuid_wrong_le: ASSERTION FAILED at lib/test_uuid.c:68 Expected guid_parse(*data, &le) == 0, but guid_parse(*data, &le) == -22 parsing of 'c33f4995-3701-450e-9fbf206a2e98e576 ' should've failed # uuid_wrong_le: not ok 1 - c33f4995-3701-450e-9fbf206a2e98e576 # uuid_wrong_le: ASSERTION FAILED at lib/test_uuid.c:68 Expected guid_parse(*data, &le) == 0, but guid_parse(*data, &le) == -22 parsing of '64b4371c-77c1-48f9-8221-29f054XX023b' should've failed # uuid_wrong_le: not ok 2 - 64b4371c-77c1-48f9-8221-29f054XX023b # uuid_wrong_le: ASSERTION FAILED at lib/test_uuid.c:68 Expected guid_parse(*data, &le) == 0, but guid_parse(*data, &le) == -22 parsing of '0cb4ddff-a545-4401-9d06-688af53e' should've failed # uuid_wrong_le: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e not ok 4 - uuid_wrong_le Changes from v2: - Clarify in commit message the new test cases setup v2: https://lore.kernel.org/lkml/20210609233730.164082-1-andrealmeid@collabora.com/ Changes from v1: - Test suite name: uuid_test -> uuid - Config name: TEST_UUID -> UUID_KUNIT_TEST - Config entry in the Kconfig file left where it is - Converted tests to use _MSG variant v1: https://lore.kernel.org/lkml/20210605215215.171165-1-andrealmeid@collabora.com/ André Almeida (1): lib: Convert UUID runtime test to KUnit lib/Kconfig.debug | 11 +++- lib/Makefile | 2 +- lib/test_uuid.c | 137 +++++++++++++++++++--------------------------- 3 files changed, 67 insertions(+), 83 deletions(-)