From patchwork Fri Dec 8 19:12:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 13485784 X-Patchwork-Delegate: brendanhiggins@google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TtOpdMwp" Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4148610F8 for ; Fri, 8 Dec 2023 11:12:29 -0800 (PST) Received: by mail-qv1-xf4a.google.com with SMTP id 6a1803df08f44-67ab4382912so43928506d6.1 for ; Fri, 08 Dec 2023 11:12:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702062748; x=1702667548; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=nVvef+BcVgEroNYpsUvlaSbQE1KPpVwrXco8mT3u6R8=; b=TtOpdMwpk3HmyGxPMn0lXTJ1YACZLpsGKVjALtvXPgYgZcY36t9V9cUvKebcqvVtLv rHzmU5w01VqPobGCR9NkYT/l+IzVWoLeh2hExeW2HZGo/kXu7oZ/KaqdW8qCO+AsT+yH rlObRSjtVUEdJG6ZycKmb+i0RZQOqhj+/StTJKaltqK56ZepK/fyzwWC7IlFA8Y2vX8U /aVX/poyzT/swMPh+JUF/qC7N4EvnO0Tbui9mDqtqJyZVTgVN+bPQT/ffOAccpXFWuLx 6NE+ekEuL8Vx3+qoo9kQp+w8QpwZSIMOhtJW8+GxOuecEzCqpy0eKfoBOpEN0bGFxnGy Oo6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702062748; x=1702667548; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=nVvef+BcVgEroNYpsUvlaSbQE1KPpVwrXco8mT3u6R8=; b=gCl+9S6hfxNkNSWof0f7nWl4EYAS+ZC5EDhisfI3D8pxxBgJalgYIahf/pKAYJcS5O 7X4A/PawmberrtXRpNli5VdorOHArhVPJU+0jOu5hYAn0C/OlvSiFEMpL1hu9enL+xfI gQ99cYzVyTpLBHMspfr2L86nU0Tl8C1TkmImVnjQhX6T7sSPDNPTOC7Zi7+XErvNRB4O T4VlUPLuGQ/3/0tH1GNdzWcReARn7iafYgileNIznPfZ2bPb9Yq6k8XwaMEUsYT4jJcv UvTXxSklDojy7lDCJI8EnUnkslgTxLQt2bPctOXih04lJBa6Aq54AKyHPuw5MC3BhO3F K9Tg== X-Gm-Message-State: AOJu0Yz+6PC26nEvCMpzZ6dwcwKGZlPWqXZDSWPj2XAmuPUTCUGySO6b FW0e3Eps2efLhw85XGGNLgo8Fm5ALQ== X-Google-Smtp-Source: AGHT+IGuOdD3gIEzMskxj65xx+SL0VrhBoEpExw0ALUkNEYuZ/zPOt9y7oxmzmGADGUzBYXekOw1J3gmSA== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a0c:f7d1:0:b0:67a:dbad:8ebd with SMTP id f17-20020a0cf7d1000000b0067adbad8ebdmr11575qvo.5.1702062748417; Fri, 08 Dec 2023 11:12:28 -0800 (PST) Date: Fri, 8 Dec 2023 19:12:21 +0000 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231208191222.630119-1-rmoar@google.com> Subject: [PATCH v3 1/2] kunit: tool: fix parsing of test attributes From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, Rae Moar Add parsing of attributes as diagnostic data. Fixes issue with test plan being parsed incorrectly as diagnostic data when located after suite-level attributes. Note that if there does not exist a test plan line, the diagnostic lines between the suite header and the first result will be saved in the suite log rather than the first test case log. Signed-off-by: Rae Moar --- tools/testing/kunit/kunit_parser.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86 diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 79d8832c862a..ce34be15c929 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -450,7 +450,7 @@ def parse_diagnostic(lines: LineStream) -> List[str]: Log of diagnostic lines """ log = [] # type: List[str] - non_diagnostic_lines = [TEST_RESULT, TEST_HEADER, KTAP_START, TAP_START] + non_diagnostic_lines = [TEST_RESULT, TEST_HEADER, KTAP_START, TAP_START, TEST_PLAN] while lines and not any(re.match(lines.peek()) for re in non_diagnostic_lines): log.append(lines.pop()) @@ -726,6 +726,7 @@ def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: # test plan test.name = "main" ktap_line = parse_ktap_header(lines, test) + test.log.extend(parse_diagnostic(lines)) parse_test_plan(lines, test) parent_test = True else: @@ -737,6 +738,7 @@ def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: if parent_test: # If KTAP version line and/or subtest header is found, attempt # to parse test plan and print test header + test.log.extend(parse_diagnostic(lines)) parse_test_plan(lines, test) print_test_header(test) expected_count = test.expected_count From patchwork Fri Dec 8 19:12:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rae Moar X-Patchwork-Id: 13485785 X-Patchwork-Delegate: brendanhiggins@google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rLlkJo3m" Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D4BC10D for ; Fri, 8 Dec 2023 11:12:31 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5d942a656b7so30502847b3.1 for ; Fri, 08 Dec 2023 11:12:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702062750; x=1702667550; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=o4FGZcnm1uluLFPaSEUNrvTfpJlOw0WcvKHKK9pE/vY=; b=rLlkJo3m3jGjf4q0m3BDBDAuxYMa2f0ouPf3UlWAfJxjDvmafE8Hr3+z5hrvR4YXLB f6q/y4xmbA0+AaF0DiGQSeNlhmFV6Q48S58sg0pvOIK1AlyT/n1Hthf905KMqjR4PkJh 4XRgezhWv2PVzxPvIWB5gf+2c3mNjn1pC2ujTFwg3ey8ACvQkB5K+bYNOU+qK9a6Cdm9 LPs36/hM7Ps2Jks5mcXzN+aUGrX1lL3SxC14Y0H72LVq18HhBY9OiuCbobSxt4gjGfOt iJDHvH3KMSLoYTzWIevbjnzkJNxC5ZGBD1P4Y6VkKxfOIhzktlklk++1X/0Njiin09Yi bpFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702062750; x=1702667550; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=o4FGZcnm1uluLFPaSEUNrvTfpJlOw0WcvKHKK9pE/vY=; b=gi7H8SI86dF0XyyiPsc4AD6fR4kcz0YPO3GRR+BFeA2bEQbi/WNPnqAkspxvH2/l17 HJ37PGfRvGEdTgW47gos40nrSgClhh2oOe+iccQ7uyRIr8QGE6s+xV/4nn1WgFIlF3u+ 4pj9MxFrYy5YfD4xzehGvwpCrdimJjcW+mAxDcAlOinSEYWrB1esIZkbB8qytdkbXXSq EeqQHP/DcExqWqaYDySKKdMNG7ApPIHTvKq4eaNYRArAML2oVVOOs3wyQOirHzCbwM5h G753DB67WhHZwhKrj/FUGicopzVTqPUmH36wq51/83xHvtpdj57qqlG5hCI/VDLZFp4t kIyA== X-Gm-Message-State: AOJu0YzklbT09sVpPbTH8bOGT8s0ZRoxsnCUsphUH39fblOjVHjbHWtY Rq0cGifqrSrq9J635st4l/u3HucSBg== X-Google-Smtp-Source: AGHT+IFz9RNgqFFJNGwhiJxIZm6Fz0EpoDETu6m3K4Q6CZzE7fMeSwt4lm+teSXQXGq7RC8mPelDicLLRA== X-Received: from rmoar-specialist.c.googlers.com ([fda3:e722:ac3:cc00:2b:7d90:c0a8:45d3]) (user=rmoar job=sendgmr) by 2002:a05:690c:c1c:b0:5d8:8d39:f67e with SMTP id cl28-20020a05690c0c1c00b005d88d39f67emr5720ywb.7.1702062750743; Fri, 08 Dec 2023 11:12:30 -0800 (PST) Date: Fri, 8 Dec 2023 19:12:22 +0000 In-Reply-To: <20231208191222.630119-1-rmoar@google.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231208191222.630119-1-rmoar@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231208191222.630119-2-rmoar@google.com> Subject: [PATCH v3 2/2] kunit: tool: add test for parsing attributes From: Rae Moar To: shuah@kernel.org, davidgow@google.com, dlatypov@google.com, brendan.higgins@linux.dev Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, Rae Moar Add test for parsing attributes to kunit_tool_test.py. Test checks attributes are parsed and saved in the test logs. This test also checks that the attributes have not interfered with the parsing of other test information, specifically the suite header as the test plan was being incorrectely parsed. Signed-off-by: Rae Moar --- tools/testing/kunit/kunit_tool_test.py | 16 ++++++++++++++++ .../kunit/test_data/test_parse_attributes.log | 9 +++++++++ 2 files changed, 25 insertions(+) create mode 100644 tools/testing/kunit/test_data/test_parse_attributes.log diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index b28c1510be2e..2beb7327e53f 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -331,6 +331,22 @@ class KUnitParserTest(unittest.TestCase): kunit_parser.parse_run_tests(file.readlines()) self.print_mock.assert_any_call(StrContains('suite (1 subtest)')) + def test_parse_attributes(self): + ktap_log = test_data_path('test_parse_attributes.log') + with open(ktap_log) as file: + result = kunit_parser.parse_run_tests(file.readlines()) + + # Test should pass with no errors + self.assertEqual(result.counts, kunit_parser.TestCounts(passed=1, errors=0)) + self.assertEqual(kunit_parser.TestStatus.SUCCESS, result.status) + + # Ensure suite header is parsed correctly + self.print_mock.assert_any_call(StrContains('suite (1 subtest)')) + + # Ensure attributes in correct test log + self.assertContains('# module: example', result.subtests[0].log) + self.assertContains('# test.speed: slow', result.subtests[0].subtests[0].log) + def test_show_test_output_on_failure(self): output = """ KTAP version 1 diff --git a/tools/testing/kunit/test_data/test_parse_attributes.log b/tools/testing/kunit/test_data/test_parse_attributes.log new file mode 100644 index 000000000000..74df125d0863 --- /dev/null +++ b/tools/testing/kunit/test_data/test_parse_attributes.log @@ -0,0 +1,9 @@ +KTAP version 1 +1..1 + KTAP version 1 + # Subtest: suite + # module: example + 1..1 + # test.speed: slow + ok 1 test +ok 1 suite