From patchwork Wed Mar 15 00:52:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86465C7618A for ; Wed, 15 Mar 2023 00:52:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230194AbjCOAws (ORCPT ); Tue, 14 Mar 2023 20:52:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230176AbjCOAwp (ORCPT ); Tue, 14 Mar 2023 20:52:45 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 014365C9D8; Tue, 14 Mar 2023 17:52:37 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8AD7161A98; Wed, 15 Mar 2023 00:52:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8771C4339B; Wed, 15 Mar 2023 00:52:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841557; bh=kDPMutoppQB9uvDQRbQbLkzA8OMhjuIiQEV5sbIGGCo=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=m4I+mujFqvk9GAIZfitZBblXKvGxDf3iupZizaUOhYQE92i1dMvYvvHUPDzIgzd51 pxu+Ix7x5wAZAwh7CtYKT5uvDpSSBfF1XcGMLCAWC+6lq58SJUZIRfxx25mdCwfVc2 +6tFM/1u3pkTYuiHqXWU1J6g62fCFrqFkSvFZGwTCH6oS7t3J9QtCzRqF8MXolnXQ4 50+AyX08wyymJOKDrlK4Kp2q/VfWRYz9+riMoiiZTGzkNcJa4OXfJ5zq3pbULwMoKv 3kNDCXBpxfeY126bNQoXu8BhzxUR5uGsJlNyYDzPJbesuSTWvzN7O57qm6x6UkoJ+8 vDVuvPzsioR3Q== Subject: [PATCH 01/15] check: generate section reports between tests From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: Leah Rumancik , Leah Rumancik , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:52:36 -0700 Message-ID: <167884155648.2482843.4182317488319546153.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Generate the section report between tests so that the summary report always reflects the outcome of the most recent test. Two usecases are envisioned here -- if a cluster-based test runner anticipates that the testrun could crash the VM, they can set REPORT_DIR to (say) an NFS mount to preserve the intermediate results. If the VM does indeed crash, the scheduler can examine the state of the crashed VM and move the tests to another VM. The second usecase is a reporting agent that runs in the VM to upload live results to a test dashboard. Signed-off-by: Darrick J. Wong Reviewed-by: Leah Rumancik Tested-by: Leah Rumancik --- check | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/check b/check index 0bf5b22e06..14b398fd73 100755 --- a/check +++ b/check @@ -844,6 +844,15 @@ function run_section() fi seqres="$REPORT_DIR/$seqnum" + # Generate the entire section report with whatever test results + # we have so far. Leave the $sect_time parameter empty so that + # it's a little more obvious that this test run is incomplete. + if $do_report; then + _make_section_report "$section" "${#try[*]}" \ + "${#bad[*]}" "${#notrun[*]}" \ + "" &> /dev/null + fi + mkdir -p $RESULT_DIR rm -f ${RESULT_DIR}/require_scratch* rm -f ${RESULT_DIR}/require_test* From patchwork Wed Mar 15 00:52:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D44DC7618A for ; Wed, 15 Mar 2023 00:53:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229525AbjCOAxA (ORCPT ); Tue, 14 Mar 2023 20:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229511AbjCOAw7 (ORCPT ); Tue, 14 Mar 2023 20:52:59 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72129580DF; Tue, 14 Mar 2023 17:52:45 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D686CB81C3C; Wed, 15 Mar 2023 00:52:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 956BFC4339C; Wed, 15 Mar 2023 00:52:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841562; bh=6xuL0iZpbpNe1aGEaEvrrl+l0+uunevUefzRUNb0YhY=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ZdeUEw1hl6EDFBBoCE1EFWA0cuNBM3YTkkJdPy1FM9A3WCZn5MSSz7YBOB8bYfz+b TNXDrrPbY+tXdjzpxbgs0pH4ZElUfhqquJLmxSgH54DiQVjN7nD3vt25xjsXrNWfij SX6ew+HIF6hkFg0j05TZAKOgVrm3RAImxfCw/jySmNew5kYX14unq4cK4goWWltgoH ZSYVskoN3vslLy8g5DErYzwtwGeBMeQkRhgbdhUfVl3hcajJUSiy0AbP37TJqLe5ik yVrdXQ4F9ozB2P2Cwtgo8zv/1Qt65838v+w6POCfyE0/7PyaaMWcCD5ew3osfbUAmz DOCQ8hoUhj3rw== Subject: [PATCH 02/15] report: derive an xml schema for the xunit report From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:52:42 -0700 Message-ID: <167884156213.2482843.1661122394114481049.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong The "xunit" report format emits an XML document that more or less follows the junit xml schema. However, there are two major exceptions: 1. fstests does not emit an @errors attribute on the testsuite element because we don't have the concept of unanticipated errors such as "unchecked throwables". 2. The system-out/system-err elements sound like they belong under the testcase element, though the schema itself imprecisely says "while the test was executed". The schema puts them under the top-level testsuite element, but we put them under the testcase element. Define an xml schema for the xunit report format, and update the xml headers to link to the schema file. This enables consumers of the reports to check mechanically that the incoming document follows the format. Signed-off-by: Darrick J. Wong --- common/report | 15 +++- doc/xunit.xsd | 226 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 238 insertions(+), 3 deletions(-) create mode 100644 doc/xunit.xsd diff --git a/common/report b/common/report index 64f9c86648..1d84650270 100644 --- a/common/report +++ b/common/report @@ -48,9 +48,18 @@ _xunit_make_section_report() if [ -z "$date_time" ]; then date_time=$(date +"%F %T") fi - local stats="failures=\"$bad_count\" skipped=\"$notrun_count\" tests=\"$tests_count\" time=\"$sect_time\"" - local hw_info="hostname=\"$HOST\" timestamp=\"${date_time/ /T}\" " - echo "" >> $REPORT_DIR/result.xml + + local fstests_ns="https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git" + cat >> $REPORT_DIR/result.xml << ENDL + +ENDL # Properties echo -e "\t" >> $REPORT_DIR/result.xml diff --git a/doc/xunit.xsd b/doc/xunit.xsd new file mode 100644 index 0000000000..ba97ccd67d --- /dev/null +++ b/doc/xunit.xsd @@ -0,0 +1,226 @@ + + + + fstests xunit test result schema, derived from https://github.com/windyroad/JUnit-Schema + + + + + + + + + + Contains an aggregation of testsuite results + + + + + + + + + + Derived from testsuite/@name in the non-aggregated documents + + + + + Starts at '0' for the first testsuite and is incremented by 1 for each following testsuite + + + + + + + + + + + + Contains the results of executing a testsuite + + + + + Properties (e.g., environment settings) set during test execution + + + + + + + + + + + + + + + + + + + + + + + + + Indicates that the test was skipped. + + + + + + + The message specifying why the test case was skipped + + + + + + + + + Indicates that the test errored. An errored test is one that had an unanticipated problem. e.g., an unchecked throwable; or a problem with the implementation of the test. Contains as a text node relevant data for the error, e.g., a stack trace + + + + + + + The error message. e.g., if a java exception is thrown, the return value of getMessage() + + + + + The type of error that occured. e.g., if a java execption is thrown the full class name of the exception. + + + + + + + + + Indicates that the test failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals. Contains as a text node relevant data for the failure, e.g., a stack trace + + + + + + + The message specified in the assert + + + + + The type of the assert. + + + + + + + + + + + Data that was written to the .full log file while the test was executed. + + + + + + + + + + Kernel log or data that was compared to the golden output file after the test was executed. + + + + + + + + + + + + Name of the test method + + + + + Full class name for the class the test method is in. + + + + + Time taken (in seconds) to execute the test + + + + + + + + Full class name of the test for non-aggregated testsuite documents. Class name without the package for aggregated testsuites documents + + + + + + + + + + when the test was executed. Timezone may not be specified. + + + + + Host on which the tests were executed. 'localhost' should be used if the hostname cannot be determined. + + + + + + + + + + The total number of tests in the suite + + + + + The total number of tests in the suite that failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals + + + + + The total number of ignored or skipped tests in the suite. + + + + + Time taken (in seconds) to execute the tests in the suite + + + + + + + + + From patchwork Wed Mar 15 00:52:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAD44C76195 for ; Wed, 15 Mar 2023 00:53:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229866AbjCOAxC (ORCPT ); Tue, 14 Mar 2023 20:53:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229511AbjCOAxB (ORCPT ); Tue, 14 Mar 2023 20:53:01 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 543E43C78E; Tue, 14 Mar 2023 17:52:49 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DB01461A9D; Wed, 15 Mar 2023 00:52:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42544C433D2; Wed, 15 Mar 2023 00:52:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841568; bh=Qik54vovpc327jWLgD9UI6QzowU+B6J69BoZPVm2b8I=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=kqa6V/j2tluNCFBiJTxb6BtdfAsesmEGAuuNMotEh4z0qoJ7ArQgRKhLKUulmGbvG 1JmIsiM0vdCKMcMET9ygr/vRf3a7R60M4s6L4ofC/vGgjWo2wP7CGLukbNFmoNd/x0 2x01DvGdSaOrPyVFX6L1OWtSIyC1bheaHTUMO/NmeaFSfFrTlz7KQBA9wFVTbiEfJQ 3MV1gstYLKxGuYV7gfoI24s11Hb0K/e76y6cTxI6o3VAjPVsdc0x7s0ca8PGINzgRX GvH1XMFRKf+BZLO3YsRjqIm6QEQv6AvKrrh/AKaU0eIirRhlvRS5VcWYWgvj5bRvsH fHQgU0ri/pC+A== Subject: [PATCH 03/15] report: capture the time zone in the test report timestamp From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: Qu Wenruo , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:52:47 -0700 Message-ID: <167884156778.2482843.3569229630033004210.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Make sure we put the time zone of the system running the test in the timestamp that is recorded in the xunit report. `date "+%F %T"' reports the local time zone, not UTC. Signed-off-by: Darrick J. Wong Reviewed-by: Qu Wenruo --- common/report | 9 ++++++--- doc/xunit.xsd | 4 ++-- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/common/report b/common/report index 1d84650270..1817132d51 100644 --- a/common/report +++ b/common/report @@ -38,6 +38,7 @@ _xunit_make_section_report() local bad_count="$3" local notrun_count="$4" local sect_time="$5" + local timestamp if [ $sect_name == '-no-sections-' ]; then sect_name='global' @@ -45,8 +46,10 @@ _xunit_make_section_report() local report=$tmp.report.xunit.$sect_name.xml # Header echo "" > $REPORT_DIR/result.xml - if [ -z "$date_time" ]; then - date_time=$(date +"%F %T") + if [ -n "$date_time" ]; then + timestamp="$(date -Iseconds --date="$date_time")" + else + timestamp="$(date -Iseconds)" fi local fstests_ns="https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git" @@ -58,7 +61,7 @@ _xunit_make_section_report() name="xfstests" failures="$bad_count" skipped="$notrun_count" tests="$tests_count" time="$sect_time" - hostname="$HOST" timestamp="${date_time/ /T}"> + hostname="$HOST" timestamp="$timestamp"> ENDL # Properties diff --git a/doc/xunit.xsd b/doc/xunit.xsd index ba97ccd67d..9295c5dc82 100644 --- a/doc/xunit.xsd +++ b/doc/xunit.xsd @@ -12,7 +12,7 @@ - + @@ -184,7 +184,7 @@ - when the test was executed. Timezone may not be specified. + when the test was executed. Timezone must be specified as an offset from UTC. From patchwork Wed Mar 15 00:52:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89885C7618A for ; Wed, 15 Mar 2023 00:53:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229629AbjCOAxK (ORCPT ); Tue, 14 Mar 2023 20:53:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229796AbjCOAxK (ORCPT ); Tue, 14 Mar 2023 20:53:10 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF5B83D924; Tue, 14 Mar 2023 17:52:56 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 34B37B81C3D; Wed, 15 Mar 2023 00:52:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D20B5C433D2; Wed, 15 Mar 2023 00:52:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841573; bh=2dLtk/dgWRSizQWYnWSkBBdxwheAzIh/VcgGY5g7LpY=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=q0E1XaaeR14SQC7rK4bhb3P3DaTV2fm8NnCnqJXij7dQ8LAQi9LKvsu8qokymAFgc nezt8MozHWUQ+QQmT1sTU8/LhwIbdGSPlwlG0Mm+hLCU1lDWT3ec8Mn3xUvyzvwjUC 42FXKSDRMfno0U1lZXwlG/N4I6qw0hWiR8UbDJ8TOGM5uvyEhTIvJpb1byngq1Bsey iItyOSueO66sZhAdiewrKPhqMXKenIiUmX7tN+MntiJTqvlGXEq0N3Z54theUBBXum FMN3KxC4wpYZlKuMR4VZH0N3F6Avdfs0tB6bmrV5OKGigDADmq4BJItvkzFkmgEiKw tRFTf88gqXlZw== Subject: [PATCH 04/15] report: clarify the meaning of the timestamp attribute From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:52:53 -0700 Message-ID: <167884157344.2482843.8376813921962028640.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong We've never specified what the timestamp attribute of the testsuite element actually means, and it history is rather murky. Prior to the introduction of the xml report format in commit f9fde7db2f, the "date_time" variable was used only to scrape dmesg via the /dev/kmsg device after each test. If /dev/kmsg was not a writable path, the variable was not set at all. In this case, the report timestamp would be blank. In commit ffdecf7498a1, Ted changed the xunit report code to handle empty date_time values by setting date_time to the time of report generation. This change was done to handle the case where no tests are run at all. However, it did not change the behavior that date_time is not set if /dev/kmsg is not writable. Clear up all this confusion by defining the timestamp attribute to reflect the start time of the most recent test, regardless of the state of /dev/kmsg. If no tests are run, then define the attribute to be the time of report generation. Signed-off-by: Darrick J. Wong --- check | 1 + common/report | 8 +++++--- doc/xunit.xsd | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/check b/check index 14b398fd73..f2be3d7d7d 100755 --- a/check +++ b/check @@ -917,6 +917,7 @@ function run_section() # to be reported for each test (echo 1 > $DEBUGFS_MNT/clear_warn_once) > /dev/null 2>&1 + test_start_time="$(date +"%F %T")" if [ "$DUMP_OUTPUT" = true ]; then _run_seq 2>&1 | tee $tmp.out # Because $? would get tee's return code diff --git a/common/report b/common/report index 1817132d51..8e19e9f557 100644 --- a/common/report +++ b/common/report @@ -46,8 +46,8 @@ _xunit_make_section_report() local report=$tmp.report.xunit.$sect_name.xml # Header echo "" > $REPORT_DIR/result.xml - if [ -n "$date_time" ]; then - timestamp="$(date -Iseconds --date="$date_time")" + if [ -n "$test_start_time" ]; then + timestamp="$(date -Iseconds --date="$test_start_time")" else timestamp="$(date -Iseconds)" fi @@ -61,7 +61,9 @@ _xunit_make_section_report() name="xfstests" failures="$bad_count" skipped="$notrun_count" tests="$tests_count" time="$sect_time" - hostname="$HOST" timestamp="$timestamp"> + hostname="$HOST" + timestamp="$timestamp" +> ENDL # Properties diff --git a/doc/xunit.xsd b/doc/xunit.xsd index 9295c5dc82..653f486871 100644 --- a/doc/xunit.xsd +++ b/doc/xunit.xsd @@ -184,7 +184,7 @@ - when the test was executed. Timezone must be specified as an offset from UTC. + Time that the last testcase was started. If no tests are started, this is the time the report was generated. Timezone must be specified as an offset from UTC. From patchwork Wed Mar 15 00:52:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6071DC6FD1F for ; Wed, 15 Mar 2023 00:53:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229482AbjCOAxT (ORCPT ); Tue, 14 Mar 2023 20:53:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229988AbjCOAxS (ORCPT ); Tue, 14 Mar 2023 20:53:18 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46F7A59430; Tue, 14 Mar 2023 17:53:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DF05CB81C3A; Wed, 15 Mar 2023 00:53:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7919CC433EF; Wed, 15 Mar 2023 00:52:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841579; bh=W2HFy7gevwuyCLmk5A6qMgfDk4GwnyEbZWcPH1HRhGk=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=clmVZzlRXM8a/Nv4c85SmS+QilPu155kgL5jaCwx0x4JvPal27kIpCjwUeVbTmvaT TsOB2/wcvup4Omw3px3cGX3H0u6Pxwq+MTANK1ZFziD3CRIw1eAV4bKJuOVIdIaM3V Hy4OfQ6IaaZGbA7EEXAbtABIFRjDOch2bFSXlhuHMlnSBSNT8oZ2A9x79RpDUCho1S JT58HOkFUKauV/44Ko5AIXZbaQWgCswmxSh+vr7zGsuATTMW2Q43ZGtmYts1Mlrfar wv4bKTy/HBWHydQ98rrDnm4WOu3xkdBDD1JvOIFcWMB4cQ/vlRrYuP2q8BnHd07Dkf NF0eppsdE9GQw== Subject: [PATCH 05/15] report: record fstests start and report generation timestamps From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:52:59 -0700 Message-ID: <167884157904.2482843.3935162326036259013.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Report two new timestamps in the xml report: the time that ./check was started, and the time that the report was generated. We introduce new timestamps to minimize breakage with parsing scripts. Signed-off-by: Darrick J. Wong --- check | 1 + common/report | 4 +++- doc/xunit.xsd | 10 ++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/check b/check index f2be3d7d7d..1a58a2b269 100755 --- a/check +++ b/check @@ -668,6 +668,7 @@ _run_seq() { _detect_kmemleak _prepare_test_list +fstests_start_time="$(date +"%F %T")" if $OPTIONS_HAVE_SECTIONS; then trap "_summary; exit \$status" 0 1 2 3 15 diff --git a/common/report b/common/report index 8e19e9f557..be991b55f5 100644 --- a/common/report +++ b/common/report @@ -62,7 +62,9 @@ _xunit_make_section_report() name="xfstests" failures="$bad_count" skipped="$notrun_count" tests="$tests_count" time="$sect_time" hostname="$HOST" - timestamp="$timestamp" + start_timestamp="$(date -Iseconds --date="$fstests_start_time")" + timestamp="$timestamp" + report_timestamp="$(date -Iseconds)" > ENDL diff --git a/doc/xunit.xsd b/doc/xunit.xsd index 653f486871..3ed72f2f86 100644 --- a/doc/xunit.xsd +++ b/doc/xunit.xsd @@ -187,6 +187,16 @@ Time that the last testcase was started. If no tests are started, this is the time the report was generated. Timezone must be specified as an offset from UTC. + + + Time that the report was generated. + + + + + Time that fstests was started. + + Host on which the tests were executed. 'localhost' should be used if the hostname cannot be determined. From patchwork Wed Mar 15 00:53:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72A4EC7618D for ; Wed, 15 Mar 2023 00:53:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230215AbjCOAxY (ORCPT ); Tue, 14 Mar 2023 20:53:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230063AbjCOAxX (ORCPT ); Tue, 14 Mar 2023 20:53:23 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FA8E41B75; Tue, 14 Mar 2023 17:53:06 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B75DF61A9D; Wed, 15 Mar 2023 00:53:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20E99C433D2; Wed, 15 Mar 2023 00:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841585; bh=5ZjDhw74kNEdDtZQQatDNNzQ8pK7OUwxcc6xRLiOFg4=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=M5v1BNKGdIXeMG1W+2+UnJeX81uKs7wPTeZ10nNA47W7rJ8PfgVG1iPz1W5vkuYvC n0nmo7r1XCMrrJ7EOc5ew0msHDb8WVEqPq6O0t+VGu6PTJfxLKCWeDvLZOsu8ZRQRM GyYK6rq2LZkevUyujdHsaVfZWsLxliiPDCgNgtaU2a5bC5ifvAO2nnG/OqT/7k4+wo u84yCYrFBR7H36bW2+I3uYMwlGe/QusWzepEK5VhkZRC1e7vrOnP/09s3n+4ITx9RZ S6htJR5oKoBZTaM7HskLhX30hzf3KYD2bKK8m9O2mhgofgFjB2vvpThnt802qi4fx8 GhOcsaVFRCTUg== Subject: [PATCH 06/15] report: encode cdata sections correctly From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:04 -0700 Message-ID: <167884158467.2482843.6284263235769240995.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong The XML report format captures the contents of .full and .out.bad files in CDATA sections. CDATA sections are supposed to be a stream of verbatim data, terminated with a "]]>". Hence XML entities such as quotation marks and angle brackes should not be escaped, and an actual bracket-bracket-gt sequence in those files /does/ need escaping. Create a separate filtering function so that these files are encoded properly. Signed-off-by: Darrick J. Wong --- common/report | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/common/report b/common/report index be991b55f5..eb169175bc 100644 --- a/common/report +++ b/common/report @@ -19,6 +19,11 @@ encode_xml() -e 's/"/\"/g' } +encode_cdata() +{ + cat -v | sed -e 's/]]>/]]]]>/g' +} + # # Xunit format report functions _xunit_add_property() @@ -128,7 +133,7 @@ _xunit_make_testcase_report() if [ -z "$quiet" -a -s "$full_file" ]; then echo -e "\t\t" >> $report printf '>$report - cat "$full_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report + cat "$full_file" | tr -dc '[:print:][:space:]' | encode_cdata >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report fi @@ -137,13 +142,13 @@ _xunit_make_testcase_report() elif [ -f "$dmesg_file" ]; then echo -e "\t\t" >> $report printf '>$report - cat "$dmesg_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report + cat "$dmesg_file" | tr -dc '[:print:][:space:]' | encode_cdata >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report elif [ -s "$outbad_file" ]; then echo -e "\t\t" >> $report printf '>$report - $diff "$out_src" "$outbad_file" | encode_xml >>$report + $diff "$out_src" "$outbad_file" | encode_cdata >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report fi From patchwork Wed Mar 15 00:53:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175169 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 564DCC6FD1F for ; Wed, 15 Mar 2023 00:53:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229988AbjCOAx3 (ORCPT ); Tue, 14 Mar 2023 20:53:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230143AbjCOAx0 (ORCPT ); Tue, 14 Mar 2023 20:53:26 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D22925966; Tue, 14 Mar 2023 17:53:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 195F6B81C39; Wed, 15 Mar 2023 00:53:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2415C433EF; Wed, 15 Mar 2023 00:53:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841590; bh=aDoHPwIX71sr2bOvQRnxSv2gH4rLK/XOdaCIF4+rU80=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=lNtX6z7vFl3U2oHhkdA/76CZuXT+u6npK/+GFfqQ6jApcZwecDVBqNaCcEIC3gKcQ p50aohBMPuusvbJYvbWvlvsuWcO1AX5pNP0Dz1EtXZGYeqTM84gYW92dCHFgfaexm7 D2wUpSkNw4KY7dZzoy5yPzpssstYIOlzDmuFnmew7A5kR71F1Qory0jAWK72wYXcxT P322Jp47iw0TT+HuL6T0KerBTaTf3iH9aahOLxm3j57qxSbgjt9HJ025Se4YXihac8 bXrmQmsPtdnjcGJmZPPzrqNhGaScTEbxoEcCbL2kJSrLlMoqOeKFIZfkDYXEefg5zj qclK9yDjDLE1g== Subject: [PATCH 07/15] report: encode the kernel log as a separate xml element From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:10 -0700 Message-ID: <167884159031.2482843.9926414972357367277.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Record the .dmesg file in a new element instead of multiplexing it with . This means that the xml report can now capture kernel log and bad golden output. Signed-off-by: Darrick J. Wong --- common/report | 11 +++++------ doc/xunit.xsd | 14 ++++++++++++-- 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/common/report b/common/report index eb169175bc..2e5959312a 100644 --- a/common/report +++ b/common/report @@ -137,15 +137,14 @@ _xunit_make_testcase_report() printf ']]>\n' >>$report echo -e "\t\t" >> $report fi - if [ -n "$quiet" ]; then - : - elif [ -f "$dmesg_file" ]; then - echo -e "\t\t" >> $report + if [ -z "$quiet" -a -f "$dmesg_file" ]; then + echo -e "\t\t" >> $report printf '>$report cat "$dmesg_file" | tr -dc '[:print:][:space:]' | encode_cdata >>$report printf ']]>\n' >>$report - echo -e "\t\t" >> $report - elif [ -s "$outbad_file" ]; then + echo -e "\t\t" >> $report + fi + if [ -z "$quiet" -a -s "$outbad_file" ]; then echo -e "\t\t" >> $report printf '>$report $diff "$out_src" "$outbad_file" | encode_cdata >>$report diff --git a/doc/xunit.xsd b/doc/xunit.xsd index 3ed72f2f86..d287eaf5a2 100644 --- a/doc/xunit.xsd +++ b/doc/xunit.xsd @@ -131,7 +131,7 @@ - + Data that was written to the .full log file while the test was executed. @@ -144,7 +144,17 @@ - Kernel log or data that was compared to the golden output file after the test was executed. + Data that was compared to the golden output file after the test was executed. + + + + + + + + + + Kernel log recorded while the test was executed. From patchwork Wed Mar 15 00:53:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175170 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 933C9C76195 for ; Wed, 15 Mar 2023 00:53:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230265AbjCOAxa (ORCPT ); Tue, 14 Mar 2023 20:53:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230063AbjCOAx2 (ORCPT ); Tue, 14 Mar 2023 20:53:28 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7217F44AC; Tue, 14 Mar 2023 17:53:17 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0BC4C61A87; Wed, 15 Mar 2023 00:53:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 658B3C433D2; Wed, 15 Mar 2023 00:53:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841596; bh=NklK8j5aIk23WQ11zKifZreUOFsN+M1nMiw6yA+yz64=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=DlWQSi7ObvIJWUG2RKZXpKmvVMHafJSvvFCh+T3i2s5sllIW4uA0Fk1LTJCxDrGHz mfsHKDby0xbR7Fgt3awQ9Zs8a2+x4lrShvkL5JwKvysFOoiY/gG0GKJnrRdUyYKe/M C356WDvf1k94NP2i+l3vg0GEp1o34dCkEWTT2N5fzpM5raswhE95iTnuvx82Dml2oY N5dUUsKN+S1YcHXWWl3mkX6i3Y9btooz/9LVTzxlly0vcnQ1GznirgTGP2+Ki3dK+L r9r32W5r9HyJNGB5OMA7sXtUxfIQYp4B+Gv/AC8ViiV0wlZvyAGQow3X9ERDizWgNx 03LdNp6N0A7BQ== Subject: [PATCH 08/15] report: sort properties by name From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:15 -0700 Message-ID: <167884159597.2482843.3047170173182476876.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong When we're generating a junit xml report, always sort the properties by name. This makes it easier for humans to find a particular property. Signed-off-by: Darrick J. Wong --- common/report | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/common/report b/common/report index 2e5959312a..3ec2d88178 100644 --- a/common/report +++ b/common/report @@ -32,7 +32,7 @@ _xunit_add_property() local value="${!name}" if [ ! -z "$value" ]; then - echo -e "\t\t" >> $REPORT_DIR/result.xml + echo -e "\t\t" fi } _xunit_make_section_report() @@ -77,7 +77,7 @@ ENDL echo -e "\t" >> $REPORT_DIR/result.xml for p in "${REPORT_ENV_LIST[@]}"; do _xunit_add_property "$p" - done + done | sort >> $REPORT_DIR/result.xml echo -e "\t" >> $REPORT_DIR/result.xml if [ -f $report ]; then cat $report >> $REPORT_DIR/result.xml From patchwork Wed Mar 15 00:53:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FE7CC76196 for ; Wed, 15 Mar 2023 00:53:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230122AbjCOAxb (ORCPT ); Tue, 14 Mar 2023 20:53:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230054AbjCOAxa (ORCPT ); Tue, 14 Mar 2023 20:53:30 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B04B3D93C; Tue, 14 Mar 2023 17:53:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A358C61A8D; Wed, 15 Mar 2023 00:53:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 08FD9C433D2; Wed, 15 Mar 2023 00:53:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841602; bh=FA4EEx8gfdDz4k6GWGgK8hBjfnTqt2wh6MFCE1JniaM=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=R1wney6T0tITevVQTazuCIWy2tRWxSK2bijlffdC2anP3lwh2aXLQvqdgAXWBghLJ ImpCJv5gak2cAYOXjr3vOmXlz3lRLdyIPla8G1HA7MbiyEfBov/ciKjPk6LwXnfJxR AZK7OL3dyK/cB4yYL/LSjIagmmycUWxk8gWmcsZ9qDTdwSNnodVDn7RayCMybHNPGG cilKYwuNZYfY52jWPPgDhcvmnkRpIrZ9/j8MT4kRuNIAkiq6V79bZoUpKQhK1/ONXD Hb+g3PVfNahtoB2WEZKS9IRjsSHfd8H/1yMIJQBac0l7rHhZEvhBZ7gqrHwgCuC/mj P6ZBzFS/OIKFQ== Subject: [PATCH 09/15] report: pass property value to _xunit_add_property From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:21 -0700 Message-ID: <167884160159.2482843.3272337401757797525.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Change this helper to require the caller to pass the value as the second parameter. This prepares us to start reporting a lot more information about a test run, not all of which are encoded as bash variables. Signed-off-by: Darrick J. Wong --- common/report | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/common/report b/common/report index 3ec2d88178..2ab83928db 100644 --- a/common/report +++ b/common/report @@ -29,12 +29,13 @@ encode_cdata() _xunit_add_property() { local name="$1" - local value="${!name}" + local value="$2" - if [ ! -z "$value" ]; then - echo -e "\t\t" - fi + test -z "$value" && return + + echo -e "\t\t" } + _xunit_make_section_report() { # xfstest:section ==> xunit:testsuite @@ -76,7 +77,7 @@ ENDL # Properties echo -e "\t" >> $REPORT_DIR/result.xml for p in "${REPORT_ENV_LIST[@]}"; do - _xunit_add_property "$p" + _xunit_add_property "$p" "${!p}" done | sort >> $REPORT_DIR/result.xml echo -e "\t" >> $REPORT_DIR/result.xml if [ -f $report ]; then From patchwork Wed Mar 15 00:53:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 266C3C7619A for ; Wed, 15 Mar 2023 00:53:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230039AbjCOAxc (ORCPT ); Tue, 14 Mar 2023 20:53:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbjCOAxc (ORCPT ); Tue, 14 Mar 2023 20:53:32 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CDFB44AC; Tue, 14 Mar 2023 17:53:30 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D5EC7B81C37; Wed, 15 Mar 2023 00:53:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 978E5C433D2; Wed, 15 Mar 2023 00:53:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841607; bh=RiD9GhkALst5gh+pM8o1R7jQftIUnOhJS1DqyE/m0/Q=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=HH2C9F/rSyCi926c3OgVeVJN2lrxHzUFK4kjUbm2wBbqLmXU0MQIK97xDdHJrAJap 3gwNxgb8nmHeBa8wKMgG+pkS94i2c+038znHzFilu1TtvHWkSkEdbzihHxgMG7gIZ3 31uBBIwA59vvOOFpOBJN4MxplGcmgjjZN5UQBANHQElKGJUpspFmK1ZCWEtZDQn6+1 6Ffp9wNBnYwYJRiCZdzXKkjxqu20ijYrMcnTLilV4cQkxClOp/kDAFmjoPHJkpAB0R t+4rEVNrKae8PovcU82FUAs023eTVD06NegXAi3pjlk0SLseJwqLjnmKm+q1qiXwnT 0ot+2k1vip8Mg== Subject: [PATCH 10/15] report: encode xml entities in property values From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:27 -0700 Message-ID: <167884160721.2482843.18113696253114083867.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Avoid trouble with the properties reported in the xml reports by translating xml-tricky characters in the property values into their xml entity equivalents. IOWs, if someone sets a property "NAME" to the value 'BOBBY"; DROP TABLES;', the xml will be formatted: Thus avoiding XML problems. Signed-off-by: Darrick J. Wong --- common/report | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/common/report b/common/report index 2ab83928db..946ee4887c 100644 --- a/common/report +++ b/common/report @@ -33,7 +33,10 @@ _xunit_add_property() test -z "$value" && return - echo -e "\t\t" + local xname="$(echo "$name" | encode_xml)" + local xvalue="$(echo "$value" | encode_xml)" + + echo -e "\t\t" } _xunit_make_section_report() From patchwork Wed Mar 15 00:53:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEAF2C6FD1F for ; Wed, 15 Mar 2023 00:53:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229528AbjCOAxh (ORCPT ); Tue, 14 Mar 2023 20:53:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbjCOAxh (ORCPT ); Tue, 14 Mar 2023 20:53:37 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C09FC2007D; Tue, 14 Mar 2023 17:53:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 72911B81C37; Wed, 15 Mar 2023 00:53:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 33C05C433EF; Wed, 15 Mar 2023 00:53:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841613; bh=i7xX0hc8MsLy1WOFx+zcMT7yce6kJuM32RN5aTkt17s=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=f4N8Gfz5U6wFjJAh496mY1l9wm40+LbzN9zVLZr2+uO1o1Tu+vyf71aQVxSS6/kBm e6WHtd1LuDdDtRFzPd/4cNrpsm42RTcHOO5uzPnr6KsB2sXwGUYUILmiYpeJVQ1q27 fsiIEVGgSsHorHlmnxzrnrQeK8MjEtrxzpg9mkGp+51YSLTZrBEGrFgLRNHHH1VvlK wmD1c3zXaE649KZdPvtAO4fcsembznYDCrD+8YS2L8M1YFKu617pWfH2bfCaZCbunp sFDooTLCOG9RCj/svcbWMQynIlbMCQaa32khK8XgMEjsQfFNnL0DApfjgtNCpKAHfT eLZno9UDya9EQ== Subject: [PATCH 11/15] report: collect basic information about a test run From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:32 -0700 Message-ID: <167884161278.2482843.3263104858658925696.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Record various generic information about an fstests run when generating a junit xml report. This includes the cpu architecture, the kernel revision, the CPU, memory, and numa node counts, and some information about the block devices passed in. Signed-off-by: Darrick J. Wong --- common/report | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/common/report b/common/report index 946ee4887c..90d4f980d1 100644 --- a/common/report +++ b/common/report @@ -24,6 +24,42 @@ encode_cdata() cat -v | sed -e 's/]]>/]]]]>/g' } +# Fill out REPORT_VARS with information about the block device referred to by +# the passed-in bash variable. +__generate_blockdev_report_vars() { + local bdev_var="$1" + local bdev="${!bdev_var}" + + test -z "$bdev" && return + test -b "$bdev" || return + local sysfs_bdev="$(_sysfs_dev "$bdev")" + + REPORT_VARS["${bdev_var}_SIZE_KB"]="$(( "$(blockdev --getsz "$bdev")" / 2 ))" + REPORT_VARS["${bdev_var}_ROTATIONAL"]="$(cat "$sysfs_bdev/queue/rotational" 2>/dev/null)" + REPORT_VARS["${bdev_var}_DAX"]="$(cat "$sysfs_bdev/queue/dax" 2>/dev/null)" + REPORT_VARS["${bdev_var}_DISCARD"]="$(sed -e 's/[1-9][0-9]*/1/g' "$sysfs_bdev/queue/discard_max_bytes" 2>/dev/null)" + REPORT_VARS["${bdev_var}_WRITE_ZEROES"]="$(sed -e 's/[1-9][0-9]*/1/g' "$sysfs_bdev/queue/write_zeroes_max_bytes" 2>/dev/null)" + REPORT_VARS["${bdev_var}_PHYS_BLOCK_BYTES"]="$(cat "$sysfs_bdev/queue/physical_block_size" 2>/dev/null)" + REPORT_VARS["${bdev_var}_LBA_BYTES"]="$(cat "$sysfs_bdev/queue/logical_block_size" 2>/dev/null)" + REPORT_VARS["${bdev_var}_ZONES"]="$(cat "$sysfs_bdev/queue/nr_zones" 2>/dev/null)" +} + +# Fill out REPORT_VARS with tidbits about our test runner configuration. +# Caller is required to declare REPORT_VARS to be an associative array. +__generate_report_vars() { + REPORT_VARS["ARCH"]="$(uname -m)" + REPORT_VARS["KERNEL"]="$(uname -r)" + REPORT_VARS["CPUS"]="$(getconf _NPROCESSORS_ONLN 2>/dev/null)" + REPORT_VARS["MEM_KB"]="$(grep MemTotal: /proc/meminfo | awk '{print $2}')" + REPORT_VARS["SWAP_KB"]="$(grep SwapTotal: /proc/meminfo | awk '{print $2}')" + + test -e /sys/devices/system/node/possible && \ + REPORT_VARS["NUMA_NODES"]="$(cat /sys/devices/system/node/possible 2>/dev/null)" + + __generate_blockdev_report_vars "TEST_DEV" + __generate_blockdev_report_vars "SCRATCH_DEV" +} + # # Xunit format report functions _xunit_add_property() @@ -77,11 +113,17 @@ _xunit_make_section_report() > ENDL + declare -A REPORT_VARS + __generate_report_vars + # Properties echo -e "\t" >> $REPORT_DIR/result.xml + (for key in "${!REPORT_VARS[@]}"; do + _xunit_add_property "$key" "${REPORT_VARS["$key"]}" + done; for p in "${REPORT_ENV_LIST[@]}"; do _xunit_add_property "$p" "${!p}" - done | sort >> $REPORT_DIR/result.xml + done) | sort >> $REPORT_DIR/result.xml echo -e "\t" >> $REPORT_DIR/result.xml if [ -f $report ]; then cat $report >> $REPORT_DIR/result.xml From patchwork Wed Mar 15 00:53:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D913C6FD1F for ; Wed, 15 Mar 2023 00:53:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230063AbjCOAxl (ORCPT ); Tue, 14 Mar 2023 20:53:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbjCOAxk (ORCPT ); Tue, 14 Mar 2023 20:53:40 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7FF42007D; Tue, 14 Mar 2023 17:53:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 65B0B61AA3; Wed, 15 Mar 2023 00:53:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BEA03C433D2; Wed, 15 Mar 2023 00:53:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841618; bh=ZhIzbDwUl8nk/wvJRijfR3fmg5j+ZuK4A7Q/zgfwFJc=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=TXcaiDv9Mox6r1cRFPPeqCOpQzCZo2RxwpaZl3U6bJeDkdccGerWUajGlftwpqB2n RM8y0lNVwSXA/NarVY+ZeCsWdEqAFSTEPnKlIpzek7nnLZPI4fOe6xN8JrlGNAzs3l nYHlCMnW3RvQtt5OuHu7SCRhOQ2qi+UHq/RAbmkydSTX2XiIsOTf4g0F364T9VrXxI gVJRtN0FjaSLA1TCo4FcMyY+kaE6nNhb7DuUh57ZnohJvLMORb0IToruV9L5rTCa0W 8/B1PSX6eLc+tuAwgqaqv9MMPok85STNrmECfOjTvs6lAnWyxeKHZEY9MBWVBcyJzB 5r7x1ls/gwhQg== Subject: [PATCH 12/15] report: record optional environment variables From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:38 -0700 Message-ID: <167884161838.2482843.6219764673500069919.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong These environment variables are documented as being significant, but optional. If they're set to a non-empty string, record them in the reports. Signed-off-by: Darrick J. Wong --- common/report | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/common/report b/common/report index 90d4f980d1..af3c04db56 100644 --- a/common/report +++ b/common/report @@ -9,6 +9,11 @@ REPORT_ENV_LIST=("SECTION" "FSTYP" "PLATFORM" "MKFS_OPTIONS" "MOUNT_OPTIONS" \ "TIME_FACTOR" "LOAD_FACTOR" "TEST_DIR" "TEST_DEV" \ "SCRATCH_DEV" "SCRATCH_MNT" "OVL_UPPER" "OVL_LOWER" "OVL_WORK") +# Variables that are captured in the report /if/ they are set. +REPORT_ENV_LIST_OPT=("TAPE_DEV" "RMT_TAPE_DEV" "FSSTRES_AVOID" "FSX_AVOID" + "KCONFIG_PATH" "PERF_CONFIGNAME" "MIN_FSSIZE" + "IDMAPPED_MOUNTS") + encode_xml() { cat -v | \ @@ -58,6 +63,11 @@ __generate_report_vars() { __generate_blockdev_report_vars "TEST_DEV" __generate_blockdev_report_vars "SCRATCH_DEV" + + # Optional environmental variables + for varname in "${REPORT_ENV_LIST_OPT[@]}"; do + test -n "${!varname}" && REPORT_VARS["${varname}"]="${!varname}" + done } # From patchwork Wed Mar 15 00:53:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEEE5C6FD1F for ; Wed, 15 Mar 2023 00:53:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230259AbjCOAxq (ORCPT ); Tue, 14 Mar 2023 20:53:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbjCOAxq (ORCPT ); Tue, 14 Mar 2023 20:53:46 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61B1521965; Tue, 14 Mar 2023 17:53:45 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F119B61A8A; Wed, 15 Mar 2023 00:53:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59E2AC433D2; Wed, 15 Mar 2023 00:53:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841624; bh=Fp0knP7QNKI+B0t53gTOPPfHszUwTpIapZtTyIFU/yw=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=SatWer3XH4Q843ndVWlxz5oQBJxuSShsyMUDKhg15vEfk1abBAlidF7vi8PWgmAPb vzIyzJ+LCX/emFu0RZGUKHTt2ziffTYNqfSzDtLIb6yw9sjfy3Jp6cGOPh2RLtE3Da EOlo4e69DMw0nEzdwStIQGv3cHE0BW6U/wTdzJMUPd3ab8w1XyYA4tOZQwG+SyhEGO OZqAoBL+hjnbvnJ+PNOE/+I7Zyu1HhJIpSaSsSr2rDeBoHnOS4Zssb61LBRkMAUEK8 bw0WYMtZr/2BIvWowyvllIZyuymtPeJWscBmv7XXMjC0Y1bGvA347EF0T3ANfoHGmg aju++36eCogPA== Subject: [PATCH 13/15] report: record xfs-specific information about a test run From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:43 -0700 Message-ID: <167884162396.2482843.6834126356123933920.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Report various XFS-specific information about a test run. Signed-off-by: Darrick J. Wong --- common/report | 3 +++ common/xfs | 11 +++++++++++ 2 files changed, 14 insertions(+) diff --git a/common/report b/common/report index af3c04db56..86274af887 100644 --- a/common/report +++ b/common/report @@ -64,6 +64,9 @@ __generate_report_vars() { __generate_blockdev_report_vars "TEST_DEV" __generate_blockdev_report_vars "SCRATCH_DEV" + # Add per-filesystem variables to the report variable list + test "$FSTYP" = "xfs" && __generate_xfs_report_vars + # Optional environmental variables for varname in "${REPORT_ENV_LIST_OPT[@]}"; do test -n "${!varname}" && REPORT_VARS["${varname}"]="${!varname}" diff --git a/common/xfs b/common/xfs index e679af824f..e8e4832cea 100644 --- a/common/xfs +++ b/common/xfs @@ -2,6 +2,17 @@ # XFS specific common functions. # +__generate_xfs_report_vars() { + __generate_blockdev_report_vars TEST_RTDEV + __generate_blockdev_report_vars TEST_LOGDEV + __generate_blockdev_report_vars SCRATCH_RTDEV + __generate_blockdev_report_vars SCRATCH_LOGDEV + + REPORT_VARS["XFS_ALWAYS_COW"]="$(cat /sys/fs/xfs/debug/always_cow 2>/dev/null)" + REPORT_VARS["XFS_LARP"]="$(cat /sys/fs/xfs/debug/larp 2>/dev/null)" + REPORT_ENV_LIST_OPT+=("TEST_XFS_REPAIR_REBUILD" "TEST_XFS_SCRUB_REBUILD") +} + _setup_large_xfs_fs() { fs_size=$1 From patchwork Wed Mar 15 00:53:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C51C6FD1D for ; Wed, 15 Mar 2023 00:53:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229456AbjCOAxx (ORCPT ); Tue, 14 Mar 2023 20:53:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230239AbjCOAxv (ORCPT ); Tue, 14 Mar 2023 20:53:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F22D21965; Tue, 14 Mar 2023 17:53:51 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A3BBC61A8D; Wed, 15 Mar 2023 00:53:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0807EC433EF; Wed, 15 Mar 2023 00:53:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841630; bh=PFtlph1E4q/CEQjbKuStRk3Cq9HLNkxvt+LC4KuoBDE=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=K48q+ewv/vub02KBQUctnCs+91xP5AuKG//Ppyp7HujkKpet5i1NBCef+zmRqBYMR c/MAxWZAFBX4B2by9EhbDwfBxvspDvWZ4h7sslm3/LY8eFITxKv7V7WSRtVqzCkn7l k+OkJp3eY7q7Cl8QiJczdLDVRPrqCQmbAyJKOQa4oMH0rOZ+Y1ZzzKsRFf5ayG04DD FulE6V0lZRWAFRRYm6i7E/B385LFSZU593ibp2EmcIflVpP7vX/ZA7ie9Hsm+kn7jN +vxEyn0vu3SCUpNVOEsddcb39SJxpMYEPpVRvwzXZVsJdO7zY2l8m9KBrg4VpfSmDp 0S5UqGbFhZQUA== Subject: [PATCH 14/15] report: record ext*-specific information about a test run From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:49 -0700 Message-ID: <167884162954.2482843.191249534178669922.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Report various ext* specific information about a test run. Signed-off-by: Darrick J. Wong --- common/ext4 | 5 +++++ common/report | 1 + 2 files changed, 6 insertions(+) diff --git a/common/ext4 b/common/ext4 index 8fd6dbc682..3dcbfe17c9 100644 --- a/common/ext4 +++ b/common/ext4 @@ -2,6 +2,11 @@ # ext4 specific common functions # +__generate_ext4_report_vars() { + __generate_blockdev_report_vars TEST_LOGDEV + __generate_blockdev_report_vars SCRATCH_LOGDEV +} + _setup_large_ext4_fs() { local fs_size=$1 diff --git a/common/report b/common/report index 86274af887..db15aec54f 100644 --- a/common/report +++ b/common/report @@ -66,6 +66,7 @@ __generate_report_vars() { # Add per-filesystem variables to the report variable list test "$FSTYP" = "xfs" && __generate_xfs_report_vars + [[ "$FSTYP" == ext[0-9]* ]] && __generate_ext4_report_vars # Optional environmental variables for varname in "${REPORT_ENV_LIST_OPT[@]}"; do From patchwork Wed Mar 15 00:53:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13175177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09330C6FD1D for ; Wed, 15 Mar 2023 00:53:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230239AbjCOAx6 (ORCPT ); Tue, 14 Mar 2023 20:53:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229532AbjCOAx5 (ORCPT ); Tue, 14 Mar 2023 20:53:57 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A390319C72; Tue, 14 Mar 2023 17:53:56 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 40D0361A8D; Wed, 15 Mar 2023 00:53:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9D3D7C433EF; Wed, 15 Mar 2023 00:53:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678841635; bh=djGUBsNNLaiAk71y23+h1TexBx0pNbwBkGur7nq4Y1Y=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=BpuXKUz2gGVRnQErRIH2yXlCNoqJh+9O19xmHMvHJFEJNkqZ+W01GA9rCTjz8v2bY XejTSSbg1w69a2rLDf9D6SRTH7socEQ9I5lBxLZR5hYW4xseJ117jUA0wT53BZn80g +bNjal7PjljvnzkYYh6+3JDmXnj8CkqCNrF3mbKSK1i48uBNEGl1OqoV94w/gzT8fI ZjtwEJaqpnUeLZ1NBFYTeebdiYXhgheId5fp6SPR3iyJ0Tl/ddITa7tY3BkPUl52o1 XwJ/6El6utmvKp+RFLAfj28Ro8EqTByzCOgcrt3IV3W7b9iMozAUrNcnymutzLhhaC zu52NhxdVdXKg== Subject: [PATCH 15/15] report: allow test runners to inject arbitrary values From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu Date: Tue, 14 Mar 2023 17:53:55 -0700 Message-ID: <167884163520.2482843.13683454023771021049.stgit@magnolia> In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia> References: <167884155064.2482843.4310780034948240980.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Per Ted's request, add to the test section reporting code the ability for test runners to point to a file containing colon-separated key value pairs. These key value pairs will be recorded in the report file as extra properties. Requested-by: Theodore Ts'o Signed-off-by: Darrick J. Wong --- README | 3 +++ common/report | 10 ++++++++++ 2 files changed, 13 insertions(+) diff --git a/README b/README index 1ca506492b..4ee877a962 100644 --- a/README +++ b/README @@ -268,6 +268,9 @@ Misc: this option is supported for all filesystems currently only -overlay is expected to run without issues. For other filesystems additional patches and fixes to the test suite might be needed. + - Set REPORT_VARS_FILE to a file containing colon-separated name-value pairs + that will be recorded in the test section report. Names must be unique. + Whitespace surrounding the colon will be removed. ______________________ USING THE FSQA SUITE diff --git a/common/report b/common/report index db15aec54f..23ddbb096d 100644 --- a/common/report +++ b/common/report @@ -49,9 +49,19 @@ __generate_blockdev_report_vars() { REPORT_VARS["${bdev_var}_ZONES"]="$(cat "$sysfs_bdev/queue/nr_zones" 2>/dev/null)" } +__import_report_vars() { + local fname="$1" + + while IFS=':' read key value; do + REPORT_VARS["${key%% }"]="${value## }" + done < "$1" +} + # Fill out REPORT_VARS with tidbits about our test runner configuration. # Caller is required to declare REPORT_VARS to be an associative array. __generate_report_vars() { + test "$REPORT_VARS_FILE" && __import_report_vars "$REPORT_VARS_FILE" + REPORT_VARS["ARCH"]="$(uname -m)" REPORT_VARS["KERNEL"]="$(uname -r)" REPORT_VARS["CPUS"]="$(getconf _NPROCESSORS_ONLN 2>/dev/null)"