From patchwork Tue Dec 20 00:01:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077304 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99F8CC4332F for ; Tue, 20 Dec 2022 00:01:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232531AbiLTABX (ORCPT ); Mon, 19 Dec 2022 19:01:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232796AbiLTABN (ORCPT ); Mon, 19 Dec 2022 19:01:13 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B960F1115B; Mon, 19 Dec 2022 16:01:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 27E29B80F2B; Tue, 20 Dec 2022 00:01:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E83C8C433D2; Tue, 20 Dec 2022 00:01:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494469; bh=s1JQfyn4mxTewWB2Hp9Jp98ubgWVp0W9RQ5OF2paesM=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=cWt3F6ZJC9Vfp4h2G5JvMiXFLFoh3pZ2dL5ZEwQ0FgFvFNCO1Q7iaQRdI53A0zsyh iYrUPE5CbCUdaT+HjEjjAqnkRpAKfAKl7nYgZEVSNcojxD2wTyUVNrXfqkqnfSPqWU xVt3h12WQf4RyTreHKYpLcGlmr/Io4TR1Bi8ArZBb9Fa55pZHIaFtjmGH0w8XJ5QQi 8T/HHjvHXlGfxVHuzzQQVtcdOzI9DLDkBVQJKc11hBXscuDJzPpa0sr2iQpnqyktsW J18wPe0TGPi7d3UeB0PD/9fT/cAS3kST/b5rqV9Tb0mJypjZ7Ru7oAkmkwaFwlf2UF TlyBDlb9/DpKg== Subject: [PATCH 1/8] check: generate section reports between tests From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:09 -0800 Message-ID: <167149446946.332657.17186597494532662986.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Generate the section report between tests so that the summary report always reflects the outcome of the most recent test. Two usecases are envisioned here -- if a cluster-based test runner anticipates that the testrun could crash the VM, they can set REPORT_DIR to (say) an NFS mount to preserve the intermediate results. If the VM does indeed crash, the scheduler can examine the state of the crashed VM and move the tests to another VM. The second usecase is a reporting agent that runs in the VM to upload live results to a test dashboard. Signed-off-by: Darrick J. Wong --- check | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/check b/check index 1ff0f44af1..70a0b537b1 100755 --- a/check +++ b/check @@ -842,6 +842,16 @@ function run_section() fi seqres="$REPORT_DIR/$seqnum" + # Generate the entire section report with whatever test results + # we have so far. Leave the $sect_time parameter empty so that + # it's a little more obvious that this test run is incomplete. + if $do_report; then + local sect_now=`_wallclock` + _make_section_report "$section" "${#try[*]}" \ + "${#bad[*]}" "${#notrun[*]}" \ + "" &> /dev/null + fi + mkdir -p $RESULT_DIR rm -f ${RESULT_DIR}/require_scratch* rm -f ${RESULT_DIR}/require_test* From patchwork Tue Dec 20 00:01:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077306 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D960C001B2 for ; Tue, 20 Dec 2022 00:01:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229779AbiLTABY (ORCPT ); Mon, 19 Dec 2022 19:01:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229934AbiLTABT (ORCPT ); Mon, 19 Dec 2022 19:01:19 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13825636C; Mon, 19 Dec 2022 16:01:18 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id BFA70B80F2B; Tue, 20 Dec 2022 00:01:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81DC9C433D2; Tue, 20 Dec 2022 00:01:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494475; bh=2x5b/wBpOZqBmRd/+nwuQbj3MESZ+1aRRSeUSt1bDg0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=f4fKhwuFo/pmegkLI1vrJnozqPIhe6ycX5DNnQGuUirTQmmZsIzxtyc5V6+CtOYcb cJeOHrhjkTSt9AhNry8KA/6+QecYIoQv2DQfe98fBgd49ut2J6Ny+OXJ9Q/VPeO0+m 1S/SQzGsk8FHlxWI4QJfv7vwQrzmg4BHtPRKhtall6eTroAyEGBFHQZrTcAzg1jL5Z nDhPzW0fwxWfCn4isvilfrcGwnH90v6oIy3h31g5nyQlr9cK4lmDF9AxqTKDI3hSps ivyaKfkOQfYXQlLuZCVFjjODGV9kjIb2yMYeNkV5J7nS2fCjh1YHBmxR7cnSn3LQ9b vVgE9q7+tqKNQ== Subject: [PATCH 2/8] report: derive an xml schema for the xunit report From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:15 -0800 Message-ID: <167149447509.332657.12495196329565215003.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong The "xunit" report format emits an XML document that more or less follows the junit xml schema. However, there are two major exceptions: 1. fstests does not emit an @errors attribute on the testsuite element because we don't have the concept of unanticipated errors such as "unchecked throwables". 2. The system-out/system-err elements sound like they belong under the testcase element, though the schema itself imprecisely says "while the test was executed". The schema puts them under the top-level testsuite element, but we put them under the testcase element. Define an xml schema for the xunit report format, and update the xml headers to link to the schema file. This enables consumers of the reports to check mechanically that the incoming document follows the format. Signed-off-by: Darrick J. Wong --- common/report | 15 +++- doc/xunit.xsd | 226 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 238 insertions(+), 3 deletions(-) create mode 100644 doc/xunit.xsd diff --git a/common/report b/common/report index 64f9c86648..1d84650270 100644 --- a/common/report +++ b/common/report @@ -48,9 +48,18 @@ _xunit_make_section_report() if [ -z "$date_time" ]; then date_time=$(date +"%F %T") fi - local stats="failures=\"$bad_count\" skipped=\"$notrun_count\" tests=\"$tests_count\" time=\"$sect_time\"" - local hw_info="hostname=\"$HOST\" timestamp=\"${date_time/ /T}\" " - echo "" >> $REPORT_DIR/result.xml + + local fstests_ns="https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git" + cat >> $REPORT_DIR/result.xml << ENDL + +ENDL # Properties echo -e "\t" >> $REPORT_DIR/result.xml diff --git a/doc/xunit.xsd b/doc/xunit.xsd new file mode 100644 index 0000000000..0aef8a9839 --- /dev/null +++ b/doc/xunit.xsd @@ -0,0 +1,226 @@ + + + + fstests xunit test result schema, derived from https://github.com/windyroad/JUnit-Schema + + + + + + + + + + Contains an aggregation of testsuite results + + + + + + + + + + Derived from testsuite/@name in the non-aggregated documents + + + + + Starts at '0' for the first testsuite and is incremented by 1 for each following testsuite + + + + + + + + + + + + Contains the results of executing a testsuite + + + + + Properties (e.g., environment settings) set during test execution + + + + + + + + + + + + + + + + + + + + + + + + + Indicates that the test was skipped. + + + + + + + The message specifying why the test case was skipped + + + + + + + + + Indicates that the test errored. An errored test is one that had an unanticipated problem. e.g., an unchecked throwable; or a problem with the implementation of the test. Contains as a text node relevant data for the error, e.g., a stack trace + + + + + + + The error message. e.g., if a java exception is thrown, the return value of getMessage() + + + + + The type of error that occured. e.g., if a java execption is thrown the full class name of the exception. + + + + + + + + + Indicates that the test failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals. Contains as a text node relevant data for the failure, e.g., a stack trace + + + + + + + The message specified in the assert + + + + + The type of the assert. + + + + + + + + + + + Data that was written to standard out while the test was executed + + + + + + + + + + Data that was written to standard error while the test was executed + + + + + + + + + + + + Name of the test method + + + + + Full class name for the class the test method is in. + + + + + Time taken (in seconds) to execute the test + + + + + + + + Full class name of the test for non-aggregated testsuite documents. Class name without the package for aggregated testsuites documents + + + + + + + + + + when the test was executed. Timezone may not be specified. + + + + + Host on which the tests were executed. 'localhost' should be used if the hostname cannot be determined. + + + + + + + + + + The total number of tests in the suite + + + + + The total number of tests in the suite that failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals + + + + + The total number of ignored or skipped tests in the suite. + + + + + Time taken (in seconds) to execute the tests in the suite + + + + + + + + + From patchwork Tue Dec 20 00:01:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA086C10F1D for ; Tue, 20 Dec 2022 00:01:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232633AbiLTAB1 (ORCPT ); Mon, 19 Dec 2022 19:01:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229703AbiLTABY (ORCPT ); Mon, 19 Dec 2022 19:01:24 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABCE42678; Mon, 19 Dec 2022 16:01:23 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 53CF2B81047; Tue, 20 Dec 2022 00:01:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 183F9C433D2; Tue, 20 Dec 2022 00:01:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494481; bh=xpryf+HUH4ukVMf6blbU+BdCHMxVjAUxWheoYhXuQFA=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=LaeTJ0c+0HnktKZCes4Wq9d0VpE1S31LvS1PSqcIjAccXy1tq9ZcMEE74yt3oTk6M 7H3mnpr5do2BGCzwgrCHd5dyc0mC5MnhlW2Up/rMVdyMbUmqcQm/QgKI3A1uZgYpTN 4vIqDSD7wxOWdFD0Qis7nPBg631sOSKBpHNsWRb4g9xEa4hPHI/GbFqBN+7fp8kYmm /BJn5g9gIbiVbhGHjFbO7oTD0Qf4p24CdZkSWpu3BAO4TXCgrAgNQsTwz+TLZGRdy8 RY2uPAQNGCgRuDhNs9niyd3jf1aeF+xcoNVIDztgk6r/idOCcmutcl8LPIUyAJ2rXd s07CvysbkNkBw== Subject: [PATCH 3/8] report: capture the time zone in the test report timestamp From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:20 -0800 Message-ID: <167149448068.332657.14583277548579655582.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Make sure we put the time zone of the system running the test in the timestamp that is recorded in the xunit report. `date "+%F %T"' reports the local time zone, not UTC. Signed-off-by: Darrick J. Wong Reviewed-by: Qu Wenruo --- common/report | 9 ++++++--- doc/xunit.xsd | 2 +- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/common/report b/common/report index 1d84650270..1817132d51 100644 --- a/common/report +++ b/common/report @@ -38,6 +38,7 @@ _xunit_make_section_report() local bad_count="$3" local notrun_count="$4" local sect_time="$5" + local timestamp if [ $sect_name == '-no-sections-' ]; then sect_name='global' @@ -45,8 +46,10 @@ _xunit_make_section_report() local report=$tmp.report.xunit.$sect_name.xml # Header echo "" > $REPORT_DIR/result.xml - if [ -z "$date_time" ]; then - date_time=$(date +"%F %T") + if [ -n "$date_time" ]; then + timestamp="$(date -Iseconds --date="$date_time")" + else + timestamp="$(date -Iseconds)" fi local fstests_ns="https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git" @@ -58,7 +61,7 @@ _xunit_make_section_report() name="xfstests" failures="$bad_count" skipped="$notrun_count" tests="$tests_count" time="$sect_time" - hostname="$HOST" timestamp="${date_time/ /T}"> + hostname="$HOST" timestamp="$timestamp"> ENDL # Properties diff --git a/doc/xunit.xsd b/doc/xunit.xsd index 0aef8a9839..67f586b816 100644 --- a/doc/xunit.xsd +++ b/doc/xunit.xsd @@ -12,7 +12,7 @@ - + From patchwork Tue Dec 20 00:01:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2A56C4332F for ; Tue, 20 Dec 2022 00:01:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232470AbiLTAB3 (ORCPT ); Mon, 19 Dec 2022 19:01:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229703AbiLTAB2 (ORCPT ); Mon, 19 Dec 2022 19:01:28 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CEEB266D; Mon, 19 Dec 2022 16:01:27 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 377466117E; Tue, 20 Dec 2022 00:01:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9DF6FC433D2; Tue, 20 Dec 2022 00:01:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494486; bh=Wku4FtOgk/DHwwvpv+yxM3NQ/iaBGbXawl52x+/5Fp4=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=C3CiaSUf8l/EmiOKs+XJI8N1QhEyGBrQazlPGI26eSXQVD9EFyynWC2TfTFUsZeTm 1+NpWnaggVF/uvYZ9OltsGHueyIl8OBF+mgzqSiR9vrEFYC5PKQlM6rSxQwIO+u5Ns oZIFOX+TT9rZgPuXLwc4tm9KXS5+KRx7ua/oepMEMAOMsgtSg25HSbScIxpe7nZzfe Dbv85bxYjWZ+RtB3dj7HxwdRJi4lJ+w+L4sP5runVGX4ZzTck3G4izACvmkJdoBKfF DMDqdoqeqPLj8O4jfRrU+gNkR196e8H5yLIHFhHSn6FnYGd0poqcrwRWZnJ2g4wMTg XdN1okRgdfupg== Subject: [PATCH 4/8] report: sort properties by name From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:26 -0800 Message-ID: <167149448624.332657.10588092617370450616.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong When we're generating a junit xml report, always sort the properties by name. This makes it easier for humans to find a particular property. Signed-off-by: Darrick J. Wong --- common/report | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/common/report b/common/report index 1817132d51..b415a2641d 100644 --- a/common/report +++ b/common/report @@ -27,7 +27,7 @@ _xunit_add_property() local value="${!name}" if [ ! -z "$value" ]; then - echo -e "\t\t" >> $REPORT_DIR/result.xml + echo -e "\t\t" fi } _xunit_make_section_report() @@ -68,7 +68,7 @@ ENDL echo -e "\t" >> $REPORT_DIR/result.xml for p in "${REPORT_ENV_LIST[@]}"; do _xunit_add_property "$p" - done + done | sort >> $REPORT_DIR/result.xml echo -e "\t" >> $REPORT_DIR/result.xml if [ -f $report ]; then cat $report >> $REPORT_DIR/result.xml From patchwork Tue Dec 20 00:01:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82DA5C4332F for ; Tue, 20 Dec 2022 00:01:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229703AbiLTABf (ORCPT ); Mon, 19 Dec 2022 19:01:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232473AbiLTABe (ORCPT ); Mon, 19 Dec 2022 19:01:34 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C326B266D; Mon, 19 Dec 2022 16:01:33 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C762A611B3; Tue, 20 Dec 2022 00:01:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35BD1C433F0; Tue, 20 Dec 2022 00:01:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494492; bh=osCYtWlvvFAP0wEvCU8bqtrlJt6VUmNOanZLEEPHx7M=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=jR5XzhJ/4NfEBj9I7x6ZxSZyTEXXtNDMsa8oBcA+gW/CPHUARuKz7WMcO/jAIvU5o 60S6ifX9sDOtCZceHDmi4vVONZQuumOZ4XPGbKNkW9o5HQ8Fkpr1PR88+iG51I1yw8 SL3hKGLxDJM5CGvs4CKcjIB83WRnFUMaq/h2WXy2GNlRDcpgn2OZew1PLSZZtCH9b4 gHpvnrQirXYwOww1r9fhKmqV24ts8Zo4S2hWRfTlztW6A+jZTrknidqG+YUeHS1kYz AU5KQN6s4zYmWgKRm1Sebm9h3qPW+zjYvNIDeNTZ4F5RB+DnCPf+XytBdH0Ym/+UzO rwxQ9/XgM6rgA== Subject: [PATCH 5/8] report: pass property value to _xunit_add_property From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:31 -0800 Message-ID: <167149449179.332657.7712352434986216407.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Change this helper to require the caller to pass the value as the second parameter. This prepares us to start reporting a lot more information about a test run, not all of which are encoded as bash variables. Signed-off-by: Darrick J. Wong --- common/report | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/common/report b/common/report index b415a2641d..642e0426a6 100644 --- a/common/report +++ b/common/report @@ -24,9 +24,9 @@ encode_xml() _xunit_add_property() { local name="$1" - local value="${!name}" + local value="$2" - if [ ! -z "$value" ]; then + if [ -n "$value" ]; then echo -e "\t\t" fi } @@ -67,7 +67,7 @@ ENDL # Properties echo -e "\t" >> $REPORT_DIR/result.xml for p in "${REPORT_ENV_LIST[@]}"; do - _xunit_add_property "$p" + _xunit_add_property "$p" "${!p}" done | sort >> $REPORT_DIR/result.xml echo -e "\t" >> $REPORT_DIR/result.xml if [ -f $report ]; then From patchwork Tue Dec 20 00:01:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 096C7C4332F for ; Tue, 20 Dec 2022 00:01:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232671AbiLTABl (ORCPT ); Mon, 19 Dec 2022 19:01:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232476AbiLTABj (ORCPT ); Mon, 19 Dec 2022 19:01:39 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD89A2613; Mon, 19 Dec 2022 16:01:38 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5A5666117E; Tue, 20 Dec 2022 00:01:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B77F2C433D2; Tue, 20 Dec 2022 00:01:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494497; bh=3WVr78pi6CwfpZxITi+fPNPnolf+nJlIy6DgEKcOyO0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=sVYKkKLcV8IG0Si2lexhLkPIk9svdC4LfLbH+g3b2pc20UziK436Ue4tniqcFfzx6 zC+zdOqmc8dGUZH6kF9JzoMH+0/pyjFP433tVFj8xDw4tKO4NcY71eE126HrRTXAv7 KjPKL0Nzj1iPvzaW6L53ewxzsTlFAk/Q5eh0xJuYO+wMRwvgPctQds9ljr/W6zTklQ ve8h9HUV7AMbEIkCMhwC/5NaHQXuMemuPTsEXDsOmC2sJo9WnHk0WhVK7m5vfnX7+U 2RU5Zq5Ox40feb9olSasNl1dOrSU3vaGDeeU5LsNGe6JcFgOJh16HDuxCZZY5bNk8N TX1avkz2Dxaow== Subject: [PATCH 6/8] report: collect basic information about a test run From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:37 -0800 Message-ID: <167149449737.332657.1308561091226926848.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Record various generic information about an fstests run when generating a junit xml report. This includes the cpu architecture, the kernel revision, the CPU, memory, and numa node counts, and some information about the block devices passed in. Signed-off-by: Darrick J. Wong --- common/report | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/common/report b/common/report index 642e0426a6..68646a7709 100644 --- a/common/report +++ b/common/report @@ -19,6 +19,42 @@ encode_xml() -e 's/"/\"/g' } +# Fill out REPORT_VARS with information about the block device referred to by +# the passed-in bash variable. +__generate_blockdev_report_vars() { + local bdev_var="$1" + local bdev="${!bdev_var}" + + test -z "$bdev" && return + test -b "$bdev" || return + local sysfs_bdev="$(_sysfs_dev "$bdev")" + + REPORT_VARS["${bdev_var}_SIZE_KB"]="$(( "$(blockdev --getsz "$bdev")" / 2 ))" + REPORT_VARS["${bdev_var}_ROTATIONAL"]="$(cat "$sysfs_bdev/queue/rotational" 2>/dev/null)" + REPORT_VARS["${bdev_var}_DAX"]="$(cat "$sysfs_bdev/queue/dax" 2>/dev/null)" + REPORT_VARS["${bdev_var}_DISCARD"]="$(sed -e 's/[1-9][0-9]*/1/g' "$sysfs_bdev/queue/discard_max_bytes" 2>/dev/null)" + REPORT_VARS["${bdev_var}_WRITE_ZEROES"]="$(sed -e 's/[1-9][0-9]*/1/g' "$sysfs_bdev/queue/write_zeroes_max_bytes" 2>/dev/null)" + REPORT_VARS["${bdev_var}_PHYS_BLOCK_BYTES"]="$(cat "$sysfs_bdev/queue/physical_block_size" 2>/dev/null)" + REPORT_VARS["${bdev_var}_LBA_BYTES"]="$(cat "$sysfs_bdev/queue/logical_block_size" 2>/dev/null)" + REPORT_VARS["${bdev_var}_ZONES"]="$(cat "$sysfs_bdev/queue/nr_zones" 2>/dev/null)" +} + +# Fill out REPORT_VARS with tidbits about our test runner configuration. +# Caller is required to declare REPORT_VARS to be an associative array. +__generate_report_vars() { + REPORT_VARS["ARCH"]="$(uname -m)" + REPORT_VARS["KERNEL"]="$(uname -r)" + REPORT_VARS["CPUS"]="$(getconf _NPROCESSORS_ONLN 2>/dev/null)" + REPORT_VARS["MEM_KB"]="$(grep MemTotal: /proc/meminfo | awk '{print $2}')" + REPORT_VARS["SWAP_KB"]="$(grep SwapTotal: /proc/meminfo | awk '{print $2}')" + + test -e /sys/devices/system/node/possible && \ + REPORT_VARS["NUMA_NODES"]="$(cat /sys/devices/system/node/possible 2>/dev/null)" + + __generate_blockdev_report_vars "TEST_DEV" + __generate_blockdev_report_vars "SCRATCH_DEV" +} + # # Xunit format report functions _xunit_add_property() @@ -64,11 +100,17 @@ _xunit_make_section_report() hostname="$HOST" timestamp="$timestamp"> ENDL + declare -A REPORT_VARS + __generate_report_vars + # Properties echo -e "\t" >> $REPORT_DIR/result.xml + (for key in "${!REPORT_VARS[@]}"; do + _xunit_add_property "$key" "${REPORT_VARS["$key"]}" + done; for p in "${REPORT_ENV_LIST[@]}"; do _xunit_add_property "$p" "${!p}" - done | sort >> $REPORT_DIR/result.xml + done) | sort >> $REPORT_DIR/result.xml echo -e "\t" >> $REPORT_DIR/result.xml if [ -f $report ]; then cat $report >> $REPORT_DIR/result.xml From patchwork Tue Dec 20 00:01:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D239C4332F for ; Tue, 20 Dec 2022 00:01:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232476AbiLTABr (ORCPT ); Mon, 19 Dec 2022 19:01:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232444AbiLTABq (ORCPT ); Mon, 19 Dec 2022 19:01:46 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04A3D2614; Mon, 19 Dec 2022 16:01:46 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B26C9B80ED7; Tue, 20 Dec 2022 00:01:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5CDA3C433D2; Tue, 20 Dec 2022 00:01:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494503; bh=LvuwLi4xewL992WDyJbgDS8QS3Kn+O6Rns7/6pVlOx4=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=OLoeJLJXjFaWW64VWYMXS1ZRUCKbuPzt3crVE9dIxcBt14ndRqCEtCbtHEQjC0EGE NKuqgbjrTB3iCDIPvtKP6FRCVfIx3Kzm1038e3Zj4u5DauwNkUZXG720HZ4itCpPSo MX66GaV/d+r/Dbk81uPRnWUbUVZ+8iBP1gywvnKsSuahCnUPDN0PMIxwJZ3WZ9Lg0C dl2C/l1NXYLVbxGYOnH10EIK/pZlCaHz3UaBmalfECfnVX95J0ulyONQmDIlK8+Fyf j+/uy/yoRSYG4Ay3+VnCD0an4k8HrN2nD9t2jwnuwwIAuE8FdcO0HPRptJvfePdlo5 OzPbCoxcWplAA== Subject: [PATCH 7/8] report: record xfs-specific information about a test run From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:42 -0800 Message-ID: <167149450292.332657.3110931385669293441.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Report various XFS-specific information about a test run. Signed-off-by: Darrick J. Wong --- common/report | 2 ++ common/xfs | 10 ++++++++++ 2 files changed, 12 insertions(+) diff --git a/common/report b/common/report index 68646a7709..f2b60c317e 100644 --- a/common/report +++ b/common/report @@ -53,6 +53,8 @@ __generate_report_vars() { __generate_blockdev_report_vars "TEST_DEV" __generate_blockdev_report_vars "SCRATCH_DEV" + + test "$FSTYP" = "xfs" && __generate_xfs_report_vars } # diff --git a/common/xfs b/common/xfs index 7eee76c0ee..add3a42fa4 100644 --- a/common/xfs +++ b/common/xfs @@ -2,6 +2,16 @@ # XFS specific common functions. # +__generate_xfs_report_vars() { + __generate_blockdev_report_vars TEST_RTDEV + __generate_blockdev_report_vars TEST_LOGDEV + __generate_blockdev_report_vars SCRATCH_RTDEV + __generate_blockdev_report_vars SCRATCH_LOGDEV + + REPORT_VARS["XFS_ALWAYS_COW"]="$(cat /sys/fs/xfs/debug/always_cow 2>/dev/null)" + REPORT_VARS["XFS_LARP"]="$(cat /sys/fs/xfs/debug/larp 2>/dev/null)" +} + _setup_large_xfs_fs() { fs_size=$1 From patchwork Tue Dec 20 00:01:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13077311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1C41C4167B for ; Tue, 20 Dec 2022 00:01:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231356AbiLTABx (ORCPT ); Mon, 19 Dec 2022 19:01:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229454AbiLTABw (ORCPT ); Mon, 19 Dec 2022 19:01:52 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 885672613; Mon, 19 Dec 2022 16:01:51 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 42954B80ED7; Tue, 20 Dec 2022 00:01:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7A5DC433D2; Tue, 20 Dec 2022 00:01:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671494509; bh=m3HrovVCofOfCjGm0j6I8S2e13RlE/wl0/sN7owzzho=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=pDMkrPp3hWRiS86HQyujQEI24gWsHwWy7t0FVJZLyXFWVwRHhwuyMiDkLOQ267cBf FGRkC/SCXKdeVOV9wh83QmrQoPzrZHLAHjBVzi32Ot2UYmjAgzxpDVLeu52VZapzhk VPX8aBGclpgyabnFmkUQgolQxLhYgtc70zF+np8LcDvxYcrYHJY56viHM7vV8MZ0F8 lD8Yb3vnv2fkIT/HzzPW9r2kzMAXUR5xixhtxOMf4tOtRFYlgik86bpxMu5r3FoN+g NOEIIGbmUc4DmAcKZFpIuocsLQl0EFPFZ9ftH6W9O+afKXd3S9nhVdM50cDl2qGr+f 26iRZxQBnYhxw== Subject: [PATCH 8/8] report: record ext*-specific information about a test run From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me, leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com Date: Mon, 19 Dec 2022 16:01:48 -0800 Message-ID: <167149450855.332657.5873316810748864701.stgit@magnolia> In-Reply-To: <167149446381.332657.9402608531757557463.stgit@magnolia> References: <167149446381.332657.9402608531757557463.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Report various ext* specific information about a test run. Signed-off-by: Darrick J. Wong --- common/ext4 | 5 +++++ common/report | 1 + 2 files changed, 6 insertions(+) diff --git a/common/ext4 b/common/ext4 index 8fd6dbc682..3dcbfe17c9 100644 --- a/common/ext4 +++ b/common/ext4 @@ -2,6 +2,11 @@ # ext4 specific common functions # +__generate_ext4_report_vars() { + __generate_blockdev_report_vars TEST_LOGDEV + __generate_blockdev_report_vars SCRATCH_LOGDEV +} + _setup_large_ext4_fs() { local fs_size=$1 diff --git a/common/report b/common/report index f2b60c317e..199851084c 100644 --- a/common/report +++ b/common/report @@ -55,6 +55,7 @@ __generate_report_vars() { __generate_blockdev_report_vars "SCRATCH_DEV" test "$FSTYP" = "xfs" && __generate_xfs_report_vars + [[ "$FSTYP" == ext[0-9]* ]] && __generate_ext4_report_vars } #