From patchwork Fri Jul 8 08:51:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Disseldorp X-Patchwork-Id: 12910744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA986C433EF for ; Fri, 8 Jul 2022 08:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237692AbiGHIvv (ORCPT ); Fri, 8 Jul 2022 04:51:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237572AbiGHIvt (ORCPT ); Fri, 8 Jul 2022 04:51:49 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E67EF823B9 for ; Fri, 8 Jul 2022 01:51:48 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 998941FEF9; Fri, 8 Jul 2022 08:51:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1657270307; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=edUJPb2YHWx7/4NM+Iy+t/w1rUsBJkqir0JpYw7neVI=; b=mKGpLpqBV6YBcV+Ld4kbDqC9pXhW3LBKIzYV+RTcIxuA4ft+Uu9Tl6fAYZysiQIkO1Yheg rcZXGMKTzjOm1bH9a0GLK349hwtBgD9OKZ2895QK5vUxeHvm/poq9MwwuBPX2Pnh1njJmR QaugyXKXcaTPO3pa+uTc2/DiP1RZNXo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1657270307; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=edUJPb2YHWx7/4NM+Iy+t/w1rUsBJkqir0JpYw7neVI=; b=JzDxmB+7PX2cdJfERh8aaOUxHITVj5wFn3ba/F2zJPr0nkVi34HKKNdjizbib3XW/F25/f oNpb1J8k4VzA6oAw== Received: from echidna.suse.de (unknown [10.163.47.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 6DA6A2C141; Fri, 8 Jul 2022 08:51:47 +0000 (UTC) From: David Disseldorp To: fstests@vger.kernel.org, tytso@mit.edu Cc: djwong@kernel.org, zlang@redhat.com, David Disseldorp Subject: [PATCH v4 1/5] report: use array for REPORT_ENV_LIST Date: Fri, 8 Jul 2022 10:51:38 +0200 Message-Id: <20220708085142.20991-2-ddiss@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220708085142.20991-1-ddiss@suse.de> References: <20220708085142.20991-1-ddiss@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org There's no need for multiple assignments. Signed-off-by: David Disseldorp Reviewed-by: Darrick J. Wong --- common/report | 26 +++++--------------------- 1 file changed, 5 insertions(+), 21 deletions(-) diff --git a/common/report b/common/report index 84d9e0a7..2b8285d8 100644 --- a/common/report +++ b/common/report @@ -4,26 +4,10 @@ # List of xfstests's enviroment variables to include reports ## TODO automate list population inside common/conf -REPORT_ENV_LIST="$REPORT_ENV_LIST SECTION" -REPORT_ENV_LIST="$REPORT_ENV_LIST FSTYP" -REPORT_ENV_LIST="$REPORT_ENV_LIST PLATFORM" -REPORT_ENV_LIST="$REPORT_ENV_LIST MKFS_OPTIONS" -REPORT_ENV_LIST="$REPORT_ENV_LIST MOUNT_OPTIONS" - -REPORT_ENV_LIST="$REPORT_ENV_LIST HOST_OPTIONS" -REPORT_ENV_LIST="$REPORT_ENV_LIST CHECK_OPTIONS" -REPORT_ENV_LIST="$REPORT_ENV_LIST XFS_MKFS_OPTIONS" -REPORT_ENV_LIST="$REPORT_ENV_LIST TIME_FACTOR" -REPORT_ENV_LIST="$REPORT_ENV_LIST LOAD_FACTOR" - -REPORT_ENV_LIST="$REPORT_ENV_LIST TEST_DIR" -REPORT_ENV_LIST="$REPORT_ENV_LIST TEST_DEV" -REPORT_ENV_LIST="$REPORT_ENV_LIST SCRATCH_DEV" -REPORT_ENV_LIST="$REPORT_ENV_LIST SCRATCH_MNT" - -REPORT_ENV_LIST="$REPORT_ENV_LIST OVL_UPPER" -REPORT_ENV_LIST="$REPORT_ENV_LIST OVL_LOWER" -REPORT_ENV_LIST="$REPORT_ENV_LIST OVL_WORK" +REPORT_ENV_LIST=("SECTION" "FSTYP" "PLATFORM" "MKFS_OPTIONS" "MOUNT_OPTIONS" \ + "HOST_OPTIONS" "CHECK_OPTIONS" "XFS_MKFS_OPTIONS" \ + "TIME_FACTOR" "LOAD_FACTOR" "TEST_DIR" "TEST_DEV" \ + "SCRATCH_DEV" "SCRATCH_MNT" "OVL_UPPER" "OVL_LOWER" "OVL_WORK") encode_xml() { @@ -70,7 +54,7 @@ _xunit_make_section_report() # Properties echo -e "\t" >> $REPORT_DIR/result.xml - for p in $REPORT_ENV_LIST;do + for p in "${REPORT_ENV_LIST[@]}"; do _xunit_add_property "$p" done echo -e "\t" >> $REPORT_DIR/result.xml From patchwork Fri Jul 8 08:51:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Disseldorp X-Patchwork-Id: 12910746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0393C43334 for ; Fri, 8 Jul 2022 08:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237439AbiGHIvx (ORCPT ); Fri, 8 Jul 2022 04:51:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237650AbiGHIvu (ORCPT ); Fri, 8 Jul 2022 04:51:50 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2365E823BB for ; Fri, 8 Jul 2022 01:51:49 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id CF0261FEFE; Fri, 8 Jul 2022 08:51:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1657270307; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMqChZaE6/7jkdCdOmPysyseqK04lpb5vwnq+37Wy6I=; b=ZYcxjEjhMJHaWLrNI4M1FjD0KyyXShK+v91Mleq4t7gIw6BKCbwDFaNNR3E7Q773H8OlTh 7YBXJKwtSdwUe7OTCRCeK64XpcY1egh3hbsqyB4aHBu5mgS8QGit9syegyeQ74tCIfS8lD RNvIO+ZEE0vP88f2uoOhZLUJ+G8SAMg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1657270307; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMqChZaE6/7jkdCdOmPysyseqK04lpb5vwnq+37Wy6I=; b=2Wyrv3JyeH1/9neOAexBE0inbhx02PDVKnWi6UgD2UpIGNiX6cwjyj8di2PpJ+NDCJ8Jlf uYeI5ekWyTinz9Dg== Received: from echidna.suse.de (unknown [10.163.47.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id A29D02C142; Fri, 8 Jul 2022 08:51:47 +0000 (UTC) From: David Disseldorp To: fstests@vger.kernel.org, tytso@mit.edu Cc: djwong@kernel.org, zlang@redhat.com, David Disseldorp Subject: [PATCH v4 2/5] report: pass through most details as function parameters Date: Fri, 8 Jul 2022 10:51:39 +0200 Message-Id: <20220708085142.20991-3-ddiss@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220708085142.20991-1-ddiss@suse.de> References: <20220708085142.20991-1-ddiss@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Report generation currently involves reaching into a whole bunch of globals for things like section name and start/end times. Pass these through as explicit function parameters to avoid unintentional breakage. One minor fix included is the default xunit error message, which used $sequm instead of $seqnum. Signed-off-by: David Disseldorp Reviewed-by: Darrick J. Wong --- check | 14 +++++++---- common/report | 64 +++++++++++++++++++++++++++++---------------------- 2 files changed, 47 insertions(+), 31 deletions(-) diff --git a/check b/check index 4b0ebad6..dbad6dde 100755 --- a/check +++ b/check @@ -432,7 +432,9 @@ _wrapup() if $showme && $needwrap; then if $do_report; then # $showme = all selected tests are notrun (no tries) - _make_section_report "${#notrun[*]}" "0" "${#notrun[*]}" + _make_section_report "$section" "${#notrun[*]}" "0" \ + "${#notrun[*]}" \ + "$((sect_stop - sect_start))" fi needwrap=false elif $needwrap; then @@ -493,7 +495,9 @@ _wrapup() fi echo "" >>$tmp.summary if $do_report; then - _make_section_report "${#try[*]}" "${#bad[*]}" "${#notrun[*]}" + _make_section_report "$section" "${#try[*]}" \ + "${#bad[*]}" "${#notrun[*]}" \ + "$((sect_stop - sect_start))" fi needwrap=false fi @@ -736,7 +740,8 @@ function run_section() bad+=("$seqnum") fi if $do_report && [[ ! $tc_status =~ ^(init|expunge)$ ]]; then - _make_testcase_report "$prev_seq" "$tc_status" + _make_testcase_report "$section" "$seqnum" \ + "$tc_status" "$((stop - start))" fi prev_seq="$seq" @@ -940,7 +945,8 @@ function run_section() bad+=("$seqnum") fi if $do_report && [[ ! $tc_status =~ ^(init|expunge)$ ]]; then - _make_testcase_report "$prev_seq" "$tc_status" + _make_testcase_report "$section" "$seqnum" "$tc_status" \ + "$((stop - start))" fi sect_stop=`_wallclock` diff --git a/common/report b/common/report index 2b8285d8..5ca41bc4 100644 --- a/common/report +++ b/common/report @@ -33,11 +33,11 @@ _xunit_add_property() _xunit_make_section_report() { # xfstest:section ==> xunit:testsuite - local tests_count="$1" - local bad_count="$2" - local notrun_count="$3" - local sect_name=$section - local sect_time=`expr $sect_stop - $sect_start` + local sect_name="$1" + local tests_count="$2" + local bad_count="$3" + local notrun_count="$4" + local sect_time="$5" if [ $sect_name == '-no-sections-' ]; then sect_name='global' @@ -67,12 +67,10 @@ _xunit_make_section_report() _xunit_make_testcase_report() { - local test_seq="$1" - local test_status="$2" - local test_time=`expr $stop - $start` - local strip="$SRC_DIR/" - local test_name=${test_seq#$strip} - local sect_name=$section + local sect_name="$1" + local test_name="$2" + local test_status="$3" + local test_time="$4" # TODO: other places may also win if no-section mode will be named like 'default/global' if [ $sect_name == '-no-sections-' ]; then @@ -86,8 +84,9 @@ _xunit_make_testcase_report() "pass") ;; "notrun") - if [ -f $seqres.notrun ]; then - local msg=`cat $seqres.notrun | encode_xml` + local notrun_file="${REPORT_DIR}/${test_name}.notrun" + if [ -f "$notrun_file" ]; then + local msg=`cat "$notrun_file" | encode_xml` echo -e "\t\t" >> $report else echo -e "\t\t" >> $report @@ -97,27 +96,31 @@ _xunit_make_testcase_report() echo -e "\t\t" >> $report ;; "fail") + local out_src="${SRC_DIR}/${test_name}.out" + local full_file="${REPORT_DIR}/${test_name}.full" + local dmesg_file="${REPORT_DIR}/${test_name}.dmesg" + local outbad_file="${REPORT_DIR}/${test_name}.out.bad" if [ -z "$_err_msg" ]; then - _err_msg="Test $sequm failed, reason unknown" + _err_msg="Test $test_name failed, reason unknown" fi echo -e "\t\t" >> $report - if [ -s $seqres.full ]; then + if [ -s "$full_file" ]; then echo -e "\t\t" >> $report printf '>$report - cat $seqres.full | tr -dc '[:print:][:space:]' | encode_xml >>$report + cat "$full_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report fi - if [ -f $seqres.dmesg ]; then + if [ -f "$dmesg_file" ]; then echo -e "\t\t" >> $report printf '>$report - cat $seqres.dmesg | tr -dc '[:print:][:space:]' | encode_xml >>$report + cat "$dmesg_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report - elif [ -s $seqres.out.bad ]; then + elif [ -s "$outbad_file" ]; then echo -e "\t\t" >> $report printf '>$report - $diff $test_seq.out $seqres.out.bad | encode_xml >>$report + $diff "$out_src" "$outbad_file" | encode_xml >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report fi @@ -134,13 +137,17 @@ _xunit_make_testcase_report() # Common report generator entry points _make_section_report() { - local tests_count="$1" - local bad_count="$2" - local notrun_count="$3" + local sect_name="$1" + local tests_count="$2" + local bad_count="$3" + local notrun_count="$4" + local sect_time="$5" for report in $REPORT_LIST; do case "$report" in "xunit") - _xunit_make_section_report "$tests_count" "$bad_count" "$notrun_count" + _xunit_make_section_report "$sect_name" "$tests_count" \ + "$bad_count" "$notrun_count" \ + "$sect_time" ;; *) _dump_err "format '$report' is not supported" @@ -151,12 +158,15 @@ _make_section_report() _make_testcase_report() { - local test_seq="$1" - local test_status="$2" + local sect_name="$1" + local test_seq="$2" + local test_status="$3" + local test_time="$4" for report in $REPORT_LIST; do case "$report" in "xunit") - _xunit_make_testcase_report "$test_seq" "$test_status" + _xunit_make_testcase_report "$sect_name" "$test_seq" \ + "$test_status" "$test_time" ;; *) _dump_err "report format '$report' is not supported" From patchwork Fri Jul 8 08:51:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Disseldorp X-Patchwork-Id: 12910747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8E81CCA481 for ; Fri, 8 Jul 2022 08:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237650AbiGHIvy (ORCPT ); Fri, 8 Jul 2022 04:51:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237646AbiGHIvu (ORCPT ); Fri, 8 Jul 2022 04:51:50 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69803823BC for ; Fri, 8 Jul 2022 01:51:49 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 102F921DFA; Fri, 8 Jul 2022 08:51:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1657270308; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JaYOpiHBsI2/xr9CV4WPU1qjh7pwMJa9O1tNo72CBts=; b=cqx63RPT+5giLsA6HK/LmHd9/3uiS5kfCeDBf5FArqD45V9oyRa2m+e8PWsEVC1v0e+m+j xfEtucGIoOPTGnj01nUIjtuDi3RrG7jLljBtpx8JGj66CrW5oKH37BwzGTEHiM57sNBmwt lyyKZHWeDxLph/TrMisqzJhdzMDuKLU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1657270308; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JaYOpiHBsI2/xr9CV4WPU1qjh7pwMJa9O1tNo72CBts=; b=Q6YEOKfQTmtusqALUvi57zJ6Rh35tdpjKgkPucecLvJ0L0eQdgk3zECcEM3wxBhr+p3pEr mcTXwFCOy8CPeFBQ== Received: from echidna.suse.de (unknown [10.163.47.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id D77A92C143; Fri, 8 Jul 2022 08:51:47 +0000 (UTC) From: David Disseldorp To: fstests@vger.kernel.org, tytso@mit.edu Cc: djwong@kernel.org, zlang@redhat.com, David Disseldorp Subject: [PATCH v4 3/5] check: make a few variables local Date: Fri, 8 Jul 2022 10:51:40 +0200 Message-Id: <20220708085142.20991-4-ddiss@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220708085142.20991-1-ddiss@suse.de> References: <20220708085142.20991-1-ddiss@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org The variables aren't used outside of function scope. Also convert one timestamp output to use the helper. Signed-off-by: David Disseldorp Reviewed-by: Darrick J. Wong --- check | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/check b/check index dbad6dde..08857f7e 100755 --- a/check +++ b/check @@ -179,10 +179,10 @@ get_all_tests() # the function from that list. trim_test_list() { - test_list="$*" + local test_list="$*" rm -f $tmp.grep - numsed=0 + local numsed=0 for t in $test_list do if [ $numsed -gt 100 ]; then @@ -207,7 +207,7 @@ _wallclock() _timestamp() { - now=`date "+%T"` + local now=`date "+%T"` echo -n " [$now]" } @@ -603,7 +603,7 @@ fi function run_section() { - local section=$1 + local section=$1 skip OLD_FSTYP=$FSTYP OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS @@ -820,7 +820,7 @@ function run_section() rm -f core $seqres.notrun start=`_wallclock` - $timestamp && echo -n " ["`date "+%T"`"]" + $timestamp && _timestamp [ ! -x $seq ] && chmod u+x $seq # ensure we can run it $LOGGER_PROG "run xfstest $seqnum" if [ -w /dev/kmsg ]; then From patchwork Fri Jul 8 08:51:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Disseldorp X-Patchwork-Id: 12910748 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 828C4CCA480 for ; Fri, 8 Jul 2022 08:51:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237635AbiGHIvz (ORCPT ); Fri, 8 Jul 2022 04:51:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237666AbiGHIvu (ORCPT ); Fri, 8 Jul 2022 04:51:50 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8212F823BD for ; Fri, 8 Jul 2022 01:51:49 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 42D0D21E61; Fri, 8 Jul 2022 08:51:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1657270308; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DC9ecCb45lbZHyuhGlTZOma9QMAHB3GmxyhN52VHZSs=; b=j1sDO4Z7sqKL8amifnPDdcZsES+41DbGE6Tazqzc+1cSNyS1YU31XbLPYbLOFUHL+PU6ZJ EUh9f6rP3wxw6zhY30Dpv3/LyUjauxaWzUc0v2RNha5bYZaR+4qenUkAStUEi5u+Iccpm6 oVo7aWU+2+nYHhzLS0yFvJ8GP1Pk/Q4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1657270308; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DC9ecCb45lbZHyuhGlTZOma9QMAHB3GmxyhN52VHZSs=; b=CapzXVHw4pUvjGATqv/BpnD3k1IS4KYEzlmiSuy+OZxE/yWMB199LFkyjwm1cBc4mpsfML g/oNm8vNO5rA8DBg== Received: from echidna.suse.de (unknown [10.163.47.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 182992C145; Fri, 8 Jul 2022 08:51:48 +0000 (UTC) From: David Disseldorp To: fstests@vger.kernel.org, tytso@mit.edu Cc: djwong@kernel.org, zlang@redhat.com, David Disseldorp Subject: [PATCH v4 4/5] check: append bad / notrun arrays in helper function Date: Fri, 8 Jul 2022 10:51:41 +0200 Message-Id: <20220708085142.20991-5-ddiss@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220708085142.20991-1-ddiss@suse.de> References: <20220708085142.20991-1-ddiss@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Currently the @try, @bad and @notrun arrays are appended with seqnum at different points in the main run_section() loop: - @try: shortly prior to test script execution - @notrun: on list (check -n), or after .notrun flagged test completion - @bad: at the start of subsequent test loop and loop exit For future loop-test-following-failure functionality it makes sense to combine some of these steps. This change moves both @notrun and @bad appends into a helper function which is called at the end of each loop iteration. Signed-off-by: David Disseldorp Reviewed-by: Darrick J. Wong --- check | 68 ++++++++++++++++++++++++++++++++++------------------------- 1 file changed, 39 insertions(+), 29 deletions(-) diff --git a/check b/check index 08857f7e..6dbdb2a8 100755 --- a/check +++ b/check @@ -553,6 +553,32 @@ _expunge_test() return 0 } +# Retain in @bad / @notrun the result of the just-run @test_seq. @try array +# entries are added prior to execution. +_stash_test_status() { + local test_seq="$1" + local test_status="$2" + + if $do_report && [[ $test_status != "expunge" ]]; then + _make_testcase_report "$section" "$test_seq" \ + "$test_status" "$((stop - start))" + fi + + case "$test_status" in + fail) + bad+=("$test_seq") + ;; + list|notrun) + notrun+=("$test_seq") + ;; + pass|expunge) + ;; + *) + echo "Unexpected test $test_seq status: $test_status" + ;; + esac +} + # Can we run systemd scopes? HAVE_SYSTEMD_SCOPES= systemctl reset-failed "fstests-check" &>/dev/null @@ -732,19 +758,8 @@ function run_section() seqres="$check" _check_test_fs - local tc_status="init" - prev_seq="" + local tc_status for seq in $list ; do - # Run report for previous test! - if [ "$tc_status" == "fail" ]; then - bad+=("$seqnum") - fi - if $do_report && [[ ! $tc_status =~ ^(init|expunge)$ ]]; then - _make_testcase_report "$section" "$seqnum" \ - "$tc_status" "$((stop - start))" - fi - - prev_seq="$seq" if [ ! -f $seq ]; then # Try to get full name in case the user supplied only # seq id and the test has a name. A bit of hassle to @@ -784,20 +799,21 @@ function run_section() if $showme; then _expunge_test $seqnum if [ $? -eq 1 ]; then - tc_status="expunge" - continue + tc_status="expunge" + else + echo + start=0 + stop=0 + tc_status="list" fi - echo - start=0 - stop=0 - tc_status="list" - notrun+=("$seqnum") + _stash_test_status "$seqnum" "$tc_status" continue fi tc_status="pass" if [ ! -f $seq ]; then echo " - no such test?" + _stash_test_status "$seqnum" "$tc_status" continue fi @@ -808,6 +824,7 @@ function run_section() _expunge_test $seqnum if [ $? -eq 1 ]; then tc_status="expunge" + _stash_test_status "$seqnum" "$tc_status" continue fi @@ -857,8 +874,8 @@ function run_section() $timestamp && echo " [not run]" && \ echo -n " $seqnum -- " cat $seqres.notrun - notrun+=("$seqnum") tc_status="notrun" + _stash_test_status "$seqnum" "$tc_status" # Unmount the scratch fs so that we can wipe the scratch # dev state prior to the next test run. @@ -903,6 +920,7 @@ function run_section() if [ ! -f $seq.out ]; then _dump_err "no qualified output" tc_status="fail" + _stash_test_status "$seqnum" "$tc_status" continue; fi @@ -938,17 +956,9 @@ function run_section() rm -f $seqres.hints fi fi + _stash_test_status "$seqnum" "$tc_status" done - # make sure we record the status of the last test we ran. - if [ "$tc_status" == "fail" ]; then - bad+=("$seqnum") - fi - if $do_report && [[ ! $tc_status =~ ^(init|expunge)$ ]]; then - _make_testcase_report "$section" "$seqnum" "$tc_status" \ - "$((stop - start))" - fi - sect_stop=`_wallclock` interrupt=false _wrapup From patchwork Fri Jul 8 08:51:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Disseldorp X-Patchwork-Id: 12910749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D56A8C43334 for ; Fri, 8 Jul 2022 08:51:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237646AbiGHIv4 (ORCPT ); Fri, 8 Jul 2022 04:51:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237651AbiGHIvu (ORCPT ); Fri, 8 Jul 2022 04:51:50 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7386823BE for ; Fri, 8 Jul 2022 01:51:49 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 7778D21E73; Fri, 8 Jul 2022 08:51:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1657270308; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z32yBV3xcygo/Qzl/kOZ/6GVDykIHQMRoXQ4H9nbuu0=; b=AjP4/A45zk94DeLPSW4G79qiwQqOrNwCHkj73hX9xEBwpvwX4Cw16Fv9ZNjDCb53f8QZIz dp0V8FuFhSH9qCGofC7LkgDAgJCu6odHWqEa0+0N3rFHQmMPqTO/mSM2NvkaHgOqr2d/fb 7zaK/c4mrZoo6b2QBbnHKXZZIs2Y2QE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1657270308; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z32yBV3xcygo/Qzl/kOZ/6GVDykIHQMRoXQ4H9nbuu0=; b=wrpR6cAnIP5tP7+XSbh3Ay+5koNcbMgbcSgHIKpaYMM5H85T4xDbw67JJTF3l07oi9bk9W mHGQRwP4bYhSjvAg== Received: from echidna.suse.de (unknown [10.163.47.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 4C5B82C146; Fri, 8 Jul 2022 08:51:48 +0000 (UTC) From: David Disseldorp To: fstests@vger.kernel.org, tytso@mit.edu Cc: djwong@kernel.org, zlang@redhat.com, David Disseldorp Subject: [PATCH v4 5/5] check: add -L parameter to rerun failed tests Date: Fri, 8 Jul 2022 10:51:42 +0200 Message-Id: <20220708085142.20991-6-ddiss@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20220708085142.20991-1-ddiss@suse.de> References: <20220708085142.20991-1-ddiss@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org If check is run with -L , then a failed test will be rerun times before proceeding to the next test. Following completion of the rerun loop, aggregate pass/fail statistics are printed. Rerun tests will be tracked as a single failure in overall pass/fail metrics (via @try and @bad), with .out.bad, .dmesg, .core, .hints, .notrun and .full saved using a .rerun# suffix. Suggested-by: Theodore Ts'o Link: https://lwn.net/Articles/897061/ Signed-off-by: David Disseldorp Reviewed-by: Darrick J. Wong --- check | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 54 insertions(+), 3 deletions(-) diff --git a/check b/check index 6dbdb2a8..5f6d86b4 100755 --- a/check +++ b/check @@ -26,6 +26,7 @@ do_report=false DUMP_OUTPUT=false iterations=1 istop=false +loop_on_fail=0 # This is a global variable used to pass test failure text to reporting gunk _err_msg="" @@ -78,6 +79,7 @@ check options --large-fs optimise scratch device for large filesystems -s section run only specified section from config file -S section exclude the specified section from the config file + -L loop tests times following a failure, measuring aggregate pass/fail metrics testlist options -g group[,group...] include tests from these groups @@ -336,6 +338,9 @@ while [ $# -gt 0 ]; do ;; --large-fs) export LARGE_SCRATCH_DEV=yes ;; --extra-space=*) export SCRATCH_DEV_EMPTY_SPACE=${r#*=} ;; + -L) [[ $2 =~ ^[0-9]+$ ]] || usage + loop_on_fail=$2; shift + ;; -*) usage ;; *) # not an argument, we've got tests now. @@ -553,6 +558,19 @@ _expunge_test() return 0 } +# retain files which would be overwritten in subsequent reruns of the same test +_stash_fail_loop_files() { + local seq_prefix="${REPORT_DIR}/${1}" + local cp_suffix="$2" + + for i in ".full" ".dmesg" ".out.bad" ".notrun" ".core" ".hints"; do + rm -f "${seq_prefix}${i}${cp_suffix}" + if [ -f "${seq_prefix}${i}" ]; then + cp "${seq_prefix}${i}" "${seq_prefix}${i}${cp_suffix}" + fi + done +} + # Retain in @bad / @notrun the result of the just-run @test_seq. @try array # entries are added prior to execution. _stash_test_status() { @@ -564,8 +582,35 @@ _stash_test_status() { "$test_status" "$((stop - start))" fi + if ((${#loop_status[*]} > 0)); then + # continuing or completing rerun-on-failure loop + _stash_fail_loop_files "$test_seq" ".rerun${#loop_status[*]}" + loop_status+=("$test_status") + if ((${#loop_status[*]} > loop_on_fail)); then + printf "%s aggregate results across %d runs: " \ + "$test_seq" "${#loop_status[*]}" + awk "BEGIN { + n=split(\"${loop_status[*]}\", arr);"' + for (i = 1; i <= n; i++) + stats[arr[i]]++; + for (x in stats) + printf("%s=%d (%.1f%%)", + (i-- > n ? x : ", " x), + stats[x], 100 * stats[x] / n); + }' + echo + loop_status=() + fi + return # only stash @bad result for initial failure in loop + fi + case "$test_status" in fail) + if ((loop_on_fail > 0)); then + # initial failure, start rerun-on-failure loop + _stash_fail_loop_files "$test_seq" ".rerun0" + loop_status+=("$test_status") + fi bad+=("$test_seq") ;; list|notrun) @@ -758,8 +803,12 @@ function run_section() seqres="$check" _check_test_fs - local tc_status - for seq in $list ; do + loop_status=() # track rerun-on-failure state + local tc_status ix + local -a _list=( $list ) + for ((ix = 0; ix < ${#_list[*]}; !${#loop_status[*]} && ix++)); do + seq="${_list[$ix]}" + if [ ! -f $seq ]; then # Try to get full name in case the user supplied only # seq id and the test has a name. A bit of hassle to @@ -829,7 +878,9 @@ function run_section() fi # record that we really tried to run this test. - try+=("$seqnum") + if ((!${#loop_status[*]})); then + try+=("$seqnum") + fi awk 'BEGIN {lasttime=" "} \ $1 == "'$seqnum'" {lasttime=" " $2 "s ... "; exit} \