From patchwork Wed Jul 20 16:43:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Theodore Ts'o X-Patchwork-Id: 12924299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4989CCA482 for ; Wed, 20 Jul 2022 16:44:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229437AbiGTQoR (ORCPT ); Wed, 20 Jul 2022 12:44:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233723AbiGTQoH (ORCPT ); Wed, 20 Jul 2022 12:44:07 -0400 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C6F566AE1 for ; Wed, 20 Jul 2022 09:44:06 -0700 (PDT) Received: from cwcc.thunk.org (pool-173-48-118-63.bstnma.fios.verizon.net [173.48.118.63]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 26KGi1sA023369 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Jul 2022 12:44:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1658335442; bh=YGUTV65UfHjpOyg+ZFMAj1kz+xJRBw2bMpjFRnZYBmQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=FVISuY2Oxm61lVG0jQzUHAzVlhmRMbHY9ZaXag0dp//86509a2AyhMfDKHXRXUjFy +a96nMCHC5Gg6eVEs+pRlKV8tKLdiamy8H4gZ9LE1UX2ERYz8v6lUYQWTs4N3qqPEw NI21NGgIP8arqsv5r5fruQtYYaRD/EDzXdlVnRt+sd5UP/n31lin27I6lPx90Uw5MN OYxLUKyYeqmhDmv9HehkeYrG+vKSZLMB3YSFIgy/VenjW59stiNOua7ZrIcZijy39d RuMk1GsCUcc3qwwrsQKeXcmABcoiMrg5GUuo2baOqjEBHDnhpllEnPC6Zk7kuyCyH3 eKq/g24mP/cdw== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 079E615C3EC1; Wed, 20 Jul 2022 12:44:01 -0400 (EDT) From: "Theodore Ts'o" To: fstests@vger.kernel.org Cc: "Theodore Ts'o" , David Disseldorp Subject: [PATCH 1/4] report: add support for the xunit-quiet format Date: Wed, 20 Jul 2022 12:43:53 -0400 Message-Id: <20220720164356.4078789-2-tytso@mit.edu> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220720164356.4078789-1-tytso@mit.edu> References: <20220720164356.4078789-1-tytso@mit.edu> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org The xunit-quiet format excludes the NNN.{full,dmesg,bad} files in and nodes which are included in the xunit report format. For test runners that save the entire results directory to preserve all of the test artifacts, capturing the NNN.{full,dmesg,bad} in the results.xml file is redundant. In addition, if the NNN.bad is too large, it can cause the junitparser python library to refuse to parse the XML file to prevent potential denial of service attacks[1]. A simple way to avoid this problem is to simply to omit the and nodes in the results.xml file. [1] https://gitlab.com/gitlab-org/gitlab/-/issues/268035 Signed-off-by: Theodore Ts'o Reviewed-by: David Disseldorp Reviewed-by: Darrick J. Wong --- check | 2 +- common/report | 21 ++++++++++++++------- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/check b/check index 5f6d86b4..0b2f10ed 100755 --- a/check +++ b/check @@ -75,7 +75,7 @@ check options -I iterate the test list times, but stops iterating further in case of any test failure -d dump test output to stdout -b brief test summary - -R fmt[,fmt] generate report in formats specified. Supported format: [xunit] + -R fmt[,fmt] generate report in formats specified. Supported formats: xunit, xunit-quiet --large-fs optimise scratch device for large filesystems -s section run only specified section from config file -S section exclude the specified section from the config file diff --git a/common/report b/common/report index 5ca41bc4..64f9c866 100644 --- a/common/report +++ b/common/report @@ -71,11 +71,16 @@ _xunit_make_testcase_report() local test_name="$2" local test_status="$3" local test_time="$4" + local report_format="$5" + local quiet + + if [ "$report_format" = xunit-quiet ]; then + quiet=yes + fi # TODO: other places may also win if no-section mode will be named like 'default/global' if [ $sect_name == '-no-sections-' ]; then sect_name='global' - fi local report=$tmp.report.xunit.$sect_name.xml @@ -104,14 +109,16 @@ _xunit_make_testcase_report() _err_msg="Test $test_name failed, reason unknown" fi echo -e "\t\t" >> $report - if [ -s "$full_file" ]; then + if [ -z "$quiet" -a -s "$full_file" ]; then echo -e "\t\t" >> $report printf '>$report cat "$full_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report printf ']]>\n' >>$report echo -e "\t\t" >> $report fi - if [ -f "$dmesg_file" ]; then + if [ -n "$quiet" ]; then + : + elif [ -f "$dmesg_file" ]; then echo -e "\t\t" >> $report printf '>$report cat "$dmesg_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report @@ -144,7 +151,7 @@ _make_section_report() local sect_time="$5" for report in $REPORT_LIST; do case "$report" in - "xunit") + "xunit"|"xunit-quiet") _xunit_make_section_report "$sect_name" "$tests_count" \ "$bad_count" "$notrun_count" \ "$sect_time" @@ -164,9 +171,9 @@ _make_testcase_report() local test_time="$4" for report in $REPORT_LIST; do case "$report" in - "xunit") + "xunit"|"xunit-quiet") _xunit_make_testcase_report "$sect_name" "$test_seq" \ - "$test_status" "$test_time" + "$test_status" "$test_time" "$report" ;; *) _dump_err "report format '$report' is not supported" @@ -178,7 +185,7 @@ _make_testcase_report() _assert_report_list() { for report in $REPORT_LIST; do case "$report" in - "xunit") + "xunit"|"xunit-quiet") ;; *) _fatal "report format '$report' is not supported" From patchwork Wed Jul 20 16:43:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Theodore Ts'o X-Patchwork-Id: 12924301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45E34C43334 for ; Wed, 20 Jul 2022 16:45:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231816AbiGTQpG (ORCPT ); Wed, 20 Jul 2022 12:45:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233747AbiGTQoH (ORCPT ); Wed, 20 Jul 2022 12:44:07 -0400 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAA3766AC9 for ; Wed, 20 Jul 2022 09:44:04 -0700 (PDT) Received: from cwcc.thunk.org (pool-173-48-118-63.bstnma.fios.verizon.net [173.48.118.63]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 26KGi10Z023368 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Jul 2022 12:44:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1658335442; bh=RsacFu/gkpReCLWTpyvQT/8up/a9sHbIrRf+StS7fB8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=edi0wyxjZWZVVwaVX55YmnWFXD+ZHRVgJzOxe/7g2CJJnAqGvDrMkX9J/wG2EhwQQ szkcL+Vq8y7xqhbT+UEzfu8v83wS2q/rsP69+LnUfPEUbgF/a1uvqUu2ml6n/eSPHX K5IwaUv+rnDNQCJ832aeoTajR49U1guAzmJ5WF+/HjTMZEz75Chf6VdFP62px4E60V rmKomQEiV6z0Ev5kP5hK/eW5YbjpDAa9tnWHd0ZSOCcGcNux/ZfpNLt0kILHj2+Fvf NZt/r3A/kVV2VMSigqKADCJf/MoEeyDmeo/30IF0hqfdZrlmhsUcCwya1YpuvJWxDL U4eGUgrbLwNzg== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 0926C15C3EC3; Wed, 20 Jul 2022 12:44:01 -0400 (EDT) From: "Theodore Ts'o" To: fstests@vger.kernel.org Cc: "Theodore Ts'o" Subject: [PATCH 2/4] generic/556: add a check to make sure ext4 supports encrypted casefolding Date: Wed, 20 Jul 2022 12:43:54 -0400 Message-Id: <20220720164356.4078789-3-tytso@mit.edu> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220720164356.4078789-1-tytso@mit.edu> References: <20220720164356.4078789-1-tytso@mit.edu> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Some older kernels support ext4 file systems with encryption enabled, and with casefold enabled, but not file systems have both encryption *and* casefolding enabled. On those kernels, generic/556 will fail. Fortunately, we can test if ext4 supports encrypted casefold via the presence of /sys/fs/ext4/features/encrypted_casefold. Signed-off-by: Theodore Ts'o --- common/casefold | 17 +++++++++++++++++ tests/generic/556 | 1 + 2 files changed, 18 insertions(+) diff --git a/common/casefold b/common/casefold index d9126f4c..e3e89508 100644 --- a/common/casefold +++ b/common/casefold @@ -43,6 +43,23 @@ _require_scratch_casefold() _require_command "$LSATTR_PROG" lsattr } +_require_encrypted_casefold () +{ + case $FSTYP in + ext4) + if test ! -f /sys/fs/ext4/features/casefold ; then + _notrun "casefolding not supported" + fi + if test ! -f /sys/fs/ext4/features/encryption ; then + _notrun "file system encryption not supported" + fi + if test ! -f /sys/fs/ext4/features/encrypted_casefold ; then + _notrun "encrypted casefolding not supported" + fi + ;; + esac +} + _scratch_mkfs_casefold() { case $FSTYP in diff --git a/tests/generic/556 b/tests/generic/556 index 404a3243..80563502 100755 --- a/tests/generic/556 +++ b/tests/generic/556 @@ -13,6 +13,7 @@ _begin_fstest auto quick casefold . ./common/attr _supported_fs generic +_require_encrypted_casefold _require_scratch_nocheck _require_scratch_casefold _require_symlinks From patchwork Wed Jul 20 16:43:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Theodore Ts'o X-Patchwork-Id: 12924300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49EC4C43334 for ; Wed, 20 Jul 2022 16:44:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232606AbiGTQo3 (ORCPT ); Wed, 20 Jul 2022 12:44:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233723AbiGTQoV (ORCPT ); Wed, 20 Jul 2022 12:44:21 -0400 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9896365D69 for ; Wed, 20 Jul 2022 09:44:20 -0700 (PDT) Received: from cwcc.thunk.org (pool-173-48-118-63.bstnma.fios.verizon.net [173.48.118.63]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 26KGi1L9023372 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Jul 2022 12:44:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1658335442; bh=Ok/jP8rC+NKtItaJ6xxmvUqSk7vDAZAX3JyDEcSt2N4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TSK26SlPt3BU7cdMu0f+z/EWJvnWS3ecjSF/utqI0VE2MyRRdz4a1GMsA21ka+VEV ALCum7sAsPNFnjiArbcvc4ls8q8V35GESQZMg6LKiIu52DfR/ILpfU5IblsCiABY7t l5Ztyxyc8JUiKL2gnw5YX+soJ02bhKH7F91xCuQ9BhMgpRDaFpOo/Z1QuoW8vl6B37 pT/E4Vl0vygi58ayxgfBW8hoNOqDL57fFZ0xUjYvIjimyQ1Sd2Ye0zeP1ZOLuT7EAa HZzK27RmZM241cWCErU1V5DMWLHDpqH/gBqUKdpS79NWl2FLO2lEHyFL3kPSNm3NEz pu1/VJiSEnLIA== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 0B0B815C3EC7; Wed, 20 Jul 2022 12:44:01 -0400 (EDT) From: "Theodore Ts'o" To: fstests@vger.kernel.org Cc: "Theodore Ts'o" Subject: [PATCH 3/4] dmflakey: don't run dmflakey tests with an external log device Date: Wed, 20 Jul 2022 12:43:55 -0400 Message-Id: <20220720164356.4078789-4-tytso@mit.edu> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220720164356.4078789-1-tytso@mit.edu> References: <20220720164356.4078789-1-tytso@mit.edu> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org dmflakey works by dropping all writes before unmounting to simulate a crash/power loss. This doesn't work if there is an external log device, since we only drop writes to the primary block device, and not the external log device. Fixing this for real would require somehow arranging to atomically loading a new dmflakey table for two block devices at the same time, so for now, just skip tests using dmflakey if the external log device is enabled. Signed-off-by: Theodore Ts'o --- common/dmflakey | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/common/dmflakey b/common/dmflakey index 52da3b10..c2be78e9 100644 --- a/common/dmflakey +++ b/common/dmflakey @@ -12,6 +12,10 @@ _init_flakey() { # Scratch device local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV` + + if test "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ; then + _notrun "dmflakey tests don't work with an external log device" + fi FLAKEY_DEV=/dev/mapper/flakey-test FLAKEY_TABLE="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 180 0" FLAKEY_TABLE_DROP="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 0 180 1 drop_writes" From patchwork Wed Jul 20 16:43:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Theodore Ts'o X-Patchwork-Id: 12924303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 757BECCA480 for ; Wed, 20 Jul 2022 16:45:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232601AbiGTQpH (ORCPT ); Wed, 20 Jul 2022 12:45:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233592AbiGTQoH (ORCPT ); Wed, 20 Jul 2022 12:44:07 -0400 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94AC365D56 for ; Wed, 20 Jul 2022 09:44:04 -0700 (PDT) Received: from cwcc.thunk.org (pool-173-48-118-63.bstnma.fios.verizon.net [173.48.118.63]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 26KGi14S023371 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Jul 2022 12:44:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1658335442; bh=iCnwOw0mW1GMjl/nYcdjKKDpZAilqI7XsqfHaDyvrZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=P0G5mygfQ7iUmqB4QciybLZ1iWyznZm8W4FuiSyVT9Vhij1BRKoOQi0ImpGmJsu+Q gV4dasCuKAjKYIml7m/jS+IF9fRtvrB90JvN4O7MWWMWRem0M4rodtet8XhhVBlsna kbOUNxSu5fe+oEwEg9aRUpE2LrWfgvykZzlyv3HSGwRYjRK43zGDK7LYWyMxTthCtG yWYF33SrsfGM/xSPzKG7pNrrLsETbaQETfz6Hp3kpUcqCJ6cZAXS7bazFSoBQlg39P 65irZ5r+qoNfEa7XRCGdE0M52cEOf86uvrKitvRlspXOE/HhwsFLL+T+uRciXxm7uS FqkeN0O+NLOrw== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 0D9D815C3ECF; Wed, 20 Jul 2022 12:44:01 -0400 (EDT) From: "Theodore Ts'o" To: fstests@vger.kernel.org Cc: "Theodore Ts'o" Subject: [PATCH 4/4] generic/475: skip test when using an external log device Date: Wed, 20 Jul 2022 12:43:56 -0400 Message-Id: <20220720164356.4078789-5-tytso@mit.edu> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220720164356.4078789-1-tytso@mit.edu> References: <20220720164356.4078789-1-tytso@mit.edu> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org This test tests log recovery by kicking off fstress on the scratch fs, then switching out the underlying the device with dm-error to see what happens when the disk goes down. But when there is an external log device writes to the log device still succeed, while writes to the rest of the file system fail, and this leads to file system corruption. So skip this test when there is an external log. Signed-off-by: Theodore Ts'o --- tests/generic/475 | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/generic/475 b/tests/generic/475 index c426402e..43c26303 100755 --- a/tests/generic/475 +++ b/tests/generic/475 @@ -34,6 +34,10 @@ _require_scratch _require_dm_target error _require_command "$KILLALL_PROG" "killall" +if test "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ; then + _notrun "Cannot have external log device" +fi + echo "Silence is golden." _scratch_mkfs >> $seqres.full 2>&1