From patchwork Fri Dec 15 02:08:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10113605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6778D60231 for ; Fri, 15 Dec 2017 02:14:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 534E229EC8 for ; Fri, 15 Dec 2017 02:14:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 47B5029ECA; Fri, 15 Dec 2017 02:14:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC2AA29EC8 for ; Fri, 15 Dec 2017 02:14:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754819AbdLOCOI (ORCPT ); Thu, 14 Dec 2017 21:14:08 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:60551 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754594AbdLOCOF (ORCPT ); Thu, 14 Dec 2017 21:14:05 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.21/8.16.0.21) with SMTP id vBF2CZSe112210; Fri, 15 Dec 2017 02:14:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2017-10-26; bh=zWj0kJNizPymovu9uhUhWVjVKc54RDZwcNA0xW4mtY8=; b=mkBJkiSI/vC8suHLYQt0yDqTmm7rx71HQiAnPfr/6n3zNBGaqShYSHiBynqOmDqS8e/D iQlYbpQ0wXAIFhKKj1appzsbl39qXyq8nPjzQ/8IKBfuDeMSUjJQGZJ79aIv2J9lC3T2 BOpx0GbgB01s3EUlXEoytr+qrnvvCt3rw3Uc2j8aciVVIYpqN7VfZZ+SAf38wqGOrOp2 Pg1oONR/BFG+qQtTca/NauWQvTix1VVwlgGBcJ7Gg0MrOaW84TdqFjX0gsA0NF20K7jb 4sbE0Tjj3w02bkmMlmTzOq7NBj9v5gbriXHW0CKjdobu2Y+QYELuRxeZXkyMUdAUv1jR dA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2ev5ct0077-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Dec 2017 02:14:01 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id vBF2901H011849 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Dec 2017 02:09:00 GMT Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id vBF290V7021392; Fri, 15 Dec 2017 02:09:00 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Dec 2017 18:08:59 -0800 Date: Thu, 14 Dec 2017 18:08:59 -0800 From: "Darrick J. Wong" To: eguan@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Subject: [PATCH v3 8/8] xfs/068: fix variability problems in file/dir count output Message-ID: <20171215020859.GN6896@magnolia> References: <151314499003.18893.8687182548758898133.stgit@magnolia> <151314505158.18893.11894289091110903029.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <151314505158.18893.11894289091110903029.stgit@magnolia> User-Agent: Mutt/1.5.24 (2015-08-30) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8745 signatures=668648 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1712150025 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In these tests we use a fixed sequence of operations in fsstress to create a directory tree and then exercise xfsdump/xfsrestore on that. However, this changes every time someone adds a new fsstress command, or someone runs with FSSTRESS_AVOID, etc. Therefore, check the counts directly from xfsrestore output instead of relying on the golden output and do much more rigorous checking of the dir tree complexity and the intactness of the dirs and files after restoring them. Signed-off-by: Darrick J. Wong --- v3: moar checking --- common/dump | 40 ++++++++++++++++++++++++++++++++++++++-- tests/xfs/027.out | 2 +- tests/xfs/068.out | 2 +- 3 files changed, 40 insertions(+), 4 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/common/dump b/common/dump index 898aaa4..db2e156 100644 --- a/common/dump +++ b/common/dump @@ -316,7 +316,8 @@ _create_dumpdir_stress_num() echo "-----------------------------------------------" echo "fsstress : $_param" echo "-----------------------------------------------" - if ! $here/ltp/fsstress $_param -s 1 $FSSTRESS_AVOID -n $_count -d $dump_dir >$tmp.out 2>&1 + echo $FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n $_count -d $dump_dir >> $seqres.full + if ! $FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n $_count -d $dump_dir >$tmp.out 2>&1 then echo " fsstress (count=$_count) returned $? - see $seqres.full" @@ -1240,9 +1241,44 @@ _do_dump_restore() echo "xfsdump|xfsrestore ..." restore_opts="$_restore_debug$restore_args - $restore_dir" dump_opts="$_dump_debug$dump_args -s $dump_sdir - $SCRATCH_MNT" + + # We expect there to be one more dir (and inode) than what's in dump_dir. + # Construct the string we expect to see in the output, since fsstress + # will create different directory structures every time someone adds + # a new command, runs with a different FSSTRESS_AVOID, etc. + expected_dirs=$(find $dump_dir -type d | wc -l) + expected_inodes=$(find $dump_dir | wc -l) + expected_str=": $((expected_dirs + 1)) directories and $((expected_inodes + 1)) entries processed" + + if [ $expected_dirs -lt 100 ] || [ $expected_inodes -lt 600 ]; then + echo "Oddly small dir tree$expected_str" + fi + + # Measure the md5 of every file... + (cd $dump_dir ; find . -type f -print0 | xargs -0 md5sum) > $seqres.md5 + (cd $dump_dir ; find | sort) > $seqres.fstree + echo "xfsdump $dump_opts | xfsrestore $restore_opts" | _dir_filter - $XFSDUMP_PROG $dump_opts 2>$tmp.dump.mlog | $XFSRESTORE_PROG $restore_opts 2>&1 | tee -a $seqres.full | _dump_filter + $XFSDUMP_PROG $dump_opts 2>$tmp.dump.mlog | $XFSRESTORE_PROG $restore_opts > $tmp.restore.mlog 2>&1 + cat $tmp.restore.mlog >> $seqres.full + echo "xfsrestore output should contain$expected_str" >> $seqres.full + cat $tmp.restore.mlog | _dump_filter | sed -e 's/: \([0-9]*\) directories and \([0-9]*\) entries/: XXX directories and YYY entries/g' _dump_filter <$tmp.dump.mlog + + # Did we actually restore as many dirs/files as we had? + if ! grep -q "$expected_str" $tmp.restore.mlog; then + echo "mismatch counts between directory tree and restored filesystem" + grep "directories and.*entries processed" $tmp.restore.mlog | sed -e 's/^.*:/found:/g' + echo "expected$expected_str" + fi + + # Does the directory tree match? + diff -u $seqres.fstree <(cd $restore_dir/$dump_sdir ; find | sort) + + # Measure the md5 of every restored file... + (cd $restore_dir/$dump_sdir ; md5sum --quiet -c $seqres.md5) + + rm -rf $seqres.md5 $seqres.fstree $tmp.restore.mlog $tmp.dump.mlog } # diff --git a/tests/xfs/027.out b/tests/xfs/027.out index ba425a3..7665021 100644 --- a/tests/xfs/027.out +++ b/tests/xfs/027.out @@ -19,7 +19,7 @@ xfsrestore: session id: ID xfsrestore: media ID: ID xfsrestore: searching media for directory dump xfsrestore: reading directories -xfsrestore: 3 directories and 39 entries processed +xfsrestore: XXX directories and YYY entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: restore complete: SECS seconds elapsed diff --git a/tests/xfs/068.out b/tests/xfs/068.out index fa3a552..f53c555 100644 --- a/tests/xfs/068.out +++ b/tests/xfs/068.out @@ -22,7 +22,7 @@ xfsrestore: session id: ID xfsrestore: media ID: ID xfsrestore: searching media for directory dump xfsrestore: reading directories -xfsrestore: 383 directories and 1335 entries processed +xfsrestore: XXX directories and YYY entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: restore complete: SECS seconds elapsed