From patchwork Fri Dec 15 02:17:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10113613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8E537602B3 for ; Fri, 15 Dec 2017 02:24:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 785C829E89 for ; Fri, 15 Dec 2017 02:24:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B88729E90; Fri, 15 Dec 2017 02:24:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C4A3F29E89 for ; Fri, 15 Dec 2017 02:24:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754711AbdLOCYd (ORCPT ); Thu, 14 Dec 2017 21:24:33 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:37030 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754594AbdLOCYc (ORCPT ); Thu, 14 Dec 2017 21:24:32 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.21/8.16.0.21) with SMTP id vBF2OSmq120169; Fri, 15 Dec 2017 02:24:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2017-10-26; bh=EQb/0UpVjEUxQva3B3VxDDh5U/H2apAiAztYEfkCxp8=; b=bGzIO2weYyd+kFnV00ioXAkvncqxHcd1W3Zlrpturfa+YkBf3A7zA2FOG75/sB4nAQDx cBsKmpt7d9Vvjs0sFldR3Z1pKh+jJaylFejT2pUMjIr7zpI4QHDdPpvA2dNI7BYKbb3Y QQbUbygZjGktZFPK9C9rIdGN8b439MiFQmbjpdblr7M8KgnrF92a5hEQTqHEkXEtE3VR DgDDSxqbmARX431/n+36qCOU1+4ILk05mvsHi9I++iKsYm/2C63a+Vv3ejci5gsHB8kH 5goNnDnFzjju4LwuZco6LJkJ3qYJiMoOgphpkiYn7LbFa4/VqABRA3/gu1M4TZTozA7o ag== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2ev5ct013g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Dec 2017 02:24:28 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id vBF2HU6X015482 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Dec 2017 02:17:30 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id vBF2HTTI018062; Fri, 15 Dec 2017 02:17:29 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Dec 2017 18:17:29 -0800 Date: Thu, 14 Dec 2017 18:17:27 -0800 From: "Darrick J. Wong" To: eguan@redhat.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Subject: [PATCH v4 8/8] xfs/068: fix variability problems in file/dir count output Message-ID: <20171215021727.GV19219@magnolia> References: <151314499003.18893.8687182548758898133.stgit@magnolia> <151314505158.18893.11894289091110903029.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <151314505158.18893.11894289091110903029.stgit@magnolia> User-Agent: Mutt/1.5.24 (2015-08-30) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8745 signatures=668648 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1712150027 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In these tests we use a fixed sequence of operations in fsstress to create a directory tree and then exercise xfsdump/xfsrestore on that. However, this changes every time someone adds a new fsstress command, or someone runs with FSSTRESS_AVOID, etc. Therefore, check the counts directly from xfsrestore output instead of relying on the golden output; check that the paths of every file in the directory tree match; and check that the md5 of every regular file matches. Signed-off-by: Darrick J. Wong --- v4: add lots more checking of everything we restored --- common/dump | 36 ++++++++++++++++++++++++++++++++++-- tests/xfs/027.out | 2 +- tests/xfs/068 | 9 +++++++++ tests/xfs/068.out | 2 +- 4 files changed, 45 insertions(+), 4 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/common/dump b/common/dump index 898aaa4..f4edd53 100644 --- a/common/dump +++ b/common/dump @@ -316,7 +316,8 @@ _create_dumpdir_stress_num() echo "-----------------------------------------------" echo "fsstress : $_param" echo "-----------------------------------------------" - if ! $here/ltp/fsstress $_param -s 1 $FSSTRESS_AVOID -n $_count -d $dump_dir >$tmp.out 2>&1 + echo $FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n $_count -d $dump_dir >> $seqres.full + if ! $FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n $_count -d $dump_dir >$tmp.out 2>&1 then echo " fsstress (count=$_count) returned $? - see $seqres.full" @@ -1240,9 +1241,40 @@ _do_dump_restore() echo "xfsdump|xfsrestore ..." restore_opts="$_restore_debug$restore_args - $restore_dir" dump_opts="$_dump_debug$dump_args -s $dump_sdir - $SCRATCH_MNT" + + # We expect there to be one more dir (and inode) than what's in dump_dir. + # Construct the string we expect to see in the output, since fsstress + # will create different directory structures every time someone adds + # a new command, runs with a different FSSTRESS_AVOID, etc. + expected_dirs=$(find $dump_dir -type d | wc -l) + expected_inodes=$(find $dump_dir | wc -l) + expected_str=": $((expected_dirs + 1)) directories and $((expected_inodes + 1)) entries processed" + + # Measure the md5 of every file... + (cd $dump_dir ; find . -type f -print0 | xargs -0 md5sum) > $seqres.md5 + (cd $dump_dir ; find | sort) > $seqres.fstree + echo "xfsdump $dump_opts | xfsrestore $restore_opts" | _dir_filter - $XFSDUMP_PROG $dump_opts 2>$tmp.dump.mlog | $XFSRESTORE_PROG $restore_opts 2>&1 | tee -a $seqres.full | _dump_filter + $XFSDUMP_PROG $dump_opts 2>$tmp.dump.mlog | $XFSRESTORE_PROG $restore_opts > $tmp.restore.mlog 2>&1 + cat $tmp.restore.mlog >> $seqres.full + echo "xfsrestore output should contain$expected_str" >> $seqres.full + cat $tmp.restore.mlog | _dump_filter | sed -e 's/: \([0-9]*\) directories and \([0-9]*\) entries/: XXX directories and YYY entries/g' _dump_filter <$tmp.dump.mlog + + # Did we actually restore as many dirs/files as we had? + if ! grep -q "$expected_str" $tmp.restore.mlog; then + echo "mismatch counts between directory tree and restored filesystem" + grep "directories and.*entries processed" $tmp.restore.mlog | sed -e 's/^.*:/found:/g' + echo "expected$expected_str" + fi + + # Does the directory tree match? + diff -u $seqres.fstree <(cd $restore_dir/$dump_sdir ; find | sort) + + # Measure the md5 of every restored file... + (cd $restore_dir/$dump_sdir ; md5sum --quiet -c $seqres.md5) + + rm -rf $seqres.md5 $seqres.fstree $tmp.restore.mlog $tmp.dump.mlog } # diff --git a/tests/xfs/027.out b/tests/xfs/027.out index ba425a3..7665021 100644 --- a/tests/xfs/027.out +++ b/tests/xfs/027.out @@ -19,7 +19,7 @@ xfsrestore: session id: ID xfsrestore: media ID: ID xfsrestore: searching media for directory dump xfsrestore: reading directories -xfsrestore: 3 directories and 39 entries processed +xfsrestore: XXX directories and YYY entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: restore complete: SECS seconds elapsed diff --git a/tests/xfs/068 b/tests/xfs/068 index 7151e28..9ecb836 100755 --- a/tests/xfs/068 +++ b/tests/xfs/068 @@ -44,6 +44,15 @@ _supported_fs xfs _supported_os Linux _create_dumpdir_stress_num 4096 + +# Let's make sure there's a largeish number of dirs/files here. +expected_dirs=$(find $dump_dir -type d | wc -l) +expected_inodes=$(find $dump_dir | wc -l) + +if [ $expected_dirs -lt 100 ] || [ $expected_inodes -lt 600 ]; then + echo "Oddly small dir tree: $((expected_dirs + 1)) directories and $((expected_inodes + 1)) entries processed" +fi + _do_dump_restore # success, all done diff --git a/tests/xfs/068.out b/tests/xfs/068.out index fa3a552..f53c555 100644 --- a/tests/xfs/068.out +++ b/tests/xfs/068.out @@ -22,7 +22,7 @@ xfsrestore: session id: ID xfsrestore: media ID: ID xfsrestore: searching media for directory dump xfsrestore: reading directories -xfsrestore: 383 directories and 1335 entries processed +xfsrestore: XXX directories and YYY entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: restore complete: SECS seconds elapsed