From patchwork Tue Jun 18 21:07:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11002807 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA347112C for ; Tue, 18 Jun 2019 21:07:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9DFD6212E8 for ; Tue, 18 Jun 2019 21:07:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 92C7428862; Tue, 18 Jun 2019 21:07:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3290627F4B for ; Tue, 18 Jun 2019 21:07:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731071AbfFRVHU (ORCPT ); Tue, 18 Jun 2019 17:07:20 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:38140 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731068AbfFRVHN (ORCPT ); Tue, 18 Jun 2019 17:07:13 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL4Fge045604; Tue, 18 Jun 2019 21:07:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=R7eHZdFo5VpeykbIJ+fHXXz1JWICxA6vjXnC+u8sv8M=; b=D4qdrMqWo1kOAL/Go0lcpeO/iK1USHAi8c/sgV+W6ybWpTVIjtXxf9InuwGblM90qf56 lkF7J1TS8/sPlNFuRMNvePpgG98w+xtkKfNwpm7VVm2jvmmZaGjG+K4krEfcpUw/HxwC pftpZ7V/tUppBaFdnCs7pIYciB7DT/jCuHNmfrhNPlFmeG7FnD2d0/XlwALI+6PC8dNn H+AIZlmIXe973BJRNXVNkUEYl0LB1WajOfAt8TsrjjXnClpSjcoiXDShpQbG8H2/8bR2 BQbdeCnWnPvokvzfNd9dAzzsIfdV8nyh/twUelyRttPtjm9Tv1/VFY5QlxHc/tB0dMcy FA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 2t4saqertb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:11 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL6R7i061269; Tue, 18 Jun 2019 21:07:10 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 2t5mgc6mku-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:10 +0000 Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x5IL7APT000793; Tue, 18 Jun 2019 21:07:10 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 18 Jun 2019 14:07:09 -0700 Subject: [PATCH 1/4] dump: _cleanup_dump should only check the scratch fs if the test required it From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 18 Jun 2019 14:07:08 -0700 Message-ID: <156089202883.345809.17656192140244878661.stgit@magnolia> In-Reply-To: <156089201978.345809.17444450351199726553.stgit@magnolia> References: <156089201978.345809.17444450351199726553.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong _cleanup_dump always tries to check the scratch fs, even if the caller didn't actually _require_scratch. If a previous test wrote garbage to the scratch device then the dump test will fail here when repair stumbles over the garbage. This was observed by running xfs/016 and xfs/036 in succession. xfs/016 writes 0xc6 to the scratch device and tries to format a small log. If the log is too small the format fails and the test will _notrun. The subsequent xfs/036 will _notrun and then _cleanup_dump if no tape device is set, at which point we try to check the scratch device and logprint aborts due to the abnormal log size (0xc6c6c6c6). Signed-off-by: Darrick J. Wong Reviewed-by: Allison Collins --- common/dump | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/common/dump b/common/dump index 7c4c9cd8..2b8e0893 100644 --- a/common/dump +++ b/common/dump @@ -250,7 +250,7 @@ _cleanup_dump() mv $dir.$seq $dir done - if [ $status -ne $NOTRUNSTS ]; then + if [ -f ${RESULT_DIR}/require_scratch ] && [ $status -ne $NOTRUNSTS ]; then # Sleep added to stop _check_scratch_fs from complaining that the # scratch_dev is still busy sleep 10 From patchwork Tue Jun 18 21:07:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11002831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18543924 for ; Tue, 18 Jun 2019 21:07:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09FF4212E8 for ; Tue, 18 Jun 2019 21:07:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E7AEA27F4B; Tue, 18 Jun 2019 21:07:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7FF9B22F3E for ; Tue, 18 Jun 2019 21:07:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730869AbfFRVHX (ORCPT ); Tue, 18 Jun 2019 17:07:23 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:54868 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731068AbfFRVHX (ORCPT ); Tue, 18 Jun 2019 17:07:23 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL43oL143790; Tue, 18 Jun 2019 21:07:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=ZaTpbmM0TGO+VrZ+A34FSGarLjneylHsUL3BmqFp2e0=; b=zTmacvvLnAMps8aiQ2UWfVZ0X2/utvXVBxnHsYUoyJHNgJcB61cj2+aU+KRX3qzWUu5h 4t6xBsMEK+r/B6S97CcNfRlUPkn9QB3QCxUpO5vyJnjsNmHwN0DygEubZHF/xg+o4HBv IuA/0ij4sCEmag06zJrRmci4KAKxyW+6FmujUo/5HWVKa8ePq4Rl4UrTCrP1jiKrFT+O XgVyFnJl39vQCWmpM/WD8ysRVtq2Zt7zvelj34XxwuixiQoedpKKVPT7g6OlsbqidlCv KD+9ePvzBxQ3MT+c+EgsAgYwIbL1GW4lQ7ZVuMHeyfg8sVh76tdRqcnERPWJb+qyu27S tg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 2t4rmp6t0n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:18 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL78jM128828; Tue, 18 Jun 2019 21:07:17 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3020.oracle.com with ESMTP id 2t5h5tyqn4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:17 +0000 Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5IL7G4t023052; Tue, 18 Jun 2019 21:07:16 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 18 Jun 2019 14:07:16 -0700 Subject: [PATCH 2/4] xfs: rework min log size helper From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 18 Jun 2019 14:07:15 -0700 Message-ID: <156089203509.345809.3448903728041546348.stgit@magnolia> In-Reply-To: <156089201978.345809.17444450351199726553.stgit@magnolia> References: <156089201978.345809.17444450351199726553.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong The recent _scratch_find_xfs_min_logblocks helper has a major thinko in it -- it relies on feeding a too-small size to _scratch_do_mkfs so that mkfs will tell us the minimum log size. Unfortunately, _scratch_do_mkfs will see that first failure and retry the mkfs without MKFS_OPTIONS, which means that we return the minimum log size for the default mkfs settings without MKFS_OPTIONS. This is a problem if someone's running fstests with a set of MKFS_OPTIONS that affects the minimum log size. To fix this, open-code the _scratch_do_mkfs retry behavior so that we only do the "retry without MKFS_OPTIONS" behavior if the mkfs failed for a reason other than the minimum log size check. Signed-off-by: Darrick J. Wong --- common/rc | 13 ++++++++++--- common/xfs | 23 +++++++++++++++++++++-- 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/common/rc b/common/rc index 25203bb4..a38b7f02 100644 --- a/common/rc +++ b/common/rc @@ -438,6 +438,14 @@ _scratch_mkfs_options() echo $SCRATCH_OPTIONS $MKFS_OPTIONS $* $SCRATCH_DEV } +# Format the scratch device directly. First argument is the mkfs command. +# Second argument are all the parameters. stdout goes to $tmp.mkfsstd and +# stderr goes to $tmp.mkfserr. +__scratch_do_mkfs() +{ + eval "$1 $2 $SCRATCH_DEV" 2>$tmp.mkfserr 1>$tmp.mkfsstd +} + # Do the actual mkfs work on SCRATCH_DEV. Firstly mkfs with both MKFS_OPTIONS # and user specified mkfs options, if that fails (due to conflicts between mkfs # options), do a second mkfs with only user provided mkfs options. @@ -456,8 +464,7 @@ _scratch_do_mkfs() # save mkfs output in case conflict means we need to run again. # only the output for the mkfs that applies should be shown - eval "$mkfs_cmd $MKFS_OPTIONS $extra_mkfs_options $SCRATCH_DEV" \ - 2>$tmp.mkfserr 1>$tmp.mkfsstd + __scratch_do_mkfs "$mkfs_cmd" "$MKFS_OPTIONS $extra_mkfs_options" mkfs_status=$? # a mkfs failure may be caused by conflicts between $MKFS_OPTIONS and @@ -471,7 +478,7 @@ _scratch_do_mkfs() ) >> $seqres.full # running mkfs again. overwrite previous mkfs output files - eval "$mkfs_cmd $extra_mkfs_options $SCRATCH_DEV" \ + __scratch_do_mkfs "$mkfs_cmd" "$extra_mkfs_options" \ 2>$tmp.mkfserr 1>$tmp.mkfsstd mkfs_status=$? fi diff --git a/common/xfs b/common/xfs index f8dafc6c..8733e2ae 100644 --- a/common/xfs +++ b/common/xfs @@ -87,16 +87,33 @@ _scratch_find_xfs_min_logblocks() # minimum log size. local XFS_MIN_LOG_BYTES=2097152 - _scratch_do_mkfs "$mkfs_cmd" "cat" $* -N -l size=$XFS_MIN_LOG_BYTES \ - 2>$tmp.mkfserr 1>$tmp.mkfsstd + # Try formatting the filesystem with all the options given and the + # minimum log size. We hope either that this succeeds or that mkfs + # tells us the required minimum log size for the feature set. + # + # We cannot use _scratch_do_mkfs because it will retry /any/ failed + # mkfs with MKFS_OPTIONS removed even if the only "failure" was that + # the log was too small. + local extra_mkfs_options="$* -N -l size=$XFS_MIN_LOG_BYTES" + __scratch_do_mkfs "$mkfs_cmd" "$MKFS_OPTIONS $extra_mkfs_options" local mkfs_status=$? + # If the format fails for a reason other than the log being too small, + # try again without MKFS_OPTIONS because that's what _scratch_do_mkfs + # will do if we pass in the log size option. + if [ $mkfs_status -ne 0 ] && + ! grep -q 'log size.*too small, minimum' $tmp.mkfserr; then + __scratch_do_mkfs "$mkfs_cmd" "$extra_mkfs_options" + local mkfs_status=$? + fi + # mkfs suceeded, so we must pick out the log block size to do the # unit conversion if [ $mkfs_status -eq 0 ]; then local blksz="$(grep '^log.*bsize' $tmp.mkfsstd | \ sed -e 's/log.*bsize=\([0-9]*\).*$/\1/g')" echo $((XFS_MIN_LOG_BYTES / blksz)) + rm -f $tmp.mkfsstd $tmp.mkfserr return fi @@ -104,6 +121,7 @@ _scratch_find_xfs_min_logblocks() if grep -q 'minimum size is' $tmp.mkfserr; then grep 'minimum size is' $tmp.mkfserr | \ sed -e 's/^.*minimum size is \([0-9]*\) blocks/\1/g' + rm -f $tmp.mkfsstd $tmp.mkfserr return fi @@ -111,6 +129,7 @@ _scratch_find_xfs_min_logblocks() echo "Cannot determine minimum log size" >&2 cat $tmp.mkfsstd >> $seqres.full cat $tmp.mkfserr >> $seqres.full + rm -f $tmp.mkfsstd $tmp.mkfserr } _scratch_mkfs_xfs() From patchwork Tue Jun 18 21:07:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11002817 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD5BD924 for ; Tue, 18 Jun 2019 21:07:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99F44212E8 for ; Tue, 18 Jun 2019 21:07:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8DD6E288E0; Tue, 18 Jun 2019 21:07:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 03971212E8 for ; Tue, 18 Jun 2019 21:07:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731119AbfFRVH1 (ORCPT ); Tue, 18 Jun 2019 17:07:27 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:34902 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730794AbfFRVH1 (ORCPT ); Tue, 18 Jun 2019 17:07:27 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL4Epu034895; Tue, 18 Jun 2019 21:07:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=PgQv0Bxuz8ap1iForh/dPXBQ3mVwr7eIIWVPaZY0VdQ=; b=W2/dlHuEwTyylv6w8DW3Id4KYszh+azeRJ+B0sWpGuGnEm9cEWf4u1FVVTQ3gYwvZpCq nz7ZQNcyMeo0AGdJJqqUsrJPBtau3ODgO0SMaoU7nk6/wOkbKbdaTMGiJrBo/bmRKQ8a vTqeSedjkXxyfXmppUxtBugEoT09HKqlCJ2yuwx5CpXLTHJxziKl9kUU7N6rWjoMvGVN ppkA0d5oWgj5BRtmdNMLBdL/nM1vb0gZROQe3YFO5SCr7tIzaBOKxJCOYp7wU9QGpufo vJm3xxiXdmnrvOxzBkin0TrgrGyTs3dLdmkety4mjX4uS0ECwpYk0vdJc6s9fdlUBPrS Og== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2130.oracle.com with ESMTP id 2t4r3tpuvu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:24 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL7CeF000768; Tue, 18 Jun 2019 21:07:24 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 2t5cpe9q5e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:23 +0000 Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5IL7Mv6023083; Tue, 18 Jun 2019 21:07:22 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 18 Jun 2019 14:07:22 -0700 Subject: [PATCH 3/4] xfs/016: calculate minimum log size and end locations From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 18 Jun 2019 14:07:21 -0700 Message-ID: <156089204146.345809.516891823391869532.stgit@magnolia> In-Reply-To: <156089201978.345809.17444450351199726553.stgit@magnolia> References: <156089201978.345809.17444450351199726553.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong xfs/016 looks for corruption in the log when the log wraps. However, it hardcodes the minimum log size and the "95%" point where it wants to start the "nudge and check for corruption" part of the test. New features require larger logs, which causes the test to fail when it can't mkfs with the smaller log size and when that 95% point doesn't put us within 20x "_log_traffic 2"s of the end of the log. Fix the first problem by using the new min log size helper and replace the 95% figure with an estimate of where we need to be to guarantee that the 20x loop wraps the log. Signed-off-by: Darrick J. Wong Reviewed-by: Allison Collins --- tests/xfs/016 | 50 ++++++++++++++++++++++++++++++++++++++------------ tests/xfs/016.out | 1 + 2 files changed, 39 insertions(+), 12 deletions(-) diff --git a/tests/xfs/016 b/tests/xfs/016 index 3407a4b1..aed37dca 100755 --- a/tests/xfs/016 +++ b/tests/xfs/016 @@ -44,10 +44,21 @@ _block_filter() _init() { + echo "*** determine log size" + local sz_mb=50 + local dsize="-d size=${sz_mb}m" + local lsize="-l size=$(_scratch_find_xfs_min_logblocks $dsize)b" + local force_opts="$dsize $lsize" + _scratch_mkfs_xfs $force_opts >> $seqres.full 2>&1 + + # set log_size and log_size_bb globally + log_size_bb=`_log_size` + log_size=$((log_size_bb * 512)) + echo "log_size_bb = $log_size_bb log_size = $log_size" >> $seqres.full + echo "*** reset partition" - $here/src/devzero -b 2048 -n 50 -v 198 $SCRATCH_DEV + $here/src/devzero -b 2048 -n $sz_mb -v 198 $SCRATCH_DEV # write 0xc6 echo "*** mkfs" - force_opts="-dsize=50m -lsize=$log_size" # # Do not discard blocks as we check for patterns in free space. # @@ -65,6 +76,9 @@ _init() . $tmp.mkfs [ $logsunit -ne 0 ] && \ _notrun "Cannot run this test using log MKFS_OPTIONS specified" + + # quotas generate extra log traffic so force it off + _qmount_option noquota } _log_traffic() @@ -157,6 +171,7 @@ _check_corrupt() # get standard environment, filters and checks . ./common/rc . ./common/filter +. ./common/quota # real QA test starts here _supported_fs xfs @@ -164,10 +179,6 @@ _supported_os Linux rm -f $seqres.full -# mkfs sizes -log_size=3493888 -log_size_bb=`expr $log_size / 512` - _require_scratch _init @@ -188,18 +199,29 @@ echo "log sunit = $lsunit" >>$seqres.full [ $head -eq 2 -o $head -eq $((lsunit/512)) ] || \ _fail "!!! unexpected initial log position $head vs. $((lsunit/512))" -# find how how many blocks per op for 100 ops +# find how how many blocks per op for 200 ops # ignore the fact that it will also include an unmount record etc... # this should be small overall echo " lots of traffic for sampling" >>$seqres.full -sample_size_ops=100 +sample_size_ops=200 _log_traffic $sample_size_ops head1=`_log_head` num_blocks=`expr $head1 - $head` blocks_per_op=`echo "scale=3; $num_blocks / $sample_size_ops" | bc` +echo "log position = $head1; old log position: $head" >> $seqres.full echo "blocks_per_op = $blocks_per_op" >>$seqres.full -num_expected_ops=`echo "$log_size_bb / $blocks_per_op" | bc` + +# Since this is a log wrapping test, it's critical to push the log head to +# the point where it will wrap around within twenty rounds of log traffic. +near_end_min=$(echo "$log_size_bb - (10 * $blocks_per_op / 1)" | bc) +echo "near_end_min = $near_end_min" >>$seqres.full + +# Estimate the number of ops needed to get the log head close to but not past +# near_end_min. We'd rather fall short and have to step our way closer to the +# end than run past the end. +num_expected_ops=$(( 8 * $(echo "$log_size_bb / $blocks_per_op" | bc) / 10)) echo "num_expected_ops = $num_expected_ops" >>$seqres.full + num_expected_to_go=`echo "$num_expected_ops - $sample_size_ops" | bc` echo "num_expected_to_go = $num_expected_to_go" >>$seqres.full @@ -208,13 +230,17 @@ _log_traffic $num_expected_to_go head=`_log_head` echo "log position = $head" >>$seqres.full -# e.g. 3891 -near_end_min=`echo "0.95 * $log_size_bb" | bc | sed 's/\..*//'` -echo "near_end_min = $near_end_min" >>$seqres.full +# If we fell short of near_end_min, step our way towards it. +while [ $head -lt $near_end_min ]; do + echo " bump traffic from $head towards $near_end_min" >> $seqres.full + _log_traffic 10 > /dev/null 2>&1 + head=$(_log_head) +done [ $head -gt $near_end_min -a $head -lt $log_size_bb ] || \ _fail "!!! unexpected near end log position $head" +# Try to wrap the log, checking for corruption with each advance. for c in `seq 0 20` do echo " little traffic" >>$seqres.full diff --git a/tests/xfs/016.out b/tests/xfs/016.out index f7844cdf..f4c8f88d 100644 --- a/tests/xfs/016.out +++ b/tests/xfs/016.out @@ -1,4 +1,5 @@ QA output created by 016 +*** determine log size *** reset partition Wrote 51200.00Kb (value 0xc6) *** mkfs From patchwork Tue Jun 18 21:07:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 11002823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C82EF112C for ; Tue, 18 Jun 2019 21:07:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8515212E8 for ; Tue, 18 Jun 2019 21:07:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC9C2288E6; Tue, 18 Jun 2019 21:07:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4FAEA212E8 for ; Tue, 18 Jun 2019 21:07:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731136AbfFRVHe (ORCPT ); Tue, 18 Jun 2019 17:07:34 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:34964 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730791AbfFRVHe (ORCPT ); Tue, 18 Jun 2019 17:07:34 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL4QUO034959; Tue, 18 Jun 2019 21:07:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=XD9x3x9ptC27CgjT6JyrJ5jlZjiLlyTRByn7Afz/F50=; b=P+rsWC7xrzMFqmjibxcrMOQbj50p/dSZCOkEwKVLiJ4Px+/oisgfmQN2JlPmTzsMvqDj JCjo7aB2wf8rXUiIyHZNIiK8eAjLw8oMCFB8PBLT/43LFThodOsZTAYBWeF4rlAFYp1I So2aGLU6rjRR4GviDm7OpJHff2tmaMJl93DTWJGdHSSAU4R7wlk5RkUMG+EZYyHsyBZn sqW6hngKDPNr3bEe8gp7absBGBWP/C6P3Hyvxy7kUiLWTbJah+QAJNnMEAhSIO2X1o1U BaIX0V8cNIAAo9sV3nkSCpfGCJSbvachY2mLezFoYYciSxYG7kfrT+k39hwCINswjqPr kw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 2t4r3tpuw5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:31 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5IL796w128942; Tue, 18 Jun 2019 21:07:31 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 2t5h5tyqqc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Jun 2019 21:07:31 +0000 Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5IL7Uvh028636; Tue, 18 Jun 2019 21:07:30 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 18 Jun 2019 14:07:30 -0700 Subject: [PATCH 4/4] xfs/119: fix MKFS_OPTIONS exporting From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 18 Jun 2019 14:07:27 -0700 Message-ID: <156089204777.345809.18314859473454869520.stgit@magnolia> In-Reply-To: <156089201978.345809.17444450351199726553.stgit@magnolia> References: <156089201978.345809.17444450351199726553.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9292 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906180167 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong This test originally exported its own MKFS_OPTIONS to force the tested filesystem config to the mkfs defaults + test-specific log size options. This overrides whatever the test runner might have set in MKFS_OPTIONS. In commit 2fd273886b525 ("xfs: refactor minimum log size formatting code") we fail to export our test-specific MKFS_OPTIONS before calculating the minimum log size, which leads to the wrong min log size being calculated once we fixed the helper to be smarter about mkfs options. Signed-off-by: Darrick J. Wong Reviewed-by: Allison Collins --- tests/xfs/119 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/xfs/119 b/tests/xfs/119 index 8825a5c3..f245a0a6 100755 --- a/tests/xfs/119 +++ b/tests/xfs/119 @@ -38,7 +38,8 @@ _require_scratch # this may hang sync -logblks=$(_scratch_find_xfs_min_logblocks -l version=2,su=64k) +export MKFS_OPTIONS="-l version=2,su=64k" +logblks=$(_scratch_find_xfs_min_logblocks) export MKFS_OPTIONS="-l version=2,size=${logblks}b,su=64k" export MOUNT_OPTIONS="-o logbsize=64k" _scratch_mkfs_xfs >/dev/null