From patchwork Wed May 17 22:36:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liu Bo X-Patchwork-Id: 9732063 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B954D603ED for ; Wed, 17 May 2017 23:37:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E74F287D6 for ; Wed, 17 May 2017 23:37:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 534DF287F4; Wed, 17 May 2017 23:37:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7AEDC287E9 for ; Wed, 17 May 2017 23:37:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754191AbdEQXg7 (ORCPT ); Wed, 17 May 2017 19:36:59 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:43749 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754039AbdEQXgy (ORCPT ); Wed, 17 May 2017 19:36:54 -0400 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v4HNaoo6024262 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 17 May 2017 23:36:50 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v4HNanwQ027818 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 17 May 2017 23:36:50 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v4HNanS7025473; Wed, 17 May 2017 23:36:49 GMT Received: from localhost.us.oracle.com (/10.211.47.181) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 17 May 2017 16:36:49 -0700 From: Liu Bo To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana , Eryu Guan Subject: [PATCH v2 5/5] fstests: regression test for nocsum buffered read's repair Date: Wed, 17 May 2017 16:36:10 -0600 Message-Id: <20170517223610.3916-6-bo.li.liu@oracle.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170517223610.3916-1-bo.li.liu@oracle.com> References: <20170517223610.3916-1-bo.li.liu@oracle.com> X-Source-IP: aserv0022.oracle.com [141.146.126.234] Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is to test whether buffered read retry-repair code is able to work in raid1 case as expected. Please note that without checksum, btrfs doesn't know if the data used to repair is correct, so repair is more of resync which makes sure that both of the copy has the same content. Commit 20a7db8ab3f2 ("btrfs: add dummy callback for readpage_io_failed and drop checks") introduced the regression. The upstream fix is Btrfs: bring back repair during read Signed-off-by: Liu Bo --- v2: - In order to make the tests deterministic, change to check pid for odd/even instead of doing grep with key words in dmesg tests/btrfs/143 | 150 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/143.out | 39 ++++++++++++++ tests/btrfs/group | 1 + 3 files changed, 190 insertions(+) create mode 100755 tests/btrfs/143 create mode 100644 tests/btrfs/143.out diff --git a/tests/btrfs/143 b/tests/btrfs/143 new file mode 100755 index 0000000..d62e5c7 --- /dev/null +++ b/tests/btrfs/143 @@ -0,0 +1,150 @@ +#! /bin/bash +# FS QA Test 143 +# +# Regression test for btrfs buffered read's repair during read without checksum. +# +# This is to test whether buffered read retry-repair code is able to work in +# raid1 case as expected. +# +# Please note that without checksum, btrfs doesn't know if the data used to +# repair is correct, so repair is more of resync which makes sure that both +# of the copy has the same content. +# +# Commit 20a7db8ab3f2 ("btrfs: add dummy callback for readpage_io_failed and drop +# checks") introduced the regression. +# +# The upstream fix is +# commit 9d0d1c8b1c9d ("Btrfs: bring back repair during read") +# +#----------------------------------------------------------------------- +# Copyright (c) 2017 Liu Bo. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs btrfs +_supported_os Linux +_require_fail_make_request +_require_scratch_dev_pool 2 + +_require_btrfs_command inspect-internal dump-tree +_require_command "$FILEFRAG_PROG" filefrag + +get_physical() +{ + # $1 is logical address + # print chunk tree and find devid 2 which is $SCRATCH_DEV + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \ + grep $1 -A 6 | awk '($1 ~ /stripe/ && $3 ~ /devid/ && $4 ~ /1/) { print $6 }' +} + +SYSFS_BDEV=`_sysfs_dev $SCRATCH_DEV` + +start_fail() +{ + echo 100 > $DEBUGFS_MNT/fail_make_request/probability + # the 1st one fails the first bio which is reading 4k (or more due to + # readahead), and the 2nd one fails the retry of validation so that it + # triggers read-repair + echo 2 > $DEBUGFS_MNT/fail_make_request/times + echo 0 > $DEBUGFS_MNT/fail_make_request/verbose + echo 1 > $SYSFS_BDEV/make-it-fail +} + +stop_fail() +{ + echo 0 > $DEBUGFS_MNT/fail_make_request/probability + echo 0 > $DEBUGFS_MNT/fail_make_request/times + echo 0 > $SYSFS_BDEV/make-it-fail +} + +_scratch_dev_pool_get 2 +# step 1, create a raid1 btrfs which contains one 128k file. +echo "step 1......mkfs.btrfs" >>$seqres.full + +mkfs_opts="-d raid1 -b 1G" +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 + +# -o nospace_cache makes sure data is written to the start position of the data +# chunk +_scratch_mount -o nospace_cache,nodatasum + +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" | _filter_xfs_io + +# step 2, corrupt the first 64k of one copy (on SCRATCH_DEV which is the first +# one in $SCRATCH_DEV_POOL +echo "step 2......corrupt file extent" >>$seqres.full + +${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full +logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1` +physical_on_scratch=`get_physical ${logical_in_btrfs}` + +_scratch_unmount +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $physical_on_scratch 64K" $SCRATCH_DEV | _filter_xfs_io + +_scratch_mount -o nospace_cache + +# step 3, 128k buffered read (this read can repair bad copy) +echo "step 3......repair the bad copy" >>$seqres.full + +# since raid1 consists of two copies, and the following read may read the good +# copy directly, so lets loop until repair gets triggered and discard output +# that buffered reads give +while true; do + # start_fail only fails the following buffered read so the repair is + # supposed to work. + echo 3 > /proc/sys/vm/drop_caches + start_fail + $XFS_IO_PROG -c "pread 0 4K" "$SCRATCH_MNT/foobar" > /dev/null & + pid=$! + wait + stop_fail + [ $((pid % 2)) == 1 ] && break +done + +_scratch_unmount + +# check if the repair works +$XFS_IO_PROG -c "pread -v -b 512 $physical_on_scratch 512" $SCRATCH_DEV | _filter_xfs_io + +_scratch_dev_pool_put +# success, all done +status=0 +exit diff --git a/tests/btrfs/143.out b/tests/btrfs/143.out new file mode 100644 index 0000000..97bf45c --- /dev/null +++ b/tests/btrfs/143.out @@ -0,0 +1,39 @@ +QA output created by 143 +wrote 131072/131072 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 136708096 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +08260000: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260010: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260020: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260030: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260040: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260050: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260060: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260070: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260080: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260090: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082600a0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082600b0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082600c0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082600d0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082600e0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082600f0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260100: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260110: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260120: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260130: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260140: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260150: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260160: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260170: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260180: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +08260190: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082601a0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082601b0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082601c0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082601d0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082601e0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +082601f0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................ +read 512/512 bytes at offset 136708096 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/btrfs/group b/tests/btrfs/group index a4f97cd..6f19619 100644 --- a/tests/btrfs/group +++ b/tests/btrfs/group @@ -144,3 +144,4 @@ 140 auto quick 141 auto quick 142 auto quick +143 auto quick