From patchwork Wed Apr 16 15:01:49 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 4001771 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 2C51B9F2BA for ; Wed, 16 Apr 2014 14:01:35 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E7A802034A for ; Wed, 16 Apr 2014 14:01:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D57EB20306 for ; Wed, 16 Apr 2014 14:01:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1161249AbaDPOB3 (ORCPT ); Wed, 16 Apr 2014 10:01:29 -0400 Received: from mail-wg0-f51.google.com ([74.125.82.51]:46907 "EHLO mail-wg0-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161077AbaDPOB2 (ORCPT ); Wed, 16 Apr 2014 10:01:28 -0400 Received: by mail-wg0-f51.google.com with SMTP id k14so10827780wgh.34 for ; Wed, 16 Apr 2014 07:01:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=U7YXgssESJ46C9XxdygTTZJs4iTdeq2FQ3rDmhVlpSQ=; b=od66bsI8ID1NnB5XTgQnDhe3WeP/NVjuND3Uz+EWGKZnwJQPqOKkHE8UuVEenmoMog 6cldyrFz0N6eOp8/4soWc1tnbsTWwH5LxGTCr3Y/8NC//LeG2gcIMGSGUDihTB4o9fLc rb78fwjtD7EOOoYwnNJXi+GA4mTCnpUCmfeWCmtiytqrovxkS1AiBJGZ4QZANJEBoJgL BmqamtKwKiuQXyA36Fo1smTgAmxM7MhfDWSOMCdW2r31z+uMzZGa2Ck7Sw+ahzBhdoSp +fRD2b/DWTF2TRORiWrcR1hHQAVrQ/m/fdxtdIRm0oDA1vg1pA0G1LIRMUQvcAZAkf0Q 5SOQ== X-Received: by 10.180.98.1 with SMTP id ee1mr19853693wib.10.1397656886798; Wed, 16 Apr 2014 07:01:26 -0700 (PDT) Received: from debian-vm3.lan (bl10-198-83.dsl.telepac.pt. [85.243.198.83]) by mx.google.com with ESMTPSA id ct2sm34478575wjb.33.2014.04.16.07.01.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 16 Apr 2014 07:01:25 -0700 (PDT) From: Filipe David Borba Manana To: xfs@oss.sgi.com Cc: linux-btrfs@vger.kernel.org, jbacik@fb.com, Filipe David Borba Manana Subject: [PATCH v2] xfstests: btrfs, test send's ability to punch holes and prealloc extents Date: Wed, 16 Apr 2014 16:01:49 +0100 Message-Id: <1397660509-8237-1-git-send-email-fdmanana@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1397580201-27475-1-git-send-email-fdmanana@gmail.com> References: <1397580201-27475-1-git-send-email-fdmanana@gmail.com> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This test verifies that after an incremental btrfs send the replicated file has the same exact hole and data structure as in the origin filesystem. This didn't use to be the case before the send stream version 2 - holes were sent as write operations of 0 valued bytes instead of punching holes with the fallocate system call, and pre-allocated extents were sent as well as write operations of 0 valued bytes instead of intructions for the receiver to use the fallocate system call. It also checks that prealloc extents that lie beyond the file's size are replicated by an incremental send. Signed-off-by: Filipe David Borba Manana --- V2: Addressed Dave's comments, and updated btrfs send invocation, by specifying the new command line option (-a) that enables use of fallocate - added function _require_btrfs_send_fallocate_flag() to skip the test when an old version of btrfs-progs is found. common/rc | 9 ++++ tests/btrfs/047 | 121 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/047.out | 27 ++++++++++++ tests/btrfs/group | 1 + 4 files changed, 158 insertions(+) create mode 100755 tests/btrfs/047 create mode 100644 tests/btrfs/047.out diff --git a/common/rc b/common/rc index acf419b..e94e51c 100644 --- a/common/rc +++ b/common/rc @@ -2262,6 +2262,15 @@ _run_btrfs_util_prog() run_check $BTRFS_UTIL_PROG $* } +_require_btrfs_send_fallocate_flag() +{ + $BTRFS_UTIL_PROG send 2>&1 | \ + grep '^[ \t]*\-a[ \t]\+.* fallocate ' > /dev/null 2>&1 + if [ $? -ne 0 ]; then + _notrun "Missing btrfs-progs send -a command line option, skipped this test" + fi +} + init_rc() { if [ "$iam" == new ] diff --git a/tests/btrfs/047 b/tests/btrfs/047 new file mode 100755 index 0000000..c8171a5 --- /dev/null +++ b/tests/btrfs/047 @@ -0,0 +1,121 @@ +#! /bin/bash +# FS QA Test No. btrfs/047 +# +# Verify that after an incremental btrfs send the replicated file has +# the same exact hole and data structure as in the origin filesystem. +# This didn't use to be the case before the send stream version 2 - +# holes were sent as write operations of 0 valued bytes instead of punching +# holes with the fallocate system call, and pre-allocated extents were sent +# as well as write operations of 0 valued bytes instead of intructions for +# the receiver to use the fallocate system call. Also check that prealloc +# extents that lie beyond the file's size are replicated by an incremental +# send. +# +# More specifically, this structure preserving guarantee was added by the +# following linux kernel commits: +# +# Btrfs: send, use fallocate command to punch holes +# Btrfs: send, use fallocate command to allocate extents +# +#----------------------------------------------------------------------- +# Copyright (c) 2014 Filipe Manana. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -fr $send_files_dir + rm -fr $tmp +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/punch + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch +_require_fssum +_require_xfs_io_fiemap +_require_btrfs_send_fallocate_flag +_need_to_be_root + +send_files_dir=$TEST_DIR/btrfs-test-$seq + +rm -f $seqres.full +rm -fr $send_files_dir +mkdir $send_files_dir + +_scratch_mkfs >/dev/null 2>&1 +_scratch_mount + +$XFS_IO_PROG -f -c "pwrite -S 0x01 -b 300000 0 300000" $SCRATCH_MNT/foo \ + | _filter_xfs_io + +_run_btrfs_util_prog subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/mysnap1 + +$XFS_IO_PROG -c "fpunch 100000 50000" $SCRATCH_MNT/foo +$XFS_IO_PROG -c "falloc 100000 50000" $SCRATCH_MNT/foo +$XFS_IO_PROG -c "pwrite -S 0xff -b 1000 120000 1000" $SCRATCH_MNT/foo \ + | _filter_xfs_io +$XFS_IO_PROG -c "fpunch 250000 20000" $SCRATCH_MNT/foo + +$XFS_IO_PROG -c "falloc -k 300000 1000000" $SCRATCH_MNT/foo +$XFS_IO_PROG -c "falloc -k 9000000 2000000" $SCRATCH_MNT/foo + +_run_btrfs_util_prog subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/mysnap2 + +_run_btrfs_util_prog send -a $SCRATCH_MNT/mysnap1 -f $send_files_dir/1.snap +_run_btrfs_util_prog send -a -p $SCRATCH_MNT/mysnap1 $SCRATCH_MNT/mysnap2 \ + -f $send_files_dir/2.snap + +md5sum $SCRATCH_MNT/mysnap2/foo | _filter_scratch +# List all hole and data segments. +$XFS_IO_PROG -r -c "seek -r -a 0" $SCRATCH_MNT/mysnap2/foo +# List all extents, we're interested here in prealloc extents that lie beyond +# the file's size. +$XFS_IO_PROG -r -c "fiemap -l" $SCRATCH_MNT/mysnap2/foo | _filter_fiemap + +_scratch_unmount +_check_scratch_fs + +_scratch_mkfs >/dev/null 2>&1 +_scratch_mount + +_run_btrfs_util_prog receive $SCRATCH_MNT -f $send_files_dir/1.snap +_run_btrfs_util_prog receive $SCRATCH_MNT -f $send_files_dir/2.snap + +md5sum $SCRATCH_MNT/mysnap2/foo | _filter_scratch +# List all hole and data segments. +$XFS_IO_PROG -r -c "seek -r -a 0" $SCRATCH_MNT/mysnap2/foo +# List all extents, we're interested here in prealloc extents that lie beyond +# the file's size. +$XFS_IO_PROG -r -c "fiemap -l" $SCRATCH_MNT/mysnap2/foo | _filter_fiemap + +_check_scratch_fs + +status=0 +exit diff --git a/tests/btrfs/047.out b/tests/btrfs/047.out new file mode 100644 index 0000000..f298b58 --- /dev/null +++ b/tests/btrfs/047.out @@ -0,0 +1,27 @@ +QA output created by 047 +wrote 300000/300000 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1000/1000 bytes at offset 120000 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +5a88aa0da62c8999c16515050947a70a SCRATCH_MNT/mysnap2/foo +Whence Result +DATA 0 +HOLE 102400 +DATA 118784 +HOLE 122880 +DATA 147456 +HOLE 253952 +DATA 266240 +HOLE 300000 +0: [496..17575]: hole +5a88aa0da62c8999c16515050947a70a SCRATCH_MNT/mysnap2/foo +Whence Result +DATA 0 +HOLE 102400 +DATA 118784 +HOLE 122880 +DATA 147456 +HOLE 253952 +DATA 266240 +HOLE 300000 +0: [496..17575]: hole diff --git a/tests/btrfs/group b/tests/btrfs/group index 3a6d34e..d4e3fc1 100644 --- a/tests/btrfs/group +++ b/tests/btrfs/group @@ -49,3 +49,4 @@ 044 auto quick 045 auto quick 046 auto quick +047 auto quick