From patchwork Fri Jun 1 12:59:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 10443301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3B658602BC for ; Fri, 1 Jun 2018 13:00:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 271B928D8F for ; Fri, 1 Jun 2018 13:00:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1A20328D96; Fri, 1 Jun 2018 13:00:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3365228D95 for ; Fri, 1 Jun 2018 13:00:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751068AbeFANAA (ORCPT ); Fri, 1 Jun 2018 09:00:00 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37492 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750850AbeFAM77 (ORCPT ); Fri, 1 Jun 2018 08:59:59 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0936F818BAEE; Fri, 1 Jun 2018 12:59:59 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.234]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAF0B20244E0; Fri, 1 Jun 2018 12:59:58 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 7E2872239B4; Fri, 1 Jun 2018 08:59:58 -0400 (EDT) Date: Fri, 1 Jun 2018 08:59:58 -0400 From: Vivek Goyal To: fstests@vger.kernel.org, linux-unionfs@vger.kernel.org Cc: Amir Goldstein , miklos@szeredi.hu, guaneryu@gmail.com Subject: [PATCH V4] xfstest: overlay: Add tests for overlay metadata only copy up feature Message-ID: <20180601125958.GB3259@redhat.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.9.1 (2017-09-22) X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 01 Jun 2018 12:59:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 01 Jun 2018 12:59:59 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'vgoyal@redhat.com' RCPT:'' Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi, Find attached V4 of the patch. I have removed hard coded assumptions about number of blocks on lower. Instead now I query for number of blocks allocated by filesystem on lower. Add tests for metadata only copy up feature. Signed-off-by: Vivek Goyal Reviewed-by: Amir Goldstein --- common/overlay | 1 tests/overlay/060 | 290 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/060.out | 42 +++++++ tests/overlay/group | 1 4 files changed, 334 insertions(+) -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: xfstests-dev/tests/overlay/060 =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ xfstests-dev/tests/overlay/060 2018-06-01 08:54:03.779228635 -0400 @@ -0,0 +1,290 @@ +#! /bin/bash +# FS QA Test No. 060 +# +# Test metadata only copy up functionality. +# +#----------------------------------------------------------------------- +# Copyright (C) 2018 Red Hat, Inc. All Rights Reserved. +# Author: Vivek Goyal +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +# We use non-default scratch underlying overlay dirs, we need to check +# them explicity after test. +_require_scratch_nocheck +_require_scratch_overlay_features index redirect_dir metacopy + +# remove all files from previous tests +_scratch_mkfs + +# File size on lower +lowername="lowerfile" +lowerlink="lowerfile-link" +lowerdata="lower" +lowersize="4096" + +# Number of blocks allocated by filesystem on lower. Will be queried later. +lowerblocks="" + +udirname="pureupper" +ufile="upperfile" + +# Check metacopy xattr +check_metacopy() +{ + local target=$1 exist=$2 + local out_f target_f + local msg + + out_f=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_XATTR_METACOPY $target 2>&1 | _filter_scratch) + + if [ "$exist" == "y" ];then + [ "$out_f" == "" ] && return + echo "Metacopy xattr does not exist on ${target}. stdout=$out_f" + return + fi + + if [ "$out_f" == "" ];then + echo "Metacopy xattr exists on ${target} unexpectedly." + return + fi + + target_f=`echo $target | _filter_scratch` + msg="$target_f: trusted.overlay.metacopy: No such attribute" + + [ "$out_f" == "$msg" ] && return + + echo "Error while checking xattr on ${target}. stdout=$out" +} + +# Check redirect xattr +check_redirect() +{ + local target=$1 + local expect=$2 + + value=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_XATTR_REDIRECT $target) + + [[ "$value" == "$expect" ]] || echo "Redirect xattr incorrect. Expected=\"$expect\", actual=\"$value\"" +} + +# Check size +check_file_size() +{ + local target=$1 expected_size=$2 actual_size + + actual_size=$(stat -c "%s" $target) + + [ "$actual_size" == "$expected_size" ] || echo "Expected file size $expected_size but actual size is $actual_size" +} + +check_file_blocks() +{ + local target=$1 expected_blocks=$2 nr_blocks + + nr_blocks=$(stat -c "%b" $target) + + [ "$nr_blocks" == "$expected_blocks" ] || echo "Expected $expected_blocks blocks but actual number of blocks is ${nr_blocks}." +} + +check_file_contents() +{ + local target=$1 expected=$2 + local actual target_f + + target_f=`echo $target | _filter_scratch` + + read actual<$target + + [ "$actual" == "$expected" ] || echo "Expected file $target_f contents to be \"$expected\" but actual contents are \"$actual\"" +} + +check_file_size_contents() +{ + local target=$1 expected_size=$2 expected_content=$3 + + check_file_size $target $expected_size + check_file_contents $target $expected_content +} + +mount_overlay() +{ + local _lowerdir=$1 + + _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on +} + +umount_overlay() +{ + $UMOUNT_PROG $SCRATCH_MNT +} + +# Assumes it is called with overlay mounted. +test_common() +{ + local _lowerdir=$1 _target=$2 _size=$3 _blocks=$4 _data="$5" + local _redirect=$6 + + echo "check properties of metadata copied up file" + check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" + check_file_blocks $SCRATCH_MNT/$_target $_blocks + + # Do a mount cycle and check size and contents again. + echo "Unmount and Mount again" + umount_overlay + mount_overlay $_lowerdir + echo "check properties of metadata copied up file" + check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" + check_file_blocks $SCRATCH_MNT/$_target $_blocks + + # Make sure copied up file is a metacopy file. + umount_overlay + check_metacopy $upperdir/$_target "y" + check_file_size_contents $upperdir/$_target $_size "" + [ -n "$_redirect" ] && check_redirect $upperdir/$_target "$_redirect" + + # Trigger data copy up and check absence of metacopy xattr. + mount_overlay $_lowerdir + $XFS_IO_PROG -c "open -a $SCRATCH_MNT/$_target" + echo "check properties of data copied up file" + check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data" + umount_overlay + check_metacopy $upperdir/$_target "n" + check_file_size_contents $upperdir/$_target $_size "$_data" +} + +create_basic_files() +{ + _scratch_mkfs + mkdir -p $lowerdir $lowerdir2 $upperdir $workdir $workdir2 + mkdir -p $upperdir/$udirname + echo "$lowerdata" > $lowerdir/$lowername + chmod 600 $lowerdir/$lowername + # Create a file of size lowersize. + $XFS_IO_PROG -c "falloc 0 $lowersize" $lowerdir/$lowername + $XFS_IO_PROG -c "fsync" $lowerdir/$lowername + + # Query number of lower + lowerblocks=$(stat -c "%b" $lowerdir/$lowername) +} + +create_lower_link() +{ + ln $lowerdir/$lowername $lowerdir/$lowerlink +} + +prepare_midlayer() +{ + _scratch_mkfs + create_basic_files + # Create midlayer + _overlay_scratch_mount_dirs $lowerdir $lowerdir2 $workdir2 -o redirect_dir=on,index=on,metacopy=on + # Trigger a metacopy + chmod 400 $SCRATCH_MNT/$lowername + umount_overlay +} + +# Create test directories +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2 +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir +workdir2=$OVL_BASE_SCRATCH_MNT/workdir2 + +# Tests start here +echo "== Check Simple Metacopy ==" +create_basic_files +mount_overlay $lowerdir +chmod 400 $SCRATCH_MNT/$lowername +test_common $lowerdir $lowername $lowersize $lowerblocks "$lowerdata" + +# Test midlayer metacopy +echo -e "\n== Check Midlayer Metacopy ==" +prepare_midlayer +mount_overlay "$lowerdir2:$lowerdir" +chmod 400 $SCRATCH_MNT/$lowername +test_common "$lowerdir2:$lowerdir" $lowername $lowersize $lowerblocks \ + "$lowerdata" + +# Test Rename Redirect +echo -e "\n== Check Rename Redirect ==" +create_basic_files +mount_overlay $lowerdir +mv $SCRATCH_MNT/$lowername $SCRATCH_MNT/$ufile +test_common $lowerdir $ufile $lowersize $lowerblocks "$lowerdata" "$lowername" + +# Test midlayer rename redirect +echo -e "\n== Check Midlayer Rename Redirect ==" +prepare_midlayer +mount_overlay "$lowerdir2:$lowerdir" +mv $SCRATCH_MNT/$lowername $SCRATCH_MNT/$ufile +test_common "$lowerdir2:$lowerdir" $ufile $lowersize $lowerblocks "$lowerdata" \ + "$lowername" + +# Test Link Redirect +echo -e "\n== Check Link Redirect ==" +create_basic_files +mount_overlay $lowerdir +ln $SCRATCH_MNT/$lowername $SCRATCH_MNT/$udirname/$ufile +test_common $lowerdir $udirname/$ufile $lowersize $lowerblocks "$lowerdata" \ + "/$lowername" + +# Test midlayer link redirect +echo -e "\n== Check Midlayer Link Redirect ==" +prepare_midlayer +mount_overlay "$lowerdir2:$lowerdir" +ln $SCRATCH_MNT/$lowername $SCRATCH_MNT/$udirname/$ufile +test_common "$lowerdir2:$lowerdir" $udirname/$ufile $lowersize $lowerblocks \ + "$lowerdata" "/$lowername" + +# Test lower link file gets absolute redirect upon rename +echo -e "\n== Check Lower Link Rename Absolute Redirect ==" +create_basic_files +create_lower_link +mount_overlay $lowerdir +mv $SCRATCH_MNT/$lowerlink $SCRATCH_MNT/$ufile +test_common $lowerdir $ufile $lowersize $lowerblocks "$lowerdata" "/$lowerlink" + +# success, all done +status=0 +exit Index: xfstests-dev/common/overlay =================================================================== --- xfstests-dev.orig/common/overlay 2018-05-29 15:34:50.772714715 -0400 +++ xfstests-dev/common/overlay 2018-05-29 15:34:52.431714715 -0400 @@ -10,6 +10,7 @@ export OVL_XATTR_IMPURE="trusted.overlay export OVL_XATTR_ORIGIN="trusted.overlay.origin" export OVL_XATTR_NLINK="trusted.overlay.nlink" export OVL_XATTR_UPPER="trusted.overlay.upper" +export OVL_XATTR_METACOPY="trusted.overlay.metacopy" # helper function to do the actual overlayfs mount operation _overlay_mount_dirs() Index: xfstests-dev/tests/overlay/060.out =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ xfstests-dev/tests/overlay/060.out 2018-05-29 15:34:52.431714715 -0400 @@ -0,0 +1,42 @@ +QA output created by 060 +== Check Simple Metacopy == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Midlayer Metacopy == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Rename Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Midlayer Rename Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Link Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Midlayer Link Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Lower Link Rename Absolute Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file Index: xfstests-dev/tests/overlay/group =================================================================== --- xfstests-dev.orig/tests/overlay/group 2018-05-29 15:34:52.432714715 -0400 +++ xfstests-dev/tests/overlay/group 2018-05-29 15:35:13.736714715 -0400 @@ -62,3 +62,4 @@ 057 auto quick redirect 058 auto quick exportfs 059 auto quick copyup +060 auto quick metacopy