From patchwork Tue May 8 17:38:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 10386711 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E0C1C602C2 for ; Tue, 8 May 2018 17:38:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC72429036 for ; Tue, 8 May 2018 17:38:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C06AE2903D; Tue, 8 May 2018 17:38:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E82A729063 for ; Tue, 8 May 2018 17:38:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752869AbeEHRiO (ORCPT ); Tue, 8 May 2018 13:38:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:34024 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752068AbeEHRiN (ORCPT ); Tue, 8 May 2018 13:38:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1B0AE8D6EC; Tue, 8 May 2018 17:38:13 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.159]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02B86AFD72; Tue, 8 May 2018 17:38:13 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id A80AB2239C3; Tue, 8 May 2018 13:38:12 -0400 (EDT) Date: Tue, 8 May 2018 13:38:12 -0400 From: Vivek Goyal To: fstests@vger.kernel.org, linux-unionfs@vger.kernel.org Cc: Amir Goldstein , Miklos Szeredi , guaneryu@gmail.com Subject: [PATCH] xfstest: overlay: Add tests for overlay metadata only copy up feature Message-ID: <20180508173812.GH3501@redhat.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.9.1 (2017-09-22) X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 08 May 2018 17:38:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 08 May 2018 17:38:13 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'vgoyal@redhat.com' RCPT:'' Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add tests for proposed metadata only copy up feature in overlayfs. Signed-off-by: Vivek Goyal --- common/overlay | 1 tests/overlay/059 | 281 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/059.out | 42 +++++++ tests/overlay/group | 1 4 files changed, 325 insertions(+) -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: xfstests-dev/tests/overlay/059 =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ xfstests-dev/tests/overlay/059 2018-05-08 13:32:13.828919038 -0400 @@ -0,0 +1,281 @@ +#! /bin/bash +# FS QA Test No. 059 +# +# Test metadata only copy up functionality. +# +#----------------------------------------------------------------------- +# Copyright (C) 2018 Red Hat, Inc. All Rights Reserved. +# Author: Vivek Goyal +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +# We use non-default scratch underlying overlay dirs, we need to check +# them explicity after test. +_require_scratch_nocheck +_require_scratch_overlay_features redirect_dir + +# remove all files from previous tests +_scratch_mkfs + +# File size on lower +lowername="lowerfile" +lowerlink="lowerfile-link" +lowersize="16384" +lowerdata="lower" + +udirname="pureupper" +ufile="upperfile" + +# Check metacopy xattr +check_metacopy() +{ + local target=$1 exist=$2 + local out_f target_f + local msg + + out_f=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_XATTR_METACOPY $target 2>&1 | _filter_scratch) + + if [ "$exist" == "y" ];then + [ "$out_f" == "" ] && return + echo "Metacopy xattr does not exist on ${target}. stdout=$out_f" + return + fi + + if [ "$out_f" == "" ];then + echo "Metacopy xattr exists on ${target} unexpectedly." + return + fi + + target_f=`echo $target | _filter_scratch` + msg="$target_f: trusted.overlay.metacopy: No such attribute" + + [ "$out_f" == "$msg" ] && return + + echo "Error while checking xattr on ${target}. stdout=$out" +} + +# Check redirect xattr +check_redirect() +{ + local target=$1 + local expect=$2 + + value=$($GETFATTR_PROG --absolute-names --only-values -n \ + $OVL_XATTR_REDIRECT $target) + + [[ "$value" == "$expect" ]] || echo "Redirect xattr incorrect. Expected=\"$expect\", actual=\"$value\"" +} + +# Check size +check_size() +{ + local target=$1 expected_size=$2 + local size nr_blocks blocksize stat_data + + stat_data=$(stat -c "%s %b %B" $target) + size=`echo $stat_data | cut -d " " -f1` + nr_blocks=`echo $stat_data | cut -d " " -f2` + blocksize=`echo $stat_data | cut -d " " -f3` + + [ "$size" == "$expected_size" ] || echo "Expected file size $expected_size but actual size is $size" + + size=$(( $nr_blocks * $blocksize )) + + [ "$size" == "$expected_size" ] || echo "Expected file size $expected_size but actual size is ${size}. nr_blocks=$nr_blocks blocksize=$blocksize" +} + +check_contents() +{ + local target=$1 expected=$2 + local actual target_f + + target_f=`echo $target | _filter_scratch` + + read actual<$target + + [ "$actual" == "$expected" ] || echo "Expected file $target_f contents to be \"$expected\" but actual contents are \"$actual\"" +} + +check_file_properties() +{ + local target=$1 expected_size=$2 expected_content=$3 + + check_size $target $expected_size + check_contents $target $expected_content +} + +mount_overlay() +{ + local _lowerdir=$1 + + _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on +} + +umount_overlay() +{ + $UMOUNT_PROG $SCRATCH_MNT +} + +# Assumes it is called with overlay mounted. +test_common() +{ + local _lowerdir=$1 _target=$2 _size=$3 _data="$4" _redirect=$5 + + echo "check properties of metadata copied up file" + check_file_properties $SCRATCH_MNT/$_target $_size "$_data" + + # Do a mount cycle and check size and contents again. + echo "Unmount and Mount again" + umount_overlay + mount_overlay $_lowerdir + echo "check properties of metadata copied up file" + check_file_properties $SCRATCH_MNT/$_target $_size "$_data" + + # Make sure copied up file is a metacopy file. + umount_overlay + check_metacopy $upperdir/$_target "y" + [ -n "$_redirect" ] && check_redirect $upperdir/$_target "$_redirect" + + # Trigger data copy up and check absence of metacopy xattr. + mount_overlay $_lowerdir + $XFS_IO_PROG -c "open -a $SCRATCH_MNT/$_target" + echo "check properties of data copied up file" + check_file_properties $SCRATCH_MNT/$_target $_size "$_data" + umount_overlay + check_metacopy $upperdir/$_target "n" +} + +create_basic_files() +{ + _scratch_mkfs + mkdir -p $lowerdir $lowerdir2 $upperdir $workdir $workdir2 + mkdir -p $upperdir/$udirname + echo "$lowerdata" > $lowerdir/$lowername + chmod 600 $lowerdir/$lowername + # Create a file of size lowersize. + $XFS_IO_PROG -c "falloc 0 $lowersize" $lowerdir/$lowername +} + +create_lower_link() +{ + ln $lowerdir/$lowername $lowerdir/$lowerlink +} + +prepare_midlayer() +{ + _scratch_mkfs + create_basic_files + # Create midlayer + _overlay_scratch_mount_dirs $lowerdir $lowerdir2 $workdir2 -o redirect_dir=on,index=on,metacopy=on + # Trigger a metacopy + chmod 400 $SCRATCH_MNT/$lowername + umount_overlay +} + +# Create test directories +lowerdir=$OVL_BASE_SCRATCH_MNT/lower +lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2 +upperdir=$OVL_BASE_SCRATCH_MNT/upper +workdir=$OVL_BASE_SCRATCH_MNT/workdir +workdir2=$OVL_BASE_SCRATCH_MNT/workdir2 + +# Test simple metacopy +create_basic_files + +# Make sure metacopy feature is supported by kernel, otherwise do not +# run test. +mount_overlay $lowerdir || _notrun "mount overlay with metacopy failed" +umount_overlay + +# Tests start here +echo "== Check Simple Metacopy ==" +mount_overlay $lowerdir +chmod 400 $SCRATCH_MNT/$lowername +test_common $lowerdir $lowername $lowersize "$lowerdata" + +# Test midlayer metacopy +echo -e "\n== Check Midlayer Metacopy ==" +prepare_midlayer +mount_overlay "$lowerdir2:$lowerdir" +chmod 400 $SCRATCH_MNT/$lowername +test_common "$lowerdir2:$lowerdir" $lowername $lowersize "$lowerdata" + +# Test Rename Redirect +echo -e "\n== Check Rename Redirect ==" +create_basic_files +mount_overlay $lowerdir +mv $SCRATCH_MNT/$lowername $SCRATCH_MNT/$ufile +test_common $lowerdir $ufile $lowersize "$lowerdata" "$lowername" + +# Test midlayer rename redirect +echo -e "\n== Check Midlayer Rename Redirect ==" +prepare_midlayer +mount_overlay "$lowerdir2:$lowerdir" +mv $SCRATCH_MNT/$lowername $SCRATCH_MNT/$ufile +test_common "$lowerdir2:$lowerdir" $ufile $lowersize "$lowerdata" "$lowername" + +# Test Link Redirect +echo -e "\n== Check Link Redirect ==" +create_basic_files +mount_overlay $lowerdir +ln $SCRATCH_MNT/$lowername $SCRATCH_MNT/$udirname/$ufile +test_common $lowerdir $udirname/$ufile $lowersize "$lowerdata" "/$lowername" + +# Test midlayer link redirect +echo -e "\n== Check Midlayer Link Redirect ==" +prepare_midlayer +mount_overlay "$lowerdir2:$lowerdir" +ln $SCRATCH_MNT/$lowername $SCRATCH_MNT/$udirname/$ufile +test_common "$lowerdir2:$lowerdir" $udirname/$ufile $lowersize "$lowerdata" \ + "/$lowername" + +# Test lower link file gets absolute redirect upon rename +echo -e "\n== Check Lower Link Rename Absolute Redirect ==" +create_basic_files +create_lower_link +mount_overlay $lowerdir +mv $SCRATCH_MNT/$lowerlink $SCRATCH_MNT/$ufile +test_common $lowerdir $ufile $lowersize "$lowerdata" "/$lowerlink" + +# success, all done +status=0 +exit Index: xfstests-dev/common/overlay =================================================================== --- xfstests-dev.orig/common/overlay 2018-05-08 13:31:59.492919038 -0400 +++ xfstests-dev/common/overlay 2018-05-08 13:32:13.828919038 -0400 @@ -10,6 +10,7 @@ export OVL_XATTR_IMPURE="trusted.overlay export OVL_XATTR_ORIGIN="trusted.overlay.origin" export OVL_XATTR_NLINK="trusted.overlay.nlink" export OVL_XATTR_UPPER="trusted.overlay.upper" +export OVL_XATTR_METACOPY="trusted.overlay.metacopy" # helper function to do the actual overlayfs mount operation _overlay_mount_dirs() Index: xfstests-dev/tests/overlay/059.out =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ xfstests-dev/tests/overlay/059.out 2018-05-08 13:32:13.829919038 -0400 @@ -0,0 +1,42 @@ +QA output created by 059 +== Check Simple Metacopy == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Midlayer Metacopy == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Rename Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Midlayer Rename Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Link Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Midlayer Link Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file + +== Check Lower Link Rename Absolute Redirect == +check properties of metadata copied up file +Unmount and Mount again +check properties of metadata copied up file +check properties of data copied up file Index: xfstests-dev/tests/overlay/group =================================================================== --- xfstests-dev.orig/tests/overlay/group 2018-05-08 13:31:59.492919038 -0400 +++ xfstests-dev/tests/overlay/group 2018-05-08 13:32:13.829919038 -0400 @@ -61,3 +61,4 @@ 056 auto quick fsck 057 auto quick redirect 058 auto quick exportfs +059 auto quick