From patchwork Wed Jul 5 12:02:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amir Goldstein X-Patchwork-Id: 9826433 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EFB41603B5 for ; Wed, 5 Jul 2017 12:02:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E783F27F54 for ; Wed, 5 Jul 2017 12:02:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC98B284C8; Wed, 5 Jul 2017 12:02:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5ACA627F54 for ; Wed, 5 Jul 2017 12:02:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752763AbdGEMCD (ORCPT ); Wed, 5 Jul 2017 08:02:03 -0400 Received: from mail-wm0-f68.google.com ([74.125.82.68]:33636 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752547AbdGEMCB (ORCPT ); Wed, 5 Jul 2017 08:02:01 -0400 Received: by mail-wm0-f68.google.com with SMTP id j85so31975328wmj.0; Wed, 05 Jul 2017 05:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ioSMnIRBfqrVG4cQpe4fX7vsKkJs7KrKopHQH7np8co=; b=pG3dszu6nqpbUAvtTyNLgefIuITmvXZbBnu0Ry4GOoKs4cBr9NlMBr18HT+ryxNVRL NRf6ZrIehqkEvzkBYFZdTYBfixnUNk1auMm/hV/KdtomwBRV1yQVl2QuxErQoezbS0GL y9fLJoTSLMcU2THSw2+MCvAvzmeugIfONzgpo7S+S+anDr0PSCoosMUauXyUiZJ1PGWT INILPTOwcQmuhJ64YzgWeCdwDYWLiNTKM72O7ikj/xV1CaPcz6cbi1SIaCVmm3b/UGlR oAjsWPzVSdhmLGMf/PP0loESrf9bdmgSfiW/W3X8ikd3T07mUQLI4zZevB+Jyi3Yet6H GTzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ioSMnIRBfqrVG4cQpe4fX7vsKkJs7KrKopHQH7np8co=; b=ndenwYZ0QKhb3izEYVbw+M6blhh7lFQaYSdxiW5b4Hh5lPzfLgCIO/8/MPJdlIpJMB HjEO7slq0tMku1Cj94hhWkT2vJct8wXMkc2xU9PEdZ9H2LK2vNkn7Ct+Amifk15qI0CT q2QFQMonSieYwx0p+ukQyvSQfNEqKMohcoA9Rcgq0z56nkLCUJZ0aAyaiawm/P5+Zfww M/C3QZyexCQ3xsz7xvDv1F2EUfN8rvo3Y0Z50R32TWUw/XzN7KJrgWV0fokcIIFUtcsQ rFoGKxpwbqHUsxvPdYp6rYbdu6sSm8aO5rwTdv9i+t6e9FQ/fxaS2OSQF/k3/hzIiQqR EKWA== X-Gm-Message-State: AKS2vOxEA0OBOcFlppKWx6CE48kcUC34w3yBYDDGQIs6fZzytSWqz8u4 ucVs1pwAxte4Fw== X-Received: by 10.28.134.11 with SMTP id i11mr31683450wmd.77.1499256114157; Wed, 05 Jul 2017 05:01:54 -0700 (PDT) Received: from amir-VirtualBox.ctera.local (bzq-166-168-31-246.red.bezeqint.net. [31.168.166.246]) by smtp.gmail.com with ESMTPSA id 143sm155578wmg.9.2017.07.05.05.01.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 05 Jul 2017 05:01:53 -0700 (PDT) From: Amir Goldstein To: Eryu Guan Cc: Miklos Szeredi , linux-unionfs@vger.kernel.org, fstests@vger.kernel.org Subject: [PATCH v3 5/7] overlay: test concurrent copy up of lower hardlinks Date: Wed, 5 Jul 2017 15:02:04 +0300 Message-Id: <1499256126-21866-6-git-send-email-amir73il@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1499256126-21866-1-git-send-email-amir73il@gmail.com> References: <1499256126-21866-1-git-send-email-amir73il@gmail.com> Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Two tasks make a modification concurrently on two hardlinks of a large lower inode. The copy up should be triggered by one of the tasks and the other should be waiting for copy up to complete. Both copy up targets should end up being upper hardlinks and both metadata changes should be visible in both hardlinks. With kernel <= v4.12, hardlinks are broken on copy up, meaning that copy up is performed independetly and the resulting upper copy up targets each have only one of the the metadata changes visible. Signed-off-by: Amir Goldstein --- tests/overlay/032 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/overlay/032.out | 4 ++ tests/overlay/group | 1 + 3 files changed, 114 insertions(+) create mode 100755 tests/overlay/032 create mode 100644 tests/overlay/032.out diff --git a/tests/overlay/032 b/tests/overlay/032 new file mode 100755 index 0000000..d986ef2 --- /dev/null +++ b/tests/overlay/032 @@ -0,0 +1,109 @@ +#! /bin/bash +# FS QA Test 032 +# +# Test concurrent copy up of lower hardlinks. +# +# Two tasks make a metadata change concurrently on two hardlinks of a large +# lower inode. The copy up should be triggers by one of the tasks and the +# other should be waiting for copy up to complete. Both copy up targets +# should end up being upper hardlinks and both metadata changes should be +# applied. +# +#----------------------------------------------------------------------- +# Copyright (C) 2017 CTERA Networks. All Rights Reserved. +# Author: Amir Goldstein +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs overlay +_supported_os Linux +_require_scratch + +# Remove all files from previous tests +_scratch_mkfs + +# overlay copy_up doesn't deal with sparse file well, holes will be filled by +# zeros, so if both hardlinks are broken on copy up, we need (2*1G) free space +# on $OVL_BASE_SCRATCH_MNT. +_require_fs_space $OVL_BASE_SCRATCH_MNT $((1024*1024*2)) + +# Create a large file in lower with 2 hardlinks. +# Make the file have non zero blocks, so copy up won't be able to do +# a naive sparse file copy up optimization. +lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER +mkdir -p $lowerdir +$XFS_IO_PROG -fc "pwrite 1g 4k" $lowerdir/zero >> $seqres.full +ln $lowerdir/zero $lowerdir/one +ln $lowerdir/zero $lowerdir/two + +_scratch_mount + +do_cmd() +{ + echo "`date +%T` $1..." >> $seqres.full + eval "$1" + echo "`date +%T` ...$1" >> $seqres.full +} + +# Perform one modification on each hardlink (size and owner) +do_cmd "echo >> $SCRATCH_MNT/one" & +# +# When hardlinks are broken and overlayfs supports concurrent copy up, +# $seqres.full will show that file two copy up started ~2s after file one +# copy up started and ended ~2s after file one copy up ended. +# With synchronized copy up of lower inodes, $seqres.full will show that +# file two copy up ended at the same time as file one copy up. +# Without sparse file copy up optimizations, copy of 1g on a standard disk +# is expected to take more than 2s. +# If underlying filesystem supports clone, overlay clone up with take less +# than 1s and this test will not be doing concurrent copy up of hardlinks, +# but rather consequent copy up of hardlinks. +# +sleep 2 +do_cmd "chown 100 $SCRATCH_MNT/two" & + +wait + +# Expect all hardlinks to show both metadata modifications (owner and size). +# List : +for f in zero one two; do + _ls_l -n $SCRATCH_MNT/$f | awk '{ print $2, $3, $5, $9 }' | _filter_scratch +done + +status=0 +exit diff --git a/tests/overlay/032.out b/tests/overlay/032.out new file mode 100644 index 0000000..a4cdc13 --- /dev/null +++ b/tests/overlay/032.out @@ -0,0 +1,4 @@ +QA output created by 032 +3 100 1073745921 SCRATCH_MNT/zero +3 100 1073745921 SCRATCH_MNT/one +3 100 1073745921 SCRATCH_MNT/two diff --git a/tests/overlay/group b/tests/overlay/group index 28df5b6..2baba3a 100644 --- a/tests/overlay/group +++ b/tests/overlay/group @@ -34,3 +34,4 @@ 029 auto quick 030 auto quick perms 031 auto quick whiteout +032 auto quick copyup hardlink