From patchwork Fri Feb 14 15:18:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mauricio Faria de Oliveira X-Patchwork-Id: 11382453 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D13F109A for ; Fri, 14 Feb 2020 15:18:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6CA1624670 for ; Fri, 14 Feb 2020 15:18:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729555AbgBNPS5 (ORCPT ); Fri, 14 Feb 2020 10:18:57 -0500 Received: from youngberry.canonical.com ([91.189.89.112]:54676 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729482AbgBNPS5 (ORCPT ); Fri, 14 Feb 2020 10:18:57 -0500 Received: from mail-qk1-f200.google.com ([209.85.222.200]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1j2ck2-00022A-Q1 for fstests@vger.kernel.org; Fri, 14 Feb 2020 15:18:54 +0000 Received: by mail-qk1-f200.google.com with SMTP id w126so916513qkb.23 for ; Fri, 14 Feb 2020 07:18:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bpn1DjbE2auZrikA/d7VQIODPI3wrxIMw6GlG2uZhVI=; b=Po2ryi1SNrKYkQpIzB0InT2mrh+S9CC/CIp+oaiQyHxdZdwSsG+ezAbll29V3z1OfG YxBOxmqwOsIea+AdRpK14YasDkwWL1wQmCZRa6KFDZQn7xT2QsjrrlRCgU3Q+BAHYuXP wJWcHFcUyD8bpHknsOaMXPEiELgLvXl7Qn2drmq6yvG/YPF62gxCtUJ2mvvwgfR+8cKx 4d1SWclzWwsjAAC2YVD2T80Phv1xH3Bg3BdVR5xrrXQnYgkBoZB/49C5wM/Bo2YGee9g t2ss+9kHVrDqyAeQ2zC7rVg+NCLohq9UFgvOXaoNdlNDSZraqdW4R5uie3IK/3bvMGKF w12Q== X-Gm-Message-State: APjAAAXDLjr6thz8PamaYd9EO+ZShoKQhz7GtLspBD+S+fsK5zncYgEg bHgrSb9hR4W4b7dQ2dePqMBNIkmUwQncvCcNQcU/zvOCvFkWuAROdEP0E0IU1wGDghzAlBcoCrA 6H4nLVZ2ZWRPrH30D4x1sxM1lVUqSmXcrKLU= X-Received: by 2002:ac8:195d:: with SMTP id g29mr2940191qtk.65.1581693533684; Fri, 14 Feb 2020 07:18:53 -0800 (PST) X-Google-Smtp-Source: APXvYqwSjpsTgwTbfmXgLXQ6udOZpV8PN8VNKbgXvjshMzyWP6vfINsGpT2YBwSbxGp7zE5faqNHBQ== X-Received: by 2002:ac8:195d:: with SMTP id g29mr2940172qtk.65.1581693533352; Fri, 14 Feb 2020 07:18:53 -0800 (PST) Received: from localhost.localdomain ([2804:14c:4e7:1017:a97b:ff5a:85e:d3eb]) by smtp.gmail.com with ESMTPSA id s22sm3403933qke.19.2020.02.14.07.18.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2020 07:18:53 -0800 (PST) From: Mauricio Faria de Oliveira To: fstests@vger.kernel.org Cc: Amir Goldstein Subject: [PATCH v2 1/5] common/overlay,rc,config: introduce OVL_FSTYP variable and aufs Date: Fri, 14 Feb 2020 12:18:44 -0300 Message-Id: <20200214151848.8328-2-mfo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200214151848.8328-1-mfo@canonical.com> References: <20200214151848.8328-1-mfo@canonical.com> MIME-Version: 1.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org Recently I was looking for an aufs test suite, and reached out to Okajima, but 'There is no public test suite specific to aufs.' [1], and it looks like 'xfstests/tests/generic' should be enough [1, 2]. Thus, building on top existing xfstests support for overlay just introduce the OVL_FSTYP variable, and the default value "overlay" can be changed to "aufs" (uses overlay's upperdir as a rw-branch and lowerdir as a ro-branch; workdir is not used.) This is indeed a workaround^W simple change that does the job vs. creating a new FSTYP "aufs" and mechanically changing the number of places that check for "overlay" to just handle "aufs" as well. (so the effort is still small as aufs has no specific tests now.) This also allows testing fuse-overlayfs with the next patches. The changes are minimal -- just translate overlay mount options and use $OVL_FSTYP as filesystem type for checking/mount/umount; then report it in log headers and document it in README.overlay. Currently, running './check -overlay' tests (excluding a few [3] which either hang or keep looping) the numbers for aufs on loop devices on v5.4-based Ubuntu kernel are: - Ran: 645 tests - Not run: 483 tests - Failures: 22 tests So, hopefully this may help with a starting point for an public test suite for aufs. Thanks to Amir Goldstein for feedback/improvements and pointers to support fuse-overlayfs as well [v2]. [1] https://sourceforge.net/p/aufs/mailman/message/36918721/ [2] https://sourceforge.net/p/aufs/mailman/message/36918932/ [3] Steps: $ export OVL_FSTYP=aufs $ export FSTYP=ext4 $ export TEST_DEV=/dev/loop0 $ export TEST_DIR=/mnt/test $ export SCRATCH_DEV=/dev/loop1 $ export SCRATCH_MNT=/mnt/scratch $ sudo mkfs.$FSTYP -F $TEST_DEV $ sudo mkfs.$FSTYP -F $SCRATCH_DEV $ sudo mkdir $TEST_DIR $SCRATCH_MNT $ cat </tmp/exclude-tests generic/013 generic/070 generic/075 generic/112 generic/127 generic/461 generic/476 generic/522 generic/530 overlay/019 EOF $ sudo -E ./check -overlay -E /tmp/exclude-tests Signed-off-by: Mauricio Faria de Oliveira --- README.overlay | 4 ++++ common/config | 2 ++ common/overlay | 11 ++++++++--- common/rc | 6 ++++++ 4 files changed, 20 insertions(+), 3 deletions(-) diff --git a/README.overlay b/README.overlay index 30b5ddb2d1c3..08a39b8830c9 100644 --- a/README.overlay +++ b/README.overlay @@ -50,3 +50,7 @@ In the example above, MOUNT_OPTIONS will be used to mount the base scratch fs, TEST_FS_MOUNT_OPTS will be used to mount the base test fs, OVERLAY_MOUNT_OPTIONS will be used to mount both test and scratch overlay and OVERLAY_FSCK_OPTIONS will be used to check both test and scratch overlay. + +To test other filesystem types (experimental) configure the OVL_FSTYP variable: + + OVL_FSTYP=aufs diff --git a/common/config b/common/config index 9a9c77602b54..d92a78003295 100644 --- a/common/config +++ b/common/config @@ -71,6 +71,8 @@ export OVL_LOWER="ovl-lower" export OVL_WORK="ovl-work" # overlay mount point parent must be the base fs root export OVL_MNT="ovl-mnt" +# overlay mount filesystem type (for testing other fs) +export OVL_FSTYP=${OVL_FSTYP:-overlay} # From e2fsprogs/e2fsck/e2fsck.h: # Exit code used by fsck-type programs diff --git a/common/overlay b/common/overlay index 65c639e9c6d8..a1076926c23f 100644 --- a/common/overlay +++ b/common/overlay @@ -18,10 +18,15 @@ _overlay_mount_dirs() local lowerdir=$1 local upperdir=$2 local workdir=$3 + local options shift 3 - $MOUNT_PROG -t overlay -o lowerdir=$lowerdir -o upperdir=$upperdir \ - -o workdir=$workdir `_common_dev_mount_options $*` + options="-o lowerdir=$lowerdir -o upperdir=$upperdir -o workdir=$workdir" + if [ "$OVL_FSTYP" = "aufs" ]; then + options="-o br=$upperdir=rw -o br=$lowerdir=ro" + fi + + $MOUNT_PROG -t $OVL_FSTYP $options `_common_dev_mount_options $*` } # Mount with same options/mnt/dev of scratch mount, but optionally @@ -302,7 +307,7 @@ _overlay_check_fs() _overlay_base_mount $* else # Check and umount overlay for dir check - ovl_mounted=`_is_dir_mountpoint $ovl_mnt` + ovl_mounted=`_is_dir_mountpoint $ovl_mnt $OVL_FSTYP` [ -z "$ovl_mounted" ] || $UMOUNT_PROG $ovl_mnt fi diff --git a/common/rc b/common/rc index b4a77a2187f4..1feae1a94f9e 100644 --- a/common/rc +++ b/common/rc @@ -1471,6 +1471,10 @@ _check_mounted_on() return 2 # 2 = mounted on wrong mnt fi + if [ -n "$type" -a "$type" = "overlay" ]; then + type="$OVL_FSTYP" + fi + if [ -n "$type" -a "`_fs_type $dev`" != "$type" ]; then echo "$devname=$dev is mounted but not a type $type filesystem" # raw $DF_PROG cannot handle NFS/CIFS/overlay correctly @@ -2841,6 +2845,8 @@ _full_fstyp_details() FSTYP="$FSTYP (non-debug)" fi fi + elif [ $FSTYP = "overlay" -a "$OVL_FSTYP" != "overlay" ]; then + FSTYP="$FSTYP ($OVL_FSTYP)" fi echo $FSTYP } From patchwork Fri Feb 14 15:18:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mauricio Faria de Oliveira X-Patchwork-Id: 11382455 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C792E109A for ; Fri, 14 Feb 2020 15:18:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A50942168B for ; Fri, 14 Feb 2020 15:18:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729516AbgBNPS7 (ORCPT ); Fri, 14 Feb 2020 10:18:59 -0500 Received: from youngberry.canonical.com ([91.189.89.112]:54679 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729546AbgBNPS7 (ORCPT ); Fri, 14 Feb 2020 10:18:59 -0500 Received: from mail-qt1-f198.google.com ([209.85.160.198]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1j2ck4-00022R-GQ for fstests@vger.kernel.org; Fri, 14 Feb 2020 15:18:56 +0000 Received: by mail-qt1-f198.google.com with SMTP id r30so1827171qtb.10 for ; Fri, 14 Feb 2020 07:18:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ME41u9Tp07L1X77zLOoHPTip6xACtBhTzDmaBFGHOOM=; b=asILEg3g1Tc3eHkG9g71F82IKveestQzCBBKJStrE0e+ox+4EjXr2mgveZDXf+NslX 3Yv9PGR85QgeGEhKj7g1EGg/LasnDbTX2nO3ZK4Xpak7bF0l+aJjrL+zYyC3tyG+ZZ8G XSrKeQoaHJ50MSoiL+gdKuB6DsnAK5CIsRsXhPsfu7c3N3QPHz8LbwaEY5MZO2UU9DR4 E8MX9VkCcfEY69V4M2rrotRI4WaEnb764wV9EK20MxRKhcwUgud+jaT8qqiKnLFzLfli AtNhmiXcAWtZgbvZePVK1v4FJXT2IhG8MNMy5ZbHHkFw7wDLkCRsSLT0Fwh/p2pWqFZu aIqw== X-Gm-Message-State: APjAAAX1+GCxOXM6brR7dTrTp4M3FQZo88D1ftLrV25u5ZVKOSm0TrJA q9WppBo6WFZiYqjvoKyAL+GvTXBWlrNQXaNSEAKff3xnv3sfycuTDECRBS+QgURH21jg2ZLe+ml kXWi6lS8aYxVarhVxL3vMX9OoaJ8e9LzPD/U= X-Received: by 2002:a0c:f193:: with SMTP id m19mr2470496qvl.154.1581693535378; Fri, 14 Feb 2020 07:18:55 -0800 (PST) X-Google-Smtp-Source: APXvYqzwDqdcbhHypUBGB7UZvecSR25qQwQby/UmpXUfXTNQxuXsUibMBbNf9N6i1Vgi/qa8vnhvDw== X-Received: by 2002:a0c:f193:: with SMTP id m19mr2470465qvl.154.1581693535051; Fri, 14 Feb 2020 07:18:55 -0800 (PST) Received: from localhost.localdomain ([2804:14c:4e7:1017:a97b:ff5a:85e:d3eb]) by smtp.gmail.com with ESMTPSA id s22sm3403933qke.19.2020.02.14.07.18.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2020 07:18:54 -0800 (PST) From: Mauricio Faria de Oliveira To: fstests@vger.kernel.org Cc: Amir Goldstein Subject: [PATCH v2 2/5] tests/overlay: mount: replace overlay hardcode with OVL_FSTYP variable Date: Fri, 14 Feb 2020 12:18:45 -0300 Message-Id: <20200214151848.8328-3-mfo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200214151848.8328-1-mfo@canonical.com> References: <20200214151848.8328-1-mfo@canonical.com> MIME-Version: 1.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org This allows other filesystem types to actually be used in these tests. Well, aufs does not support the options used in most of them anyway, so just let it fail to mount (instead of implementing common helpers for middle layers.) On fuse-overlayfs (coming) the options should be supported/compatible, and thus just work, so no further changes are needed. Suggested-by: Amir Goldstein Signed-off-by: Mauricio Faria de Oliveira Reviewed-by: Amir Goldstein --- tests/overlay/011 | 2 +- tests/overlay/035 | 2 +- tests/overlay/052 | 4 ++-- tests/overlay/053 | 4 ++-- tests/overlay/062 | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/overlay/011 b/tests/overlay/011 index 1d09341b250a..0b39416c9835 100755 --- a/tests/overlay/011 +++ b/tests/overlay/011 @@ -53,7 +53,7 @@ $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir # $upperdir overlaid on top of $lowerdir, so that "trusted.overlay.opaque" # xattr should be honored and should not be listed # mount readonly, because there's no upper and workdir -$MOUNT_PROG -t overlay -o ro -o lowerdir=$upperdir:$lowerdir $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT +$MOUNT_PROG -t $OVL_FSTYP -o ro -o lowerdir=$upperdir:$lowerdir $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT # Dump trusted.overlay xattr, we should not see the "opaque" xattr _getfattr -d -m overlay $SCRATCH_MNT/testdir diff --git a/tests/overlay/035 b/tests/overlay/035 index c0aae935bcf1..bbb158f319cd 100755 --- a/tests/overlay/035 +++ b/tests/overlay/035 @@ -52,7 +52,7 @@ mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir # Mount overlay with lower layers only. # Verify that overlay is mounted read-only and that it cannot be remounted rw. -$MOUNT_PROG -t overlay -o"lowerdir=$lowerdir2:$lowerdir1" \ +$MOUNT_PROG -t $OVL_FSTYP -o"lowerdir=$lowerdir2:$lowerdir1" \ $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT touch $SCRATCH_MNT/foo 2>&1 | _filter_scratch $MOUNT_PROG -o remount,rw $SCRATCH_MNT 2>&1 | _filter_ro_mount diff --git a/tests/overlay/052 b/tests/overlay/052 index b1cf0da64bbf..35a7b5f1a903 100755 --- a/tests/overlay/052 +++ b/tests/overlay/052 @@ -147,7 +147,7 @@ unmount_dirs # Check encode/decode/read of lower file handles on lower layers only r/o overlay. # For non-upper overlay mount, nfs_export requires disabling redirect_dir. -$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ +$MOUNT_PROG -t $OVL_FSTYP $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ -o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$middle:$lower test_file_handles $SCRATCH_MNT/lowertestdir -rp test_file_handles $SCRATCH_MNT/lowertestdir/subdir -rp @@ -158,7 +158,7 @@ unmount_dirs # Overlay lookup cannot follow the redirect from $upper/lowertestdir.new to # $lower/lowertestdir. Instead, we mount an overlay subtree rooted at these # directories. -$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ +$MOUNT_PROG -t $OVL_FSTYP $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ -o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$upper/lowertestdir.new:$lower/lowertestdir test_file_handles $SCRATCH_MNT -r test_file_handles $SCRATCH_MNT/subdir -rp diff --git a/tests/overlay/053 b/tests/overlay/053 index ff95424741ec..5ac19b32c3cb 100755 --- a/tests/overlay/053 +++ b/tests/overlay/053 @@ -169,7 +169,7 @@ unmount_dirs # Check encode/decode/read of lower file handles on lower layers only r/o overlay. # For non-upper overlay mount, nfs_export requires disabling redirect_dir. -$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ +$MOUNT_PROG -t $OVL_FSTYP $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ -o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$middle:$lower test_file_handles $SCRATCH_MNT/lowertestdir -rp test_file_handles $SCRATCH_MNT/lowertestdir/subdir -rp @@ -180,7 +180,7 @@ unmount_dirs # Overlay lookup cannot follow the redirect from $upper/lowertestdir.new to # $lower/lowertestdir. Instead, we mount an overlay subtree rooted at these # directories. -$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ +$MOUNT_PROG -t $OVL_FSTYP $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ -o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$upper/lowertestdir.new:$lower/lowertestdir test_file_handles $SCRATCH_MNT -r test_file_handles $SCRATCH_MNT/subdir -rp diff --git a/tests/overlay/062 b/tests/overlay/062 index 2c86a4b6fd1e..afd3562bfd33 100755 --- a/tests/overlay/062 +++ b/tests/overlay/062 @@ -72,7 +72,7 @@ create_test_files $lowertestdir $MOUNT_PROG --bind $lowertestdir $lowertestdir # For non-upper overlay mount, nfs_export requires disabling redirect_dir. -$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ +$MOUNT_PROG -t $OVL_FSTYP $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \ -o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$lower:$lower2 # Decode an overlay directory file handle, whose underlying lower dir dentry From patchwork Fri Feb 14 15:18:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mauricio Faria de Oliveira X-Patchwork-Id: 11382457 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BE1714E3 for ; Fri, 14 Feb 2020 15:19:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 147552168B for ; Fri, 14 Feb 2020 15:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729546AbgBNPS7 (ORCPT ); Fri, 14 Feb 2020 10:18:59 -0500 Received: from youngberry.canonical.com ([91.189.89.112]:54682 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729482AbgBNPS7 (ORCPT ); Fri, 14 Feb 2020 10:18:59 -0500 Received: from mail-qk1-f198.google.com ([209.85.222.198]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1j2ck5-00022s-VR for fstests@vger.kernel.org; Fri, 14 Feb 2020 15:18:58 +0000 Received: by mail-qk1-f198.google.com with SMTP id a132so6375908qkg.5 for ; Fri, 14 Feb 2020 07:18:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=83C2lNWz8o4Om2q4nHCqNIAoV15NBW+uboT5K2kj8rU=; b=WV5Eojzk+KVLonCH9sgdzFUMfpNDCPiu4Gi8CgfLXrO3g31uVl3LCV/seLSS2ZR9ig zEVSKB4wLq84OHIirPhjFkNdYciPCPRvghzRzx92qKoyz1nqC0LMEFgQq+0C70XrUkU7 5Gu1PaB22SIvC5pLSSBRQJCSap8APp6SIKJ3Hr6HKHXWqAj8xdIGE02deSI7a1yvtsim PpdJdSJUSGwdBLhOYyUdmMKEg8M+lN+roc7fsUqQpTuB9Hpqbt8gM6cS0/w8P2Y6Tuii 54UrOZ9m8mGC+d2ZT9XsRes8IjBmp+QImzHZtQFgVpY84QEEffiFlgAzrbcNa9tF3N6c Jgjw== X-Gm-Message-State: APjAAAVh7i3ht9bqCUQtvWaFcuCYZM67VMtm5JpXsirzTIlhLwZYiAGJ gqSfgoytF0tEeR3scn00+T1qqllStFE8fccF0RZyEEqLIEsa0gFGUkfqyqZOoucdzHMJ3/fsc3/ lq7j36bY20Q4ZHMND21TlhOQ+1fRO+wJSytc= X-Received: by 2002:a37:4e97:: with SMTP id c145mr2976556qkb.253.1581693536956; Fri, 14 Feb 2020 07:18:56 -0800 (PST) X-Google-Smtp-Source: APXvYqx98yYRfFgrmnoTK7yOcT8vEYzHYdeLFs9Ej1KFbEv1l6A+9C0pufzZcLHUFobNLA5YXrl16w== X-Received: by 2002:a37:4e97:: with SMTP id c145mr2976531qkb.253.1581693536653; Fri, 14 Feb 2020 07:18:56 -0800 (PST) Received: from localhost.localdomain ([2804:14c:4e7:1017:a97b:ff5a:85e:d3eb]) by smtp.gmail.com with ESMTPSA id s22sm3403933qke.19.2020.02.14.07.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2020 07:18:56 -0800 (PST) From: Mauricio Faria de Oliveira To: fstests@vger.kernel.org Cc: Amir Goldstein Subject: [PATCH v2 3/5] common/rc: introduce new helper function _fs_type_dev_dir() Date: Fri, 14 Feb 2020 12:18:46 -0300 Message-Id: <20200214151848.8328-4-mfo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200214151848.8328-1-mfo@canonical.com> References: <20200214151848.8328-1-mfo@canonical.com> MIME-Version: 1.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org In order to determine the fs type on fuse-overlayfs (coming) we need to search for the mount point/directory; the device is not enough. (details in the next patch.) Thus the _fs_type() function is insufficient to determine the filesystem type, as it only searches for mount device. So, introduce the _fs_type_dev_dir() function, which also searches for the mountpoint/dir in addition to the device. The fs type fix-up sed script goes into a common function. P.S.: there might be other sites that need similar changes, since the mount device is also checked elsewhere, but just with this bit tests can run, so it is good enough for now.) Signed-off-by: Mauricio Faria de Oliveira Reviewed-by: Amir Goldstein --- common/rc | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) diff --git a/common/rc b/common/rc index 1feae1a94f9e..5711eca2a1d2 100644 --- a/common/rc +++ b/common/rc @@ -1262,6 +1262,19 @@ _used() _df_device $1 | $AWK_PROG '{ sub("%", "") ; print $6 }' } +# fix filesystem type up +# +_fix_fs_type() +{ + # + # The Linux kernel shows NFSv4 filesystems in df output as + # filesystem type nfs4, although we mounted it as nfs earlier. + # Fix the filesystem type up here so that the callers don't + # have to bother with this quirk. + # + sed -e 's/nfs4/nfs/' -e 's/fuse.glusterfs/glusterfs/' +} + # return the FS type of a mounted device # _fs_type() @@ -1272,14 +1285,21 @@ _fs_type() exit 1 fi - # - # The Linux kernel shows NFSv4 filesystems in df output as - # filesystem type nfs4, although we mounted it as nfs earlier. - # Fix the filesystem type up here so that the callers don't - # have to bother with this quirk. - # - _df_device $1 | $AWK_PROG '{ print $2 }' | \ - sed -e 's/nfs4/nfs/' -e 's/fuse.glusterfs/glusterfs/' + _df_device $1 | $AWK_PROG '{ print $2 }' | _fix_fs_type +} + +# return the FS type of a mounted device +# on a mount point directory (check both) +# +_fs_type_dev_dir() +{ + if [ $# -ne 2 ] + then + echo "Usage: _fs_type_dev_dir device directory" 1>&2 + exit 1 + fi + + _df_dir $2 | $AWK_PROG -v what=$1 '($1==what) { print $2 }' | _fix_fs_type } # return the FS mount options of a mounted device From patchwork Fri Feb 14 15:18:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mauricio Faria de Oliveira X-Patchwork-Id: 11382459 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1639314E3 for ; Fri, 14 Feb 2020 15:19:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F10ED2168B for ; Fri, 14 Feb 2020 15:19:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729557AbgBNPTB (ORCPT ); Fri, 14 Feb 2020 10:19:01 -0500 Received: from youngberry.canonical.com ([91.189.89.112]:54685 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729482AbgBNPTB (ORCPT ); Fri, 14 Feb 2020 10:19:01 -0500 Received: from mail-qv1-f70.google.com ([209.85.219.70]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1j2ck7-000236-Ki for fstests@vger.kernel.org; Fri, 14 Feb 2020 15:18:59 +0000 Received: by mail-qv1-f70.google.com with SMTP id v19so2467799qvk.16 for ; Fri, 14 Feb 2020 07:18:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5gxv9naRR94OrFR2E4kqdWVBYbqQzHHgz4FT5saerk8=; b=pS+6hmRwNpcUtwEJ6iP8f2/mYvD2bXhaTtKtvQygb52g60/gNyLkP7l0/iCqV9xVYn WMXad2GIrPyzmBx2n0AraK+ziCoqBkqIarPfmMPf4opvozyEx1Oa6br1jqH+g+RotzQ7 T0Iuw3mH0lFOMI+7tkN5P8dasBJa2vArZklWVZpioLx5APrZ4Wp2Bkj9bPnHllavtkO4 0XnD6NiFMFUltu3pR3YmsiDIdVFFrPokrir0QdylViZ8DBIg/Mbb9Hhk64S9JlA7/jdG FHZay2ftYzRV8IsbFiNqeRwinXncObZJhYgREFUJbi76Ys7XGY4oGV2ESyw20I8L7Ah2 O78g== X-Gm-Message-State: APjAAAVfBvrgUyWLOl7uNvLaQAvIPi6VeYM5De9+inkqLWuezSVVxPSe 3XbSBSzvUMmU+2zT5IXUGHS0w2PTlPErwXQRxclsMQyKmcVKOBfbcHrbRArpSbVOfdqzDCOO6Nk d9EAQFYfsnbSCui2e1gXHLDDoiTcSivtNhe0= X-Received: by 2002:a37:678a:: with SMTP id b132mr2714023qkc.4.1581693538560; Fri, 14 Feb 2020 07:18:58 -0800 (PST) X-Google-Smtp-Source: APXvYqxSs3a5ecC9oWFm9RUxIRwQefN+2hgpiqqvbGMwl4bdR1pwR/cuddVO1Sfr8o3OTlLgvoQaJA== X-Received: by 2002:a37:678a:: with SMTP id b132mr2714002qkc.4.1581693538291; Fri, 14 Feb 2020 07:18:58 -0800 (PST) Received: from localhost.localdomain ([2804:14c:4e7:1017:a97b:ff5a:85e:d3eb]) by smtp.gmail.com with ESMTPSA id s22sm3403933qke.19.2020.02.14.07.18.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2020 07:18:57 -0800 (PST) From: Mauricio Faria de Oliveira To: fstests@vger.kernel.org Cc: Amir Goldstein Subject: [PATCH v2 4/5] common/rc: add quirks for fuse-overlayfs device/mount point Date: Fri, 14 Feb 2020 12:18:47 -0300 Message-Id: <20200214151848.8328-5-mfo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200214151848.8328-1-mfo@canonical.com> References: <20200214151848.8328-1-mfo@canonical.com> MIME-Version: 1.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org On fuse-overlayfs the mount device is always reported as a "fuse-overlayfs" string, instead of the parent directory of the mount point (the string expected by overlay/scripts). Unfortunately, it seems that the fuse mount option 'fsname' doesn't set the filesystem source/device on fuse-overlayfs, so the easy fix of just adding it to mount options is gone. So two quirks are used to check for a fuse-overlayfs mount, that checks the mount point/directory in _check_mounted_on() and init_rc(); latter with the new helper _fs_type_dev_dir(). With this, fuse-overlayfs can now keep going through tests! Signed-off-by: Mauricio Faria de Oliveira --- README.overlay | 1 + common/rc | 19 ++++++++++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/README.overlay b/README.overlay index 08a39b8830c9..7ef07ae6bbab 100644 --- a/README.overlay +++ b/README.overlay @@ -53,4 +53,5 @@ OVERLAY_FSCK_OPTIONS will be used to check both test and scratch overlay. To test other filesystem types (experimental) configure the OVL_FSTYP variable: + OVL_FSTYP=fuse.fuse-overlayfs OVL_FSTYP=aufs diff --git a/common/rc b/common/rc index 5711eca2a1d2..6a4cb9b6d604 100644 --- a/common/rc +++ b/common/rc @@ -1481,6 +1481,14 @@ _check_mounted_on() # find $dev as the source, and print result in "$dev $mnt" format local mount_rec=`findmnt -rncv -S $dev -o SOURCE,TARGET` + + # fuse-overlayfs dev is not $dev, check via $mnt. + if [ -z "$mount_rec" -a "$FSTYP" = "overlay" -a \ + "$OVL_FSTYP" = "fuse.fuse-overlayfs" ]; then + dev="fuse-overlayfs" + mount_rec=`findmnt -rncv -S $dev -T $mnt -o SOURCE,TARGET` + fi + [ -n "$mount_rec" ] || return 1 # 1 = not mounted # if it's mounted, make sure its on $mnt @@ -3788,8 +3796,17 @@ init_rc() fi # if $TEST_DEV is not mounted, mount it now as XFS - if [ -z "`_fs_type $TEST_DEV`" ] + if [ -n "`_fs_type $TEST_DEV`" ] then + # $TEST_DEV is mounted + true + elif [ "$FSTYP" = "overlay" -a "$OVL_FSTYP" = "fuse.fuse-overlayfs" -a \ + -n "`_fs_type_dev_dir fuse-overlayfs $TEST_DEV/$OVL_MNT`" ] + then + # (fuse-overlayfs dev is not $TEST_DEV; check via mount point) + # $TEST_DEV is mounted + true + else # $TEST_DEV is not mounted if ! _test_mount then From patchwork Fri Feb 14 15:18:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mauricio Faria de Oliveira X-Patchwork-Id: 11382461 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EF6F14E3 for ; Fri, 14 Feb 2020 15:19:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 84B1E24654 for ; Fri, 14 Feb 2020 15:19:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729558AbgBNPTD (ORCPT ); Fri, 14 Feb 2020 10:19:03 -0500 Received: from youngberry.canonical.com ([91.189.89.112]:54689 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729482AbgBNPTD (ORCPT ); Fri, 14 Feb 2020 10:19:03 -0500 Received: from mail-qt1-f198.google.com ([209.85.160.198]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1j2ck9-00023N-BA for fstests@vger.kernel.org; Fri, 14 Feb 2020 15:19:01 +0000 Received: by mail-qt1-f198.google.com with SMTP id k20so6088890qtm.11 for ; Fri, 14 Feb 2020 07:19:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3GiixWLPt3Xsm/w7r5HRS6kYyZ1TZwQaBGsuqSkCksc=; b=sLqtiFR49VJYllwb15A9nK0iAmpAv0VW03pVI8zclIh0Lb8SxAXOznTtj7DBETDgil rfTC379hvwejOOUpVuU0zUUuzlNUrFl2KdyTCh2Pi5mpWW7xLmXN17egCXCCADmaB5EG nl2klOV9Mq13K6SXUVp0fvjrBIpGNMmJVCeviAm/0U5tOVseA0cKO+wTfVWiAwra0bFi GQDDIlhW0rDA4f3a1KdOrw++kYJzz00I9dEmSuyK9GB+mChewoR8LtOJkzbohKepiBmn 1eI7zr+FUTXsZxTUS7+5gdyX3jJtkm/wSErYWcRsMSG2eCblouj4KqjEJlgai52lzfzN SeYQ== X-Gm-Message-State: APjAAAXBz4jeeUhOuwV+cAZV5j91ql5ilIw02p4nB+gPcWAk4l8EVIBw xKRlCHw8ycEJiFmB0ZrSsrHR8h9tsEjVB3RMB4ro45S6DWwk+4jyzDuDIY3STQsTSKK3UhR1t71 O+/F9s1c9xXAEdEhQAiFb7SpZl51cxhVn3zA= X-Received: by 2002:a05:620a:2119:: with SMTP id l25mr3088414qkl.248.1581693540150; Fri, 14 Feb 2020 07:19:00 -0800 (PST) X-Google-Smtp-Source: APXvYqzXrSZQbHcIdhkQnJmXTyveNUP2K55D0Dm3Xws0oOOOcoew+XDbhOk/5G8rqYSJdGqL9okHqA== X-Received: by 2002:a05:620a:2119:: with SMTP id l25mr3088391qkl.248.1581693539900; Fri, 14 Feb 2020 07:18:59 -0800 (PST) Received: from localhost.localdomain ([2804:14c:4e7:1017:a97b:ff5a:85e:d3eb]) by smtp.gmail.com with ESMTPSA id s22sm3403933qke.19.2020.02.14.07.18.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2020 07:18:59 -0800 (PST) From: Mauricio Faria de Oliveira To: fstests@vger.kernel.org Cc: Amir Goldstein Subject: [PATCH v2 5/5] common/overlay: silence some mount messages for fuse-overlayfs Date: Fri, 14 Feb 2020 12:18:48 -0300 Message-Id: <20200214151848.8328-6-mfo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200214151848.8328-1-mfo@canonical.com> References: <20200214151848.8328-1-mfo@canonical.com> MIME-Version: 1.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org When mouting fuse-overlayfs there are some messages that make the tests report failures due to output mismatch; ignore them: uid=unchanged uid=unchanged upperdir=/mnt/test/ovl-upper workdir=/mnt/test/ovl-work lowerdir=/mnt/test/ovl-lower mountpoint=/mnt/test/ovl-mnt For other filesystem types (e.g., overlay and aufs) make sure to only print non-null output, to avoid blank lines output mismatch. And return the status of the mount command, not other commands. Currently, running './check -overlay' tests (excluding generic/062) the numbers for fuse-overlayfs on loop devices on v5.4-based Ubuntu kernel with the fuse-overlayfs package from Ubuntu Eoan/19.10 are: - Ran: 530 - Not run: 395 - Failures: 29 And hopefully this helps with testing for fuse-overlayfs too. Steps: $ export OVL_FSTYP=fuse.fuse-overlayfs $ export FSTYP=ext4 $ export TEST_DEV=/dev/loop0 $ export TEST_DIR=/mnt/test $ export SCRATCH_DEV=/dev/loop1 $ export SCRATCH_MNT=/mnt/scratch $ sudo mkfs.$FSTYP -F $TEST_DEV $ sudo mkfs.$FSTYP -F $SCRATCH_DEV $ sudo mkdir $TEST_DIR $SCRATCH_MNT $ cat </tmp/exclude-tests generic/062 EOF $ sudo -E ./check -overlay -E /tmp/exclude-tests Signed-off-by: Mauricio Faria de Oliveira --- common/overlay | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/common/overlay b/common/overlay index a1076926c23f..27f3c08252ee 100644 --- a/common/overlay +++ b/common/overlay @@ -19,6 +19,8 @@ _overlay_mount_dirs() local upperdir=$2 local workdir=$3 local options + local output + local rc shift 3 options="-o lowerdir=$lowerdir -o upperdir=$upperdir -o workdir=$workdir" @@ -26,7 +28,23 @@ _overlay_mount_dirs() options="-o br=$upperdir=rw -o br=$lowerdir=ro" fi - $MOUNT_PROG -t $OVL_FSTYP $options `_common_dev_mount_options $*` + options="$options `_common_dev_mount_options $*`" + output="`$MOUNT_PROG -t $OVL_FSTYP $options 2>&1`" + rc=$? + + if [ "$OVL_FSTYP" = "fuse.fuse-overlayfs" ]; then + # Less verbosity to avoid output mismatch. + echo "$output" | grep -v \ + -e "^uid=" \ + -e "^upperdir=" \ + -e "^lowerdir=" \ + -e "^workdir=" \ + -e "^mountpoint=" + elif [ -n "$output" ]; then + echo "$output" + fi + + return $rc } # Mount with same options/mnt/dev of scratch mount, but optionally