From patchwork Wed Apr 21 00:23:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12215251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27CE4C433B4 for ; Wed, 21 Apr 2021 00:23:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAA596141D for ; Wed, 21 Apr 2021 00:23:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234566AbhDUAXm (ORCPT ); Tue, 20 Apr 2021 20:23:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:36722 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233807AbhDUAXk (ORCPT ); Tue, 20 Apr 2021 20:23:40 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6BD676141C; Wed, 21 Apr 2021 00:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1618964588; bh=n2SMwFS2H16MLEaQt6yaZgpsTRX6/bc3ElUS6w6VrK8=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=D3O4pgFUZ8BdE+steSLUnQ/3+LfxxtKJBr1Z/lN7/lavZbg4xT5owGfAm3Vy1fEx/ IRTpM6LF1PY4nEzpEFnogtmLebHxtmk7JykbY3D48kFY8w5z0wTCHkr/7Vl11SJmHo uAS/pjHD/94Ytu01iUJ+Ee9pYTURh1KDaFQQUxG8ko+sqW7t4P8B5xl03W4yiJf5uS 3+gokTA8ikmyy7Q1gY9piPoDIAluu2ffyLOoqq3pj9vf6OjMEkIETfZf+yHoDMPinX dZpLQpVn6BFWnewye1L0rm4jtm48XneQP45I/9E7S9VVHKgqBe+Klx2vzldLBB82I2 chkq+xYv/XQOw== Subject: [PATCH 1/4] generic: check userspace handling of extreme timestamps From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Amir Goldstein , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 20 Apr 2021 17:23:07 -0700 Message-ID: <161896458765.776452.776474866675909773.stgit@magnolia> In-Reply-To: <161896458140.776452.9583732658582318883.stgit@magnolia> References: <161896458140.776452.9583732658582318883.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong These two tests ensure we can store and retrieve timestamps on the extremes of the date ranges supported by userspace, and the common places where overflows can happen. They differ from generic/402 in that they don't constrain the dates tested to the range that the filesystem claims to support; we attempt various things that /userspace/ can parse, and then check that the vfs clamps and persists the values correctly. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein Reviewed-by: Allison Henderson --- tests/generic/721 | 123 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/721.out | 2 + tests/generic/722 | 125 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/722.out | 1 tests/generic/group | 6 ++ 5 files changed, 255 insertions(+), 2 deletions(-) create mode 100755 tests/generic/721 create mode 100644 tests/generic/721.out create mode 100755 tests/generic/722 create mode 100644 tests/generic/722.out diff --git a/tests/generic/721 b/tests/generic/721 new file mode 100755 index 00000000..9198b6b4 --- /dev/null +++ b/tests/generic/721 @@ -0,0 +1,123 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 721 +# +# Make sure we can store and retrieve timestamps on the extremes of the +# date ranges supported by userspace, and the common places where overflows +# can happen. +# +# This differs from generic/402 in that we don't constrain ourselves to the +# range that the filesystem claims to support; we attempt various things that +# /userspace/ can parse, and then check that the vfs clamps and persists the +# values correctly. +# +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently +# when writing timestamps to disk! This test detects this silent truncation +# and fails. If you see a failure on such a kernel, contact your distributor +# for an update. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs generic +_require_scratch + +rm -f $seqres.full + +_scratch_mkfs > $seqres.full +_scratch_mount + +# Does our userspace even support large dates? +test_bigdates=1 +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0 + +# And can we do statx? +test_statx=1 +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \ + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \ + test_statx=0 + +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full +echo "xfs_io support of statx: $test_statx" >> $seqres.full + +touchme() { + local arg="$1" + local name="$2" + + echo "$arg" > $SCRATCH_MNT/t_$name + touch -d "$arg" $SCRATCH_MNT/t_$name +} + +report() { + local files=($SCRATCH_MNT/t_*) + for file in "${files[@]}"; do + echo "${file}: $(cat "${file}")" + TZ=UTC stat -c '%y %Y %n' "${file}" + test $test_statx -gt 0 && \ + $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime' + done +} + +# -2147483648 (S32_MIN, or classic unix min) +touchme 'Dec 13 20:45:52 UTC 1901' s32_min + +# 2147483647 (S32_MAX, or classic unix max) +touchme 'Jan 19 03:14:07 UTC 2038' s32_max + +# 7956915742, all twos +touchme 'Feb 22 22:22:22 UTC 2222' all_twos + +if [ $test_bigdates -gt 0 ]; then + # 16299260424 (u64 nsec counter from s32_min, like xfs does) + touchme 'Tue Jul 2 20:20:24 UTC 2486' u64ns_from_s32_min + + # 15032385535 (u34 time if you start from s32_min, like ext4 does) + touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min + + # 17179869183 (u34 time if you start from the unix epoch) + touchme 'May 30 01:53:03 UTC 2514' u34_max + + # Latest date we can synthesize(?) + touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time + + # Earliest date we can synthesize(?) + touchme 'Jan 1 00:00:00 UTC 0' abs_min_time +fi + +# Query timestamps from incore +echo before >> $seqres.full +report > $tmp.before_remount +cat $tmp.before_remount >> $seqres.full + +_scratch_cycle_mount + +# Query timestamps from disk +echo after >> $seqres.full +report > $tmp.after_remount +cat $tmp.after_remount >> $seqres.full + +# Did they match? +cmp -s $tmp.before_remount $tmp.after_remount + +# success, all done +echo Silence is golden. +status=0 +exit diff --git a/tests/generic/721.out b/tests/generic/721.out new file mode 100644 index 00000000..b2bc6d58 --- /dev/null +++ b/tests/generic/721.out @@ -0,0 +1,2 @@ +QA output created by 721 +Silence is golden. diff --git a/tests/generic/722 b/tests/generic/722 new file mode 100755 index 00000000..305c3bd6 --- /dev/null +++ b/tests/generic/722 @@ -0,0 +1,125 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 722 +# +# Make sure we can store and retrieve timestamps on the extremes of the +# date ranges supported by userspace, and the common places where overflows +# can happen. This test also ensures that the timestamps are persisted +# correctly after a shutdown. +# +# This differs from generic/402 in that we don't constrain ourselves to the +# range that the filesystem claims to support; we attempt various things that +# /userspace/ can parse, and then check that the vfs clamps and persists the +# values correctly. +# +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently +# when writing timestamps to disk! This test detects this silent truncation +# and fails. If you see a failure on such a kernel, contact your distributor +# for an update. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs generic +_require_scratch +_require_scratch_shutdown + +rm -f $seqres.full + +_scratch_mkfs > $seqres.full +_scratch_mount + +# Does our userspace even support large dates? +test_bigdates=1 +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0 + +# And can we do statx? +test_statx=1 +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \ + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \ + test_statx=0 + +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full +echo "xfs_io support of statx: $test_statx" >> $seqres.full + +touchme() { + local arg="$1" + local name="$2" + + echo "$arg" > $SCRATCH_MNT/t_$name + touch -d "$arg" $SCRATCH_MNT/t_$name +} + +report() { + local files=($SCRATCH_MNT/t_*) + for file in "${files[@]}"; do + echo "${file}: $(cat "${file}")" + TZ=UTC stat -c '%y %Y %n' "${file}" + test $test_statx -gt 0 && \ + $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime' + done +} + +# -2147483648 (S32_MIN, or classic unix min) +touchme 'Dec 13 20:45:52 UTC 1901' s32_min + +# 2147483647 (S32_MAX, or classic unix max) +touchme 'Jan 19 03:14:07 UTC 2038' s32_max + +# 7956915742, all twos +touchme 'Feb 22 22:22:22 UTC 2222' all_twos + +if [ $test_bigdates -gt 0 ]; then + # 16299260424 (u64 nsec counter from s32_min, like xfs does) + touchme 'Tue Jul 2 20:20:24 UTC 2486' u64ns_from_s32_min + + # 15032385535 (u34 time if you start from s32_min, like ext4 does) + touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min + + # 17179869183 (u34 time if you start from the unix epoch) + touchme 'May 30 01:53:03 UTC 2514' u34_max + + # Latest date we can synthesize(?) + touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time + + # Earliest date we can synthesize(?) + touchme 'Jan 1 00:00:00 UTC 0' abs_min_time +fi + +# Query timestamps from incore +echo before >> $seqres.full +report > $tmp.before_crash +cat $tmp.before_crash >> $seqres.full + +_scratch_shutdown -f +_scratch_cycle_mount + +# Query timestamps from disk +echo after >> $seqres.full +report > $tmp.after_crash +cat $tmp.after_crash >> $seqres.full + +# Did they match? +cmp -s $tmp.before_crash $tmp.after_crash + +# success, all done +status=0 +exit diff --git a/tests/generic/722.out b/tests/generic/722.out new file mode 100644 index 00000000..83acd5cf --- /dev/null +++ b/tests/generic/722.out @@ -0,0 +1 @@ +QA output created by 722 diff --git a/tests/generic/group b/tests/generic/group index 033465f1..21ac0c8f 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -260,7 +260,7 @@ 255 auto quick prealloc punch 256 auto quick punch 257 dir auto quick -258 auto quick +258 auto quick bigtime 259 auto quick clone zero 260 auto quick trim 261 auto quick clone collapse @@ -404,7 +404,7 @@ 399 auto encrypt 400 auto quick quota 401 auto quick -402 auto quick rw +402 auto quick rw bigtime 403 auto quick attr 404 auto quick insert 405 auto mkfs thin @@ -636,3 +636,5 @@ 631 auto rw overlay rename 632 auto quick mount 633 auto quick atime attr cap idmapped io_uring mount perms rw unlink +721 auto quick atime bigtime +722 auto quick atime bigtime shutdown From patchwork Wed Apr 21 00:23:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12215253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 447A0C433ED for ; Wed, 21 Apr 2021 00:23:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C97A61424 for ; Wed, 21 Apr 2021 00:23:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234501AbhDUAXs (ORCPT ); Tue, 20 Apr 2021 20:23:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:36786 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234223AbhDUAXr (ORCPT ); Tue, 20 Apr 2021 20:23:47 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id B494F6141C; Wed, 21 Apr 2021 00:23:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1618964594; bh=/IORSqdy+Ox8dFWvtPo45m+ddaFqcKeIwkSWZfD/B8g=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=t12Xpf9LrzydG/T7x/CfrzjwxWOIbmCRmHpYo4yO1JDG6DLc4TmBnR0EbaoR/XddF QnfvuX+sQgj8/6WO4afwMIKvAi21meFBIhTu4KGc6mhd6K/uc1Tbti1xyQ4rcfNyh4 w0J1szaF3gv/2VlpHUecpqhFndkGeWgtsB3Tqq8wNCQNb+5S3QZVppPIL5O5XMiiEb dGUorCP4xVjxU/fWjqXMUrp6EOFLCzKh0hvMc+tbWKzyH4AgPKoiaDU2yLC1ghusA0 yHchPy8vvK6+GjZQjEd0aBs9lxKmVA6J4HtOTaUj0zePgxJ/3CC1ESaXysVOVKd3sn No8wJQbZEuuyA== Subject: [PATCH 2/4] xfs/122: add legacy timestamps to ondisk checker From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Amir Goldstein , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 20 Apr 2021 17:23:13 -0700 Message-ID: <161896459378.776452.5480197157832099240.stgit@magnolia> In-Reply-To: <161896458140.776452.9583732658582318883.stgit@magnolia> References: <161896458140.776452.9583732658582318883.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Add these new ondisk structures. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein Reviewed-by: Allison Henderson --- tests/xfs/122 | 1 + tests/xfs/122.out | 1 + 2 files changed, 2 insertions(+) diff --git a/tests/xfs/122 b/tests/xfs/122 index 322e1d81..c8593315 100755 --- a/tests/xfs/122 +++ b/tests/xfs/122 @@ -181,6 +181,7 @@ struct xfs_iext_cursor struct xfs_ino_geometry struct xfs_attrlist struct xfs_attrlist_ent +struct xfs_legacy_ictimestamp EOF echo 'int main(int argc, char *argv[]) {' >>$cprog diff --git a/tests/xfs/122.out b/tests/xfs/122.out index b0773756..f229465a 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -97,6 +97,7 @@ sizeof(struct xfs_inode_log_format) = 56 sizeof(struct xfs_inode_log_format_32) = 52 sizeof(struct xfs_inumbers) = 24 sizeof(struct xfs_inumbers_req) = 64 +sizeof(struct xfs_legacy_timestamp) = 8 sizeof(struct xfs_log_dinode) = 176 sizeof(struct xfs_map_extent) = 32 sizeof(struct xfs_phys_extent) = 16 From patchwork Wed Apr 21 00:23:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12215255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB2AFC433B4 for ; Wed, 21 Apr 2021 00:23:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC2BB61420 for ; Wed, 21 Apr 2021 00:23:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234223AbhDUAXx (ORCPT ); Tue, 20 Apr 2021 20:23:53 -0400 Received: from mail.kernel.org ([198.145.29.99]:36850 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233807AbhDUAXx (ORCPT ); Tue, 20 Apr 2021 20:23:53 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id DC6BF6141F; Wed, 21 Apr 2021 00:23:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1618964601; bh=RyKoLmbrbH6XqUy1EbZr5UJ84//p0tSP+g9dKjlcG6c=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=qwJLyo9NrTjYXr0xOIY0WmDCEstDGNXrdyhWZGuSROV8rDkB2WcNXYiEPPAaX11Ua Qh0FT6ntrYmSevmfyQE51uYm/ipPvo6tDOtEM59CYZt2g36n0HxK7YAjlHhQHxOaKh jo3nMWmv1WjEgPH8GgZ6C1MtVWjY6dkCfv/vFFE9SyfmPRPHE90T1GO8nwRAO6sOet BOBx+0tdpXnC3U2/8jI7WXYFD3PN+ukMupZ+TDgWCAKbK2QTwz5iOcXPm33M9neGYe 3RmfbMvDQHBQLEPoQoEELkiDvmG6x8sMNsfepY3jtJd36v58J3ZMEYszri4ry5O4pk I4OUa4W8DsajQ== Subject: [PATCH 3/4] xfs: detect time limits from filesystem From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Amir Goldstein , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 20 Apr 2021 17:23:20 -0700 Message-ID: <161896460011.776452.12958363566736119178.stgit@magnolia> In-Reply-To: <161896458140.776452.9583732658582318883.stgit@magnolia> References: <161896458140.776452.9583732658582318883.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Teach fstests to extract timestamp limits of a filesystem using the new xfs_db timelimit command. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein Reviewed-by: Allison Henderson --- common/rc | 2 +- common/xfs | 19 +++++++++++++++++++ tests/xfs/911 | 44 ++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/911.out | 15 +++++++++++++++ tests/xfs/group | 1 + 5 files changed, 80 insertions(+), 1 deletion(-) create mode 100755 tests/xfs/911 create mode 100644 tests/xfs/911.out diff --git a/common/rc b/common/rc index 11ff7635..116d7b20 100644 --- a/common/rc +++ b/common/rc @@ -2124,7 +2124,7 @@ _filesystem_timestamp_range() echo "0 $u32max" ;; xfs) - echo "$s32min $s32max" + _xfs_timestamp_range "$device" ;; btrfs) echo "$s64min $s64max" diff --git a/common/xfs b/common/xfs index 3d660858..cb6a1978 100644 --- a/common/xfs +++ b/common/xfs @@ -1165,3 +1165,22 @@ _require_scratch_xfs_inobtcount() _notrun "kernel doesn't support xfs inobtcount feature" _scratch_unmount } + +_xfs_timestamp_range() +{ + local device="$1" + local use_db=0 + local dbprog="$XFS_DB_PROG $device" + test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db + + $dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1 + if [ $use_db -eq 0 ]; then + # The "timelimit" command was added to xfs_db at the same time + # that bigtime was added to xfsprogs. Therefore, we can assume + # the old timestamp range if the command isn't present. + echo "-$((1<<31)) $(((1<<31)-1))" + else + $dbprog -f -c 'timelimit --compact' | \ + awk '{printf("%s %s", $1, $2);}' + fi +} diff --git a/tests/xfs/911 b/tests/xfs/911 new file mode 100755 index 00000000..01ddb856 --- /dev/null +++ b/tests/xfs/911 @@ -0,0 +1,44 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 911 +# +# Check that the xfs_db timelimit command prints the ranges that we expect. +# This in combination with an xfs_ondisk.h build time check in the kernel +# ensures that the kernel agrees with userspace. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_xfs_db_command timelimit + +rm -f $seqres.full + +# Format filesystem without bigtime support and populate it +_scratch_mkfs > $seqres.full +echo classic xfs timelimits +_scratch_xfs_db -c 'timelimit --classic' +echo bigtime xfs timelimits +_scratch_xfs_db -c 'timelimit --bigtime' + +# success, all done +status=0 +exit diff --git a/tests/xfs/911.out b/tests/xfs/911.out new file mode 100644 index 00000000..84dc475b --- /dev/null +++ b/tests/xfs/911.out @@ -0,0 +1,15 @@ +QA output created by 911 +classic xfs timelimits +time.min = -2147483648 +time.max = 2147483647 +dqtimer.min = 1 +dqtimer.max = 4294967295 +dqgrace.min = 0 +dqgrace.min = 4294967295 +bigtime xfs timelimits +time.min = -2147483648 +time.max = 16299260424 +dqtimer.min = 4 +dqtimer.max = 16299260424 +dqgrace.min = 0 +dqgrace.min = 4294967295 diff --git a/tests/xfs/group b/tests/xfs/group index bd47333c..b4e29bab 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -527,3 +527,4 @@ 770 auto repair 773 auto quick repair 910 auto quick inobtcount +911 auto quick bigtime From patchwork Wed Apr 21 00:23:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12215257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AD32C43460 for ; Wed, 21 Apr 2021 00:23:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D63561424 for ; Wed, 21 Apr 2021 00:23:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234575AbhDUAYA (ORCPT ); Tue, 20 Apr 2021 20:24:00 -0400 Received: from mail.kernel.org ([198.145.29.99]:36904 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234442AbhDUAX7 (ORCPT ); Tue, 20 Apr 2021 20:23:59 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 108D06141F; Wed, 21 Apr 2021 00:23:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1618964607; bh=3L50YCeYC5VGtemeh2l+P6EnpBnHXHXcmIqoLkiPYa0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ipH6KX2I64dIZGSXgNe9rU2aCak+HLjFRus8omVgtPoJrEul7hztfwxEtOSwapaDV yVqQK3wgFBbLetEY6nDqnwAToC7OzizSFZVzmIfc+pdQENY0tsddWX58gL97BGBzBw SaazLHbht8QlThia9t/CTopC8420TKgS4GGcE5WihCXpnU4Sw7YOnsV+LEejS2VnyS KF5XeY+SWFuB3l1YQVssvGsRUHEnjVcd7k7VdcaYnuvHtBbRYvaJZNLji2Bc0U+rxR +cLEav/QL5RgISff0GqwGyU2XDLQGhKxqF6tBU1eFeThdb6Iz+BUwB/IF08N5PQmDc aWNAlA2UI+9jw== Subject: [PATCH 4/4] xfs: test upgrading filesystem to bigtime From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 20 Apr 2021 17:23:26 -0700 Message-ID: <161896460627.776452.15178871770338402214.stgit@magnolia> In-Reply-To: <161896458140.776452.9583732658582318883.stgit@magnolia> References: <161896458140.776452.9583732658582318883.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Test that we can upgrade an existing filesystem to use bigtime. Signed-off-by: Darrick J. Wong Reviewed-by: Allison Henderson --- common/xfs | 16 ++++++ tests/xfs/908 | 117 ++++++++++++++++++++++++++++++++++++++++++ tests/xfs/908.out | 29 ++++++++++ tests/xfs/909 | 149 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/909.out | 6 ++ tests/xfs/group | 2 + 6 files changed, 319 insertions(+) create mode 100755 tests/xfs/908 create mode 100644 tests/xfs/908.out create mode 100755 tests/xfs/909 create mode 100644 tests/xfs/909.out diff --git a/common/xfs b/common/xfs index cb6a1978..253a31e5 100644 --- a/common/xfs +++ b/common/xfs @@ -1184,3 +1184,19 @@ _xfs_timestamp_range() awk '{printf("%s %s", $1, $2);}' fi } + +# Require that the scratch device exists, that mkfs can format with bigtime +# enabled, that the kernel can mount such a filesystem, and that xfs_info +# advertises the presence of that feature. +_require_scratch_xfs_bigtime() +{ + _require_scratch + + _scratch_mkfs -m bigtime=1 &>/dev/null || \ + _notrun "mkfs.xfs doesn't support bigtime feature" + _try_scratch_mount || \ + _notrun "kernel doesn't support xfs bigtime feature" + $XFS_INFO_PROG "$SCRATCH_MNT" | grep -q -w "bigtime=1" || \ + _notrun "bigtime feature not advertised on mount?" + _scratch_unmount +} diff --git a/tests/xfs/908 b/tests/xfs/908 new file mode 100755 index 00000000..004a8563 --- /dev/null +++ b/tests/xfs/908 @@ -0,0 +1,117 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 908 +# +# Check that we can upgrade a filesystem to support bigtime and that inode +# timestamps work properly after the upgrade. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_command "$XFS_ADMIN_PROG" "xfs_admin" +_require_scratch_xfs_bigtime +_require_xfs_repair_upgrade bigtime + +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \ + _notrun "Userspace does not support dates past 2038." + +rm -f $seqres.full + +# Make sure we can't upgrade a V4 filesystem +_scratch_mkfs -m crc=0 >> $seqres.full +_scratch_xfs_admin -O bigtime=1 2>> $seqres.full +_check_scratch_xfs_features BIGTIME + +# Make sure we're required to specify a feature status +_scratch_mkfs -m crc=1,bigtime=0,inobtcount=0 >> $seqres.full +_scratch_xfs_admin -O bigtime 2>> $seqres.full + +# Can we add bigtime and inobtcount at the same time? +_scratch_mkfs -m crc=1,bigtime=0,inobtcount=0 >> $seqres.full +_scratch_xfs_admin -O bigtime=1,inobtcount=1 2>> $seqres.full + +# Format V5 filesystem without bigtime support and populate it +_scratch_mkfs -m crc=1,bigtime=0 >> $seqres.full +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full +_scratch_mount >> $seqres.full + +touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/a +touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/b +ls -la $SCRATCH_MNT/* >> $seqres.full + +echo before upgrade: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +_scratch_unmount +_check_scratch_fs + +# Now upgrade to bigtime support +_scratch_xfs_admin -O bigtime=1 2>> $seqres.full +_check_scratch_xfs_features BIGTIME +_check_scratch_fs +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full + +# Mount again, look at our files +_scratch_mount >> $seqres.full +ls -la $SCRATCH_MNT/* >> $seqres.full + +echo after upgrade: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +# Bump one of the timestamps but stay under 2038 +touch -d 'Jan 10 19:19:19 UTC 1999' $SCRATCH_MNT/a + +echo after upgrade and bump: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +_scratch_cycle_mount + +# Did the bumped timestamp survive the remount? +ls -la $SCRATCH_MNT/* >> $seqres.full + +echo after upgrade, bump, and remount: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +# Modify the other timestamp to stretch beyond 2038 +touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b + +echo after upgrade and extension: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +_scratch_cycle_mount + +# Did the timestamp survive the remount? +ls -la $SCRATCH_MNT/* >> $seqres.full + +echo after upgrade, extension, and remount: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +# success, all done +status=0 +exit diff --git a/tests/xfs/908.out b/tests/xfs/908.out new file mode 100644 index 00000000..5e05854d --- /dev/null +++ b/tests/xfs/908.out @@ -0,0 +1,29 @@ +QA output created by 908 +Running xfs_repair to upgrade filesystem. +Large timestamp feature only supported on V5 filesystems. +FEATURES: BIGTIME:NO +Running xfs_repair to upgrade filesystem. +Running xfs_repair to upgrade filesystem. +Adding inode btree counts to filesystem. +Adding large timestamp support to filesystem. +before upgrade: +915909559 +915909559 +Running xfs_repair to upgrade filesystem. +Adding large timestamp support to filesystem. +FEATURES: BIGTIME:YES +after upgrade: +915909559 +915909559 +after upgrade and bump: +915995959 +915909559 +after upgrade, bump, and remount: +915995959 +915909559 +after upgrade and extension: +915995959 +7956915742 +after upgrade, extension, and remount: +915995959 +7956915742 diff --git a/tests/xfs/909 b/tests/xfs/909 new file mode 100755 index 00000000..66587aa7 --- /dev/null +++ b/tests/xfs/909 @@ -0,0 +1,149 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 909 +# +# Check that we can upgrade a filesystem to support bigtime and that quota +# timers work properly after the upgrade. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/quota + +# real QA test starts here +_supported_fs xfs +_require_command "$XFS_ADMIN_PROG" "xfs_admin" +_require_quota +_require_scratch_xfs_bigtime +_require_xfs_repair_upgrade bigtime + +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \ + _notrun "Userspace does not support dates past 2038." + +rm -f $seqres.full + +# Format V5 filesystem without bigtime support and populate it +_scratch_mkfs -m crc=1,bigtime=0 >> $seqres.full +_qmount_option "usrquota" +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full +_scratch_mount >> $seqres.full + +# Force the block counters for uid 1 and 2 above zero +_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full +_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full +sync +chown 1 $SCRATCH_MNT/a +chown 2 $SCRATCH_MNT/b + +# Set quota limits on uid 1 before upgrading +$XFS_QUOTA_PROG -x -c 'limit -u bsoft=12k bhard=1m 1' $SCRATCH_MNT + +# Make sure the grace period is at /some/ point in the future. We have to +# use bc because not all bashes can handle integer comparisons with 64-bit +# numbers. +repquota -upn $SCRATCH_MNT > $tmp.repquota +cat $tmp.repquota >> $seqres.full +grace="$(cat $tmp.repquota | grep '^#1' | awk '{print $6}')" +now="$(date +%s)" +res="$(echo "${grace} > ${now}" | $BC_PROG)" +test $res -eq 1 || echo "Expected timer expiry (${grace}) to be after now (${now})." + +_scratch_unmount + +# Now upgrade to bigtime support +_scratch_xfs_admin -O bigtime=1 2>> $seqres.full +_check_scratch_xfs_features BIGTIME +_check_scratch_fs +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full + +# Mount again, see if our quota timer survived +_scratch_mount + +# Set a very generous grace period and quota limits on uid 2 after upgrading +$XFS_QUOTA_PROG -x -c 'timer -u -b -d 2147483647' $SCRATCH_MNT +$XFS_QUOTA_PROG -x -c 'limit -u bsoft=10000 bhard=150000 2' $SCRATCH_MNT + +# Query the grace periods to see if they got set properly after the upgrade. +repquota -upn $SCRATCH_MNT > $tmp.repquota +cat $tmp.repquota >> $seqres.full +grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')" +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +now="$(date +%s)" + +# Make sure that uid 1's expiration is in the future... +res1="$(echo "${grace} > ${now}" | $BC_PROG)" +test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})." + +# ...and that uid 2's expiration is after uid 1's... +res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)" +test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})." + +# ...and that uid 2's expiration is after 2038 if right now is far enough +# past 1970 that our generous grace period would provide for that. +res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)" +test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038." + +_scratch_cycle_mount + +# Query the grace periods to see if they survived a remount. +repquota -upn $SCRATCH_MNT > $tmp.repquota +cat $tmp.repquota >> $seqres.full +grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')" +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +now="$(date +%s)" + +# Make sure that uid 1's expiration is in the future... +res1="$(echo "${grace} > ${now}" | $BC_PROG)" +test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})." + +# ...and that uid 2's expiration is after uid 1's... +res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)" +test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})." + +# ...and that uid 2's expiration is after 2038 if right now is far enough +# past 1970 that our generous grace period would provide for that. +res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)" +test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038." + +# Now try setting uid 2's expiration to Feb 22 22:22:22 UTC 2222 +new_expiry=$(date -d 'Feb 22 22:22:22 UTC 2222' +%s) +now=$(date +%s) +test $now -ge $new_expiry && \ + echo "Now is after February 2222? Expect problems." +expiry_delta=$((new_expiry - now)) + +echo "setting expiration to $new_expiry - $now = $expiry_delta" >> $seqres.full +$XFS_QUOTA_PROG -x -c "timer -u $expiry_delta 2" -c 'report' $SCRATCH_MNT >> $seqres.full + +# Did we get an expiration within 5s of the target range? +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +echo "grace2 is $grace2" >> $seqres.full +_within_tolerance "grace2 expiry" $grace2 $new_expiry 5 -v + +_scratch_cycle_mount + +# ...and is it still within 5s after a remount? +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +echo "grace2 is $grace2" >> $seqres.full +_within_tolerance "grace2 expiry after remount" $grace2 $new_expiry 5 -v + +# success, all done +status=0 +exit diff --git a/tests/xfs/909.out b/tests/xfs/909.out new file mode 100644 index 00000000..72bf2416 --- /dev/null +++ b/tests/xfs/909.out @@ -0,0 +1,6 @@ +QA output created by 909 +Running xfs_repair to upgrade filesystem. +Adding large timestamp support to filesystem. +FEATURES: BIGTIME:YES +grace2 expiry is in range +grace2 expiry after remount is in range diff --git a/tests/xfs/group b/tests/xfs/group index b4e29bab..49bc4016 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -526,5 +526,7 @@ 768 auto quick repair 770 auto repair 773 auto quick repair +908 auto quick bigtime +909 auto quick bigtime quota 910 auto quick inobtcount 911 auto quick bigtime