From patchwork Wed Mar 31 01:08:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12174069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CE1FC433E9 for ; Wed, 31 Mar 2021 01:09:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68E34619DB for ; Wed, 31 Mar 2021 01:09:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232988AbhCaBIw (ORCPT ); Tue, 30 Mar 2021 21:08:52 -0400 Received: from mail.kernel.org ([198.145.29.99]:42190 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232883AbhCaBIo (ORCPT ); Tue, 30 Mar 2021 21:08:44 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id B1BAD6190A; Wed, 31 Mar 2021 01:08:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1617152923; bh=WaANTfCVqIMKX2ANFOL5uqkOxncUwAaBR4ESy9DPWrw=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=M3b8cohctasCYKX4XwhdRj35JbQJfR3DTii2+CHQegndm6a+QmV2MeJLrXMskxNAC e+B8zTtOfP5ILc7DlhIA1OBhztYWd1Q3lfVu/yipTbyYM6NmhgGz6nZaRsBZpF1pnm aQnd3ChqnOOUfFAm/fgh4ONZplowFNBW1uqvnewbOfJKn50ryw4N1AdPLLlDlzJ3lj ws8Y5Ob7XLKqilprnhUYT+laczKVtK7joV9YVpKGH+tdmxwrbKWDIyIjxiEVrP+E+Y 4cUy7iCIBRhThN0GllD1uSRoocM6mqNZG4mpQJrORHFYrOYL5hxOpHNnsBTXToX/l5 bZGyUq3nxbvyA== Subject: [PATCH 1/4] generic: check userspace handling of extreme timestamps From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Amir Goldstein , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 30 Mar 2021 18:08:41 -0700 Message-ID: <161715292137.2703979.17652236028578824526.stgit@magnolia> In-Reply-To: <161715291588.2703979.11541640936666929011.stgit@magnolia> References: <161715291588.2703979.11541640936666929011.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong These two tests ensure we can store and retrieve timestamps on the extremes of the date ranges supported by userspace, and the common places where overflows can happen. They differ from generic/402 in that they don't constrain the dates tested to the range that the filesystem claims to support; we attempt various things that /userspace/ can parse, and then check that the vfs clamps and persists the values correctly. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- tests/generic/721 | 123 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/721.out | 2 + tests/generic/722 | 125 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/722.out | 1 tests/generic/group | 6 ++ 5 files changed, 255 insertions(+), 2 deletions(-) create mode 100755 tests/generic/721 create mode 100644 tests/generic/721.out create mode 100755 tests/generic/722 create mode 100644 tests/generic/722.out diff --git a/tests/generic/721 b/tests/generic/721 new file mode 100755 index 00000000..9198b6b4 --- /dev/null +++ b/tests/generic/721 @@ -0,0 +1,123 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 721 +# +# Make sure we can store and retrieve timestamps on the extremes of the +# date ranges supported by userspace, and the common places where overflows +# can happen. +# +# This differs from generic/402 in that we don't constrain ourselves to the +# range that the filesystem claims to support; we attempt various things that +# /userspace/ can parse, and then check that the vfs clamps and persists the +# values correctly. +# +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently +# when writing timestamps to disk! This test detects this silent truncation +# and fails. If you see a failure on such a kernel, contact your distributor +# for an update. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs generic +_require_scratch + +rm -f $seqres.full + +_scratch_mkfs > $seqres.full +_scratch_mount + +# Does our userspace even support large dates? +test_bigdates=1 +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0 + +# And can we do statx? +test_statx=1 +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \ + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \ + test_statx=0 + +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full +echo "xfs_io support of statx: $test_statx" >> $seqres.full + +touchme() { + local arg="$1" + local name="$2" + + echo "$arg" > $SCRATCH_MNT/t_$name + touch -d "$arg" $SCRATCH_MNT/t_$name +} + +report() { + local files=($SCRATCH_MNT/t_*) + for file in "${files[@]}"; do + echo "${file}: $(cat "${file}")" + TZ=UTC stat -c '%y %Y %n' "${file}" + test $test_statx -gt 0 && \ + $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime' + done +} + +# -2147483648 (S32_MIN, or classic unix min) +touchme 'Dec 13 20:45:52 UTC 1901' s32_min + +# 2147483647 (S32_MAX, or classic unix max) +touchme 'Jan 19 03:14:07 UTC 2038' s32_max + +# 7956915742, all twos +touchme 'Feb 22 22:22:22 UTC 2222' all_twos + +if [ $test_bigdates -gt 0 ]; then + # 16299260424 (u64 nsec counter from s32_min, like xfs does) + touchme 'Tue Jul 2 20:20:24 UTC 2486' u64ns_from_s32_min + + # 15032385535 (u34 time if you start from s32_min, like ext4 does) + touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min + + # 17179869183 (u34 time if you start from the unix epoch) + touchme 'May 30 01:53:03 UTC 2514' u34_max + + # Latest date we can synthesize(?) + touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time + + # Earliest date we can synthesize(?) + touchme 'Jan 1 00:00:00 UTC 0' abs_min_time +fi + +# Query timestamps from incore +echo before >> $seqres.full +report > $tmp.before_remount +cat $tmp.before_remount >> $seqres.full + +_scratch_cycle_mount + +# Query timestamps from disk +echo after >> $seqres.full +report > $tmp.after_remount +cat $tmp.after_remount >> $seqres.full + +# Did they match? +cmp -s $tmp.before_remount $tmp.after_remount + +# success, all done +echo Silence is golden. +status=0 +exit diff --git a/tests/generic/721.out b/tests/generic/721.out new file mode 100644 index 00000000..b2bc6d58 --- /dev/null +++ b/tests/generic/721.out @@ -0,0 +1,2 @@ +QA output created by 721 +Silence is golden. diff --git a/tests/generic/722 b/tests/generic/722 new file mode 100755 index 00000000..305c3bd6 --- /dev/null +++ b/tests/generic/722 @@ -0,0 +1,125 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 722 +# +# Make sure we can store and retrieve timestamps on the extremes of the +# date ranges supported by userspace, and the common places where overflows +# can happen. This test also ensures that the timestamps are persisted +# correctly after a shutdown. +# +# This differs from generic/402 in that we don't constrain ourselves to the +# range that the filesystem claims to support; we attempt various things that +# /userspace/ can parse, and then check that the vfs clamps and persists the +# values correctly. +# +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently +# when writing timestamps to disk! This test detects this silent truncation +# and fails. If you see a failure on such a kernel, contact your distributor +# for an update. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs generic +_require_scratch +_require_scratch_shutdown + +rm -f $seqres.full + +_scratch_mkfs > $seqres.full +_scratch_mount + +# Does our userspace even support large dates? +test_bigdates=1 +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0 + +# And can we do statx? +test_statx=1 +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \ + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \ + test_statx=0 + +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full +echo "xfs_io support of statx: $test_statx" >> $seqres.full + +touchme() { + local arg="$1" + local name="$2" + + echo "$arg" > $SCRATCH_MNT/t_$name + touch -d "$arg" $SCRATCH_MNT/t_$name +} + +report() { + local files=($SCRATCH_MNT/t_*) + for file in "${files[@]}"; do + echo "${file}: $(cat "${file}")" + TZ=UTC stat -c '%y %Y %n' "${file}" + test $test_statx -gt 0 && \ + $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime' + done +} + +# -2147483648 (S32_MIN, or classic unix min) +touchme 'Dec 13 20:45:52 UTC 1901' s32_min + +# 2147483647 (S32_MAX, or classic unix max) +touchme 'Jan 19 03:14:07 UTC 2038' s32_max + +# 7956915742, all twos +touchme 'Feb 22 22:22:22 UTC 2222' all_twos + +if [ $test_bigdates -gt 0 ]; then + # 16299260424 (u64 nsec counter from s32_min, like xfs does) + touchme 'Tue Jul 2 20:20:24 UTC 2486' u64ns_from_s32_min + + # 15032385535 (u34 time if you start from s32_min, like ext4 does) + touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min + + # 17179869183 (u34 time if you start from the unix epoch) + touchme 'May 30 01:53:03 UTC 2514' u34_max + + # Latest date we can synthesize(?) + touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time + + # Earliest date we can synthesize(?) + touchme 'Jan 1 00:00:00 UTC 0' abs_min_time +fi + +# Query timestamps from incore +echo before >> $seqres.full +report > $tmp.before_crash +cat $tmp.before_crash >> $seqres.full + +_scratch_shutdown -f +_scratch_cycle_mount + +# Query timestamps from disk +echo after >> $seqres.full +report > $tmp.after_crash +cat $tmp.after_crash >> $seqres.full + +# Did they match? +cmp -s $tmp.before_crash $tmp.after_crash + +# success, all done +status=0 +exit diff --git a/tests/generic/722.out b/tests/generic/722.out new file mode 100644 index 00000000..83acd5cf --- /dev/null +++ b/tests/generic/722.out @@ -0,0 +1 @@ +QA output created by 722 diff --git a/tests/generic/group b/tests/generic/group index ab00cc04..f199c7ba 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -260,7 +260,7 @@ 255 auto quick prealloc punch 256 auto quick punch 257 dir auto quick -258 auto quick +258 auto quick bigtime 259 auto quick clone zero 260 auto quick trim 261 auto quick clone collapse @@ -404,7 +404,7 @@ 399 auto encrypt 400 auto quick quota 401 auto quick -402 auto quick rw +402 auto quick rw bigtime 403 auto quick attr 404 auto quick insert 405 auto mkfs thin @@ -634,3 +634,5 @@ 629 auto quick rw copy_range 630 auto quick rw dedupe clone 631 auto rw overlay rename +721 auto quick atime bigtime +722 auto quick atime bigtime shutdown From patchwork Wed Mar 31 01:08:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12174065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6507FC433E8 for ; Wed, 31 Mar 2021 01:09:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 46921619DA for ; Wed, 31 Mar 2021 01:09:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232989AbhCaBIw (ORCPT ); Tue, 30 Mar 2021 21:08:52 -0400 Received: from mail.kernel.org ([198.145.29.99]:42272 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232892AbhCaBIt (ORCPT ); Tue, 30 Mar 2021 21:08:49 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 33D4261935; Wed, 31 Mar 2021 01:08:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1617152929; bh=/IORSqdy+Ox8dFWvtPo45m+ddaFqcKeIwkSWZfD/B8g=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=AnLej2B2ZSBmuTUEhZFv94QDxzXRFEPWI1Oj+/ole7lZ05IoTK/9yP7FosliOlTkV 9gybZgVWk146xE8yyZR8L2fAnnTo1gWzlJJ3dey8jpv4O+IUUvH3LRYD5zAN/HfCzz OsMCOM4L6OFN5sFqsyS2o64izEbByykIGo9VnrWKtNlc8CgJwfRWInIvG51H55PWr+ AxiFiwMinP8DWyGcbeRH4NDu9MI/fpIF5B+eQw+wIgC1qEgcLhdlfW5sePgd2oLoGe eD+CsVNLk/MSWPBAJgx0GlhpQtZg1OdlML36y3mG2473FxEDtuySDbZmFBOfoQ7Xk5 ZCTNrZVXI41yQ== Subject: [PATCH 2/4] xfs/122: add legacy timestamps to ondisk checker From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Amir Goldstein , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 30 Mar 2021 18:08:46 -0700 Message-ID: <161715292687.2703979.6393758446727428463.stgit@magnolia> In-Reply-To: <161715291588.2703979.11541640936666929011.stgit@magnolia> References: <161715291588.2703979.11541640936666929011.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Add these new ondisk structures. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- tests/xfs/122 | 1 + tests/xfs/122.out | 1 + 2 files changed, 2 insertions(+) diff --git a/tests/xfs/122 b/tests/xfs/122 index 322e1d81..c8593315 100755 --- a/tests/xfs/122 +++ b/tests/xfs/122 @@ -181,6 +181,7 @@ struct xfs_iext_cursor struct xfs_ino_geometry struct xfs_attrlist struct xfs_attrlist_ent +struct xfs_legacy_ictimestamp EOF echo 'int main(int argc, char *argv[]) {' >>$cprog diff --git a/tests/xfs/122.out b/tests/xfs/122.out index b0773756..f229465a 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -97,6 +97,7 @@ sizeof(struct xfs_inode_log_format) = 56 sizeof(struct xfs_inode_log_format_32) = 52 sizeof(struct xfs_inumbers) = 24 sizeof(struct xfs_inumbers_req) = 64 +sizeof(struct xfs_legacy_timestamp) = 8 sizeof(struct xfs_log_dinode) = 176 sizeof(struct xfs_map_extent) = 32 sizeof(struct xfs_phys_extent) = 16 From patchwork Wed Mar 31 01:08:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12174081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57D23C433E3 for ; Wed, 31 Mar 2021 01:09:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C033619CF for ; Wed, 31 Mar 2021 01:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233067AbhCaBJX (ORCPT ); Tue, 30 Mar 2021 21:09:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:42364 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232892AbhCaBIz (ORCPT ); Tue, 30 Mar 2021 21:08:55 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id B3E0361953; Wed, 31 Mar 2021 01:08:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1617152934; bh=EUtYJujld5xMWf0YgJU+o1zPR1fKikX1eGVwQtc9oDo=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=hpUgDeRtJTnws6nQY/tOadOuYQri//sFXcUT+6GQwPKEK+eR53U9j79aD+rvFg5qI jSKw8iu+GWpLsx7/jF4yfaKouvJeDbbFk4pzhaGHodMt2VppcemdjrqEAqO9+zULaa ZN6vOZC7fK3dvUlckKZbia9MshvYLLUijh5+PRlHN+7nD/yiOgHYDKa2dmn0SdSosR mVavGpIJbMmuWGvK+9NEbN31vyApgv+lSfsf69qh2/jzjEl/4sadx1orWK6dUYzYLy /oU4+hvMC8zr5GgAzY/ftqsHM+1UxejIHB0XUZ2aXma2MM9K3YCS2/M2gt/hF0WkuB MLQWN/dEOZLDQ== Subject: [PATCH 3/4] xfs: detect time limits from filesystem From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Amir Goldstein , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 30 Mar 2021 18:08:52 -0700 Message-ID: <161715293237.2703979.18399191447470372230.stgit@magnolia> In-Reply-To: <161715291588.2703979.11541640936666929011.stgit@magnolia> References: <161715291588.2703979.11541640936666929011.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Teach fstests to extract timestamp limits of a filesystem using the new xfs_db timelimit command. Signed-off-by: Darrick J. Wong Reviewed-by: Amir Goldstein --- common/rc | 2 +- common/xfs | 19 +++++++++++++++++++ tests/xfs/911 | 44 ++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/911.out | 15 +++++++++++++++ tests/xfs/group | 1 + 5 files changed, 80 insertions(+), 1 deletion(-) create mode 100755 tests/xfs/911 create mode 100644 tests/xfs/911.out diff --git a/common/rc b/common/rc index 23f86ce6..21b87557 100644 --- a/common/rc +++ b/common/rc @@ -2054,7 +2054,7 @@ _filesystem_timestamp_range() echo "0 $u32max" ;; xfs) - echo "$s32min $s32max" + _xfs_timestamp_range "$device" ;; btrfs) echo "$s64min $s64max" diff --git a/common/xfs b/common/xfs index a65eeb58..37658788 100644 --- a/common/xfs +++ b/common/xfs @@ -1133,3 +1133,22 @@ _require_xfs_scratch_inobtcount() _notrun "inobtcount not supported by scratch filesystem type: $FSTYP" _scratch_unmount } + +_xfs_timestamp_range() +{ + local device="$1" + local use_db=0 + local dbprog="$XFS_DB_PROG $device" + test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db + + $dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1 + if [ $use_db -eq 0 ]; then + # The "timelimit" command was added to xfs_db at the same time + # that bigtime was added to xfsprogs. Therefore, we can assume + # the old timestamp range if the command isn't present. + echo "-$((1<<31)) $(((1<<31)-1))" + else + $dbprog -f -c 'timelimit --compact' | \ + awk '{printf("%s %s", $1, $2);}' + fi +} diff --git a/tests/xfs/911 b/tests/xfs/911 new file mode 100755 index 00000000..01ddb856 --- /dev/null +++ b/tests/xfs/911 @@ -0,0 +1,44 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 911 +# +# Check that the xfs_db timelimit command prints the ranges that we expect. +# This in combination with an xfs_ondisk.h build time check in the kernel +# ensures that the kernel agrees with userspace. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_xfs_db_command timelimit + +rm -f $seqres.full + +# Format filesystem without bigtime support and populate it +_scratch_mkfs > $seqres.full +echo classic xfs timelimits +_scratch_xfs_db -c 'timelimit --classic' +echo bigtime xfs timelimits +_scratch_xfs_db -c 'timelimit --bigtime' + +# success, all done +status=0 +exit diff --git a/tests/xfs/911.out b/tests/xfs/911.out new file mode 100644 index 00000000..84dc475b --- /dev/null +++ b/tests/xfs/911.out @@ -0,0 +1,15 @@ +QA output created by 911 +classic xfs timelimits +time.min = -2147483648 +time.max = 2147483647 +dqtimer.min = 1 +dqtimer.max = 4294967295 +dqgrace.min = 0 +dqgrace.min = 4294967295 +bigtime xfs timelimits +time.min = -2147483648 +time.max = 16299260424 +dqtimer.min = 4 +dqtimer.max = 16299260424 +dqgrace.min = 0 +dqgrace.min = 4294967295 diff --git a/tests/xfs/group b/tests/xfs/group index 0dc8038a..e212fd46 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -525,3 +525,4 @@ 770 auto repair 773 auto quick repair 910 auto quick inobtcount +911 auto quick bigtime From patchwork Wed Mar 31 01:08:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12174079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B1D1C433E4 for ; Wed, 31 Mar 2021 01:09:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D3F3619D7 for ; Wed, 31 Mar 2021 01:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232702AbhCaBJX (ORCPT ); Tue, 30 Mar 2021 21:09:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:42414 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232997AbhCaBJA (ORCPT ); Tue, 30 Mar 2021 21:09:00 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3C84261953; Wed, 31 Mar 2021 01:09:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1617152940; bh=BjJ/bT1d2U9ska1hl1HFCpuJMwLEdI5tnMN+reKwU6g=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=N+sXKHax095ju2qNzDI6nIp8TqgLWS/KCyhPzu7KlU8pyA2TSSp9OmUTP8K7w0Pzk ccC/fX637v2LdL/4IgVeRl2mDBjf541Nmxb1tWiKOJ+8G3TfV2R6N+Avb5PyrVszMW 8A4iteeqdhxV848QSZLa6hMjNbqaD8oXvLCJ7hvxsYqT3Zpwmm2NHQzNB+G4g7/nRP HjVEWyQRCktnB1yE6jCYyuZ20yV6J/GpW7GjcdE/62OZbBQPK9RNkcR6MFN6jYuuKR z154NWSv/X4PspLrTql/JNx79PBTGOW5dNO4kaRXgeqi6wwYKyUe29Q+rj/Bb7m/b5 JoyLC/AjCqdBA== Subject: [PATCH 4/4] xfs: test upgrading filesystem to bigtime From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Tue, 30 Mar 2021 18:08:57 -0700 Message-ID: <161715293790.2703979.8248551223530213245.stgit@magnolia> In-Reply-To: <161715291588.2703979.11541640936666929011.stgit@magnolia> References: <161715291588.2703979.11541640936666929011.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Darrick J. Wong Test that we can upgrade an existing filesystem to use bigtime. Signed-off-by: Darrick J. Wong --- common/xfs | 13 +++++ tests/xfs/908 | 97 +++++++++++++++++++++++++++++++++++ tests/xfs/908.out | 20 +++++++ tests/xfs/909 | 149 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/909.out | 6 ++ tests/xfs/group | 2 + 6 files changed, 287 insertions(+) create mode 100755 tests/xfs/908 create mode 100644 tests/xfs/908.out create mode 100755 tests/xfs/909 create mode 100644 tests/xfs/909.out diff --git a/common/xfs b/common/xfs index 37658788..c430b3ac 100644 --- a/common/xfs +++ b/common/xfs @@ -1152,3 +1152,16 @@ _xfs_timestamp_range() awk '{printf("%s %s", $1, $2);}' fi } + +_require_xfs_scratch_bigtime() +{ + _require_scratch + + _scratch_mkfs -m bigtime=1 &>/dev/null || \ + _notrun "mkfs.xfs doesn't have bigtime feature" + _try_scratch_mount || \ + _notrun "bigtime not supported by scratch filesystem type: $FSTYP" + $XFS_INFO_PROG "$SCRATCH_MNT" | grep -q "bigtime=1" || \ + _notrun "bigtime feature not advertised on mount?" + _scratch_unmount +} diff --git a/tests/xfs/908 b/tests/xfs/908 new file mode 100755 index 00000000..1ad3131a --- /dev/null +++ b/tests/xfs/908 @@ -0,0 +1,97 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 908 +# +# Check that we can upgrade a filesystem to support bigtime and that inode +# timestamps work properly after the upgrade. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_command "$XFS_ADMIN_PROG" "xfs_admin" +_require_xfs_scratch_bigtime +_require_xfs_repair_upgrade bigtime + +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \ + _notrun "Userspace does not support dates past 2038." + +rm -f $seqres.full + +# Make sure we can't upgrade a V4 filesystem +_scratch_mkfs -m crc=0 >> $seqres.full +_scratch_xfs_admin -O bigtime=1 2>> $seqres.full +_check_scratch_xfs_features BIGTIME + +# Make sure we're required to specify a feature status +_scratch_mkfs -m crc=1,bigtime=0,inobtcount=0 >> $seqres.full +_scratch_xfs_admin -O bigtime 2>> $seqres.full + +# Can we add bigtime and inobtcount at the same time? +_scratch_mkfs -m crc=1,bigtime=0,inobtcount=0 >> $seqres.full +_scratch_xfs_admin -O bigtime=1,inobtcount=1 2>> $seqres.full + +# Format V5 filesystem without bigtime support and populate it +_scratch_mkfs -m crc=1,bigtime=0 >> $seqres.full +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full +_scratch_mount >> $seqres.full + +touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/a +touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/b +ls -la $SCRATCH_MNT/* >> $seqres.full + +echo before upgrade: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +_scratch_unmount +_check_scratch_fs + +# Now upgrade to bigtime support +_scratch_xfs_admin -O bigtime=1 2>> $seqres.full +_check_scratch_xfs_features BIGTIME +_check_scratch_fs +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full + +# Mount again, look at our files +_scratch_mount >> $seqres.full +ls -la $SCRATCH_MNT/* >> $seqres.full + +# Modify one of the timestamps to stretch beyond 2038 +touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b + +echo after upgrade: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +_scratch_cycle_mount + +# Did the timestamp survive the remount? +ls -la $SCRATCH_MNT/* >> $seqres.full + +echo after upgrade and remount: +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b + +# success, all done +status=0 +exit diff --git a/tests/xfs/908.out b/tests/xfs/908.out new file mode 100644 index 00000000..ab04a3fc --- /dev/null +++ b/tests/xfs/908.out @@ -0,0 +1,20 @@ +QA output created by 908 +Running xfs_repair to upgrade filesystem. +Large timestamp feature only supported on V5 filesystems. +FEATURES: BIGTIME:NO +Running xfs_repair to upgrade filesystem. +Running xfs_repair to upgrade filesystem. +Adding inode btree counts to filesystem. +Adding large timestamp support to filesystem. +before upgrade: +915909559 +915909559 +Running xfs_repair to upgrade filesystem. +Adding large timestamp support to filesystem. +FEATURES: BIGTIME:YES +after upgrade: +915909559 +7956915742 +after upgrade and remount: +915909559 +7956915742 diff --git a/tests/xfs/909 b/tests/xfs/909 new file mode 100755 index 00000000..b33e6dd5 --- /dev/null +++ b/tests/xfs/909 @@ -0,0 +1,149 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 909 +# +# Check that we can upgrade a filesystem to support bigtime and that quota +# timers work properly after the upgrade. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/quota + +# real QA test starts here +_supported_fs xfs +_require_command "$XFS_ADMIN_PROG" "xfs_admin" +_require_quota +_require_xfs_scratch_bigtime +_require_xfs_repair_upgrade bigtime + +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \ + _notrun "Userspace does not support dates past 2038." + +rm -f $seqres.full + +# Format V5 filesystem without bigtime support and populate it +_scratch_mkfs -m crc=1,bigtime=0 >> $seqres.full +_qmount_option "usrquota" +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full +_scratch_mount >> $seqres.full + +# Force the block counters for uid 1 and 2 above zero +_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full +_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full +sync +chown 1 $SCRATCH_MNT/a +chown 2 $SCRATCH_MNT/b + +# Set quota limits on uid 1 before upgrading +$XFS_QUOTA_PROG -x -c 'limit -u bsoft=12k bhard=1m 1' $SCRATCH_MNT + +# Make sure the grace period is at /some/ point in the future. We have to +# use bc because not all bashes can handle integer comparisons with 64-bit +# numbers. +repquota -upn $SCRATCH_MNT > $tmp.repquota +cat $tmp.repquota >> $seqres.full +grace="$(cat $tmp.repquota | grep '^#1' | awk '{print $6}')" +now="$(date +%s)" +res="$(echo "${grace} > ${now}" | $BC_PROG)" +test $res -eq 1 || echo "Expected timer expiry (${grace}) to be after now (${now})." + +_scratch_unmount + +# Now upgrade to bigtime support +_scratch_xfs_admin -O bigtime=1 2>> $seqres.full +_check_scratch_xfs_features BIGTIME +_check_scratch_fs +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full + +# Mount again, see if our quota timer survived +_scratch_mount + +# Set a very generous grace period and quota limits on uid 2 after upgrading +$XFS_QUOTA_PROG -x -c 'timer -u -b -d 2147483647' $SCRATCH_MNT +$XFS_QUOTA_PROG -x -c 'limit -u bsoft=10000 bhard=150000 2' $SCRATCH_MNT + +# Query the grace periods to see if they got set properly after the upgrade. +repquota -upn $SCRATCH_MNT > $tmp.repquota +cat $tmp.repquota >> $seqres.full +grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')" +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +now="$(date +%s)" + +# Make sure that uid 1's expiration is in the future... +res1="$(echo "${grace} > ${now}" | $BC_PROG)" +test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})." + +# ...and that uid 2's expiration is after uid 1's... +res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)" +test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})." + +# ...and that uid 2's expiration is after 2038 if right now is far enough +# past 1970 that our generous grace period would provide for that. +res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)" +test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038." + +_scratch_cycle_mount + +# Query the grace periods to see if they survived a remount. +repquota -upn $SCRATCH_MNT > $tmp.repquota +cat $tmp.repquota >> $seqres.full +grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')" +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +now="$(date +%s)" + +# Make sure that uid 1's expiration is in the future... +res1="$(echo "${grace} > ${now}" | $BC_PROG)" +test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})." + +# ...and that uid 2's expiration is after uid 1's... +res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)" +test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})." + +# ...and that uid 2's expiration is after 2038 if right now is far enough +# past 1970 that our generous grace period would provide for that. +res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)" +test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038." + +# Now try setting uid 2's expiration to Feb 22 22:22:22 UTC 2222 +new_expiry=$(date -d 'Feb 22 22:22:22 UTC 2222' +%s) +now=$(date +%s) +test $now -ge $new_expiry && \ + echo "Now is after February 2222? Expect problems." +expiry_delta=$((new_expiry - now)) + +echo "setting expiration to $new_expiry - $now = $expiry_delta" >> $seqres.full +$XFS_QUOTA_PROG -x -c "timer -u $expiry_delta 2" -c 'report' $SCRATCH_MNT >> $seqres.full + +# Did we get an expiration within 5s of the target range? +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +echo "grace2 is $grace2" >> $seqres.full +_within_tolerance "grace2 expiry" $grace2 $new_expiry 5 -v + +_scratch_cycle_mount + +# ...and is it still within 5s after a remount? +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')" +echo "grace2 is $grace2" >> $seqres.full +_within_tolerance "grace2 expiry after remount" $grace2 $new_expiry 5 -v + +# success, all done +status=0 +exit diff --git a/tests/xfs/909.out b/tests/xfs/909.out new file mode 100644 index 00000000..72bf2416 --- /dev/null +++ b/tests/xfs/909.out @@ -0,0 +1,6 @@ +QA output created by 909 +Running xfs_repair to upgrade filesystem. +Adding large timestamp support to filesystem. +FEATURES: BIGTIME:YES +grace2 expiry is in range +grace2 expiry after remount is in range diff --git a/tests/xfs/group b/tests/xfs/group index e212fd46..7e56b383 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -524,5 +524,7 @@ 768 auto quick repair 770 auto repair 773 auto quick repair +908 auto quick bigtime +909 auto quick bigtime quota 910 auto quick inobtcount 911 auto quick bigtime