From patchwork Tue Feb 12 02:17:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10807219 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62D3813B4 for ; Tue, 12 Feb 2019 02:18:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DD2B2B2D6 for ; Tue, 12 Feb 2019 02:18:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4075D2B2D7; Tue, 12 Feb 2019 02:18:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A20B2B2D0 for ; Tue, 12 Feb 2019 02:18:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727714AbfBLCSK (ORCPT ); Mon, 11 Feb 2019 21:18:10 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:50146 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727265AbfBLCSK (ORCPT ); Mon, 11 Feb 2019 21:18:10 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1C2Eqf2030489; Tue, 12 Feb 2019 02:18:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=kAzXgnIhfOjDMAgGpLwgS5wVOY3s8q4PvAtEKUN9/84=; b=G34az8N2rAAJA4Q8Pq4W9xaJp6T+AHlY2BFWqAzhgPrmxJtIlRn3oMA4X28kDgQMzZvi CXbxyX9Z+iMbMA2077fNDgqK/OBT2kRJVBE6crXBnKqa4OYZBk7cDkEU+hUGe7IYcxTb iiQ87y+ZnlBrWC81pQQx+Y38ryLUenhElbGpxY6sQMOWOX3ckbxCH025RXRHqekOQGh6 OOeNaMbXfTmsHHz5d2IaIQDkQiPIeHnjFhBbcWnsI2Sdri2bby20vp/1+lWbMnajKWiZ 4KMbqHIMLcLMw04hU11ufSoasikCQdU3tSChpmLG3omHaOT3NdoH/2hDzFnJueX4v6kt Sg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2qhredsad3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Feb 2019 02:18:05 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1C2I0Io004190 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Feb 2019 02:18:00 GMT Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1C2I0TD032564; Tue, 12 Feb 2019 02:18:00 GMT Received: from localhost (/10.159.237.14) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 11 Feb 2019 18:17:59 -0800 Subject: [PATCH 2/2] generic: check the behavior of programs opening a lot of O_TMPFILE files From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Mon, 11 Feb 2019 18:17:54 -0800 Message-ID: <154993787478.2062.17047634303195751186.stgit@magnolia> In-Reply-To: <154993786231.2062.8490515215533507881.stgit@magnolia> References: <154993786231.2062.8490515215533507881.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9164 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902120014 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Create a test (+ helper program) that opens as many unlinked files as it possibly can on the scratch filesystem, then closes all the files at once to stress-test unlinked file cleanup. Add an xfs-specific test to make sure that the fallback code doesn't bitrot. Signed-off-by: Darrick J. Wong --- src/Makefile | 2 - src/tmpfile.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/710 | 65 +++++++++++++++++++++++++ tests/generic/710.out | 2 + tests/generic/711 | 73 ++++++++++++++++++++++++++++ tests/generic/711.out | 2 + tests/generic/group | 2 + tests/xfs/736 | 71 +++++++++++++++++++++++++++ tests/xfs/736.out | 2 + tests/xfs/737 | 79 ++++++++++++++++++++++++++++++ tests/xfs/737.out | 2 + tests/xfs/group | 2 + 12 files changed, 428 insertions(+), 1 deletion(-) create mode 100644 src/tmpfile.c create mode 100755 tests/generic/710 create mode 100644 tests/generic/710.out create mode 100755 tests/generic/711 create mode 100644 tests/generic/711.out create mode 100755 tests/xfs/736 create mode 100644 tests/xfs/736.out create mode 100755 tests/xfs/737 create mode 100644 tests/xfs/737.out diff --git a/src/Makefile b/src/Makefile index 41826585..5fce881d 100644 --- a/src/Makefile +++ b/src/Makefile @@ -27,7 +27,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ renameat2 t_getcwd e4compact test-nextquota punch-alternating \ attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \ dio-invalidate-cache stat_test t_encrypted_d_revalidate \ - attr_replace_test swapon mkswap + attr_replace_test swapon mkswap tmpfile SUBDIRS = log-writes perf diff --git a/src/tmpfile.c b/src/tmpfile.c new file mode 100644 index 00000000..1b74dc72 --- /dev/null +++ b/src/tmpfile.c @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Copyright (C) 2019 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + * + * Test program to open unlinked files and leak them. + */ +#ifndef _GNU_SOURCE +# define _GNU_SOURCE +#endif +#include +#include +#include +#include +#include +#include +#include +#include + +static int min_fd = -1; +static int max_fd = -1; +static unsigned int nr_opened = 0; +static float start_time; + +void clock_time(float *time) +{ + static clockid_t clkid = CLOCK_MONOTONIC; + struct timespec ts; + int ret; + +retry: + ret = clock_gettime(clkid, &ts); + if (ret) { + if (clkid == CLOCK_MONOTONIC) { + clkid = CLOCK_REALTIME; + goto retry; + } + perror("clock_gettime"); + exit(2); + } + *time = ts.tv_sec + ((float)ts.tv_nsec / 1000000000); +} + +/* + * Exit the program due to an error. + * + * If we've exhausted all the file descriptors, make sure we close all the + * open fds in the order we received them in order to exploit a quirk of ext4 + * and xfs where the oldest unlinked inodes are at the /end/ of the unlinked + * lists, which will make removing the unlinked files maximally painful. + * + * If it's some other error, just die and let the kernel sort it out. + */ +void die(void) +{ + float end_time; + int fd; + + switch (errno) { + case EMFILE: + case ENFILE: + case ENOSPC: + clock_time(&end_time); + printf("Opened %u files in %.2fs.\n", nr_opened, + end_time - start_time); + fflush(stdout); + + clock_time(&start_time); + for (fd = min_fd; fd <= max_fd; fd++) + close(fd); + clock_time(&end_time); + printf("Closed %u files in %.2fs.\n", nr_opened, + end_time - start_time); + exit(0); + break; + default: + perror("open?"); + exit(2); + break; + } +} + +/* Remember how many file we open and all that. */ +void remember_fd(int fd) +{ + if (min_fd == -1 || min_fd > fd) + min_fd = fd; + if (max_fd == -1 || max_fd < fd) + max_fd = fd; + nr_opened++; +} + +/* Put an opened file on the unlinked list and leak the fd. */ +void leak_tmpfile(void) +{ + int fd = -1; + int ret; + + /* Try to create an O_TMPFILE and leak the fd. */ +#ifdef O_TMPFILE + fd = open(".", O_TMPFILE | O_RDWR, 0644); + if (fd >= 0) { + remember_fd(fd); + return; + } + if (fd < 0 && errno != EOPNOTSUPP) + die(); +#endif + + /* Oh well, create a new file, unlink it, and leak the fd. */ + fd = open("./moo", O_CREAT | O_RDWR, 0644); + if (fd < 0) + die(); + ret = unlink("./moo"); + if (ret) + die(); + remember_fd(fd); +} + +/* Try to put as many files on the unlinked list and then kill them. */ +int main(int argc, char *argv[]) +{ + clock_time(&start_time); + while (1) + leak_tmpfile(); + return 0; +} diff --git a/tests/generic/710 b/tests/generic/710 new file mode 100755 index 00000000..18aa9d34 --- /dev/null +++ b/tests/generic/710 @@ -0,0 +1,65 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2019 Oracle, Inc. All Rights Reserved. +# +# FS QA Test No. 710 +# +# Stress test creating a lot of unlinked O_TMPFILE files and closing them +# all at once, checking that we don't blow up the filesystem. This is sort +# of a performance test for the xfs unlinked inode backref patchset, but it +# applies to most other filesystems. +# +# Use only a single CPU to test the single threaded situation. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +testfile=$TEST_DIR/$seq.txt +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr +. ./common/filter + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch + +rm -f $seqres.full +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +# Set ULIMIT_NOFILE to min(file-max, 50000 files per LOAD_FACTOR) +# so that this test doesn't take forever or OOM the box +max_files=$((50000 * LOAD_FACTOR)) +max_allowable_files=$(cat /proc/sys/fs/file-max) +test $max_files -gt $max_allowable_files && max_files=$max_allowable_files +ulimit -n $max_files + +# Open a lot of unlinked files +echo create >> $seqres.full +program=$PWD/src/tmpfile +(cd $SCRATCH_MNT ; $program >> $seqres.full) + +# Unmount to prove that we can clean it all +echo umount >> $seqres.full +before=$(date +%s) +_scratch_unmount +after=$(date +%s) +echo "Unmount took $((after - before))s." >> $seqres.full + +# Mount so that we can run the usual checks +echo silence is golden +_scratch_mount +status=0 +exit diff --git a/tests/generic/710.out b/tests/generic/710.out new file mode 100644 index 00000000..e0a55170 --- /dev/null +++ b/tests/generic/710.out @@ -0,0 +1,2 @@ +QA output created by 710 +silence is golden diff --git a/tests/generic/711 b/tests/generic/711 new file mode 100755 index 00000000..11d76218 --- /dev/null +++ b/tests/generic/711 @@ -0,0 +1,73 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2019 Oracle, Inc. All Rights Reserved. +# +# FS QA Test No. 711 +# +# Stress test creating a lot of unlinked O_TMPFILE files and closing them +# all at once, checking that we don't blow up the filesystem. This is sort +# of a performance test for the xfs unlinked inode backref patchset, but it +# applies to most other filesystems. +# +# Use every CPU possible to stress the filesystem. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +testfile=$TEST_DIR/$seq.txt +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr +. ./common/filter + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch + +rm -f $seqres.full +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +# Try to load up all the CPUs, two threads per CPU. +nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 )) + +# Set ULIMIT_NOFILE to min(file-max, 50000 files per LOAD_FACTOR) +# so that this test doesn't take forever or OOM the box +max_files=$((50000 * LOAD_FACTOR)) +max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / nr_cpus )) +test $max_files -gt $max_allowable_files && max_files=$max_allowable_files +ulimit -n $max_files + +# Open a lot of unlinked files +echo create >> $seqres.full +program=$PWD/src/tmpfile +for i in $(seq 1 $nr_cpus); do + (mkdir $SCRATCH_MNT/$i ; cd $SCRATCH_MNT/$i ; $program >> $seqres.full) & +done +for i in $(seq 1 $nr_cpus); do + wait +done + +# Unmount to prove that we can clean it all +echo umount >> $seqres.full +before=$(date +%s) +_scratch_unmount +after=$(date +%s) +echo "Unmount took $((after - before))s." >> $seqres.full + +# Mount so that we can run the usual checks +echo silence is golden +_scratch_mount +status=0 +exit diff --git a/tests/generic/711.out b/tests/generic/711.out new file mode 100644 index 00000000..cbbe36e9 --- /dev/null +++ b/tests/generic/711.out @@ -0,0 +1,2 @@ +QA output created by 711 +silence is golden diff --git a/tests/generic/group b/tests/generic/group index f56eb475..26999ca1 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -529,3 +529,5 @@ 524 auto quick 525 auto quick rw 709 auto quick +710 auto quick unlink +711 auto quick unlink diff --git a/tests/xfs/736 b/tests/xfs/736 new file mode 100755 index 00000000..e33de0ae --- /dev/null +++ b/tests/xfs/736 @@ -0,0 +1,71 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2019 Oracle, Inc. All Rights Reserved. +# +# FS QA Test No. 736 +# +# Stress test creating a lot of unlinked O_TMPFILE files and closing them +# all at once, checking that we don't blow up the filesystem. This is sort +# of a performance test for the xfs unlinked inode backref patchset, but it +# applies to most other filesystems. +# +# Here we force the use of the slow iunlink bucket walk code in a single +# threaded situation. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +testfile=$TEST_DIR/$seq.txt +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr +. ./common/filter +. ./common/inject + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_xfs_io_error_injection "iunlink_fallback" +_require_scratch + +rm -f $seqres.full +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +# Set ULIMIT_NOFILE to min(file-max, 30000 files per LOAD_FACTOR) +# so that this test doesn't take forever or OOM the box +max_files=$((30000 * LOAD_FACTOR)) +max_allowable_files=$(cat /proc/sys/fs/file-max) +test $max_files -gt $max_allowable_files && max_files=$max_allowable_files +ulimit -n $max_files + +# Force xfs to use the iunlinked fallback 50% of the time +_scratch_inject_error "iunlink_fallback" "2" + +# Open a lot of unlinked files +echo create >> $seqres.full +program=$PWD/src/tmpfile +(cd $SCRATCH_MNT ; $program >> $seqres.full) + +# Unmount to prove that we can clean it all +echo umount >> $seqres.full +before=$(date +%s) +_scratch_unmount +after=$(date +%s) +echo "Unmount took $((after - before))s." >> $seqres.full + +# Mount so that we can run the usual checks +echo silence is golden +_scratch_mount +status=0 +exit diff --git a/tests/xfs/736.out b/tests/xfs/736.out new file mode 100644 index 00000000..0258a248 --- /dev/null +++ b/tests/xfs/736.out @@ -0,0 +1,2 @@ +QA output created by 736 +silence is golden diff --git a/tests/xfs/737 b/tests/xfs/737 new file mode 100755 index 00000000..47e65607 --- /dev/null +++ b/tests/xfs/737 @@ -0,0 +1,79 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0+ +# Copyright (c) 2019 Oracle, Inc. All Rights Reserved. +# +# FS QA Test No. 737 +# +# Stress test creating a lot of unlinked O_TMPFILE files and closing them +# all at once, checking that we don't blow up the filesystem. This is sort +# of a performance test for the xfs unlinked inode backref patchset, but it +# applies to most other filesystems. +# +# Here we force the use of the slow iunlink bucket walk code, using every +# CPU possible. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" +tmp=/tmp/$$ +status=1 # failure is the default! +testfile=$TEST_DIR/$seq.txt +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr +. ./common/filter +. ./common/inject + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_xfs_io_error_injection "iunlink_fallback" +_require_scratch + +rm -f $seqres.full +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +# Load up all the CPUs, two threads per CPU. +nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 )) + +# Set ULIMIT_NOFILE to min(file-max, 30000 files per cpu per LOAD_FACTOR) +# so that this test doesn't take forever or OOM the box +max_files=$((30000 * LOAD_FACTOR)) +max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / nr_cpus )) +test $max_files -gt $max_allowable_files && max_files=$max_allowable_files +ulimit -n $max_files + +# Force xfs to use the iunlinked fallback 50% of the time +_scratch_inject_error "iunlink_fallback" "2" + +# Open a lot of unlinked files +echo create >> $seqres.full +program=$PWD/src/tmpfile +for i in $(seq 1 $nr_cpus); do + (mkdir $SCRATCH_MNT/$i ; cd $SCRATCH_MNT/$i ; $program >> $seqres.full) & +done +for i in $(seq 1 $nr_cpus); do + wait +done + +# Unmount to prove that we can clean it all +echo umount >> $seqres.full +before=$(date +%s) +_scratch_unmount +after=$(date +%s) +echo "Unmount took $((after - before))s." >> $seqres.full + +# Mount so that we can run the usual checks +echo silence is golden +_scratch_mount +status=0 +exit diff --git a/tests/xfs/737.out b/tests/xfs/737.out new file mode 100644 index 00000000..bdc4966d --- /dev/null +++ b/tests/xfs/737.out @@ -0,0 +1,2 @@ +QA output created by 737 +silence is golden diff --git a/tests/xfs/group b/tests/xfs/group index 7b7d69f1..d3189cd5 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -497,3 +497,5 @@ 497 dangerous_fuzzers dangerous_scrub dangerous_online_repair 498 dangerous_fuzzers dangerous_norepair 499 auto quick +736 auto quick unlink +737 auto quick unlink