From patchwork Fri Dec 7 06:23:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717505 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D7C4018B8 for ; Fri, 7 Dec 2018 06:23:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C80862EB2F for ; Fri, 7 Dec 2018 06:23:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BC0A92EB31; Fri, 7 Dec 2018 06:23:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7ACAB2EB37 for ; Fri, 7 Dec 2018 06:23:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725966AbeLGGXa (ORCPT ); Fri, 7 Dec 2018 01:23:30 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:56314 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGXa (ORCPT ); Fri, 7 Dec 2018 01:23:30 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76J1fT111722; Fri, 7 Dec 2018 06:23:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=c2gf9OyeeVcgIARqP4d0ppJwNicXLRI22E3vP0Qsua0=; b=Xbyb0r8cSssspAS/uXYo3BUsKtb7ACoDjpglYwVvvqLJNSzwpMMXZtjszwGtDWiO6jMb tJU1m+iZq5QTNrAYUMdB3gM5AwFOmSI2+97LFBDOHlHR0QO0yLscX+QMfS75O3ZXr8my F4+/JAhnG6hyxFhdOhWYFAJPFUfGTKQrmY78AYjW3nkn3IIQIeuubw/NNBevkWGlp0Ut DVRF9APNKNJpthyk/2rqfl+ET8oANz0szM06ioBIlSbzHeF5szUj3kKwVkwzdk3fpScZ oCdqbBL8A6D7L/UG4c5qEkS5v0NjrfwE2tTYAd1pZcCytHB/5IM73mCqvYV5Q3OBqJPB 9Q== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2p3jxrv0t9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:23:28 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76NRro016329 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:23:28 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB76NRWi027412; Fri, 7 Dec 2018 06:23:27 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:23:27 -0800 Subject: [PATCH 01/12] fsstress: fix compiler warnings From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Allison Henderson , fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:23:18 -0800 Message-ID: <154416379882.15344.9436421202591701307.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=812 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070054 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix all the gcc warnings in fsstress, and resize buffers to avoid overflows. Signed-off-by: Darrick J. Wong Reviewed-By: Allison Henderson --- ltp/fsstress.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index edbb3f2f..8d0734fc 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -499,7 +499,7 @@ int main(int argc, char **argv) exit(1); } if (logname) { - char path[PATH_MAX]; + char path[PATH_MAX + NAME_MAX + 1]; snprintf(path, sizeof(path), "%s/%s", rpath, logname); if (freopen(path, "a", stdout) == NULL) { perror("freopen logfile failed"); @@ -583,7 +583,7 @@ int main(int argc, char **argv) return 0; #endif if (logname) { - char path[PATH_MAX]; + char path[PATH_MAX + NAME_MAX + 2 + 11]; snprintf(path, sizeof(path), "%s/%s.%d", rpath, logname, i); if (freopen(path, "a", stdout) == NULL) { From patchwork Fri Dec 7 06:23:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717509 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD91518B8 for ; Fri, 7 Dec 2018 06:23:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A3912EB6A for ; Fri, 7 Dec 2018 06:23:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86C2D2EB62; Fri, 7 Dec 2018 06:23:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04E532EB64 for ; Fri, 7 Dec 2018 06:23:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725952AbeLGGXi (ORCPT ); Fri, 7 Dec 2018 01:23:38 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:41088 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725948AbeLGGXi (ORCPT ); Fri, 7 Dec 2018 01:23:38 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76IwSw149215; Fri, 7 Dec 2018 06:23:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=FwPRgh+fHgPQoYOrKPR+70hjf00HoHLhgIpSh6YBQTo=; b=0bHezb7paEN3PhwGT7TAXINBJS1RMBj6hfLhmbX+UAGnZmY3Fce9tdNBH+VPdRNaNjCC zw15GLpwW41wRTJuhYGbXr+RZ7JtkC9/izOnHNGelH2k74pHjklUg9/M8LCwuGJz2bTB Qq/rJcxV2vnlYz3x482UVDs7Jf2Lqe+dG5Zuljm401I+KhrqKpg6YTieV8/OMrEGP5Sd VEIYhL0E+9AmRRj3bD5qpuX6yFolUQKmPxMWjO1269I9cMXwO+ddDn2zgsl2niLDum8+ TsTlynEx3e8rsm5RK8C8yk8JmQdlVLwC+cMrAh7xduMo51JWD+A1QAzh5C1YpTfng0K3 aA== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2p3hquc4n9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:23:36 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76NYmc016505 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:23:35 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB76NYZF025786; Fri, 7 Dec 2018 06:23:34 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:23:34 -0800 Subject: [PATCH 02/12] fsstress: check system call return values From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Allison Henderson , fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:23:32 -0800 Message-ID: <154416381284.15344.6497982978936980932.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=775 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070054 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Check the return values of various system calls and blow up if something went wrong. Signed-off-by: Darrick J. Wong Reviewed-By: Allison Henderson --- ltp/fsstress.c | 56 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 30 insertions(+), 26 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 8d0734fc..af5d125f 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -733,7 +733,7 @@ append_pathname(pathname_t *name, char *str) /* attempting to append to a dir a zero length path */ if (len && *str == '/' && name->len == 0) { fprintf(stderr, "fsstress: append_pathname failure\n"); - chdir(homedir); + assert(chdir(homedir) == 0); abort(); /* NOTREACHED */ } @@ -765,7 +765,7 @@ attr_list_path(pathname_t *name, separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = attr_list_path(&newname, buffer, buffersize, flags, cursor); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -784,7 +784,7 @@ attr_remove_path(pathname_t *name, const char *attrname, int flags) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = attr_remove_path(&newname, attrname, flags); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -805,7 +805,7 @@ attr_set_path(pathname_t *name, const char *attrname, const char *attrvalue, if (chdir(buf) == 0) { rval = attr_set_path(&newname, attrname, attrvalue, valuelength, flags); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -819,7 +819,7 @@ check_cwd(void) if (stat64(".", &statbuf) == 0 && statbuf.st_ino == top_ino) return; - chdir(homedir); + assert(chdir(homedir) == 0); fprintf(stderr, "fsstress: check_cwd failure\n"); abort(); /* NOTREACHED */ @@ -858,7 +858,7 @@ creat_path(pathname_t *name, mode_t mode) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = creat_path(&newname, mode); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -996,11 +996,15 @@ doproc(void) } } errout: - chdir(".."); + assert(chdir("..") == 0); free(homedir); if (cleanup) { + int ret; + sprintf(cmd, "rm -rf %s", buf); - system(cmd); + ret = system(cmd); + if (ret != 0) + perror("cleaning up"); cleanup_flist(); } } @@ -1216,7 +1220,7 @@ lchown_path(pathname_t *name, uid_t owner, gid_t group) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = lchown_path(&newname, owner, group); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1240,7 +1244,7 @@ link_path(pathname_t *name1, pathname_t *name2) if (strcmp(buf1, buf2) == 0) { if (chdir(buf1) == 0) { rval = link_path(&newname1, &newname2); - chdir(".."); + assert(chdir("..") == 0); } } else { if (strcmp(buf1, "..") == 0) @@ -1260,7 +1264,7 @@ link_path(pathname_t *name1, pathname_t *name2) append_pathname(&newname2, name2->path); if (chdir(buf1) == 0) { rval = link_path(&newname1, &newname2); - chdir(".."); + assert(chdir("..") == 0); } } else { free_pathname(&newname1); @@ -1268,7 +1272,7 @@ link_path(pathname_t *name1, pathname_t *name2) append_pathname(&newname1, name1->path); if (chdir(buf2) == 0) { rval = link_path(&newname1, &newname2); - chdir(".."); + assert(chdir("..") == 0); } } } @@ -1290,7 +1294,7 @@ lstat64_path(pathname_t *name, struct stat64 *sbuf) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = lstat64_path(&newname, sbuf); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1326,7 +1330,7 @@ mkdir_path(pathname_t *name, mode_t mode) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = mkdir_path(&newname, mode); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1345,7 +1349,7 @@ mknod_path(pathname_t *name, mode_t mode, dev_t dev) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = mknod_path(&newname, mode, dev); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1386,7 +1390,7 @@ open_path(pathname_t *name, int oflag) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = open_path(&newname, oflag); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1405,7 +1409,7 @@ opendir_path(pathname_t *name) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = opendir_path(&newname); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1446,7 +1450,7 @@ readlink_path(pathname_t *name, char *lbuf, size_t lbufsiz) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = readlink_path(&newname, lbuf, lbufsiz); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1470,7 +1474,7 @@ rename_path(pathname_t *name1, pathname_t *name2) if (strcmp(buf1, buf2) == 0) { if (chdir(buf1) == 0) { rval = rename_path(&newname1, &newname2); - chdir(".."); + assert(chdir("..") == 0); } } else { if (strcmp(buf1, "..") == 0) @@ -1490,7 +1494,7 @@ rename_path(pathname_t *name1, pathname_t *name2) append_pathname(&newname2, name2->path); if (chdir(buf1) == 0) { rval = rename_path(&newname1, &newname2); - chdir(".."); + assert(chdir("..") == 0); } } else { free_pathname(&newname1); @@ -1498,7 +1502,7 @@ rename_path(pathname_t *name1, pathname_t *name2) append_pathname(&newname1, name1->path); if (chdir(buf2) == 0) { rval = rename_path(&newname1, &newname2); - chdir(".."); + assert(chdir("..") == 0); } } } @@ -1520,7 +1524,7 @@ rmdir_path(pathname_t *name) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = rmdir_path(&newname); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1603,7 +1607,7 @@ stat64_path(pathname_t *name, struct stat64 *sbuf) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = stat64_path(&newname, sbuf); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1627,7 +1631,7 @@ symlink_path(const char *name1, pathname_t *name) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = symlink_path(name1, &newname); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1646,7 +1650,7 @@ truncate64_path(pathname_t *name, off64_t length) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = truncate64_path(&newname, length); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; @@ -1665,7 +1669,7 @@ unlink_path(pathname_t *name) separate_pathname(name, buf, &newname); if (chdir(buf) == 0) { rval = unlink_path(&newname); - chdir(".."); + assert(chdir("..") == 0); } free_pathname(&newname); return rval; From patchwork Fri Dec 7 06:23:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717513 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9142118B8 for ; Fri, 7 Dec 2018 06:23:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EFB12EB3B for ; Fri, 7 Dec 2018 06:23:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 689E12EB6D; Fri, 7 Dec 2018 06:23:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 23E562EB62 for ; Fri, 7 Dec 2018 06:23:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725986AbeLGGXo (ORCPT ); Fri, 7 Dec 2018 01:23:44 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33844 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725948AbeLGGXo (ORCPT ); Fri, 7 Dec 2018 01:23:44 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76Isx0115121; Fri, 7 Dec 2018 06:23:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=7V4/DYyIKkgnVolvbSENTZSdY7EwFXkHA3ZwziuGT9s=; b=KFQWZwgbD4NGorrnJBO3XmBKbSU3ttkm+qUh1TqjZ7uxSRfLS68lj7ByGvP+lFpb/NFn ah+Z6V18iwIbmnrdTkaXEKHZJknV5m3a/B1WeyCq64LMOId5ze5xbnZUlfqZAicWusV3 2JchC4V+cGbwTpydBRdb4vLxkr5nKiOpLB8JvVxaQK33KZGmuqaHMi0wz31PHkSGY12O WrzNOCLTGy8bCGmp+++IeRViVe5oYkUBHp40LZIMOWy+y14FN4nNSuShXBBEkSNcxk+a /0+lIqOaelvhP5AhB+EhllAw1IvrAvD+f30Woaa85klgEqL7/M1/4Tzy5TUvRLRAn/ME iQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2p3ftfg1pk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:23:42 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB76Ne8O007058 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:23:41 GMT Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB76NeU3027440; Fri, 7 Dec 2018 06:23:40 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:23:40 -0800 Subject: [PATCH 03/12] fsx: shut up compiler warnings From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Allison Henderson , fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:23:39 -0800 Message-ID: <154416381937.15344.9212761760427835708.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070054 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix unused variables and potential filename overflows in fsx. Signed-off-by: Darrick J. Wong Reviewed-By: Allison Henderson --- ltp/fsx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index b0157ba3..5601c70c 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -121,7 +121,7 @@ char *bname; /* basename of our test file */ char *logdev; /* -i flag */ char *logid; /* -j flag */ char dname[1024]; /* -P flag */ -char goodfile[1024]; +char goodfile[PATH_MAX]; int dirpath = 0; /* -P flag */ int fd; /* fd for our test file */ @@ -185,7 +185,7 @@ const char *replayops = NULL; const char *recordops = NULL; FILE * fsxlogf = NULL; FILE * replayopsf = NULL; -char opsfile[1024]; +char opsfile[PATH_MAX]; int badoff = -1; int closeopen = 0; @@ -541,13 +541,13 @@ mark_log(void) void dump_fsync_buffer(void) { - char fname_buffer[1024]; + char fname_buffer[PATH_MAX]; int good_fd; if (!good_buf) return; - snprintf(fname_buffer, 1024, "%s%s.mark%d", dname, + snprintf(fname_buffer, sizeof(fname_buffer), "%s%s.mark%d", dname, bname, mark_nr); good_fd = open(fname_buffer, O_WRONLY|O_CREAT|O_TRUNC, 0666); if (good_fd < 0) { @@ -1865,7 +1865,7 @@ main(int argc, char **argv) { int i, style, ch; char *endp, *tmp; - char logfile[1024]; + char logfile[PATH_MAX]; struct stat statbuf; int o_flags = O_RDWR|O_CREAT|O_TRUNC; From patchwork Fri Dec 7 06:23:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717517 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0573218B8 for ; Fri, 7 Dec 2018 06:23:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E9E722EB62 for ; Fri, 7 Dec 2018 06:23:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD3832EB6D; Fri, 7 Dec 2018 06:23:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71F612EB62 for ; Fri, 7 Dec 2018 06:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725987AbeLGGXv (ORCPT ); Fri, 7 Dec 2018 01:23:51 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:41284 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725948AbeLGGXu (ORCPT ); Fri, 7 Dec 2018 01:23:50 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76IStP149078; Fri, 7 Dec 2018 06:23:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=unsQro2QsXUp84Ms7Pc1AacZShEsKnteNEpnFSim/eE=; b=dXZeYvf7+KqVjkvb2R+ok9Er16ugp1OauOb3d4kMGfnyk0shREZXdLO7YCDpBTAqSoUw i5BUCYD9SVHhcyzb6J/9f815vOdfVnjNDVyyFCJnOGTcXEU86kJNk5KhIssdbQbBpReJ B9AiCcpBteNq5Qn/PzIM00Zb0rXCIe935Azpd+7/hTqSZWXw/B2pXkKM5W+/UTvxE2EG an3gHaIaDr8iDOx+3u4t57Vwh6nVgP2kOPCeBegpWTNRsRyB/oqL2KlGfadmuZ/vGvgT tVUFEqxBnBAMuf8MsMEyazKKUTUkZW8KMSLrSexIQzhMgvh3CbkmVU1Cw84Lj0Lq7bCm Lw== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2p3hquc4nn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:23:48 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76Nleu017010 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:23:48 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76NltN016197; Fri, 7 Dec 2018 06:23:47 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:23:46 -0800 Subject: [PATCH 04/12] fsx: always check buffer after each operation From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Allison Henderson , fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:23:45 -0800 Message-ID: <154416382576.15344.5225803629460714580.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=924 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070054 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add a new option to make fsx read the file after each operation and compare it with the good buffer to try to catch corruptions as soon as they occur. Signed-off-by: Darrick J. Wong Reviewed-By: Allison Henderson --- ltp/fsx.c | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 76 insertions(+), 10 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 5601c70c..191a5936 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -160,7 +160,8 @@ int punch_hole_calls = 1; /* -H flag disables */ int zero_range_calls = 1; /* -z flag disables */ int collapse_range_calls = 1; /* -C flag disables */ int insert_range_calls = 1; /* -I flag disables */ -int mapped_reads = 1; /* -R flag disables it */ +int mapped_reads = 1; /* -R flag disables it */ +int check_file = 0; /* -X flag enables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -561,7 +562,7 @@ dump_fsync_buffer(void) } void -check_buffers(unsigned offset, unsigned size) +check_buffers(char *buf, unsigned offset, unsigned size) { unsigned char c, t; unsigned i = 0; @@ -569,19 +570,19 @@ check_buffers(unsigned offset, unsigned size) unsigned op = 0; unsigned bad = 0; - if (memcmp(good_buf + offset, temp_buf, size) != 0) { + if (memcmp(good_buf + offset, buf, size) != 0) { prt("READ BAD DATA: offset = 0x%x, size = 0x%x, fname = %s\n", offset, size, fname); prt("OFFSET\tGOOD\tBAD\tRANGE\n"); while (size > 0) { c = good_buf[offset]; - t = temp_buf[i]; + t = buf[i]; if (c != t) { if (n < 16) { - bad = short_at(&temp_buf[i]); + bad = short_at(&buf[i]); prt("0x%05x\t0x%04x\t0x%04x", offset, short_at(&good_buf[offset]), bad); - op = temp_buf[offset & 1 ? i+1 : i]; + op = buf[offset & 1 ? i+1 : i]; prt("\t0x%05x\n", n); if (op) prt("operation# (mod 256) for " @@ -725,10 +726,9 @@ doread(unsigned offset, unsigned size) iret, size); report_failure(141); } - check_buffers(offset, size); + check_buffers(temp_buf, offset, size); } - void check_eofpage(char *s, unsigned offset, char *p, int size) { @@ -755,6 +755,65 @@ check_eofpage(char *s, unsigned offset, char *p, int size) } } +void +check_contents(void) +{ + static char *check_buf; + unsigned offset = 0; + unsigned size = file_size; + unsigned map_offset; + unsigned map_size; + char *p; + off_t ret; + unsigned iret; + + if (!check_buf) { + check_buf = (char *) malloc(maxfilelen + writebdy); + assert(check_buf != NULL); + check_buf = round_ptr_up(check_buf, writebdy, 0); + memset(check_buf, '\0', maxfilelen); + } + + if (o_direct) + size -= size % readbdy; + if (size == 0) + return; + + ret = lseek(fd, (off_t)offset, SEEK_SET); + if (ret == (off_t)-1) { + prterr("doread: lseek"); + report_failure(140); + } + + iret = fsxread(fd, check_buf, size, offset); + if (iret != size) { + if (iret == -1) + prterr("check_contents: read"); + else + prt("short check read: 0x%x bytes instead of 0x%x\n", + iret, size); + report_failure(141); + } + check_buffers(check_buf, offset, size); + + /* Map eof page, check it */ + map_offset = size - (size & PAGE_MASK); + if (map_offset == size) + map_offset -= PAGE_SIZE; + map_size = size - map_offset; + + p = mmap(0, map_size, PROT_READ, MAP_SHARED, fd, map_offset); + if (p == MAP_FAILED) { + prterr("check_contents: mmap"); + report_failure(190); + } + check_eofpage("check_contents", map_offset, p, map_size); + + if (munmap(p, map_size) != 0) { + prterr("check_contents: munmap"); + report_failure(191); + } +} void domapread(unsigned offset, unsigned size) @@ -808,7 +867,7 @@ domapread(unsigned offset, unsigned size) report_failure(191); } - check_buffers(offset, size); + check_buffers(temp_buf, offset, size); } @@ -1624,6 +1683,9 @@ test(void) break; } + if (check_file && testcalls > simulatedopcount) + check_contents(); + out: if (sizechecks && testcalls > simulatedopcount) check_size(); @@ -1684,6 +1746,7 @@ usage(void) -P: save .fsxlog .fsxops and .fsxgood files in dirpath (default ./)\n\ -S seed: for random # generator (default 1) 0 gets timestamp\n\ -W: mapped write operations DISabled\n\ + -X: Read file and compare to good buffer after every operation.\n\ -R: read() system calls only (mapped reads disabled)\n\ -Z: O_DIRECT (use -R, -W, -r and -w too)\n\ --replay-ops opsfile: replay ops from recorded .fsxops file\n\ @@ -1880,7 +1943,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FKHzCILN:OP:RS:WZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2044,6 +2107,9 @@ main(int argc, char **argv) if (!quiet) prt("mapped writes DISABLED\n"); break; + case 'X': + check_file = 1; + break; case 'Z': o_direct = O_DIRECT; o_flags |= O_DIRECT; From patchwork Fri Dec 7 06:23:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717521 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D07418B8 for ; Fri, 7 Dec 2018 06:23:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DE1D2EB62 for ; Fri, 7 Dec 2018 06:23:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4177A2EB6D; Fri, 7 Dec 2018 06:23:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1BBA2EB62 for ; Fri, 7 Dec 2018 06:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725989AbeLGGX4 (ORCPT ); Fri, 7 Dec 2018 01:23:56 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:41362 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725948AbeLGGX4 (ORCPT ); Fri, 7 Dec 2018 01:23:56 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76JxT9149778; Fri, 7 Dec 2018 06:23:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=tvstTNFjbsktVN9RfdK/uYKHx7MD5D3lWDWVT82paAo=; b=XhL0BA3Oq8yWvkEET4tATN/hsJ9FLfBYuF++e//cKc6ceYZY/PCwhul/x2e6/1iwt68T 9NZAF+OqLc9aPfrJfZMu5pCZpkyOP5EgqI/ecH4NdaNXV5/ZN+A89z37PC0oOrBEsSty 1qJgw7opW7kdTYE7/tdMaDX9kPND0pyBlTGcIPEi4z7NYgSZJHgtp/6khCKiib5B+jtJ DZvbaP/1rleU+dsMLWowxw4Ub6FMOcOb3K8io46iF+xV3BhMVt1sboL48rWBKvntL0F/ TktoE86Cq4POSYmjFQA0d2mrFYO+dwAvuv9IVr1TBQp68OfGKgRGoEk5ZLE4aqAQ3Gjm aw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2p3hquc4nw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:23:54 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB76NrGR007581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:23:54 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76NrfQ016268; Fri, 7 Dec 2018 06:23:53 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:23:53 -0800 Subject: [PATCH 05/12] fsx: use an enum to define the operation commands From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Allison Henderson , fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:23:52 -0800 Message-ID: <154416383217.15344.12016288717012656827.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=921 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070054 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Use an enum to define operation codes and the boundaries between operation classes so that we can add new commands without having to change a bunch of unrelated #defines. Signed-off-by: Darrick J. Wong Reviewed-By: Allison Henderson --- ltp/fsx.c | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 191a5936..c3db62e4 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -88,25 +88,27 @@ int logcount = 0; /* total ops */ * mode being run. */ -/* common operations */ -#define OP_READ 0 -#define OP_WRITE 1 -#define OP_MAPREAD 2 -#define OP_MAPWRITE 3 -#define OP_MAX_LITE 4 - -/* !lite operations */ -#define OP_TRUNCATE 4 -#define OP_FALLOCATE 5 -#define OP_PUNCH_HOLE 6 -#define OP_ZERO_RANGE 7 -#define OP_COLLAPSE_RANGE 8 -#define OP_INSERT_RANGE 9 -#define OP_MAX_FULL 10 - -/* integrity operations */ -#define OP_FSYNC 10 -#define OP_MAX_INTEGRITY 11 +enum { + /* common operations */ + OP_READ = 0, + OP_WRITE, + OP_MAPREAD, + OP_MAPWRITE, + OP_MAX_LITE, + + /* !lite operations */ + OP_TRUNCATE = OP_MAX_LITE, + OP_FALLOCATE, + OP_PUNCH_HOLE, + OP_ZERO_RANGE, + OP_COLLAPSE_RANGE, + OP_INSERT_RANGE, + OP_MAX_FULL, + + /* integrity operations */ + OP_FSYNC = OP_MAX_FULL, + OP_MAX_INTEGRITY, +}; #undef PAGE_SIZE #define PAGE_SIZE getpagesize() From patchwork Fri Dec 7 06:23:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717525 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B792618B8 for ; Fri, 7 Dec 2018 06:24:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A782C2EB67 for ; Fri, 7 Dec 2018 06:24:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9C2472EB6D; Fri, 7 Dec 2018 06:24:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 263012EB6A for ; Fri, 7 Dec 2018 06:24:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725963AbeLGGYL (ORCPT ); Fri, 7 Dec 2018 01:24:11 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:56896 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725948AbeLGGYL (ORCPT ); Fri, 7 Dec 2018 01:24:11 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76IZ63111286; Fri, 7 Dec 2018 06:24:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=B9rWI2/jOrKtD4gabCui9Q8XK46OJcx/qHXCsQaw3iY=; b=jDVbuVv52Hc1+5O6AO+m6SZmDFR5eYCiXf4TljEPnh9wc/tiO56W4VxSMtSixnXg+elc WT0l0n1qJ28TVp0/cIY0uWCJ//hcU2uqA3bhihNEOn6B+zlt6A6JvitxXMxAss6RFMSv 9MXRPB2D4aX+84ygqtJkMfqql6hFacCbtTn2/Z3BG4fBZmWayCmeaoVWa/h1rZTOWCTq cI2rPtNSeNSJZ6J/y1Z7qyyY4BfOra3J5EpIoWu7jt9HzgRFdb5VJARq6GEAD9fvNftl ACTL00X6NBOFnTq3+9MPRiuXDKEp37NaKsMeI061wc9xAO4ZsHAr1m9vqDQOlWECXmrB Zw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2p3jxrv0ut-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:09 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB76O96P008647 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:09 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB76O8DW025990; Fri, 7 Dec 2018 06:24:09 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:08 -0800 Subject: [PATCH 06/12] fsx: add five-argument logging function From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, Allison Henderson , fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:23:58 -0800 Message-ID: <154416383855.15344.9018861209951059097.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070054 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add a five-argument logging function to support new operations. Signed-off-by: Darrick J. Wong Reviewed-By: Allison Henderson --- ltp/fsx.c | 45 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 40 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index c3db62e4..1907036e 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -51,7 +51,8 @@ enum opflags { FL_NONE = 0, FL_SKIPPED = 1, FL_CLOSE_OPEN = 2, FL_KEEP_SIZE = 4 struct log_entry { int operation; - int args[3]; + int nr_args; + int args[4]; enum opflags flags; }; @@ -278,6 +279,27 @@ static int op_code(const char *name) return -1; } +void +log5(int operation, int arg0, int arg1, int arg2, enum opflags flags) +{ + struct log_entry *le; + + le = &oplog[logptr]; + le->operation = operation; + if (closeopen) + flags |= FL_CLOSE_OPEN; + le->args[0] = arg0; + le->args[1] = arg1; + le->args[2] = arg2; + le->args[3] = file_size; + le->nr_args = 4; + le->flags = flags; + logptr++; + logcount++; + if (logptr >= LOGSIZE) + logptr = 0; +} + void log4(int operation, int arg0, int arg1, enum opflags flags) { @@ -290,6 +312,7 @@ log4(int operation, int arg0, int arg1, enum opflags flags) le->args[0] = arg0; le->args[1] = arg1; le->args[2] = file_size; + le->nr_args = 3; le->flags = flags; logptr++; logcount++; @@ -439,11 +462,13 @@ logdump(void) i = 0; if (logopsf) { + int j; + if (lp->flags & FL_SKIPPED) fprintf(logopsf, "skip "); - fprintf(logopsf, "%s 0x%x 0x%x 0x%x", - op_name(lp->operation), - lp->args[0], lp->args[1], lp->args[2]); + fprintf(logopsf, "%s", op_name(lp->operation)); + for (j = 0; j < lp->nr_args; j++) + fprintf(logopsf, " 0x%x", lp->args[j]); if (lp->flags & FL_KEEP_SIZE) fprintf(logopsf, " keep_size"); if (lp->flags & FL_CLOSE_OPEN) @@ -1432,6 +1457,15 @@ cleanup(int sig) exit(sig); } +static int +op_args_count(int operation) +{ + switch (operation) { + default: + return 3; + } +} + static int read_op(struct log_entry *log_entry) { @@ -1464,7 +1498,8 @@ read_op(struct log_entry *log_entry) log_entry->operation = op_code(str); if (log_entry->operation == -1) goto fail; - for (i = 0; i < 3; i++) { + log_entry->nr_args = op_args_count(log_entry->operation); + for (i = 0; i < log_entry->nr_args; i++) { char *end; str = strtok(NULL, " \t\n"); From patchwork Fri Dec 7 06:24:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717529 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B241717D5 for ; Fri, 7 Dec 2018 06:24:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C7ED2EB67 for ; Fri, 7 Dec 2018 06:24:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 90E442EB6D; Fri, 7 Dec 2018 06:24:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 044562EB6A for ; Fri, 7 Dec 2018 06:24:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725988AbeLGGYU (ORCPT ); Fri, 7 Dec 2018 01:24:20 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:41714 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725948AbeLGGYU (ORCPT ); Fri, 7 Dec 2018 01:24:20 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76OIFP152999; Fri, 7 Dec 2018 06:24:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=P8iTUEdCtf/Ov/1tf0MhKMfsXo3Bet53/qiOQxd9IUM=; b=mR3/xKhbxNrV7XYl6F51Atd8hGYMEBWh41fKNJ64oLV9wARlDwqrGMrHEmyshzWaD9I4 LeuWSGUTr5io3zHbJU2ccInzU8sLeEb2nosVJmcXNHTpt53Ht6zgPozMMiN8IMuHrblZ Puenk15RHcRboe5YiV8KWYE25yXnQQO1bumGRc/iEFXs+6cIVD73IuFXMtCVcZwzXezD aG4QoEgXLeh7d5SyrNuW7LkB1qbZ1o5Ur7rzQ8kmdw+NekJtSN4uBS3igwfJMf+3VBzf pHGBrpkvUojbqf7epvyeZeJnIj3LA9R78hAHEz79exEwhrSRdvFx1pORzSL47gfP0Y4g BA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2p3hquc4pt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:18 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OFlV004984 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:16 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OFKY026115; Fri, 7 Dec 2018 06:24:15 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:15 -0800 Subject: [PATCH 07/12] fsx: add FICLONERANGE support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:24:14 -0800 Message-ID: <154416385390.15344.16754761288279379198.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add support for the clone range (i.e. reflink) ioctl to fsx. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 157 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 152 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 1907036e..d672925f 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -104,6 +104,7 @@ enum { OP_ZERO_RANGE, OP_COLLAPSE_RANGE, OP_INSERT_RANGE, + OP_CLONE_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -165,6 +166,7 @@ int collapse_range_calls = 1; /* -C flag disables */ int insert_range_calls = 1; /* -I flag disables */ int mapped_reads = 1; /* -R flag disables it */ int check_file = 0; /* -X flag enables */ +int clone_range_calls = 1; /* -J flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -258,6 +260,7 @@ static const char *op_names[] = { [OP_ZERO_RANGE] = "zero_range", [OP_COLLAPSE_RANGE] = "collapse_range", [OP_INSERT_RANGE] = "insert_range", + [OP_CLONE_RANGE] = "clone_range", [OP_FSYNC] = "fsync", }; @@ -320,7 +323,6 @@ log4(int operation, int arg0, int arg1, enum opflags flags) logptr = 0; } - void logdump(void) { @@ -342,7 +344,7 @@ logdump(void) count = LOGSIZE; } for ( ; count > 0; count--) { - bool overlap; + bool overlap, overlap2; int opnum; opnum = i+1 + (logcount/LOGSIZE)*LOGSIZE; @@ -444,6 +446,20 @@ logdump(void) if (overlap) prt("\t******IIII"); break; + case OP_CLONE_RANGE: + prt("CLONE 0x%x thru 0x%x\t(0x%x bytes) to 0x%x thru 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], + lp->args[2], lp->args[2] + lp->args[1] - 1); + overlap2 = badoff >= lp->args[2] && + badoff < lp->args[2] + lp->args[1]; + if (overlap && overlap2) + prt("\tJJJJ**JJJJ"); + else if (overlap) + prt("\tJJJJ******"); + else if (overlap2) + prt("\t******JJJJ"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1304,6 +1320,96 @@ do_insert_range(unsigned offset, unsigned length) } #endif +#ifdef FICLONERANGE +int +test_clone_range(void) +{ + struct file_clone_range fcr = { + .src_fd = fd, + }; + + if (ioctl(fd, FICLONERANGE, &fcr) && + (errno = EOPNOTSUPP || errno == ENOTTY)) { + if (!quiet) + fprintf(stderr, + "main: filesystem does not support " + "clone range, disabling!\n"); + return 0; + } + + return 1; +} + +void +do_clone_range(unsigned offset, unsigned length, unsigned dest) +{ + struct file_clone_range fcr = { + .src_fd = fd, + .src_offset = offset, + .src_length = length, + .dest_offset = dest, + }; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length clone range\n"); + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping clone range behind EOF\n"); + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if (dest + length > biggest) { + biggest = dest + length; + if (!quiet && testcalls > simulatedopcount) + prt("cloning to largest ever: 0x%x\n", dest + length); + } + + log5(OP_CLONE_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu clone\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + if (ioctl(fd, FICLONERANGE, &fcr) == -1) { + prt("clone range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_clone_range: FICLONERANGE"); + report_failure(161); + } + + memcpy(good_buf + dest, good_buf + offset, length); + if (dest > file_size) + memset(good_buf + file_size, '\0', dest - file_size); + if (dest + length > file_size) + file_size = dest + length; +} + +#else +int +test_clone_range(void) +{ + return 0; +} + +void +do_clone_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1461,6 +1567,8 @@ static int op_args_count(int operation) { switch (operation) { + case OP_CLONE_RANGE: + return 4; default: return 3; } @@ -1534,7 +1642,7 @@ read_op(struct log_entry *log_entry) int test(void) { - unsigned long offset; + unsigned long offset, offset2; unsigned long size; unsigned long rv; unsigned long op; @@ -1565,6 +1673,7 @@ test(void) op = log_entry.operation; offset = log_entry.args[0]; size = log_entry.args[1]; + offset2 = log_entry.args[2]; closeopen = !!(log_entry.flags & FL_CLOSE_OPEN); keep_size = !!(log_entry.flags & FL_KEEP_SIZE); goto have_op; @@ -1577,6 +1686,7 @@ test(void) closeopen = (rv >> 3) < (1 << 28) / closeprob; offset = random(); + offset2 = 0; size = maxoplen; if (randomoplen) size = random() % (maxoplen + 1); @@ -1602,6 +1712,17 @@ test(void) if (zero_range_calls && size && keep_size_calls) keep_size = random() % 2; break; + case OP_CLONE_RANGE: + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 = offset2 & ~(block_size - 1); + } while (llabs(offset2 - offset) < size || + offset2 + size > maxfilelen); + break; } have_op: @@ -1645,6 +1766,12 @@ test(void) goto out; } break; + case OP_CLONE_RANGE: + if (!clone_range_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -1711,6 +1838,18 @@ test(void) do_insert_range(offset, size); break; + case OP_CLONE_RANGE: + if (size == 0) { + log5(OP_CLONE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_CLONE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_clone_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -1736,7 +1875,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxAFLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxAFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -1777,6 +1916,9 @@ usage(void) #ifdef FALLOC_FL_INSERT_RANGE " -I: Do not use insert range calls\n" #endif +#ifdef FICLONERANGE +" -J: Do not use clone range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -1980,7 +2122,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2110,6 +2252,9 @@ main(int argc, char **argv) case 'I': insert_range_calls = 0; break; + case 'J': + clone_range_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); @@ -2328,6 +2473,8 @@ main(int argc, char **argv) collapse_range_calls = test_fallocate(FALLOC_FL_COLLAPSE_RANGE); if (insert_range_calls) insert_range_calls = test_fallocate(FALLOC_FL_INSERT_RANGE); + if (clone_range_calls) + clone_range_calls = test_clone_range(); while (numops == -1 || numops--) if (!test()) From patchwork Fri Dec 7 06:24:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717531 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B9F317D5 for ; Fri, 7 Dec 2018 06:24:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2846D2EB6A for ; Fri, 7 Dec 2018 06:24:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1CE1D2EB81; Fri, 7 Dec 2018 06:24:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E84C2EB6A for ; Fri, 7 Dec 2018 06:24:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726032AbeLGGYY (ORCPT ); Fri, 7 Dec 2018 01:24:24 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:57134 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGYY (ORCPT ); Fri, 7 Dec 2018 01:24:24 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76OKKS115251; Fri, 7 Dec 2018 06:24:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=gEcLU2YVGx6uc0LUEkuzFOil7AbCjotobuqzAiLlafc=; b=Aw0sBFK6wMJ8MmYB5C/NJTMS22lsa8v2aHzOfhjxmi3JN13ULEsG6DHEj3HovkuVuJnq v/LzjEby+55hz7QcJxqjs0v67I5+2OcufvLlTbiUrqbNxrk/vuNhS2qDMiJcrjAypyjs 6Dk3gsumT7EGXqIs8CZO8RMci8AGjc42NF7478c7qKmYWJegB5HyortoWk5ycHtuv7Ti uBadKKevDK76aApNMlajwtYgNtdSGFWY+sdy3CCkXYc9yPAA7OP6s4TOLG40dct9PlWy FgeCCQ6uPZiN2jKoI3V0Hqnd6TrEksXRHtRIrmW/O14tU0ACcHYfSsu5V9Cc91QE6Y0Z Jw== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2p3jxrv0v8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:22 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OLsE005187 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:22 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OLvF026232; Fri, 7 Dec 2018 06:24:21 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:21 -0800 Subject: [PATCH 08/12] fsx: add FIDEDUPERANGE support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:24:20 -0800 Message-ID: <154416386043.15344.792303127664235766.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add support for the dedupe range ioctl to fsx. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 196 insertions(+), 2 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index d672925f..e7a88c04 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -105,6 +105,7 @@ enum { OP_COLLAPSE_RANGE, OP_INSERT_RANGE, OP_CLONE_RANGE, + OP_DEDUPE_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -167,6 +168,7 @@ int insert_range_calls = 1; /* -I flag disables */ int mapped_reads = 1; /* -R flag disables it */ int check_file = 0; /* -X flag enables */ int clone_range_calls = 1; /* -J flag disables */ +int dedupe_range_calls = 1; /* -B flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -261,6 +263,7 @@ static const char *op_names[] = { [OP_COLLAPSE_RANGE] = "collapse_range", [OP_INSERT_RANGE] = "insert_range", [OP_CLONE_RANGE] = "clone_range", + [OP_DEDUPE_RANGE] = "dedupe_range", [OP_FSYNC] = "fsync", }; @@ -460,6 +463,20 @@ logdump(void) else if (overlap2) prt("\t******JJJJ"); break; + case OP_DEDUPE_RANGE: + prt("DEDUPE 0x%x thru 0x%x\t(0x%x bytes) to 0x%x thru 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], + lp->args[2], lp->args[2] + lp->args[1] - 1); + overlap2 = badoff >= lp->args[2] && + badoff < lp->args[2] + lp->args[1]; + if (overlap && overlap2) + prt("\tBBBB**BBBB"); + else if (overlap) + prt("\tBBBB******"); + else if (overlap2) + prt("\t******BBBB"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1410,6 +1427,137 @@ do_clone_range(unsigned offset, unsigned length, unsigned dest) } #endif +#ifdef FIDEDUPERANGE +int +test_dedupe_range(void) +{ + struct file_dedupe_range *fdr; + off_t new_len; + int error; + int ret = 1; + + /* Alloc memory */ + fdr = calloc(sizeof(struct file_dedupe_range_info) + + sizeof(struct file_dedupe_range), 1); + if (!fdr) { + prterr("do_dedupe_range: malloc"); + report_failure(161); + } + + /* Make sure we have at least two blocks */ + new_len = block_size * 2; + if (file_size < new_len && ftruncate(fd, new_len)) { + warn("main: ftruncate"); + exit(132); + } + + /* Try to dedupe them */ + fdr->src_length = block_size; + fdr->dest_count = 1; + fdr->info[0].dest_fd = fd; + fdr->info[0].dest_offset = block_size; + + if (ioctl(fd, FIDEDUPERANGE, fdr)) + error = errno; + else if (fdr->info[0].status < 0) + error = -fdr->info[0].status; + else + error = 0; + + if (error == EOPNOTSUPP || error == ENOTTY) { + if (!quiet) + fprintf(stderr, + "main: filesystem does not support " + "dedupe range, disabling!\n"); + ret = 0; + } + + /* Put the file back the way it was. */ + if (file_size < new_len && ftruncate(fd, file_size)) { + warn("main: ftruncate"); + exit(132); + } + + free(fdr); + return ret; +} + +void +do_dedupe_range(unsigned offset, unsigned length, unsigned dest) +{ + struct file_dedupe_range *fdr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length dedupe range\n"); + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping dedupe range behind EOF\n"); + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu dedupe\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + /* Alloc memory */ + fdr = calloc(sizeof(struct file_dedupe_range_info) + + sizeof(struct file_dedupe_range), 1); + if (!fdr) { + prterr("do_dedupe_range: malloc"); + report_failure(161); + } + + /* Dedupe data blocks */ + fdr->src_offset = offset; + fdr->src_length = length; + fdr->dest_count = 1; + fdr->info[0].dest_fd = fd; + fdr->info[0].dest_offset = dest; + + if (ioctl(fd, FIDEDUPERANGE, fdr) == -1) { + prt("dedupe range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_dedupe_range(0): FIDEDUPERANGE"); + report_failure(161); + } else if (fdr->info[0].status < 0) { + errno = -fdr->info[0].status; + prt("dedupe range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_dedupe_range(1): FIDEDUPERANGE"); + report_failure(161); + } + + free(fdr); +} + +#else +int +test_dedupe_range(void) +{ + return 0; +} + +void +do_dedupe_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1568,6 +1716,7 @@ op_args_count(int operation) { switch (operation) { case OP_CLONE_RANGE: + case OP_DEDUPE_RANGE: return 4; default: return 3; @@ -1723,6 +1872,25 @@ test(void) } while (llabs(offset2 - offset) < size || offset2 + size > maxfilelen); break; + case OP_DEDUPE_RANGE: + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, file_size); + offset2 = offset2 & ~(block_size - 1); + } while (llabs(offset2 - offset) < size || + offset2 + size > file_size); + break; + } } have_op: @@ -1772,6 +1940,12 @@ test(void) goto out; } break; + case OP_DEDUPE_RANGE: + if (!dedupe_range_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -1850,6 +2024,18 @@ test(void) do_clone_range(offset, size, offset2); break; + case OP_DEDUPE_RANGE: + if (size == 0) { + log5(OP_DEDUPE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_DEDUPE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_dedupe_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -1875,7 +2061,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxAFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxABFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -1919,6 +2105,9 @@ usage(void) #ifdef FICLONERANGE " -J: Do not use clone range calls\n" #endif +#ifdef FIDEDUPERANGE +" -B: Do not use dedupe range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -2122,7 +2311,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FJKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:FJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2255,6 +2444,9 @@ main(int argc, char **argv) case 'J': clone_range_calls = 0; break; + case 'B': + dedupe_range_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); @@ -2475,6 +2667,8 @@ main(int argc, char **argv) insert_range_calls = test_fallocate(FALLOC_FL_INSERT_RANGE); if (clone_range_calls) clone_range_calls = test_clone_range(); + if (dedupe_range_calls) + dedupe_range_calls = test_dedupe_range(); while (numops == -1 || numops--) if (!test()) From patchwork Fri Dec 7 06:24:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717537 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ABDCA18B8 for ; Fri, 7 Dec 2018 06:24:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C3CA2EB67 for ; Fri, 7 Dec 2018 06:24:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 909FC2EB6D; Fri, 7 Dec 2018 06:24:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A259E2EB6A for ; Fri, 7 Dec 2018 06:24:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726035AbeLGGYg (ORCPT ); Fri, 7 Dec 2018 01:24:36 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:57342 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGYg (ORCPT ); Fri, 7 Dec 2018 01:24:36 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76OIHE115236; Fri, 7 Dec 2018 06:24:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=ggPi9A0WXasOTY9KEj9nYl05Gf8Qr/iSuy8zSIRsTts=; b=jR4EDqSMUwwraw0p772bl2HXAGHGqYeJsuDr/ajCdESxSaXEY2BtLh0zLWXHx7AEXy2h v4nrr1zTg9BOBupqIWb0K2UMAFsxD5hYkVWU2M9JrFULpwXFmx6mCYVMwHrISPeqm0eR Xd2IGph+PQ2s/MNz0b2jCfiB1KD4jtQ2RPLslTFs0v4isZFf7cO5/I/ZeodJuKcOQH71 qXlKao8EhrRvAvgI5Kzvp0rc2cDv8w60GPODfraeFDd/jNrq/gaetmSPz4NHopdOLzOs NZKSnGKAfsOhAeagAuX8uJw5DCDL8U6A5ABwi9USl/+YywADN3P2Owk1Ej1vtxElcMsf Yg== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2p3jxrv0vn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:33 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OS61009367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:28 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OSDW016474; Fri, 7 Dec 2018 06:24:28 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:27 -0800 Subject: [PATCH 09/12] fsstress: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:24:26 -0800 Message-ID: <154416386688.15344.13463990940166415616.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Support the copy_file_range syscall in fsstress. Signed-off-by: Darrick J. Wong --- configure.ac | 2 + include/builddefs.in | 1 ltp/Makefile | 4 + ltp/fsstress.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++++ m4/package_libcdev.m4 | 19 ++++++ 5 files changed, 173 insertions(+) diff --git a/configure.ac b/configure.ac index aede4f59..19798824 100644 --- a/configure.ac +++ b/configure.ac @@ -68,6 +68,8 @@ AC_PACKAGE_WANT_OPEN_BY_HANDLE_AT AC_PACKAGE_WANT_LINUX_PRCTL_H AC_PACKAGE_WANT_LINUX_FS_H +AC_HAVE_COPY_FILE_RANGE + AC_CHECK_FUNCS([renameat2]) AC_CONFIG_HEADER(include/config.h) diff --git a/include/builddefs.in b/include/builddefs.in index fb8e912b..2605e42d 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -67,6 +67,7 @@ HAVE_DMAPI = @have_dmapi@ HAVE_ATTR_LIST = @have_attr_list@ HAVE_FIEMAP = @have_fiemap@ HAVE_FALLOCATE = @have_fallocate@ +HAVE_COPY_FILE_RANGE = @have_copy_file_range@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall diff --git a/ltp/Makefile b/ltp/Makefile index 5f784ecf..e4ca45f4 100644 --- a/ltp/Makefile +++ b/ltp/Makefile @@ -28,6 +28,10 @@ ifeq ($(HAVE_FALLOCATE), true) LCFLAGS += -DFALLOCATE endif +ifeq ($(HAVE_COPY_FILE_RANGE),yes) +LCFLAGS += -DHAVE_COPY_FILE_RANGE +endif + default: depend $(TARGETS) depend: .dep diff --git a/ltp/fsstress.c b/ltp/fsstress.c index af5d125f..99a1d733 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -26,6 +26,7 @@ #include io_context_t io_ctx; #endif +#include #ifndef FS_IOC_GETFLAGS #define FS_IOC_GETFLAGS _IOR('f', 1, long) @@ -55,6 +56,7 @@ typedef enum { OP_BULKSTAT1, OP_CHOWN, OP_CLONERANGE, + OP_COPYRANGE, OP_CREAT, OP_DEDUPERANGE, OP_DREAD, @@ -163,6 +165,7 @@ void bulkstat_f(int, long); void bulkstat1_f(int, long); void chown_f(int, long); void clonerange_f(int, long); +void copyrange_f(int, long); void creat_f(int, long); void deduperange_f(int, long); void dread_f(int, long); @@ -212,6 +215,7 @@ opdesc_t ops[] = { { OP_BULKSTAT1, "bulkstat1", bulkstat1_f, 1, 0 }, { OP_CHOWN, "chown", chown_f, 3, 1 }, { OP_CLONERANGE, "clonerange", clonerange_f, 4, 1 }, + { OP_COPYRANGE, "copyrange", copyrange_f, 4, 1 }, { OP_CREAT, "creat", creat_f, 4, 1 }, { OP_DEDUPERANGE, "deduperange", deduperange_f, 4, 1}, { OP_DREAD, "dread", dread_f, 4, 0 }, @@ -2333,6 +2337,149 @@ clonerange_f( #endif } +/* copy some arbitrary range of f1 to f2. */ +void +copyrange_f( + int opno, + long r) +{ +#ifdef HAVE_COPY_FILE_RANGE + struct pathname fpath1; + struct pathname fpath2; + struct stat64 stat1; + struct stat64 stat2; + char inoinfo1[1024]; + char inoinfo2[1024]; + loff_t lr; + loff_t off1; + loff_t off2; + loff_t max_off2; + size_t len; + int tries = 0; + int v1; + int v2; + int fd1; + int fd2; + size_t ret; + int e; + + /* Load paths */ + init_pathname(&fpath1); + if (!get_fname(FT_REGm, r, &fpath1, NULL, NULL, &v1)) { + if (v1) + printf("%d/%d: copyrange read - no filename\n", + procid, opno); + goto out_fpath1; + } + + init_pathname(&fpath2); + if (!get_fname(FT_REGm, random(), &fpath2, NULL, NULL, &v2)) { + if (v2) + printf("%d/%d: copyrange write - no filename\n", + procid, opno); + goto out_fpath2; + } + + /* Open files */ + fd1 = open_path(&fpath1, O_RDONLY); + e = fd1 < 0 ? errno : 0; + check_cwd(); + if (fd1 < 0) { + if (v1) + printf("%d/%d: copyrange read - open %s failed %d\n", + procid, opno, fpath1.path, e); + goto out_fpath2; + } + + fd2 = open_path(&fpath2, O_WRONLY); + e = fd2 < 0 ? errno : 0; + check_cwd(); + if (fd2 < 0) { + if (v2) + printf("%d/%d: copyrange write - open %s failed %d\n", + procid, opno, fpath2.path, e); + goto out_fd1; + } + + /* Get file stats */ + if (fstat64(fd1, &stat1) < 0) { + if (v1) + printf("%d/%d: copyrange read - fstat64 %s failed %d\n", + procid, opno, fpath1.path, errno); + goto out_fd2; + } + inode_info(inoinfo1, sizeof(inoinfo1), &stat1, v1); + + if (fstat64(fd2, &stat2) < 0) { + if (v2) + printf("%d/%d: copyrange write - fstat64 %s failed %d\n", + procid, opno, fpath2.path, errno); + goto out_fd2; + } + inode_info(inoinfo2, sizeof(inoinfo2), &stat2, v2); + + /* Calculate offsets */ + len = (random() % FILELEN_MAX) + 1; + if (len == 0) + len = stat1.st_blksize; + if (len > stat1.st_size) + len = stat1.st_size; + + lr = ((int64_t)random() << 32) + random(); + if (stat1.st_size == len) + off1 = 0; + else + off1 = (off64_t)(lr % MIN(stat1.st_size - len, MAXFSIZE)); + off1 %= maxfsize; + + /* + * If srcfile == destfile, randomly generate destination ranges + * until we find one that doesn't overlap the source range. + */ + max_off2 = MIN(stat2.st_size + (1024ULL * stat2.st_blksize), MAXFSIZE); + do { + lr = ((int64_t)random() << 32) + random(); + off2 = (off64_t)(lr % max_off2); + off2 %= maxfsize; + } while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len); + + while (len > 0) { + ret = syscall(__NR_copy_file_range, fd1, &off1, fd2, &off2, + len, 0); + if (ret < 0) { + if (errno != EAGAIN || tries++ >= 300) + break; + } else if (ret > len) + break; + else if (ret > 0) + len -= ret; + } + e = ret < 0 ? errno : 0; + if (v1 || v2) { + printf("%d/%d: copyrange %s%s [%lld,%lld] -> %s%s [%lld,%lld]", + procid, opno, + fpath1.path, inoinfo1, (long long)off1, (long long)len, + fpath2.path, inoinfo2, (long long)off2, (long long)len); + + if (ret < 0) + printf(" error %d", e); + else if (len && ret > len) + printf(" asked for %lld, copied %lld??\n", + (long long)len, (long long)ret); + printf("\n"); + } + +out_fd2: + close(fd2); +out_fd1: + close(fd1); +out_fpath2: + free_pathname(&fpath2); +out_fpath1: + free_pathname(&fpath1); +#endif +} + /* dedupe some arbitrary range of f1 to f2...fn. */ void deduperange_f( diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index fb123f33..14e67e18 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -98,3 +98,22 @@ AC_DEFUN([AC_HAVE_GETMNTINFO], AC_MSG_RESULT(no)) AC_SUBST(have_getmntinfo) ]) + +# +# +# Check if we have a copy_file_range system call (Linux) +# +AC_DEFUN([AC_HAVE_COPY_FILE_RANGE], + [ AC_MSG_CHECKING([for copy_file_range]) + AC_TRY_LINK([ +#define _GNU_SOURCE +#include +#include + ], [ + syscall(__NR_copy_file_range, 0, 0, 0, 0, 0, 0); + ], have_copy_file_range=yes + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(have_copy_file_range) + ]) + From patchwork Fri Dec 7 06:24:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717545 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C408118B8 for ; Fri, 7 Dec 2018 06:24:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B042E2EB81 for ; Fri, 7 Dec 2018 06:24:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A49842EB6D; Fri, 7 Dec 2018 06:24:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 162CF2EB6A for ; Fri, 7 Dec 2018 06:24:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725963AbeLGGYp (ORCPT ); Fri, 7 Dec 2018 01:24:45 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:34536 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGYp (ORCPT ); Fri, 7 Dec 2018 01:24:45 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76OJ8G118815; Fri, 7 Dec 2018 06:24:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=dJZwAw+JAmQr8ozxwMvo7+HO0JhdNxPiB+/dxNwQYNU=; b=BPNOsIUWkpc0SSKGa9MWQWRs66emKBhx3pW/4aRUv4FcK0hKRF8wnaIUYjKL1GDXGu4p fanhdtCcU6XEoSbngZK5KM6bLD2jkqkBh0nZ6XVhyCz0qzVq3PhQplWb0QOLZoLAR3Ws /YJDRoO+zNBVuGgp/7LI7XS4YN04hwjpAhBO68VAYODZoDA5JfSk5YiixqspDOl2WGET ge8/YbbSfNiw/V4Ovev1hi4MEHsoUsvL8rj8OBT0XKMSjgLbPBv4EJiwZVDrQLIVe7M+ +5/Bf/hw77iJoN6tLRVR3x7q90TO8PgrlRcfVgGpVyc8ywdeBMZvslpu5Rt1s0XT/hbQ Rg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2p3ftfg1ry-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:40 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OZqr019081 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:35 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OYtg016500; Fri, 7 Dec 2018 06:24:34 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:34 -0800 Subject: [PATCH 10/12] fsx: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, Dave Chinner Date: Thu, 06 Dec 2018 22:24:33 -0800 Message-ID: <154416387314.15344.2165201747424065702.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add support for the copy_file_range system call to fsx. Signed-off-by: Darrick J. Wong [dchinner: copy_file_range() needs to obey read/write constraints otherwise is blows up when direct IO is used] Signed-off-by: Dave Chinner --- ltp/fsx.c | 165 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 163 insertions(+), 2 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index e7a88c04..316f08eb 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -34,6 +34,7 @@ #ifdef AIO #include #endif +#include #ifndef MAP_FILE # define MAP_FILE 0 @@ -106,6 +107,7 @@ enum { OP_INSERT_RANGE, OP_CLONE_RANGE, OP_DEDUPE_RANGE, + OP_COPY_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -169,6 +171,7 @@ int mapped_reads = 1; /* -R flag disables it */ int check_file = 0; /* -X flag enables */ int clone_range_calls = 1; /* -J flag disables */ int dedupe_range_calls = 1; /* -B flag disables */ +int copy_range_calls = 1; /* -E flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -264,6 +267,7 @@ static const char *op_names[] = { [OP_INSERT_RANGE] = "insert_range", [OP_CLONE_RANGE] = "clone_range", [OP_DEDUPE_RANGE] = "dedupe_range", + [OP_COPY_RANGE] = "copy_range", [OP_FSYNC] = "fsync", }; @@ -477,6 +481,20 @@ logdump(void) else if (overlap2) prt("\t******BBBB"); break; + case OP_COPY_RANGE: + prt("COPY 0x%x thru 0x%x\t(0x%x bytes) to 0x%x thru 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], + lp->args[2], lp->args[2] + lp->args[1] - 1); + overlap2 = badoff >= lp->args[2] && + badoff < lp->args[2] + lp->args[1]; + if (overlap && overlap2) + prt("\tEEEE**EEEE"); + else if (overlap) + prt("\tEEEE******"); + else if (overlap2) + prt("\t******EEEE"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1558,6 +1576,110 @@ do_dedupe_range(unsigned offset, unsigned length, unsigned dest) } #endif +#ifdef HAVE_COPY_FILE_RANGE +int +test_copy_range(void) +{ + loff_t o1 = 0, o2 = 0; + + if (syscall(__NR_copy_file_range, fd, &o1, fd, &o2, 0, 0) == -1 && + (errno == EOPNOTSUPP || errno == ENOTTY)) { + if (!quiet) + fprintf(stderr, + "main: filesystem does not support " + "copy range, disabling!\n"); + return 0; + } + + return 1; +} + +void +do_copy_range(unsigned offset, unsigned length, unsigned dest) +{ + loff_t o1, o2; + size_t olen; + ssize_t nr; + int tries = 0; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length copy range\n"); + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping copy range behind EOF\n"); + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if (dest + length > biggest) { + biggest = dest + length; + if (!quiet && testcalls > simulatedopcount) + prt("copying to largest ever: 0x%x\n", dest + length); + } + + log5(OP_COPY_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu copy\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + o1 = offset; + o2 = dest; + olen = length; + + while (olen > 0) { + nr = syscall(__NR_copy_file_range, fd, &o1, fd, &o2, olen, 0); + if (nr < 0) { + if (errno != EAGAIN || tries++ >= 300) + break; + } else if (nr > olen) { + prt("copy range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prt("do_copy_range: asked %u, copied %u??\n", + nr, olen); + report_failure(161); + } else if (nr > 0) + olen -= nr; + } + if (nr < 0) { + prt("copy range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_copy_range:"); + report_failure(161); + } + + memcpy(good_buf + dest, good_buf + offset, length); + if (dest > file_size) + memset(good_buf + file_size, '\0', dest - file_size); + if (dest + length > file_size) + file_size = dest + length; +} + +#else +int +test_copy_range(void) +{ + return 0; +} + +void +do_copy_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1717,6 +1839,7 @@ op_args_count(int operation) switch (operation) { case OP_CLONE_RANGE: case OP_DEDUPE_RANGE: + case OP_COPY_RANGE: return 4; default: return 3; @@ -1891,6 +2014,18 @@ test(void) offset2 + size > file_size); break; } + case OP_COPY_RANGE: + TRIM_OFF_LEN(offset, size, file_size); + offset -= offset % readbdy; + if (o_direct) + size -= size % readbdy; + do { + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 -= offset2 % writebdy; + } while (llabs(offset2 - offset) < size || + offset2 + size > maxfilelen); + break; } have_op: @@ -1946,6 +2081,12 @@ test(void) goto out; } break; + case OP_COPY_RANGE: + if (!copy_range_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -2036,6 +2177,18 @@ test(void) do_dedupe_range(offset, size, offset2); break; + case OP_COPY_RANGE: + if (size == 0) { + log5(OP_COPY_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_COPY_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_copy_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -2061,7 +2214,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxABFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxABEFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -2108,6 +2261,9 @@ usage(void) #ifdef FIDEDUPERANGE " -B: Do not use dedupe range calls\n" #endif +#ifdef HAVE_COPY_FILE_RANGE +" -E: Do not use copy range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -2311,7 +2467,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:FJKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:EFJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2447,6 +2603,9 @@ main(int argc, char **argv) case 'B': dedupe_range_calls = 0; break; + case 'E': + copy_range_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); @@ -2669,6 +2828,8 @@ main(int argc, char **argv) clone_range_calls = test_clone_range(); if (dedupe_range_calls) dedupe_range_calls = test_dedupe_range(); + if (copy_range_calls) + copy_range_calls = test_copy_range(); while (numops == -1 || numops--) if (!test()) From patchwork Fri Dec 7 06:24:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717541 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4463818B8 for ; Fri, 7 Dec 2018 06:24:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3576D2EB67 for ; Fri, 7 Dec 2018 06:24:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2A2282EB6D; Fri, 7 Dec 2018 06:24:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF0612EB6A for ; Fri, 7 Dec 2018 06:24:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725985AbeLGGYn (ORCPT ); Fri, 7 Dec 2018 01:24:43 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:42150 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGYn (ORCPT ); Fri, 7 Dec 2018 01:24:43 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76OJCn153026; Fri, 7 Dec 2018 06:24:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=bYuzvfc2noHHTyDPwU24TDrhIvVN7I8a7JIntOFSYTI=; b=rV2xGtreZiNHKXeKWwCsb80VA/mI1lgNp7xX9jjDuu5A+7Tpznz1vprYZ3fKu4ii1kRy 4FJaSGh1JpVYPN+/zLRL4bszqn+LW5l2rD/sCNj7hCGEyNJwv3j5lyH5zpDYHNznwQ0m si1cOSsFb71BGrOvioifwX8z/ZWhtZHH2Vyvc7mSbxVed1xWQQAdzGMLkQ24WbgvEAaS 1JtIRNcnTciWVvuJmNNdvF4FCitqPmBXUvziFaEh+c0y4Vrg1WyrIfwUqx964e7aCdxQ 4j8rXg0C/bGywEXXv3YMIn365FjuBCk1QzRZ/UeymE3q18Nu5X7LvjAiznUOR/wC9jNf EQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2p3hquc4r6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:41 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OejQ005723 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:41 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB76Oeam027872; Fri, 7 Dec 2018 06:24:40 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:40 -0800 Subject: [PATCH 11/12] common/dump: disable copyrange From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:24:39 -0800 Message-ID: <154416387957.15344.2056457883087980703.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=507 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In _create_dumpdir_stress_num, disable copyrange in fsstress so that we dump exactly the same set of files and directories no matter how the xfs is configured. Signed-off-by: Darrick J. Wong --- common/dump | 3 +++ 1 file changed, 3 insertions(+) diff --git a/common/dump b/common/dump index c45b8a5a..4d1a1607 100644 --- a/common/dump +++ b/common/dump @@ -307,6 +307,9 @@ _create_dumpdir_stress_num() if $FSSTRESS_PROG | grep -q deduperange; then FSSTRESS_AVOID="-f deduperange=0 $FSSTRESS_AVOID" fi + if $FSSTRESS_PROG | grep -q copyrange; then + FSSTRESS_AVOID="-f copyrange=0 $FSSTRESS_AVOID" + fi echo "" echo "-----------------------------------------------" From patchwork Fri Dec 7 06:24:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717549 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AFC818B8 for ; Fri, 7 Dec 2018 06:24:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B7642EB6A for ; Fri, 7 Dec 2018 06:24:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1FB8F2EB83; Fri, 7 Dec 2018 06:24:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4FD52EB6A for ; Fri, 7 Dec 2018 06:24:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726044AbeLGGYu (ORCPT ); Fri, 7 Dec 2018 01:24:50 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:34586 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGYu (ORCPT ); Fri, 7 Dec 2018 01:24:50 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76Ocdt119162; Fri, 7 Dec 2018 06:24:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=yO3pNbB8yKNV+5ivAkcCWpY4cTjIDcYIBR52h+TW9C0=; b=Y4KAh8LTwZw+MDOUqB6/fv666c1php+M5MpPWOGVdMsN0LOMxXPoEXbvaD4xinhxcOAP FepUx3qXn5Cv5yHv810eRT99lZQek+tFi6ePVQXBl7kUNpuxD90Isd8qTyOJMR+ENWk2 Q4bytguk5a3kLJiGmabEJ28QB8giOezaoJmddoVeKvDs17ow+6SerTdQYmdrTHCoo2PI fMk6LeICk7rKiWCLYW8JNgKDtIYoSl3zEiaRBAQj4u7+LTogjGZdcYVDU3pJkR06doIX kFV38WU4JmwIEQtx7zTrpkFRvKUBpLX5YdrNzkJIv6rPV5g0UM/2459xCJxZqZaA+ap2 jA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2p3ftfg1sa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:47 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB76Olun020367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:47 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB76Ol9P027896; Fri, 7 Dec 2018 06:24:47 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:46 -0800 Subject: [PATCH 12/12] generic: long fsx soak tests From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:24:45 -0800 Message-ID: <154416388577.15344.12576557765635823075.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=985 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add a few tests to implement long soak tests of fsx. Signed-off-by: Darrick J. Wong --- tests/generic/938 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/938.out | 2 ++ tests/generic/939 | 55 ++++++++++++++++++++++++++++++++++++++++++++ tests/generic/939.out | 2 ++ tests/generic/group | 2 ++ 5 files changed, 122 insertions(+) create mode 100755 tests/generic/938 create mode 100644 tests/generic/938.out create mode 100755 tests/generic/939 create mode 100644 tests/generic/939.out diff --git a/tests/generic/938 b/tests/generic/938 new file mode 100755 index 00000000..9a860579 --- /dev/null +++ b/tests/generic/938 @@ -0,0 +1,61 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2018 Oracle. All Rights Reserved. +# +# FS QA Test 938 +# +# Long-soak directio fsx test +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test +_require_odirect + +# Run fsx for a million ops or more +nr_ops=$((1000000 * TIME_FACTOR)) +op_sz=$((128000 * LOAD_FACTOR)) +file_sz=$((600000 * LOAD_FACTOR)) +fsx_file=$TEST_DIR/fsx.$seq +min_dio_sz=$(_min_dio_alignment) + +fsx_args=(-q) +fsx_args+=(-N $nr_ops) +fsx_args+=(-p $((nr_ops / 100))) +fsx_args+=(-o $op_sz) +fsx_args+=(-l $file_sz) +fsx_args+=(-r $min_dio_sz) +fsx_args+=(-t $min_dio_sz) +fsx_args+=(-w $min_dio_sz) +fsx_args+=(-Z) + +run_fsx "${fsx_args[@]}" | sed -e '/^fsx.*/d' + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/generic/938.out b/tests/generic/938.out new file mode 100644 index 00000000..11b565ee --- /dev/null +++ b/tests/generic/938.out @@ -0,0 +1,2 @@ +QA output created by 938 +Silence is golden diff --git a/tests/generic/939 b/tests/generic/939 new file mode 100755 index 00000000..6a251fdd --- /dev/null +++ b/tests/generic/939 @@ -0,0 +1,55 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2018 Oracle. All Rights Reserved. +# +# FS QA Test 938 +# +# Long-soak buffered fsx test +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test + +# Run fsx for a million ops or more +nr_ops=$((1000000 * TIME_FACTOR)) +op_sz=$((128000 * LOAD_FACTOR)) +file_sz=$((600000 * LOAD_FACTOR)) +fsx_file=$TEST_DIR/fsx.$seq + +fsx_args=(-q) +fsx_args+=(-N $nr_ops) +fsx_args+=(-p $((nr_ops / 100))) +fsx_args+=(-o $op_sz) +fsx_args+=(-l $file_sz) + +run_fsx "${fsx_args[@]}" | sed -e '/^fsx.*/d' + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/generic/939.out b/tests/generic/939.out new file mode 100644 index 00000000..a86e90c5 --- /dev/null +++ b/tests/generic/939.out @@ -0,0 +1,2 @@ +QA output created by 939 +Silence is golden diff --git a/tests/generic/group b/tests/generic/group index eb5e4ce9..9025d24e 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -523,3 +523,5 @@ 518 auto quick clone 519 auto quick 520 auto quick log +938 soak long_rw +939 soak long_rw