From patchwork Tue Nov 20 21:56:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691303 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FE0B1709 for ; Tue, 20 Nov 2018 21:56:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D89228723 for ; Tue, 20 Nov 2018 21:56:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 813472AE54; Tue, 20 Nov 2018 21:56:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D68128723 for ; Tue, 20 Nov 2018 21:56:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725926AbeKUI2J (ORCPT ); Wed, 21 Nov 2018 03:28:09 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:48336 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725851AbeKUI2I (ORCPT ); Wed, 21 Nov 2018 03:28:08 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrY9m153534; Tue, 20 Nov 2018 21:56:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=gkEbzt0X75mV0RTARNIfHdzBB/jgI6z3YXlukup5mf4=; b=OcFBjK8PftXAKSul93X89+LNigo4iRUCme94k3fWmViCgPOVCxxOejpgEHJadNjeVGza Ql7i4KtIA7FQVEnR69MJR/ruc37NRxknQ/O7n5s2iz0MNMWm+cCc7A65Xo9Ex2VWOLX7 6at3OjiFjRvx1oqeNSRPhn44LsH7n7+ZVKuqtvbuaPQI+bpBdQxfk9NblYaMk5qGh8cH 0jApL/2GEz8Zg458ZJUZoulRmg+D8z6cONRohnd2PSFz1hWFR0zMcK/bQWkFtCZK0DWr don+RomUCMBKw7WZprFjrhjmNO3kkXID6jXSBxklyBDn1y7toSbyJe5l2sDZgtWKXpPh WQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2ntadtxdc5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:56:49 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLumYX007265 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:56:48 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAKLumh6029031; Tue, 20 Nov 2018 21:56:48 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:56:47 -0800 Subject: [PATCH 01/10] fsx: shut up compiler warnings From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:56:47 -0800 Message-ID: <154275100761.8611.14831629844790867009.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Fix unused variables and potential filename overflows in fsx. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index b0157ba3..5601c70c 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -121,7 +121,7 @@ char *bname; /* basename of our test file */ char *logdev; /* -i flag */ char *logid; /* -j flag */ char dname[1024]; /* -P flag */ -char goodfile[1024]; +char goodfile[PATH_MAX]; int dirpath = 0; /* -P flag */ int fd; /* fd for our test file */ @@ -185,7 +185,7 @@ const char *replayops = NULL; const char *recordops = NULL; FILE * fsxlogf = NULL; FILE * replayopsf = NULL; -char opsfile[1024]; +char opsfile[PATH_MAX]; int badoff = -1; int closeopen = 0; @@ -541,13 +541,13 @@ mark_log(void) void dump_fsync_buffer(void) { - char fname_buffer[1024]; + char fname_buffer[PATH_MAX]; int good_fd; if (!good_buf) return; - snprintf(fname_buffer, 1024, "%s%s.mark%d", dname, + snprintf(fname_buffer, sizeof(fname_buffer), "%s%s.mark%d", dname, bname, mark_nr); good_fd = open(fname_buffer, O_WRONLY|O_CREAT|O_TRUNC, 0666); if (good_fd < 0) { @@ -1865,7 +1865,7 @@ main(int argc, char **argv) { int i, style, ch; char *endp, *tmp; - char logfile[1024]; + char logfile[PATH_MAX]; struct stat statbuf; int o_flags = O_RDWR|O_CREAT|O_TRUNC; From patchwork Tue Nov 20 21:56:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6A551709 for ; Tue, 20 Nov 2018 21:56:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 962EC28801 for ; Tue, 20 Nov 2018 21:56:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A94C28723; Tue, 20 Nov 2018 21:56:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2624528801 for ; Tue, 20 Nov 2018 21:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725986AbeKUI2P (ORCPT ); Wed, 21 Nov 2018 03:28:15 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:48434 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725851AbeKUI2O (ORCPT ); Wed, 21 Nov 2018 03:28:14 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrL0E153210; Tue, 20 Nov 2018 21:56:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=M4HYCNRdEHSvc480uvUZaJWaaZJs9/ZR+Uufqas05fE=; b=v11aKzLjj9Jq/O6pNJEOGbgm2Uu37ijHOHusKMyTIs0TQwR4RN4FsNc8cOtsV3r/0qjB kCBHRtQ3akxESqm/uP+uTVdSXihPBpOx+lZK83gYUIORqK0TTqzS26uRmxhfJChi2qFp APacrKB4Ssh0XkmkpH+/O153ofzfL1QDIb05GwM8mKVeeU8TZ5omGjW81ZuGdBjmhPXm gtcHGXXhPiCafVTl1PWRAqAHbZKIFEXUQVFTBqlkBdWKUTlS966k4d3Kra5T4IMUvg6K WXn0GSQborL3BkrjslOfeCq/u7KCotP/Wo9R0XMwu9gxuOKv07QzUeK0uBaj0pnOnrrT yg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2ntadtxdcj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:56:55 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLusQs007454 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:56:54 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLusWX020422; Tue, 20 Nov 2018 21:56:54 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:56:53 -0800 Subject: [PATCH 02/10] fsx: always check buffer after each operation From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:56:53 -0800 Message-ID: <154275101368.8611.120389888153356093.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=738 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add a new option to make fsx read the file after each operation and compare it with the good buffer to try to catch corruptions as soon as they occur. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 56 insertions(+), 9 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 5601c70c..4b3a5d5a 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -160,7 +160,8 @@ int punch_hole_calls = 1; /* -H flag disables */ int zero_range_calls = 1; /* -z flag disables */ int collapse_range_calls = 1; /* -C flag disables */ int insert_range_calls = 1; /* -I flag disables */ -int mapped_reads = 1; /* -R flag disables it */ +int mapped_reads = 1; /* -R flag disables it */ +int check_file = 1; /* -X flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -561,7 +562,7 @@ dump_fsync_buffer(void) } void -check_buffers(unsigned offset, unsigned size) +check_buffers(char *buf, unsigned offset, unsigned size) { unsigned char c, t; unsigned i = 0; @@ -569,19 +570,19 @@ check_buffers(unsigned offset, unsigned size) unsigned op = 0; unsigned bad = 0; - if (memcmp(good_buf + offset, temp_buf, size) != 0) { + if (memcmp(good_buf + offset, buf, size) != 0) { prt("READ BAD DATA: offset = 0x%x, size = 0x%x, fname = %s\n", offset, size, fname); prt("OFFSET\tGOOD\tBAD\tRANGE\n"); while (size > 0) { c = good_buf[offset]; - t = temp_buf[i]; + t = buf[i]; if (c != t) { if (n < 16) { - bad = short_at(&temp_buf[i]); + bad = short_at(&buf[i]); prt("0x%05x\t0x%04x\t0x%04x", offset, short_at(&good_buf[offset]), bad); - op = temp_buf[offset & 1 ? i+1 : i]; + op = buf[offset & 1 ? i+1 : i]; prt("\t0x%05x\n", n); if (op) prt("operation# (mod 256) for " @@ -725,7 +726,46 @@ doread(unsigned offset, unsigned size) iret, size); report_failure(141); } - check_buffers(offset, size); + check_buffers(temp_buf, offset, size); +} + +void +check_contents(void) +{ + static char *check_buf; + unsigned offset = 0; + unsigned size = file_size; + off_t ret; + unsigned iret; + + if (!check_buf) { + check_buf = (char *) malloc(maxfilelen + writebdy); + assert(check_buf != NULL); + check_buf = round_ptr_up(check_buf, writebdy, 0); + memset(check_buf, '\0', maxfilelen); + } + + if (o_direct) + size -= size % readbdy; + if (size == 0) + return; + + ret = lseek(fd, (off_t)offset, SEEK_SET); + if (ret == (off_t)-1) { + prterr("doread: lseek"); + report_failure(140); + } + + iret = fsxread(fd, check_buf, size, offset); + if (iret != size) { + if (iret == -1) + prterr("check_contents: read"); + else + prt("short check read: 0x%x bytes instead of 0x%x\n", + iret, size); + report_failure(141); + } + check_buffers(check_buf, offset, size); } @@ -808,7 +848,7 @@ domapread(unsigned offset, unsigned size) report_failure(191); } - check_buffers(offset, size); + check_buffers(temp_buf, offset, size); } @@ -1624,6 +1664,9 @@ test(void) break; } + if (check_file) + check_contents(); + out: if (sizechecks && testcalls > simulatedopcount) check_size(); @@ -1684,6 +1727,7 @@ usage(void) -P: save .fsxlog .fsxops and .fsxgood files in dirpath (default ./)\n\ -S seed: for random # generator (default 1) 0 gets timestamp\n\ -W: mapped write operations DISabled\n\ + -X: Do not read file and compare to good buffer after every operation.\n\ -R: read() system calls only (mapped reads disabled)\n\ -Z: O_DIRECT (use -R, -W, -r and -w too)\n\ --replay-ops opsfile: replay ops from recorded .fsxops file\n\ @@ -1880,7 +1924,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FKHzCILN:OP:RS:WZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2044,6 +2088,9 @@ main(int argc, char **argv) if (!quiet) prt("mapped writes DISABLED\n"); break; + case 'X': + check_file = 0; + break; case 'Z': o_direct = O_DIRECT; o_flags |= O_DIRECT; From patchwork Tue Nov 20 21:56:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F040016B1 for ; Tue, 20 Nov 2018 21:57:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF67528723 for ; Tue, 20 Nov 2018 21:57:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D3B1C28872; Tue, 20 Nov 2018 21:57:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C78628801 for ; Tue, 20 Nov 2018 21:57:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725998AbeKUI2U (ORCPT ); Wed, 21 Nov 2018 03:28:20 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:48610 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725851AbeKUI2U (ORCPT ); Wed, 21 Nov 2018 03:28:20 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrQiY113566; Tue, 20 Nov 2018 21:57:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=s3v3Gov5cIhfOOXiXDm3biCpQKO96FxTJunf3AAA4QM=; b=mGU1k0CavxaYyXRb8l6BagAy/BcuJDt4VqSeRCbFKuOcGy1dUxwL7IRChculQ0lgBd5H 1w89nV2xXxuZGTvrpZN3B4uwUtt7n2przvsZFUFJewyHZvnPeS+4crr9s57NcyQ1dxoi 0dyTj2e2O2kq7Ij7Mrm0d1SU479DHqeo4EsLezrNfW+o+jbfX0uDG5n20UCK6F/P4Hrx v6FNHwkrDQRdmEC8twoiOFPcABXuMpz8tX35U0YNf8R5vB8N6/aE2tYqMPidIkuwB6+R ohxS8MW5D8axGqZda2JKOsmqSTOGWhc9SQanEA4JIPSK5zCaa5g52Uj2bMm66QPW1frF WQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2ntbmqp9jp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:00 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLv09L028762 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:00 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAKLv0EB020974; Tue, 20 Nov 2018 21:57:00 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:00 -0800 Subject: [PATCH 03/10] fsx: use an enum to define the operation commands From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:56:59 -0800 Message-ID: <154275101979.8611.5920262961282352956.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=801 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Use an enum to define operation codes and the boundaries between operation classes so that we can add new commands without having to change a bunch of unrelated #defines. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 40 +++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 4b3a5d5a..290b42f6 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -88,25 +88,27 @@ int logcount = 0; /* total ops */ * mode being run. */ -/* common operations */ -#define OP_READ 0 -#define OP_WRITE 1 -#define OP_MAPREAD 2 -#define OP_MAPWRITE 3 -#define OP_MAX_LITE 4 - -/* !lite operations */ -#define OP_TRUNCATE 4 -#define OP_FALLOCATE 5 -#define OP_PUNCH_HOLE 6 -#define OP_ZERO_RANGE 7 -#define OP_COLLAPSE_RANGE 8 -#define OP_INSERT_RANGE 9 -#define OP_MAX_FULL 10 - -/* integrity operations */ -#define OP_FSYNC 10 -#define OP_MAX_INTEGRITY 11 +enum { + /* common operations */ + OP_READ = 0, + OP_WRITE, + OP_MAPREAD, + OP_MAPWRITE, + OP_MAX_LITE, + + /* !lite operations */ + OP_TRUNCATE = OP_MAX_LITE, + OP_FALLOCATE, + OP_PUNCH_HOLE, + OP_ZERO_RANGE, + OP_COLLAPSE_RANGE, + OP_INSERT_RANGE, + OP_MAX_FULL, + + /* integrity operations */ + OP_FSYNC = OP_MAX_FULL, + OP_MAX_INTEGRITY, +}; #undef PAGE_SIZE #define PAGE_SIZE getpagesize() From patchwork Tue Nov 20 21:57:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 974651709 for ; Tue, 20 Nov 2018 21:57:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8689828723 for ; Tue, 20 Nov 2018 21:57:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74D212ACBC; Tue, 20 Nov 2018 21:57:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25B6228801 for ; Tue, 20 Nov 2018 21:57:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725946AbeKUI2c (ORCPT ); Wed, 21 Nov 2018 03:28:32 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:33226 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725851AbeKUI2c (ORCPT ); Wed, 21 Nov 2018 03:28:32 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLsG0N119410; Tue, 20 Nov 2018 21:57:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=5M5oK3+uvqeSey8L6oVwZ/cBa97FNCeDMMrRxeCQ7AE=; b=m7ZKMBNxYrwghk4n+3W1WBd7ShXUpI8zLp412HBlDtdfzm0cPmOMdLhvcV139U/D6qwm ylHjPkLz0aqWKadbNM/n71l1A96dGPcqc2k0AFLRULYevUMk0PP/BBRqc2GdEpRRhASl i2nLDjZb6nnhUdVhhaftHm3Ld1Aeak9xOcgyNOZYs/R5mFFiXxyqDjWpvP8/MS6lRJDF Rbqt4XqrNElxhwCmt5H3l4OBq8DoBmUy/i9XVhNyeNPxAGgrl5HE/7uHrO3ieK1UDuUE qr9Vt4lkYavrRMYrwFziLgXqtwpw5pXI+EmOH2kbb+wP8oypwZkhbbDxJsG3HZPIcMTy Gg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2120.oracle.com with ESMTP id 2ntaxq6buu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:12 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLv6Pp017327 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:07 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAKLv6cX021162; Tue, 20 Nov 2018 21:57:06 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:06 -0800 Subject: [PATCH 04/10] fsx: add five-argument logging function From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:57:05 -0800 Message-ID: <154275102589.8611.12239624125210101030.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=925 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add a five-argument logging function to support new operations. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 45 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 40 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 290b42f6..2372964d 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -51,7 +51,8 @@ enum opflags { FL_NONE = 0, FL_SKIPPED = 1, FL_CLOSE_OPEN = 2, FL_KEEP_SIZE = 4 struct log_entry { int operation; - int args[3]; + int nr_args; + int args[4]; enum opflags flags; }; @@ -278,6 +279,27 @@ static int op_code(const char *name) return -1; } +void +log5(int operation, int arg0, int arg1, int arg2, enum opflags flags) +{ + struct log_entry *le; + + le = &oplog[logptr]; + le->operation = operation; + if (closeopen) + flags |= FL_CLOSE_OPEN; + le->args[0] = arg0; + le->args[1] = arg1; + le->args[2] = arg2; + le->args[3] = file_size; + le->nr_args = 4; + le->flags = flags; + logptr++; + logcount++; + if (logptr >= LOGSIZE) + logptr = 0; +} + void log4(int operation, int arg0, int arg1, enum opflags flags) { @@ -290,6 +312,7 @@ log4(int operation, int arg0, int arg1, enum opflags flags) le->args[0] = arg0; le->args[1] = arg1; le->args[2] = file_size; + le->nr_args = 3; le->flags = flags; logptr++; logcount++; @@ -439,11 +462,13 @@ logdump(void) i = 0; if (logopsf) { + int j; + if (lp->flags & FL_SKIPPED) fprintf(logopsf, "skip "); - fprintf(logopsf, "%s 0x%x 0x%x 0x%x", - op_name(lp->operation), - lp->args[0], lp->args[1], lp->args[2]); + fprintf(logopsf, "%s", op_name(lp->operation)); + for (j = 0; j < lp->nr_args; j++) + fprintf(logopsf, " 0x%x", lp->args[j]); if (lp->flags & FL_KEEP_SIZE) fprintf(logopsf, " keep_size"); if (lp->flags & FL_CLOSE_OPEN) @@ -1413,6 +1438,15 @@ cleanup(int sig) exit(sig); } +static int +op_args_count(int operation) +{ + switch (operation) { + default: + return 3; + } +} + static int read_op(struct log_entry *log_entry) { @@ -1445,7 +1479,8 @@ read_op(struct log_entry *log_entry) log_entry->operation = op_code(str); if (log_entry->operation == -1) goto fail; - for (i = 0; i < 3; i++) { + log_entry->nr_args = op_args_count(log_entry->operation); + for (i = 0; i < log_entry->nr_args; i++) { char *end; str = strtok(NULL, " \t\n"); From patchwork Tue Nov 20 21:57:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B671D1709 for ; Tue, 20 Nov 2018 21:57:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A5D3D28723 for ; Tue, 20 Nov 2018 21:57:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 96C8B28801; Tue, 20 Nov 2018 21:57:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2AB128801 for ; Tue, 20 Nov 2018 21:57:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725851AbeKUI2c (ORCPT ); Wed, 21 Nov 2018 03:28:32 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:48850 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeKUI2c (ORCPT ); Wed, 21 Nov 2018 03:28:32 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrMZX113547; Tue, 20 Nov 2018 21:57:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=t9pxZzBg+zpNWS+ttipAGLyGAjmejfJgipNxJwyTu5s=; b=3BAImDyjx3c2Kjnif7s8rZn8WePm6WiRjXFQhrW1VMKHZhDW+Irz0o8qUEUHAb5tCTVE R2Zy+CZMkddtuKtfIyj8JcTND2WJ7W7lHDzWw3yObdJHtRyogIlT+YsE+Morfw9ugjeR HSdLvhEQXiXDSkqIyfoRH4tmGWk+667cH8tO4PQueLn3hWhJl840iuGmhePuvYiw80+W OMxFSVKXcHeojk9dlVAHAnrqqB7u0+1LJ8RPLk8IjzfQXylrcY8TaM/P56DfTBSMXq7R oECGlPTsP5VeLvl4m5Jp2eixEPctRDSvHup7koNBlo91diy/hogKhuKpPmOxvE+J2Kx7 JQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2ntbmqp9kq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:13 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvCjh025081 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:12 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAKLvC9F021216; Tue, 20 Nov 2018 21:57:12 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:12 -0800 Subject: [PATCH 05/10] fsx: add clone range From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:57:12 -0800 Message-ID: <154275103196.8611.2152558129471909566.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add support for FICLONERANGE to fsx. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 133 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 128 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 2372964d..a1a44b23 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -104,6 +104,7 @@ enum { OP_ZERO_RANGE, OP_COLLAPSE_RANGE, OP_INSERT_RANGE, + OP_CLONE_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -165,6 +166,7 @@ int collapse_range_calls = 1; /* -C flag disables */ int insert_range_calls = 1; /* -I flag disables */ int mapped_reads = 1; /* -R flag disables it */ int check_file = 1; /* -X flag disables */ +int clone_range_calls = 1; /* -J flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -258,6 +260,7 @@ static const char *op_names[] = { [OP_ZERO_RANGE] = "zero_range", [OP_COLLAPSE_RANGE] = "collapse_range", [OP_INSERT_RANGE] = "insert_range", + [OP_CLONE_RANGE] = "clone_range", [OP_FSYNC] = "fsync", }; @@ -320,7 +323,6 @@ log4(int operation, int arg0, int arg1, enum opflags flags) logptr = 0; } - void logdump(void) { @@ -342,7 +344,7 @@ logdump(void) count = LOGSIZE; } for ( ; count > 0; count--) { - bool overlap; + bool overlap, overlap2; int opnum; opnum = i+1 + (logcount/LOGSIZE)*LOGSIZE; @@ -444,6 +446,20 @@ logdump(void) if (overlap) prt("\t******IIII"); break; + case OP_CLONE_RANGE: + prt("CLONE 0x%x thru 0x%x\t(0x%x bytes) to 0x%x thru 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], + lp->args[2], lp->args[2] + lp->args[1] - 1); + overlap2 = badoff >= lp->args[2] && + badoff < lp->args[2] + lp->args[1]; + if (overlap && overlap2) + prt("\tJJJJ**JJJJ"); + else if (overlap) + prt("\tJJJJ******"); + else if (overlap2) + prt("\t******JJJJ"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1285,6 +1301,73 @@ do_insert_range(unsigned offset, unsigned length) } #endif +#ifdef FICLONERANGE +void +do_clone_range(unsigned offset, unsigned length, unsigned dest) +{ + struct file_clone_range fcr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length clone range\n"); + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping clone range behind EOF\n"); + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_CLONE_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu clone\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + fcr.src_fd = fd; + fcr.src_offset = offset; + fcr.src_length = length; + fcr.dest_offset = dest; + + if (ioctl(fd, FICLONERANGE, &fcr) == -1) { + if (errno == EOPNOTSUPP || errno == ENOTTY) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping unsupported clone range\n"); + logptr--; + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + prt("clone range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_clone_range: FICLONERANGE"); + report_failure(161); + } + + memcpy(good_buf + dest, good_buf + offset, length); + if (dest > file_size) + memset(good_buf + file_size, '\0', dest - file_size); + if (dest + length > file_size) + file_size = dest + length; +} + +#else +void +do_clone_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1442,6 +1525,8 @@ static int op_args_count(int operation) { switch (operation) { + case OP_CLONE_RANGE: + return 4; default: return 3; } @@ -1450,6 +1535,7 @@ op_args_count(int operation) static int read_op(struct log_entry *log_entry) { + char *end; char line[256]; memset(log_entry, 0, sizeof(*log_entry)); @@ -1515,7 +1601,7 @@ read_op(struct log_entry *log_entry) int test(void) { - unsigned long offset; + unsigned long offset, offset2; unsigned long size; unsigned long rv; unsigned long op; @@ -1546,6 +1632,7 @@ test(void) op = log_entry.operation; offset = log_entry.args[0]; size = log_entry.args[1]; + offset2 = log_entry.args[2]; closeopen = !!(log_entry.flags & FL_CLOSE_OPEN); keep_size = !!(log_entry.flags & FL_KEEP_SIZE); goto have_op; @@ -1558,6 +1645,7 @@ test(void) closeopen = (rv >> 3) < (1 << 28) / closeprob; offset = random(); + offset2 = 0; size = maxoplen; if (randomoplen) size = random() % (maxoplen + 1); @@ -1583,6 +1671,17 @@ test(void) if (zero_range_calls && size && keep_size_calls) keep_size = random() % 2; break; + case OP_CLONE_RANGE: + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 = offset2 & ~(block_size - 1); + } while (llabs(offset2 - offset) < size || + offset2 + size > maxfilelen); + break; } have_op: @@ -1626,6 +1725,12 @@ test(void) goto out; } break; + case OP_CLONE_RANGE: + if (!clone_range_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -1692,6 +1797,18 @@ test(void) do_insert_range(offset, size); break; + case OP_CLONE_RANGE: + if (size == 0) { + log5(OP_CLONE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_CLONE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_clone_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -1717,7 +1834,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxAFLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxAFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -1758,6 +1875,9 @@ usage(void) #ifdef FALLOC_FL_INSERT_RANGE " -I: Do not use insert range calls\n" #endif +#ifdef FICLONERANGE +" -J: Do not use clone range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -1961,7 +2081,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2091,6 +2211,9 @@ main(int argc, char **argv) case 'I': insert_range_calls = 0; break; + case 'J': + clone_range_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); From patchwork Tue Nov 20 21:57:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691323 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD2A314BD for ; Tue, 20 Nov 2018 21:57:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CABDC2ACBC for ; Tue, 20 Nov 2018 21:57:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BAB74293DC; Tue, 20 Nov 2018 21:57:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3487628872 for ; Tue, 20 Nov 2018 21:57:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725950AbeKUI2j (ORCPT ); Wed, 21 Nov 2018 03:28:39 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:48958 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeKUI2j (ORCPT ); Wed, 21 Nov 2018 03:28:39 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrLQL113539; Tue, 20 Nov 2018 21:57:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=3InRr0trBCIMmYPA1eFwDOhO4FaAR5Jboq2M1Uq2J98=; b=pevEh7nZOTGs24pAdtq7c+XZLNUtJbUdrXppQOuF7gQWbJNGcgIGpLykEbO4YF7sT78/ ToYdSYYawsa5qVhwaqvEMzHMf7K4ypWVnEBMWY+6hz8bGhGgB9PYDMlOzck8iBnvmIl5 +p1B6af/jf9ZYbQ+K0fhIZ2uvgDImiA/26W9zX6U1OvsuKetyOv5bIOFW2Rq0rsXG4mF yda31GPPbLso4dIvjlaho9HaWrZZ30QQgCXxEVxOUw/DuKHGNjno6BsjhiFcw8zV2BjA NtZ7wZXWspfxFWnhv3dfjB/o4DcYUdJZKshSit8CUKPdRta9KvOzy7P4ZMHSbajIxxio Hg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2ntbmqp9m3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:19 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvICD008728 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:18 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvIHB013778; Tue, 20 Nov 2018 21:57:18 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:18 -0800 Subject: [PATCH 06/10] fsx: add FIDEDUPERANGE support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:57:18 -0800 Message-ID: <154275103811.8611.6606584641150172917.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 141 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 139 insertions(+), 2 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index a1a44b23..6f6263e2 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -105,6 +105,7 @@ enum { OP_COLLAPSE_RANGE, OP_INSERT_RANGE, OP_CLONE_RANGE, + OP_DEDUPE_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -167,6 +168,7 @@ int insert_range_calls = 1; /* -I flag disables */ int mapped_reads = 1; /* -R flag disables it */ int check_file = 1; /* -X flag disables */ int clone_range_calls = 1; /* -J flag disables */ +int dedupe_range_calls = 1; /* -B flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -261,6 +263,7 @@ static const char *op_names[] = { [OP_COLLAPSE_RANGE] = "collapse_range", [OP_INSERT_RANGE] = "insert_range", [OP_CLONE_RANGE] = "clone_range", + [OP_DEDUPE_RANGE] = "dedupe_range", [OP_FSYNC] = "fsync", }; @@ -460,6 +463,20 @@ logdump(void) else if (overlap2) prt("\t******JJJJ"); break; + case OP_DEDUPE_RANGE: + prt("DEDUPE 0x%x thru 0x%x\t(0x%x bytes) to 0x%x thru 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], + lp->args[2], lp->args[2] + lp->args[1] - 1); + overlap2 = badoff >= lp->args[2] && + badoff < lp->args[2] + lp->args[1]; + if (overlap && overlap2) + prt("\tBBBB**BBBB"); + else if (overlap) + prt("\tBBBB******"); + else if (overlap2) + prt("\t******BBBB"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1368,6 +1385,82 @@ do_clone_range(unsigned offset, unsigned length, unsigned dest) } #endif +#ifdef FIDEDUPERANGE +void +do_dedupe_range(unsigned offset, unsigned length, unsigned dest) +{ + struct file_dedupe_range *fdr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length dedupe range\n"); + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping dedupe range behind EOF\n"); + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu dedupe\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + /* Alloc memory */ + fdr = malloc(sizeof(struct file_dedupe_range_info) + + sizeof(struct file_dedupe_range)); + if (!fdr) { + prterr("do_dedupe_range: malloc"); + report_failure(161); + } + memset(fdr, 0, (sizeof(struct file_dedupe_range_info) + + sizeof(struct file_dedupe_range))); + + /* Dedupe data blocks */ + fdr->src_offset = offset; + fdr->src_length = length; + fdr->dest_count = 1; + fdr->info[0].dest_fd = fd; + fdr->info[0].dest_offset = dest; + + if (ioctl(fd, FIDEDUPERANGE, fdr) == -1) { + if (errno == EOPNOTSUPP || errno == ENOTTY) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping unsupported dedupe range\n"); + logptr--; + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + free(fdr); + return; + } + + prt("dedupe range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_dedupe_range: FIDEDUPERANGE"); + report_failure(161); + } + + free(fdr); +} + +#else +void +do_dedupe_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1526,6 +1619,7 @@ op_args_count(int operation) { switch (operation) { case OP_CLONE_RANGE: + case OP_DEDUPE_RANGE: return 4; default: return 3; @@ -1682,6 +1776,25 @@ test(void) } while (llabs(offset2 - offset) < size || offset2 + size > maxfilelen); break; + case OP_DEDUPE_RANGE: + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, file_size); + offset2 = offset2 & ~(block_size - 1); + } while (llabs(offset2 - offset) < size || + offset2 + size > file_size); + break; + } } have_op: @@ -1731,6 +1844,12 @@ test(void) goto out; } break; + case OP_DEDUPE_RANGE: + if (!dedupe_range_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -1809,6 +1928,18 @@ test(void) do_clone_range(offset, size, offset2); break; + case OP_DEDUPE_RANGE: + if (size == 0) { + log5(OP_DEDUPE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_DEDUPE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_dedupe_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -1834,7 +1965,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxAFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxABFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -1878,6 +2009,9 @@ usage(void) #ifdef FICLONERANGE " -J: Do not use clone range calls\n" #endif +#ifdef FIDEDUPERANGE +" -B: Do not use dedupe range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -2081,7 +2215,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyAD:FJKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:FJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2214,6 +2348,9 @@ main(int argc, char **argv) case 'J': clone_range_calls = 0; break; + case 'B': + dedupe_range_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); From patchwork Tue Nov 20 21:57:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AADA14BD for ; Tue, 20 Nov 2018 21:57:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24AEA28801 for ; Tue, 20 Nov 2018 21:57:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 162432ACBB; Tue, 20 Nov 2018 21:57:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2610528801 for ; Tue, 20 Nov 2018 21:57:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725900AbeKUI2y (ORCPT ); Wed, 21 Nov 2018 03:28:54 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:49218 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeKUI2y (ORCPT ); Wed, 21 Nov 2018 03:28:54 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLs59n113817; Tue, 20 Nov 2018 21:57:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=3SAdy6YttZ73Ennb5m7cDCLMMkw5YTQEcbzqMPGUhX0=; b=PdukpybesmVs9cZdviWo/LxxxDVm19vZNtiMXNhG1ren7pTnhYWy29sVkhCkGff7nkzV rS47+ULaUJa370NL+O4qyxtvNLM4Fx2cXmcxmK5geVsvaVSVrU9ggtr+ou0pjbiImrFy +eGJHFsWY0Dez/MMWVWEnxaLdHXcbkBHlt/LDS0Pme88rATZSMIBxWWsvwHTiDIiXyHf X4vCDyBzXKdhtnlep+SIkCNIvY0J7Ii5qLLDkZfQQZQUQXBEPPTlYkJssKPT+yq6/Ztl RbfMHkk9ErDnm7du19MTc2pPSrfIAroEh1oM/D/hTI/17sryhfZ8mZdGxlgIjA8BOgYL Ow== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2ntbmqp9n3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:34 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvXcJ009228 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:33 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAKLvWbb021330; Tue, 20 Nov 2018 21:57:32 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:32 -0800 Subject: [PATCH 07/10] fsstress: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:57:24 -0800 Message-ID: <154275104427.8611.12901910984808681102.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Support the copy_file_range syscall. Signed-off-by: Darrick J. Wong --- configure.ac | 2 + include/builddefs.in | 1 ltp/Makefile | 4 + ltp/fsstress.c | 133 +++++++++++++++++++++++++++++++++++++++++++++++++ m4/package_libcdev.m4 | 19 +++++++ 5 files changed, 159 insertions(+) diff --git a/configure.ac b/configure.ac index aede4f59..19798824 100644 --- a/configure.ac +++ b/configure.ac @@ -68,6 +68,8 @@ AC_PACKAGE_WANT_OPEN_BY_HANDLE_AT AC_PACKAGE_WANT_LINUX_PRCTL_H AC_PACKAGE_WANT_LINUX_FS_H +AC_HAVE_COPY_FILE_RANGE + AC_CHECK_FUNCS([renameat2]) AC_CONFIG_HEADER(include/config.h) diff --git a/include/builddefs.in b/include/builddefs.in index fb8e912b..2605e42d 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -67,6 +67,7 @@ HAVE_DMAPI = @have_dmapi@ HAVE_ATTR_LIST = @have_attr_list@ HAVE_FIEMAP = @have_fiemap@ HAVE_FALLOCATE = @have_fallocate@ +HAVE_COPY_FILE_RANGE = @have_copy_file_range@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall diff --git a/ltp/Makefile b/ltp/Makefile index 5f784ecf..e4ca45f4 100644 --- a/ltp/Makefile +++ b/ltp/Makefile @@ -28,6 +28,10 @@ ifeq ($(HAVE_FALLOCATE), true) LCFLAGS += -DFALLOCATE endif +ifeq ($(HAVE_COPY_FILE_RANGE),yes) +LCFLAGS += -DHAVE_COPY_FILE_RANGE +endif + default: depend $(TARGETS) depend: .dep diff --git a/ltp/fsstress.c b/ltp/fsstress.c index edbb3f2f..dded0cf7 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -26,6 +26,7 @@ #include io_context_t io_ctx; #endif +#include #ifndef FS_IOC_GETFLAGS #define FS_IOC_GETFLAGS _IOR('f', 1, long) @@ -55,6 +56,7 @@ typedef enum { OP_BULKSTAT1, OP_CHOWN, OP_CLONERANGE, + OP_COPYRANGE, OP_CREAT, OP_DEDUPERANGE, OP_DREAD, @@ -163,6 +165,7 @@ void bulkstat_f(int, long); void bulkstat1_f(int, long); void chown_f(int, long); void clonerange_f(int, long); +void copyrange_f(int, long); void creat_f(int, long); void deduperange_f(int, long); void dread_f(int, long); @@ -212,6 +215,7 @@ opdesc_t ops[] = { { OP_BULKSTAT1, "bulkstat1", bulkstat1_f, 1, 0 }, { OP_CHOWN, "chown", chown_f, 3, 1 }, { OP_CLONERANGE, "clonerange", clonerange_f, 4, 1 }, + { OP_COPYRANGE, "copyrange", copyrange_f, 4, 1 }, { OP_CREAT, "creat", creat_f, 4, 1 }, { OP_DEDUPERANGE, "deduperange", deduperange_f, 4, 1}, { OP_DREAD, "dread", dread_f, 4, 0 }, @@ -2329,6 +2333,135 @@ clonerange_f( #endif } +/* copy some arbitrary range of f1 to f2. */ +void +copyrange_f( + int opno, + long r) +{ +#ifdef HAVE_COPY_FILE_RANGE + struct pathname fpath1; + struct pathname fpath2; + struct stat64 stat1; + struct stat64 stat2; + char inoinfo1[1024]; + char inoinfo2[1024]; + loff_t lr; + loff_t off1; + loff_t off2; + loff_t max_off2; + size_t len; + int v1; + int v2; + int fd1; + int fd2; + int ret; + int e; + + /* Load paths */ + init_pathname(&fpath1); + if (!get_fname(FT_REGm, r, &fpath1, NULL, NULL, &v1)) { + if (v1) + printf("%d/%d: copyrange read - no filename\n", + procid, opno); + goto out_fpath1; + } + + init_pathname(&fpath2); + if (!get_fname(FT_REGm, random(), &fpath2, NULL, NULL, &v2)) { + if (v2) + printf("%d/%d: copyrange write - no filename\n", + procid, opno); + goto out_fpath2; + } + + /* Open files */ + fd1 = open_path(&fpath1, O_RDONLY); + e = fd1 < 0 ? errno : 0; + check_cwd(); + if (fd1 < 0) { + if (v1) + printf("%d/%d: copyrange read - open %s failed %d\n", + procid, opno, fpath1.path, e); + goto out_fpath2; + } + + fd2 = open_path(&fpath2, O_WRONLY); + e = fd2 < 0 ? errno : 0; + check_cwd(); + if (fd2 < 0) { + if (v2) + printf("%d/%d: copyrange write - open %s failed %d\n", + procid, opno, fpath2.path, e); + goto out_fd1; + } + + /* Get file stats */ + if (fstat64(fd1, &stat1) < 0) { + if (v1) + printf("%d/%d: copyrange read - fstat64 %s failed %d\n", + procid, opno, fpath1.path, errno); + goto out_fd2; + } + inode_info(inoinfo1, sizeof(inoinfo1), &stat1, v1); + + if (fstat64(fd2, &stat2) < 0) { + if (v2) + printf("%d/%d: copyrange write - fstat64 %s failed %d\n", + procid, opno, fpath2.path, errno); + goto out_fd2; + } + inode_info(inoinfo2, sizeof(inoinfo2), &stat2, v2); + + /* Calculate offsets */ + len = (random() % FILELEN_MAX) + 1; + if (len == 0) + len = stat1.st_blksize; + if (len > stat1.st_size) + len = stat1.st_size; + + lr = ((int64_t)random() << 32) + random(); + if (stat1.st_size == len) + off1 = 0; + else + off1 = (off64_t)(lr % MIN(stat1.st_size - len, MAXFSIZE)); + off1 %= maxfsize; + + /* + * If srcfile == destfile, randomly generate destination ranges + * until we find one that doesn't overlap the source range. + */ + max_off2 = MIN(stat2.st_size + (1024ULL * stat2.st_blksize), MAXFSIZE); + do { + lr = ((int64_t)random() << 32) + random(); + off2 = (off64_t)(lr % max_off2); + off2 %= maxfsize; + } while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len); + + ret = copy_file_range(fd1, &off1, fd2, &off2, len, 0); + e = ret < 0 ? errno : 0; + if (v1 || v2) { + printf("%d/%d: copyrange %s%s [%lld,%lld] -> %s%s [%lld,%lld]", + procid, opno, + fpath1.path, inoinfo1, (long long)off1, (long long)len, + fpath2.path, inoinfo2, (long long)off2, (long long)len); + + if (ret < 0) + printf(" error %d", e); + printf("\n"); + } + +out_fd2: + close(fd2); +out_fd1: + close(fd1); +out_fpath2: + free_pathname(&fpath2); +out_fpath1: + free_pathname(&fpath1); +#endif +} + /* dedupe some arbitrary range of f1 to f2...fn. */ void deduperange_f( diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index fb123f33..14e67e18 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -98,3 +98,22 @@ AC_DEFUN([AC_HAVE_GETMNTINFO], AC_MSG_RESULT(no)) AC_SUBST(have_getmntinfo) ]) + +# +# +# Check if we have a copy_file_range system call (Linux) +# +AC_DEFUN([AC_HAVE_COPY_FILE_RANGE], + [ AC_MSG_CHECKING([for copy_file_range]) + AC_TRY_LINK([ +#define _GNU_SOURCE +#include +#include + ], [ + syscall(__NR_copy_file_range, 0, 0, 0, 0, 0, 0); + ], have_copy_file_range=yes + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(have_copy_file_range) + ]) + From patchwork Tue Nov 20 21:57:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691331 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE31E14BD for ; Tue, 20 Nov 2018 21:57:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD77828801 for ; Tue, 20 Nov 2018 21:57:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1A3528723; Tue, 20 Nov 2018 21:57:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 35D2528801 for ; Tue, 20 Nov 2018 21:57:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725999AbeKUI3B (ORCPT ); Wed, 21 Nov 2018 03:29:01 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:33644 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeKUI3B (ORCPT ); Wed, 21 Nov 2018 03:29:01 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrixg118965; Tue, 20 Nov 2018 21:57:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Ycf+Hn9NqqbHmXUvp4gOnb+HmVTenXwfHTCH+Gj8Usc=; b=nWDGKgMAkbzTsZtTV99ZRenTovd/8RrdRZL/oXag1Gc45Du4L3xwTy2/o8TnQqTqsj1Q kT6UK/B23FlWMG5ga9J9qlqAc/C+cEWYj0zj9DxsKm3541ELWnen39+mjOi77YmOa99z NMjf59Lz5GENr/mbIPSliOMi4TcUfGUBUqJVUMSPWM3E518I5yIvX7GhH4SMopeBmmAI gPjKdPn2RrEsrVM0fQ9gCDs9pocJwW5eCre2DN+E6twcQIjh0T8mnwh03HiGWh4dCAPq CIbg1Q0ddYUNWtJyt4rQogeHcL5Aw1Kdyu+27LnHXkueb5MbhfFV/6yDNb2/ckj4Wgpl KQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2120.oracle.com with ESMTP id 2ntaxq6bw6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:41 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvds0025870 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:40 GMT Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAKLvdcG029610; Tue, 20 Nov 2018 21:57:39 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:39 -0800 Subject: [PATCH 08/10] fsx: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:57:38 -0800 Message-ID: <154275105839.8611.5707428732705773914.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 122 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 120 insertions(+), 2 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 6f6263e2..ec3defad 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -34,6 +34,7 @@ #ifdef AIO #include #endif +#include #ifndef MAP_FILE # define MAP_FILE 0 @@ -106,6 +107,7 @@ enum { OP_INSERT_RANGE, OP_CLONE_RANGE, OP_DEDUPE_RANGE, + OP_COPY_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -169,6 +171,7 @@ int mapped_reads = 1; /* -R flag disables it */ int check_file = 1; /* -X flag disables */ int clone_range_calls = 1; /* -J flag disables */ int dedupe_range_calls = 1; /* -B flag disables */ +int copy_range_calls = 1; /* -E flag disables */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; int o_direct; /* -Z */ @@ -264,6 +267,7 @@ static const char *op_names[] = { [OP_INSERT_RANGE] = "insert_range", [OP_CLONE_RANGE] = "clone_range", [OP_DEDUPE_RANGE] = "dedupe_range", + [OP_COPY_RANGE] = "copy_range", [OP_FSYNC] = "fsync", }; @@ -477,6 +481,20 @@ logdump(void) else if (overlap2) prt("\t******BBBB"); break; + case OP_COPY_RANGE: + prt("COPY 0x%x thru 0x%x\t(0x%x bytes) to 0x%x thru 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], + lp->args[2], lp->args[2] + lp->args[1] - 1); + overlap2 = badoff >= lp->args[2] && + badoff < lp->args[2] + lp->args[1]; + if (overlap && overlap2) + prt("\tEEEE**EEEE"); + else if (overlap) + prt("\tEEEE******"); + else if (overlap2) + prt("\t******EEEE"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1461,6 +1479,73 @@ do_dedupe_range(unsigned offset, unsigned length, unsigned dest) } #endif +#ifdef HAVE_COPY_FILE_RANGE +void +do_copy_range(unsigned offset, unsigned length, unsigned dest) +{ + loff_t o1, o2; + ssize_t nr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length copy range\n"); + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping copy range behind EOF\n"); + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_COPY_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu copy\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + o1 = offset; + o2 = dest; + + nr = copy_file_range(fd, &o1, fd, &o2, length, 0); + if (nr == -1) { + if (errno == EOPNOTSUPP || errno == ENOTTY) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping unsupported copy range\n"); + logptr--; + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + prt("copy range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_copy_range:"); + report_failure(161); + } + + memcpy(good_buf + dest, good_buf + offset, nr); + if (dest > file_size) + memset(good_buf + file_size, '\0', dest - file_size); + if (dest + nr > file_size) + file_size = dest + nr; +} + +#else +void +do_copy_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1620,6 +1705,7 @@ op_args_count(int operation) switch (operation) { case OP_CLONE_RANGE: case OP_DEDUPE_RANGE: + case OP_COPY_RANGE: return 4; default: return 3; @@ -1795,6 +1881,14 @@ test(void) offset2 + size > file_size); break; } + case OP_COPY_RANGE: + TRIM_OFF_LEN(offset, size, file_size); + do { + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + } while (llabs(offset2 - offset) < size || + offset2 + size > maxfilelen); + break; } have_op: @@ -1850,6 +1944,12 @@ test(void) goto out; } break; + case OP_COPY_RANGE: + if (!copy_range_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -1940,6 +2040,18 @@ test(void) do_dedupe_range(offset, size, offset2); break; + case OP_COPY_RANGE: + if (size == 0) { + log5(OP_COPY_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_COPY_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_copy_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -1965,7 +2077,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxABFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxABEFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -2012,6 +2124,9 @@ usage(void) #ifdef FIDEDUPERANGE " -B: Do not use dedupe range calls\n" #endif +#ifdef HAVE_COPY_FILE_RANGE +" -E: Do not use copy range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -2215,7 +2330,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:FJKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:EFJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2351,6 +2466,9 @@ main(int argc, char **argv) case 'B': dedupe_range_calls = 0; break; + case 'E': + copy_range_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); From patchwork Tue Nov 20 21:57:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691335 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B7B41709 for ; Tue, 20 Nov 2018 21:57:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B41B28723 for ; Tue, 20 Nov 2018 21:57:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1FC022ACBC; Tue, 20 Nov 2018 21:57:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB89828801 for ; Tue, 20 Nov 2018 21:57:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725940AbeKUI3J (ORCPT ); Wed, 21 Nov 2018 03:29:09 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:49298 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeKUI3I (ORCPT ); Wed, 21 Nov 2018 03:29:08 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrxaq153741; Tue, 20 Nov 2018 21:57:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=1Ze1LB78qP6LeL8S6UIqFhZzLvSPDPRUIit5Fg3YneM=; b=k5CPhBLTrmzXmKklQoOdtcjZuCkHtwpveqGT3S+8xmEaPoqLykNbb5FgOp+UQpD4ejhr iyp6HHO1jDYh3s61UzcPzLmpdPinr96SMf7YzwXZviBvR9rnthBhRTDKbBf8BQLi86dl 4N5iiUmNd2F0zKX0JYQvttsvMUZOJqYMyx8u5enOLo1aD+GJeaKUKYySh7dz8Ev2B3mG lz2UH4Z33DeE58t1BLUnrbdXKMRDdUOtj0DWv+td7yLfviMWaCEszkxED6c33rgr4c7E BLJDaNyZwgFvQRdTmSkeaLaxF01ot6lyJLAWias1olaPmon7UpmaeO15rLe+lf1gte7s pQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2ntadtxdg0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:47 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvkgS018916 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:46 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvjat020856; Tue, 20 Nov 2018 21:57:45 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:45 -0800 Subject: [PATCH 09/10] fsx: clean up copy/dedupe file range support. From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, Dave Chinner Date: Tue, 20 Nov 2018 13:57:45 -0800 Message-ID: <154275106538.8611.5991693163854449275.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Dave Chinner copy_file_range() needs to obey read/write constraints otherwise is blows up when direct IO is used. FIDEDUPERANGE has a completely screwed up API for error reporting. The ioctl succeeds even if dedupe fails, so you have to check every individual dedupe operations for failure. Without this, dedupe "succeeds" on kernels filesystems that don't even support dedupe... Signed-off-by: Dave Chinner Reviewed-by: Darrick J. Wong Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index ec3defad..cea765d8 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1452,7 +1452,11 @@ do_dedupe_range(unsigned offset, unsigned length, unsigned dest) fdr->info[0].dest_fd = fd; fdr->info[0].dest_offset = dest; - if (ioctl(fd, FIDEDUPERANGE, fdr) == -1) { + if (ioctl(fd, FIDEDUPERANGE, fdr) == -1 || + fdr->info[0].status < 0) { + if (fdr->info[0].status < 0) + errno = -fdr->info[0].status; + if (errno == EOPNOTSUPP || errno == ENOTTY) { if (!quiet && testcalls > simulatedopcount) prt("skipping unsupported dedupe range\n"); @@ -1486,6 +1490,11 @@ do_copy_range(unsigned offset, unsigned length, unsigned dest) loff_t o1, o2; ssize_t nr; + offset -= offset % readbdy; + dest -= dest % writebdy; + if (o_direct) + length -= length % readbdy; + if (length == 0) { if (!quiet && testcalls > simulatedopcount) prt("skipping zero length copy range\n"); From patchwork Tue Nov 20 21:57:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10691339 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8E821709 for ; Tue, 20 Nov 2018 21:57:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D911528801 for ; Tue, 20 Nov 2018 21:57:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD3CD28723; Tue, 20 Nov 2018 21:57:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9083028801 for ; Tue, 20 Nov 2018 21:57:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726008AbeKUI3M (ORCPT ); Wed, 21 Nov 2018 03:29:12 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:33806 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeKUI3M (ORCPT ); Wed, 21 Nov 2018 03:29:12 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAKLrLPv118838; Tue, 20 Nov 2018 21:57:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=bYuzvfc2noHHTyDPwU24TDrhIvVN7I8a7JIntOFSYTI=; b=ep8uWvlHUfJixMtzLIBa9jsdLDCM8usNqeGYzFyR7V4/u51YjltuD8bxy9bpQ6mNLHHS 2ur4xOD8BiGS9UjUzumnN0Fj+FW0oprwFBbechQwDc6g801ACEMFPYhYtXa9lUQ0wvyp bDyCWXK+8jUdGhC0j3kCtkPu2iRQHWvx3MuumOXWbQND/9lzVdv0OvqWIiEH8ltS7+jZ E9FzGQCR/dHJSgvze6HN8HICt0loX2tToOJRLdfFV/JmNEt0h6VHlJwteabMHMOFfJXL MrY546VbRChl4ugK7N3cMrkC2NVkVeqj8eHVxZDEw7CkThiGlrtMFJqyqAT4l8hpGI3s WA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2120.oracle.com with ESMTP id 2ntaxq6bwq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:53 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvqIS026213 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Nov 2018 21:57:52 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wAKLvpge020903; Tue, 20 Nov 2018 21:57:52 GMT Received: from localhost (/10.145.178.163) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 20 Nov 2018 13:57:51 -0800 Subject: [PATCH 10/10] common/dump: disable copyrange From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 20 Nov 2018 13:57:51 -0800 Message-ID: <154275107150.8611.15972158351495348017.stgit@magnolia> In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9083 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=492 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811200192 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In _create_dumpdir_stress_num, disable copyrange in fsstress so that we dump exactly the same set of files and directories no matter how the xfs is configured. Signed-off-by: Darrick J. Wong --- common/dump | 3 +++ 1 file changed, 3 insertions(+) diff --git a/common/dump b/common/dump index c45b8a5a..4d1a1607 100644 --- a/common/dump +++ b/common/dump @@ -307,6 +307,9 @@ _create_dumpdir_stress_num() if $FSSTRESS_PROG | grep -q deduperange; then FSSTRESS_AVOID="-f deduperange=0 $FSSTRESS_AVOID" fi + if $FSSTRESS_PROG | grep -q copyrange; then + FSSTRESS_AVOID="-f copyrange=0 $FSSTRESS_AVOID" + fi echo "" echo "-----------------------------------------------" From patchwork Wed Nov 21 18:38:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10693021 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64C981709 for ; Wed, 21 Nov 2018 18:38:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 515FB2C087 for ; Wed, 21 Nov 2018 18:38:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 454C32C081; Wed, 21 Nov 2018 18:38:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D63AB2C07D for ; Wed, 21 Nov 2018 18:38:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729430AbeKVFOZ (ORCPT ); Thu, 22 Nov 2018 00:14:25 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51448 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726172AbeKVFOZ (ORCPT ); Thu, 22 Nov 2018 00:14:25 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wALIcreA105690; Wed, 21 Nov 2018 18:38:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : in-reply-to; s=corp-2018-07-02; bh=2xbtwK+cyK8zZfC9ov/K2aU7OfrSJCNdvWmRciRunHE=; b=Okc2bUzpfAXPcSn7a24qJVNFyXh9mGUUtGsTSYDQ7mIuXBReAGpltx8tY8P9vw+H9hn9 KLMbT9Zrvky/6TsFVwYdLUb5BmmF5qxWwkHcetU0wzTWcllFUCV0MPbldVEznxBNyBCC J0p3tKG6Ntkgoe4DACvDdBV854b1vIVMBQ7wQekQKXD2CZF2UlDNn4BkITPsIglPjJ4x f02NkaNzbWwwxJ7TZjdt4ulkIwoY8OLMX47ijP07jTe1ve1D8hLfk8wDzJzOTEzChPbj rmNHaIlA45bWgdPUjuRK8+ttuVAa2K3wW44Icw+tVNdU97+SMc/WjAf3jmwA1J0jwc61 3A== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2ntadu37qg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 21 Nov 2018 18:38:53 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wALIcqva023561 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 21 Nov 2018 18:38:52 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wALIcp7m022573; Wed, 21 Nov 2018 18:38:51 GMT Received: from localhost (/67.169.218.210) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 21 Nov 2018 10:38:51 -0800 Date: Wed, 21 Nov 2018 10:38:50 -0800 From: "Darrick J. Wong" To: guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Subject: [PATCH 11/10] generic: long fsx soak tests Message-ID: <20181121183850.GB6783@magnolia> References: <154275100143.8611.10235098565750994724.stgit@magnolia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <154275100143.8611.10235098565750994724.stgit@magnolia> User-Agent: Mutt/1.9.4 (2018-02-28) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9084 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811210161 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add a few tests to implement long soak tests of fsx. Signed-off-by: Darrick J. Wong --- tests/generic/938 | 65 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/938.out | 2 ++ tests/generic/939 | 60 +++++++++++++++++++++++++++++++++++++++++++++ tests/generic/939.out | 2 ++ tests/generic/group | 2 ++ 5 files changed, 131 insertions(+) create mode 100644 tests/generic/938 create mode 100644 tests/generic/938.out create mode 100644 tests/generic/939 create mode 100644 tests/generic/939.out diff --git a/tests/generic/938 b/tests/generic/938 new file mode 100644 index 00000000..a85a6472 --- /dev/null +++ b/tests/generic/938 @@ -0,0 +1,65 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2018 Oracle. All Rights Reserved. +# +# FS QA Test 938 +# +# Long-soak directio fsx test +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test + +# Run fsx for a million ops or more +nr_ops=$((1000000 * TIME_FACTOR)) +op_sz=$((128000 * LOAD_FACTOR)) +file_sz=$((600000 * LOAD_FACTOR)) +fsx_file=$TEST_DIR/fsx.$seq +min_dio_sz=$(_min_dio_alignment) + +fsx_args=(-q) +fsx_args+=(-N $nr_ops) +fsx_args+=(-p $((nr_ops / 100))) +fsx_args+=(-o $op_sz) +fsx_args+=(-l $file_sz) +fsx_args+=(-r $min_dio_sz) +fsx_args+=(-t $min_dio_sz) +fsx_args+=(-w $min_dio_sz) +fsx_args+=(-Z) +fsx_args+=($fsx_file) + +echo "ltp ${fsx_args[@]} $FSX_ARGS" >> $seqres.full + +if ! ltp/fsx "${fsx_args[@]}" $FSX_ARGS > $tmp.fsx 2>&1; then + cat $tmp.fsx | tee -a $seqres.full +fi + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/generic/938.out b/tests/generic/938.out new file mode 100644 index 00000000..11b565ee --- /dev/null +++ b/tests/generic/938.out @@ -0,0 +1,2 @@ +QA output created by 938 +Silence is golden diff --git a/tests/generic/939 b/tests/generic/939 new file mode 100644 index 00000000..5e341e30 --- /dev/null +++ b/tests/generic/939 @@ -0,0 +1,60 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2018 Oracle. All Rights Reserved. +# +# FS QA Test 938 +# +# Long-soak buffered fsx test +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test + +# Run fsx for a million ops or more +nr_ops=$((1000000 * TIME_FACTOR)) +op_sz=$((128000 * LOAD_FACTOR)) +file_sz=$((600000 * LOAD_FACTOR)) +fsx_file=$TEST_DIR/fsx.$seq + +fsx_args=(-q) +fsx_args+=(-N $nr_ops) +fsx_args+=(-p $((nr_ops / 100))) +fsx_args+=(-o $op_sz) +fsx_args+=(-l $file_sz) +fsx_args+=($fsx_file) + +echo "ltp ${fsx_args[@]} $FSX_ARGS" >> $seqres.full + +if ! ltp/fsx "${fsx_args[@]}" $FSX_ARGS > $tmp.fsx 2>&1; then + cat $tmp.fsx | tee -a $seqres.full +fi + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/generic/939.out b/tests/generic/939.out new file mode 100644 index 00000000..a86e90c5 --- /dev/null +++ b/tests/generic/939.out @@ -0,0 +1,2 @@ +QA output created by 939 +Silence is golden diff --git a/tests/generic/group b/tests/generic/group index 92330b52..16e5cc17 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -522,3 +522,5 @@ 517 auto quick dedupe clone 518 auto quick clone 519 auto quick +938 soak long_rw +939 soak long_rw