From patchwork Tue Nov 13 23:39:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10681627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A4FE93CF1 for ; Tue, 13 Nov 2018 23:39:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 982F12B458 for ; Tue, 13 Nov 2018 23:39:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C69B2B487; Tue, 13 Nov 2018 23:39:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8629A2B486 for ; Tue, 13 Nov 2018 23:39:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727065AbeKNJkX (ORCPT ); Wed, 14 Nov 2018 04:40:23 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36980 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726659AbeKNJkX (ORCPT ); Wed, 14 Nov 2018 04:40:23 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wADNclvH097686; Tue, 13 Nov 2018 23:39:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=jvdrTT0PzoV0cru3Qpyf/zulPb3ZxtUtH6u9S4X/zvw=; b=XOlAectVxjJAj1ER7LzL71eyPouolrB+N3wCvA9SaAl5za0Uc1MICZIDhyDw3jJINm+N SwOWRpa06e9TvFNZ3m0vB0RwLqAcw7tMo7hQjtBJJgbaeQLjsVJfYnebhji5/jUvwS0j lyIkhHWIIwKhZXxQ6+EZ23aReH81O800Qr5tvthWgaD+sp6LV++apYgcpF96PkERHpqU hh/VJ0ZCMbwZLUIwmlkxBjEBgTPh1seHOu8BoorYE94POopJ+Py9juBg5L0Cy3gKMvql RSIf+ueEeoYSp7Q2Aw3HBMomZRBWDmtZNDFE+y/nGaK6XSkuBDq3I3i4K0XMGKtsiZG9 Yw== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2nr7cs0927-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:39:46 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wADNdjg8023775 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:39:45 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wADNdiFI024681; Tue, 13 Nov 2018 23:39:44 GMT Received: from localhost (/10.159.147.150) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 13 Nov 2018 15:39:44 -0800 Subject: [PATCH 1/6] fsx: add clone range From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 13 Nov 2018 15:39:43 -0800 Message-ID: <154215238338.21151.852724936803809717.stgit@magnolia> In-Reply-To: <154215237717.21151.11976488103599724788.stgit@magnolia> References: <154215237717.21151.11976488103599724788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9076 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811130211 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Add support for FICLONERANGE to fsx. Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 182 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 158 insertions(+), 24 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index b0157ba3..2b0a2b28 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -88,25 +88,28 @@ int logcount = 0; /* total ops */ * mode being run. */ -/* common operations */ -#define OP_READ 0 -#define OP_WRITE 1 -#define OP_MAPREAD 2 -#define OP_MAPWRITE 3 -#define OP_MAX_LITE 4 - -/* !lite operations */ -#define OP_TRUNCATE 4 -#define OP_FALLOCATE 5 -#define OP_PUNCH_HOLE 6 -#define OP_ZERO_RANGE 7 -#define OP_COLLAPSE_RANGE 8 -#define OP_INSERT_RANGE 9 -#define OP_MAX_FULL 10 - -/* integrity operations */ -#define OP_FSYNC 10 -#define OP_MAX_INTEGRITY 11 +enum { + /* common operations */ + OP_READ = 0, + OP_WRITE, + OP_MAPREAD, + OP_MAPWRITE, + OP_MAX_LITE, + + /* !lite operations */ + OP_TRUNCATE = OP_MAX_LITE, + OP_FALLOCATE, + OP_PUNCH_HOLE, + OP_ZERO_RANGE, + OP_COLLAPSE_RANGE, + OP_INSERT_RANGE, + OP_CLONE_RANGE, + OP_MAX_FULL, + + /* integrity operations */ + OP_FSYNC = OP_MAX_FULL, + OP_MAX_INTEGRITY, +}; #undef PAGE_SIZE #define PAGE_SIZE getpagesize() @@ -160,6 +163,7 @@ int punch_hole_calls = 1; /* -H flag disables */ int zero_range_calls = 1; /* -z flag disables */ int collapse_range_calls = 1; /* -C flag disables */ int insert_range_calls = 1; /* -I flag disables */ +int remap_calls = 1; /* -J flag disables */ int mapped_reads = 1; /* -R flag disables it */ int integrity = 0; /* -i flag */ int fsxgoodfd = 0; @@ -254,6 +258,7 @@ static const char *op_names[] = { [OP_ZERO_RANGE] = "zero_range", [OP_COLLAPSE_RANGE] = "collapse_range", [OP_INSERT_RANGE] = "insert_range", + [OP_CLONE_RANGE] = "clone_range", [OP_FSYNC] = "fsync", }; @@ -275,6 +280,25 @@ static int op_code(const char *name) return -1; } +void +log5(int operation, int arg0, int arg1, int arg2, enum opflags flags) +{ + struct log_entry *le; + + le = &oplog[logptr]; + le->operation = operation; + if (closeopen) + flags |= FL_CLOSE_OPEN; + le->args[0] = arg0; + le->args[1] = arg1; + le->args[2] = arg2; + le->flags = flags; + logptr++; + logcount++; + if (logptr >= LOGSIZE) + logptr = 0; +} + void log4(int operation, int arg0, int arg1, enum opflags flags) { @@ -294,7 +318,6 @@ log4(int operation, int arg0, int arg1, enum opflags flags) logptr = 0; } - void logdump(void) { @@ -418,6 +441,13 @@ logdump(void) if (overlap) prt("\t******IIII"); break; + case OP_CLONE_RANGE: + prt("CLONE 0x%x thru 0x%x\t(0x%x bytes) to 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], lp->args[2]); + if (overlap) + prt("\t******IIII"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1218,6 +1248,73 @@ do_insert_range(unsigned offset, unsigned length) } #endif +#ifdef FICLONERANGE +void +do_clone_range(unsigned offset, unsigned length, unsigned dest) +{ + struct file_clone_range fcr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length clone range\n"); + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping clone range behind EOF\n"); + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_CLONE_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu clone\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + fcr.src_fd = fd; + fcr.src_offset = offset; + fcr.src_length = length; + fcr.dest_offset = dest; + + if (ioctl(fd, FICLONERANGE, &fcr) == -1) { + if (errno == EOPNOTSUPP || errno == ENOTTY) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping unsupported clone range\n"); + logptr--; + log5(OP_CLONE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + prt("clone range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_clone_range: FICLONERANGE"); + report_failure(161); + } + + memcpy(good_buf + dest, good_buf + offset, length); + if (dest > file_size) + memset(good_buf + file_size, '\0', dest - file_size); + if (dest + length > file_size) + file_size = dest + length; +} + +#else +void +do_clone_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1374,6 +1471,7 @@ cleanup(int sig) static int read_op(struct log_entry *log_entry) { + char *end; char line[256]; memset(log_entry, 0, sizeof(*log_entry)); @@ -1404,8 +1502,6 @@ read_op(struct log_entry *log_entry) if (log_entry->operation == -1) goto fail; for (i = 0; i < 3; i++) { - char *end; - str = strtok(NULL, " \t\n"); if (!str) goto fail; @@ -1438,7 +1534,7 @@ read_op(struct log_entry *log_entry) int test(void) { - unsigned long offset; + unsigned long offset, offset2; unsigned long size; unsigned long rv; unsigned long op; @@ -1469,6 +1565,7 @@ test(void) op = log_entry.operation; offset = log_entry.args[0]; size = log_entry.args[1]; + offset2 = log_entry.args[2]; closeopen = !!(log_entry.flags & FL_CLOSE_OPEN); keep_size = !!(log_entry.flags & FL_KEEP_SIZE); goto have_op; @@ -1481,6 +1578,7 @@ test(void) closeopen = (rv >> 3) < (1 << 28) / closeprob; offset = random(); + offset2 = 0; size = maxoplen; if (randomoplen) size = random() % (maxoplen + 1); @@ -1506,6 +1604,17 @@ test(void) if (zero_range_calls && size && keep_size_calls) keep_size = random() % 2; break; + case OP_CLONE_RANGE: + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 = offset2 & ~(block_size - 1); + } while (llabs(offset2 - offset) < size || + offset2 + size > maxfilelen); + break; } have_op: @@ -1549,6 +1658,12 @@ test(void) goto out; } break; + case OP_CLONE_RANGE: + if (!remap_calls) { + log5(op, offset, size, offset2, FL_SKIPPED); + goto out; + } + break; } switch (op) { @@ -1615,6 +1730,18 @@ test(void) do_insert_range(offset, size); break; + case OP_CLONE_RANGE: + if (size == 0) { + log5(OP_CLONE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_CLONE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_clone_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; @@ -1637,7 +1764,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxAFLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxAFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -1678,6 +1805,10 @@ usage(void) #ifdef FALLOC_FL_INSERT_RANGE " -I: Do not use insert range calls\n" #endif +#if defined(FICLONERANGE) || defined(FIDEDUPERANGE) || \ + defined(HAVE_COPY_FILE_RANGE) +" -J: Do not use clone/dedupe/copy range calls\n" +#endif " -L: fsxLite - no file creations & no file size changes\n\ -N numops: total # operations to do (default infinity)\n\ -O: use oplen (see -o flag) for every op (default random)\n\ @@ -2010,6 +2141,9 @@ main(int argc, char **argv) case 'I': insert_range_calls = 0; break; + case 'J': + remap_calls = 0; + break; case 'L': lite = 1; o_flags &= ~(O_CREAT|O_TRUNC); From patchwork Tue Nov 13 23:39:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10681631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBC4D46E4 for ; Tue, 13 Nov 2018 23:39:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC4832B458 for ; Tue, 13 Nov 2018 23:39:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDBA52B486; Tue, 13 Nov 2018 23:39:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58EC12B4A0 for ; Tue, 13 Nov 2018 23:39:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728210AbeKNJkd (ORCPT ); Wed, 14 Nov 2018 04:40:33 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:37100 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727770AbeKNJkc (ORCPT ); Wed, 14 Nov 2018 04:40:32 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wADNd8J0098103; Tue, 13 Nov 2018 23:39:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=CCspqmgq6rcZ1jq4J5wVnBCehxGn1nlO3/9yyYIifpY=; b=txvjsDTEvPW5O2k3W0s3zq6AXdwyUR/3woBIJNBrR8EB3Urfc5iThOYTn2ByWyODJ9cp fTP9Rbsid7TEnqTspf6jgqLiHcluqX/2p88LuXzDP5/9/VCE9gZO0JjKBZRRQMb2RR9i AhbC9wynKZTCB8nfyB7LWHW7VaSo4EYxyKPkjSRkpEDPeobGSmD93OtxmP9gigF5OsmY C0Q8x2+936pJB0gKvPPjl66mV+GLw6XIl73dyydTggSMr9nPfXvJO/Fixz8NjLURavF5 GBeYgODfV0kR0luQVFvB+3EneSnfqti74nRWOjoCG0mJUtdRH8YyGm/+91zxvDppiBjk 4w== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2nr7cs092s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:39:55 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wADNds6K023962 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:39:54 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wADNds6b024748; Tue, 13 Nov 2018 23:39:54 GMT Received: from localhost (/10.159.147.150) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 13 Nov 2018 15:39:53 -0800 Subject: [PATCH 2/6] fsx: add FIDEDUPERANGE support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 13 Nov 2018 15:39:49 -0800 Message-ID: <154215238968.21151.11312208246254421488.stgit@magnolia> In-Reply-To: <154215237717.21151.11976488103599724788.stgit@magnolia> References: <154215237717.21151.11976488103599724788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9076 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811130211 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 117 insertions(+) diff --git a/ltp/fsx.c b/ltp/fsx.c index 2b0a2b28..6f2b3364 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -104,6 +104,7 @@ enum { OP_COLLAPSE_RANGE, OP_INSERT_RANGE, OP_CLONE_RANGE, + OP_DEDUPE_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -259,6 +260,7 @@ static const char *op_names[] = { [OP_COLLAPSE_RANGE] = "collapse_range", [OP_INSERT_RANGE] = "insert_range", [OP_CLONE_RANGE] = "clone_range", + [OP_DEDUPE_RANGE] = "dedupe_range", [OP_FSYNC] = "fsync", }; @@ -448,6 +450,13 @@ logdump(void) if (overlap) prt("\t******IIII"); break; + case OP_DEDUPE_RANGE: + prt("DEDUPE 0x%x thru 0x%x\t(0x%x bytes) to 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], lp->args[2]); + if (overlap) + prt("\t******IIII"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1315,6 +1324,82 @@ do_clone_range(unsigned offset, unsigned length, unsigned dest) } #endif +#ifdef FIDEDUPERANGE +void +do_dedupe_range(unsigned offset, unsigned length, unsigned dest) +{ + struct file_dedupe_range *fdr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length dedupe range\n"); + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping dedupe range behind EOF\n"); + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu dedupe\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + /* Alloc memory */ + fdr = malloc(sizeof(struct file_dedupe_range_info) + + sizeof(struct file_dedupe_range)); + if (!fdr) { + prterr("do_dedupe_range: malloc"); + report_failure(161); + } + memset(fdr, 0, (sizeof(struct file_dedupe_range_info) + + sizeof(struct file_dedupe_range))); + + /* Dedupe data blocks */ + fdr->src_offset = offset; + fdr->src_length = length; + fdr->dest_count = 1; + fdr->info[0].dest_fd = fd; + fdr->info[0].dest_offset = dest; + + if (ioctl(fd, FIDEDUPERANGE, fdr) == -1) { + if (errno == EOPNOTSUPP || errno == ENOTTY) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping unsupported dedupe range\n"); + logptr--; + log5(OP_DEDUPE_RANGE, offset, length, dest, FL_SKIPPED); + free(fdr); + return; + } + + prt("dedupe range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_dedupe_range: FIDEDUPERANGE"); + report_failure(161); + } + + free(fdr); +} + +#else +void +do_dedupe_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1615,6 +1700,25 @@ test(void) } while (llabs(offset2 - offset) < size || offset2 + size > maxfilelen); break; + case OP_DEDUPE_RANGE: + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, file_size); + offset2 = offset2 & ~(block_size - 1); + } while (llabs(offset2 - offset) < size || + offset2 + size > file_size); + break; + } } have_op: @@ -1659,6 +1763,7 @@ test(void) } break; case OP_CLONE_RANGE: + case OP_DEDUPE_RANGE: if (!remap_calls) { log5(op, offset, size, offset2, FL_SKIPPED); goto out; @@ -1742,6 +1847,18 @@ test(void) do_clone_range(offset, size, offset2); break; + case OP_DEDUPE_RANGE: + if (size == 0) { + log5(OP_DEDUPE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_DEDUPE_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_dedupe_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; From patchwork Tue Nov 13 23:39:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10681641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D66B41759 for ; Tue, 13 Nov 2018 23:40:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA6082B458 for ; Tue, 13 Nov 2018 23:40:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BEF4B2B49F; Tue, 13 Nov 2018 23:40:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 880132B486 for ; Tue, 13 Nov 2018 23:40:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728686AbeKNJkv (ORCPT ); Wed, 14 Nov 2018 04:40:51 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:55328 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727770AbeKNJkv (ORCPT ); Wed, 14 Nov 2018 04:40:51 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wADNeDEQ139819; Tue, 13 Nov 2018 23:40:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=3SAdy6YttZ73Ennb5m7cDCLMMkw5YTQEcbzqMPGUhX0=; b=RVk8sKhkqhTMYkBCGfu558/ksq4GDK3IpbWdL71JbYr/Xx8JT9ywTmsBO7dRownMQjZQ ZkFg6AIyPpMpi9L7XeGsFJOdOLx9mmCYqW5iGpMZ5PapVQ9PkyUnF1A+tqANF2462Epv 46lEGXZsGYhyE3Y+2ZZrrH/p1HB27OXg9kILbjjUMOuQVs+PzQotxrmyc//IOuW8xkmP oJGlwWd1l+KI8OElwOtKqmeBmG5ML6FYgFvFmEP9XyAq+wKFrnzxk9GQUtA1fFNuw8tv KqI1jas9OxlrVnpGT4THSO70+/h8j+PYDvUzyJVcakmXGGkH1oL/Vr9Su2yTGKhW/WEx Nw== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2nr7cs092n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:13 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wADNe0aE024093 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:00 GMT Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wADNe0mk016643; Tue, 13 Nov 2018 23:40:00 GMT Received: from localhost (/10.159.147.150) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 13 Nov 2018 15:40:00 -0800 Subject: [PATCH 3/6] fsstress: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 13 Nov 2018 15:39:59 -0800 Message-ID: <154215239907.21151.16590308915145355962.stgit@magnolia> In-Reply-To: <154215237717.21151.11976488103599724788.stgit@magnolia> References: <154215237717.21151.11976488103599724788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9076 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811130211 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Support the copy_file_range syscall. Signed-off-by: Darrick J. Wong --- configure.ac | 2 + include/builddefs.in | 1 ltp/Makefile | 4 + ltp/fsstress.c | 133 +++++++++++++++++++++++++++++++++++++++++++++++++ m4/package_libcdev.m4 | 19 +++++++ 5 files changed, 159 insertions(+) diff --git a/configure.ac b/configure.ac index aede4f59..19798824 100644 --- a/configure.ac +++ b/configure.ac @@ -68,6 +68,8 @@ AC_PACKAGE_WANT_OPEN_BY_HANDLE_AT AC_PACKAGE_WANT_LINUX_PRCTL_H AC_PACKAGE_WANT_LINUX_FS_H +AC_HAVE_COPY_FILE_RANGE + AC_CHECK_FUNCS([renameat2]) AC_CONFIG_HEADER(include/config.h) diff --git a/include/builddefs.in b/include/builddefs.in index fb8e912b..2605e42d 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -67,6 +67,7 @@ HAVE_DMAPI = @have_dmapi@ HAVE_ATTR_LIST = @have_attr_list@ HAVE_FIEMAP = @have_fiemap@ HAVE_FALLOCATE = @have_fallocate@ +HAVE_COPY_FILE_RANGE = @have_copy_file_range@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall diff --git a/ltp/Makefile b/ltp/Makefile index 5f784ecf..e4ca45f4 100644 --- a/ltp/Makefile +++ b/ltp/Makefile @@ -28,6 +28,10 @@ ifeq ($(HAVE_FALLOCATE), true) LCFLAGS += -DFALLOCATE endif +ifeq ($(HAVE_COPY_FILE_RANGE),yes) +LCFLAGS += -DHAVE_COPY_FILE_RANGE +endif + default: depend $(TARGETS) depend: .dep diff --git a/ltp/fsstress.c b/ltp/fsstress.c index edbb3f2f..dded0cf7 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -26,6 +26,7 @@ #include io_context_t io_ctx; #endif +#include #ifndef FS_IOC_GETFLAGS #define FS_IOC_GETFLAGS _IOR('f', 1, long) @@ -55,6 +56,7 @@ typedef enum { OP_BULKSTAT1, OP_CHOWN, OP_CLONERANGE, + OP_COPYRANGE, OP_CREAT, OP_DEDUPERANGE, OP_DREAD, @@ -163,6 +165,7 @@ void bulkstat_f(int, long); void bulkstat1_f(int, long); void chown_f(int, long); void clonerange_f(int, long); +void copyrange_f(int, long); void creat_f(int, long); void deduperange_f(int, long); void dread_f(int, long); @@ -212,6 +215,7 @@ opdesc_t ops[] = { { OP_BULKSTAT1, "bulkstat1", bulkstat1_f, 1, 0 }, { OP_CHOWN, "chown", chown_f, 3, 1 }, { OP_CLONERANGE, "clonerange", clonerange_f, 4, 1 }, + { OP_COPYRANGE, "copyrange", copyrange_f, 4, 1 }, { OP_CREAT, "creat", creat_f, 4, 1 }, { OP_DEDUPERANGE, "deduperange", deduperange_f, 4, 1}, { OP_DREAD, "dread", dread_f, 4, 0 }, @@ -2329,6 +2333,135 @@ clonerange_f( #endif } +/* copy some arbitrary range of f1 to f2. */ +void +copyrange_f( + int opno, + long r) +{ +#ifdef HAVE_COPY_FILE_RANGE + struct pathname fpath1; + struct pathname fpath2; + struct stat64 stat1; + struct stat64 stat2; + char inoinfo1[1024]; + char inoinfo2[1024]; + loff_t lr; + loff_t off1; + loff_t off2; + loff_t max_off2; + size_t len; + int v1; + int v2; + int fd1; + int fd2; + int ret; + int e; + + /* Load paths */ + init_pathname(&fpath1); + if (!get_fname(FT_REGm, r, &fpath1, NULL, NULL, &v1)) { + if (v1) + printf("%d/%d: copyrange read - no filename\n", + procid, opno); + goto out_fpath1; + } + + init_pathname(&fpath2); + if (!get_fname(FT_REGm, random(), &fpath2, NULL, NULL, &v2)) { + if (v2) + printf("%d/%d: copyrange write - no filename\n", + procid, opno); + goto out_fpath2; + } + + /* Open files */ + fd1 = open_path(&fpath1, O_RDONLY); + e = fd1 < 0 ? errno : 0; + check_cwd(); + if (fd1 < 0) { + if (v1) + printf("%d/%d: copyrange read - open %s failed %d\n", + procid, opno, fpath1.path, e); + goto out_fpath2; + } + + fd2 = open_path(&fpath2, O_WRONLY); + e = fd2 < 0 ? errno : 0; + check_cwd(); + if (fd2 < 0) { + if (v2) + printf("%d/%d: copyrange write - open %s failed %d\n", + procid, opno, fpath2.path, e); + goto out_fd1; + } + + /* Get file stats */ + if (fstat64(fd1, &stat1) < 0) { + if (v1) + printf("%d/%d: copyrange read - fstat64 %s failed %d\n", + procid, opno, fpath1.path, errno); + goto out_fd2; + } + inode_info(inoinfo1, sizeof(inoinfo1), &stat1, v1); + + if (fstat64(fd2, &stat2) < 0) { + if (v2) + printf("%d/%d: copyrange write - fstat64 %s failed %d\n", + procid, opno, fpath2.path, errno); + goto out_fd2; + } + inode_info(inoinfo2, sizeof(inoinfo2), &stat2, v2); + + /* Calculate offsets */ + len = (random() % FILELEN_MAX) + 1; + if (len == 0) + len = stat1.st_blksize; + if (len > stat1.st_size) + len = stat1.st_size; + + lr = ((int64_t)random() << 32) + random(); + if (stat1.st_size == len) + off1 = 0; + else + off1 = (off64_t)(lr % MIN(stat1.st_size - len, MAXFSIZE)); + off1 %= maxfsize; + + /* + * If srcfile == destfile, randomly generate destination ranges + * until we find one that doesn't overlap the source range. + */ + max_off2 = MIN(stat2.st_size + (1024ULL * stat2.st_blksize), MAXFSIZE); + do { + lr = ((int64_t)random() << 32) + random(); + off2 = (off64_t)(lr % max_off2); + off2 %= maxfsize; + } while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len); + + ret = copy_file_range(fd1, &off1, fd2, &off2, len, 0); + e = ret < 0 ? errno : 0; + if (v1 || v2) { + printf("%d/%d: copyrange %s%s [%lld,%lld] -> %s%s [%lld,%lld]", + procid, opno, + fpath1.path, inoinfo1, (long long)off1, (long long)len, + fpath2.path, inoinfo2, (long long)off2, (long long)len); + + if (ret < 0) + printf(" error %d", e); + printf("\n"); + } + +out_fd2: + close(fd2); +out_fd1: + close(fd1); +out_fpath2: + free_pathname(&fpath2); +out_fpath1: + free_pathname(&fpath1); +#endif +} + /* dedupe some arbitrary range of f1 to f2...fn. */ void deduperange_f( diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index fb123f33..14e67e18 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -98,3 +98,22 @@ AC_DEFUN([AC_HAVE_GETMNTINFO], AC_MSG_RESULT(no)) AC_SUBST(have_getmntinfo) ]) + +# +# +# Check if we have a copy_file_range system call (Linux) +# +AC_DEFUN([AC_HAVE_COPY_FILE_RANGE], + [ AC_MSG_CHECKING([for copy_file_range]) + AC_TRY_LINK([ +#define _GNU_SOURCE +#include +#include + ], [ + syscall(__NR_copy_file_range, 0, 0, 0, 0, 0, 0); + ], have_copy_file_range=yes + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(have_copy_file_range) + ]) + From patchwork Tue Nov 13 23:40:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10681635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 464A246E4 for ; Tue, 13 Nov 2018 23:40:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A9B42B458 for ; Tue, 13 Nov 2018 23:40:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2F7E62B4A1; Tue, 13 Nov 2018 23:40:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6D062B486 for ; Tue, 13 Nov 2018 23:40:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728330AbeKNJku (ORCPT ); Wed, 14 Nov 2018 04:40:50 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:55314 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726408AbeKNJku (ORCPT ); Wed, 14 Nov 2018 04:40:50 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wADNeCbr139789; Tue, 13 Nov 2018 23:40:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=qIhxKHX1n81H1dFSs3E4AUBOG4IVve5czZQUPYpn0KM=; b=1IdyU4OERlMgNApWg+aRtpWPTCOKaV63ix36uFXvabLPlYcKme0VzthGJsv2Jbq1DvLu Adrigv7IOttjBgFJCHIQxRg+3UP5AVOhOa8m5O94tFjRV8u9LXSJwXyMBy9dlWU7f5JJ vsN4IvuM9ShgsePUxvZLgMe2woBuPm6xpA2p7PQGzjIoxs/4GPgMYoAazRLjep4/xQMs yu7UDmf4wfeO+ByMn6RtbjCsK0F3sZpqDWR72oZ/UXJXTt+da8efx8twlNRW2MNuoryb 78pz6L0GkBYq/jnpeouhaEEcKzzQejTqqG4QC0iItPE5VvS4QzOkT47MUSFT2upyBfYW kw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2nr7cs0938-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:12 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wADNe6N0031143 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:07 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wADNe6G8016695; Tue, 13 Nov 2018 23:40:06 GMT Received: from localhost (/10.159.147.150) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 13 Nov 2018 15:40:06 -0800 Subject: [PATCH 4/6] fsx: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 13 Nov 2018 15:40:05 -0800 Message-ID: <154215240539.21151.10456496832542456055.stgit@magnolia> In-Reply-To: <154215237717.21151.11976488103599724788.stgit@magnolia> References: <154215237717.21151.11976488103599724788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9076 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811130211 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/ltp/fsx.c b/ltp/fsx.c index 6f2b3364..3161ba12 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -34,6 +34,7 @@ #ifdef AIO #include #endif +#include #ifndef MAP_FILE # define MAP_FILE 0 @@ -105,6 +106,7 @@ enum { OP_INSERT_RANGE, OP_CLONE_RANGE, OP_DEDUPE_RANGE, + OP_COPY_RANGE, OP_MAX_FULL, /* integrity operations */ @@ -261,6 +263,7 @@ static const char *op_names[] = { [OP_INSERT_RANGE] = "insert_range", [OP_CLONE_RANGE] = "clone_range", [OP_DEDUPE_RANGE] = "dedupe_range", + [OP_COPY_RANGE] = "copy_range", [OP_FSYNC] = "fsync", }; @@ -457,6 +460,13 @@ logdump(void) if (overlap) prt("\t******IIII"); break; + case OP_COPY_RANGE: + prt("COPY 0x%x thru 0x%x\t(0x%x bytes) to 0x%x", + lp->args[0], lp->args[0] + lp->args[1] - 1, + lp->args[1], lp->args[2]); + if (overlap) + prt("\t******IIII"); + break; case OP_FSYNC: prt("FSYNC"); break; @@ -1400,6 +1410,73 @@ do_dedupe_range(unsigned offset, unsigned length, unsigned dest) } #endif +#ifdef HAVE_COPY_FILE_RANGE +void +do_copy_range(unsigned offset, unsigned length, unsigned dest) +{ + loff_t o1, o2; + ssize_t nr; + + if (length == 0) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping zero length copy range\n"); + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + if ((loff_t)offset >= file_size) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping copy range behind EOF\n"); + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + log5(OP_COPY_RANGE, offset, length, dest, FL_NONE); + + if (testcalls <= simulatedopcount) + return; + + if ((progressinterval && testcalls % progressinterval == 0) || + (debug && (monitorstart == -1 || monitorend == -1 || + dest <= monitorstart || dest + length <= monitorend))) { + prt("%lu copy\tfrom 0x%x to 0x%x, (0x%x bytes) at 0x%x\n", + testcalls, offset, offset+length, length, dest); + } + + o1 = offset; + o2 = dest; + + nr = copy_file_range(fd, &o1, fd, &o2, length, 0); + if (nr == -1) { + if (errno == EOPNOTSUPP || errno == ENOTTY) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping unsupported copy range\n"); + logptr--; + log5(OP_COPY_RANGE, offset, length, dest, FL_SKIPPED); + return; + } + + prt("copy range: 0x%x to 0x%x at 0x%x\n", offset, + offset + length, dest); + prterr("do_copy_range:"); + report_failure(161); + } + + memcpy(good_buf + dest, good_buf + offset, nr); + if (dest > file_size) + memset(good_buf + file_size, '\0', dest - file_size); + if (dest + nr > file_size) + file_size = dest + nr; +} + +#else +void +do_copy_range(unsigned offset, unsigned length, unsigned dest) +{ + return; +} +#endif + #ifdef HAVE_LINUX_FALLOC_H /* fallocate is basically a no-op unless extending, then a lot like a truncate */ void @@ -1719,6 +1796,14 @@ test(void) offset2 + size > file_size); break; } + case OP_COPY_RANGE: + TRIM_OFF_LEN(offset, size, file_size); + do { + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + } while (llabs(offset2 - offset) < size || + offset2 + size > maxfilelen); + break; } have_op: @@ -1764,6 +1849,7 @@ test(void) break; case OP_CLONE_RANGE: case OP_DEDUPE_RANGE: + case OP_COPY_RANGE: if (!remap_calls) { log5(op, offset, size, offset2, FL_SKIPPED); goto out; @@ -1859,6 +1945,18 @@ test(void) do_dedupe_range(offset, size, offset2); break; + case OP_COPY_RANGE: + if (size == 0) { + log5(OP_COPY_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + if (offset2 + size > maxfilelen) { + log5(OP_COPY_RANGE, offset, size, offset2, FL_SKIPPED); + goto out; + } + + do_copy_range(offset, size, offset2); + break; case OP_FSYNC: dofsync(); break; From patchwork Tue Nov 13 23:40:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10681639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73F033CF1 for ; Tue, 13 Nov 2018 23:40:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63E512B458 for ; Tue, 13 Nov 2018 23:40:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5830A2B4A0; Tue, 13 Nov 2018 23:40:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD18C2B49F for ; Tue, 13 Nov 2018 23:40:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727770AbeKNJkw (ORCPT ); Wed, 14 Nov 2018 04:40:52 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:53336 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726408AbeKNJkw (ORCPT ); Wed, 14 Nov 2018 04:40:52 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wADNeDrp107170; Tue, 13 Nov 2018 23:40:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=fZxSJxbO5oHBjORjSt1JYYIUIPQrYELnRPhHAky7s2c=; b=2rH3b5JD0/tuBovlHXmzXYqLErsTjYGfqL3putPteqdlGjDmvzNsDsJrOhUQdq7eWE2d G1XcfUt/vc60oqMserZL/oJ5+nOm8KSQXbCgrA4iP/Ny0E3q270xYqo8wL5hWfn7mA37 qrZRF81uK3ZQ3j1j3jjdc5fHeTDPL+i/riSR3eT19cr8bRoUKjRWL0mOEfMC3EGhlmn9 yi6Sv4V2MapuQwTmhx8tVD15WBGt6W8FXVTUIw6UOPlxyj/YpJMNkIW/MvCC98hF3lmD NuYLnZy+uqInuloqWGqEOx5DZqWOpR2Fp1b9X7+JL+qiPm3ak8a8q3Ua3ybEXW9Qx5yw Sw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2120.oracle.com with ESMTP id 2nr7cs097y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:13 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wADNeD2S001309 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:13 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wADNeCuf002217; Tue, 13 Nov 2018 23:40:12 GMT Received: from localhost (/10.159.147.150) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 13 Nov 2018 15:40:12 -0800 Subject: [PATCH 5/6] fsx: clean up copy/dedupe file range support. From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, Dave Chinner Date: Tue, 13 Nov 2018 15:40:11 -0800 Message-ID: <154215241163.21151.5676816385137982668.stgit@magnolia> In-Reply-To: <154215237717.21151.11976488103599724788.stgit@magnolia> References: <154215237717.21151.11976488103599724788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9076 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811130211 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Dave Chinner copy_file_range() needs to obey read/write constraints otherwise is blows up when direct IO is used. FIDEDUPERANGE has a completely screwed up API for error reporting. The ioctl succeeds even if dedupe fails, so you have to check every individual dedupe operations for failure. Without this, dedupe "succeeds" on kernels filesystems that don't even support dedupe... Signed-off-by: Dave Chinner Reviewed-by: Darrick J. Wong Signed-off-by: Darrick J. Wong --- ltp/fsx.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 3161ba12..f6cc6e95 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1383,7 +1383,11 @@ do_dedupe_range(unsigned offset, unsigned length, unsigned dest) fdr->info[0].dest_fd = fd; fdr->info[0].dest_offset = dest; - if (ioctl(fd, FIDEDUPERANGE, fdr) == -1) { + if (ioctl(fd, FIDEDUPERANGE, fdr) == -1 || + fdr->info[0].status < 0) { + if (fdr->info[0].status < 0) + errno = -fdr->info[0].status; + if (errno == EOPNOTSUPP || errno == ENOTTY) { if (!quiet && testcalls > simulatedopcount) prt("skipping unsupported dedupe range\n"); @@ -1417,6 +1421,11 @@ do_copy_range(unsigned offset, unsigned length, unsigned dest) loff_t o1, o2; ssize_t nr; + offset -= offset % readbdy; + dest -= dest % writebdy; + if (o_direct) + length -= length % readbdy; + if (length == 0) { if (!quiet && testcalls > simulatedopcount) prt("skipping zero length copy range\n"); From patchwork Tue Nov 13 23:40:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10681645 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 123D646E4 for ; Tue, 13 Nov 2018 23:40:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F32F22B486 for ; Tue, 13 Nov 2018 23:40:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E77492B4A0; Tue, 13 Nov 2018 23:40:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9A232B486 for ; Tue, 13 Nov 2018 23:40:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726999AbeKNJk5 (ORCPT ); Wed, 14 Nov 2018 04:40:57 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:55424 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726408AbeKNJk5 (ORCPT ); Wed, 14 Nov 2018 04:40:57 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wADNeCmZ139802; Tue, 13 Nov 2018 23:40:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=bYuzvfc2noHHTyDPwU24TDrhIvVN7I8a7JIntOFSYTI=; b=KN+aiU3nQgEFRRPt28oC8b0+tvYTM9+hoZzYnk2cmE8pbXlUmT6rfUIsgIfNM9qO8t/N Itc6/B9qUc8BtEhTAbU1REd5w25lUboNWAWiTIG9z4rt4dA20XdWOCorxinwBqqyIWzg KKreDCVjM6e5nsBZZvQVhElfI+RgfG+H0Gg7Zk9+wCTsOeJaQcFi7qsBB5Xut1nMFbkZ bvtU9MAvw9wz1MLnDeQEMPN+vw1rSEXKaQsYm2xIgLvnIRZZrwlgb4c0iVj+zknlVuYb F+/Ql7oQal4hF/94hA0FIkhtrswWNBNF5xQffKR2/mfqHVcECvW6w5UFz39LBFLvtOI3 lA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2nr7cs093s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:19 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wADNeJTN015871 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Nov 2018 23:40:19 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wADNeJNh024890; Tue, 13 Nov 2018 23:40:19 GMT Received: from localhost (/10.159.147.150) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 13 Nov 2018 15:40:19 -0800 Subject: [PATCH 6/6] common/dump: disable copyrange From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Tue, 13 Nov 2018 15:40:17 -0800 Message-ID: <154215241795.21151.2305359735347835725.stgit@magnolia> In-Reply-To: <154215237717.21151.11976488103599724788.stgit@magnolia> References: <154215237717.21151.11976488103599724788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9076 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=484 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1811130211 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong In _create_dumpdir_stress_num, disable copyrange in fsstress so that we dump exactly the same set of files and directories no matter how the xfs is configured. Signed-off-by: Darrick J. Wong --- common/dump | 3 +++ 1 file changed, 3 insertions(+) diff --git a/common/dump b/common/dump index c45b8a5a..4d1a1607 100644 --- a/common/dump +++ b/common/dump @@ -307,6 +307,9 @@ _create_dumpdir_stress_num() if $FSSTRESS_PROG | grep -q deduperange; then FSSTRESS_AVOID="-f deduperange=0 $FSSTRESS_AVOID" fi + if $FSSTRESS_PROG | grep -q copyrange; then + FSSTRESS_AVOID="-f copyrange=0 $FSSTRESS_AVOID" + fi echo "" echo "-----------------------------------------------"