From patchwork Fri Dec 7 06:24:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10717535 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A45D517D5 for ; Fri, 7 Dec 2018 06:24:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90B3A2EB81 for ; Fri, 7 Dec 2018 06:24:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 84A952EB78; Fri, 7 Dec 2018 06:24:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76E702EB67 for ; Fri, 7 Dec 2018 06:24:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725948AbeLGGYg (ORCPT ); Fri, 7 Dec 2018 01:24:36 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:57342 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725952AbeLGGYg (ORCPT ); Fri, 7 Dec 2018 01:24:36 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB76OIHE115236; Fri, 7 Dec 2018 06:24:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=ggPi9A0WXasOTY9KEj9nYl05Gf8Qr/iSuy8zSIRsTts=; b=jR4EDqSMUwwraw0p772bl2HXAGHGqYeJsuDr/ajCdESxSaXEY2BtLh0zLWXHx7AEXy2h v4nrr1zTg9BOBupqIWb0K2UMAFsxD5hYkVWU2M9JrFULpwXFmx6mCYVMwHrISPeqm0eR Xd2IGph+PQ2s/MNz0b2jCfiB1KD4jtQ2RPLslTFs0v4isZFf7cO5/I/ZeodJuKcOQH71 qXlKao8EhrRvAvgI5Kzvp0rc2cDv8w60GPODfraeFDd/jNrq/gaetmSPz4NHopdOLzOs NZKSnGKAfsOhAeagAuX8uJw5DCDL8U6A5ABwi9USl/+YywADN3P2Owk1Ej1vtxElcMsf Yg== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2p3jxrv0vn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 06:24:33 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OS61009367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 06:24:28 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB76OSDW016474; Fri, 7 Dec 2018 06:24:28 GMT Received: from localhost (/10.159.153.113) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 22:24:27 -0800 Subject: [PATCH 09/12] fsstress: add copy_file_range support From: "Darrick J. Wong" To: guaneryu@gmail.com, darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org Date: Thu, 06 Dec 2018 22:24:26 -0800 Message-ID: <154416386688.15344.13463990940166415616.stgit@magnolia> In-Reply-To: <154416379233.15344.4377231692638417530.stgit@magnolia> References: <154416379233.15344.4377231692638417530.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070055 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong Support the copy_file_range syscall in fsstress. Signed-off-by: Darrick J. Wong --- configure.ac | 2 + include/builddefs.in | 1 ltp/Makefile | 4 + ltp/fsstress.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++++ m4/package_libcdev.m4 | 19 ++++++ 5 files changed, 173 insertions(+) diff --git a/configure.ac b/configure.ac index aede4f59..19798824 100644 --- a/configure.ac +++ b/configure.ac @@ -68,6 +68,8 @@ AC_PACKAGE_WANT_OPEN_BY_HANDLE_AT AC_PACKAGE_WANT_LINUX_PRCTL_H AC_PACKAGE_WANT_LINUX_FS_H +AC_HAVE_COPY_FILE_RANGE + AC_CHECK_FUNCS([renameat2]) AC_CONFIG_HEADER(include/config.h) diff --git a/include/builddefs.in b/include/builddefs.in index fb8e912b..2605e42d 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -67,6 +67,7 @@ HAVE_DMAPI = @have_dmapi@ HAVE_ATTR_LIST = @have_attr_list@ HAVE_FIEMAP = @have_fiemap@ HAVE_FALLOCATE = @have_fallocate@ +HAVE_COPY_FILE_RANGE = @have_copy_file_range@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall diff --git a/ltp/Makefile b/ltp/Makefile index 5f784ecf..e4ca45f4 100644 --- a/ltp/Makefile +++ b/ltp/Makefile @@ -28,6 +28,10 @@ ifeq ($(HAVE_FALLOCATE), true) LCFLAGS += -DFALLOCATE endif +ifeq ($(HAVE_COPY_FILE_RANGE),yes) +LCFLAGS += -DHAVE_COPY_FILE_RANGE +endif + default: depend $(TARGETS) depend: .dep diff --git a/ltp/fsstress.c b/ltp/fsstress.c index af5d125f..99a1d733 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -26,6 +26,7 @@ #include io_context_t io_ctx; #endif +#include #ifndef FS_IOC_GETFLAGS #define FS_IOC_GETFLAGS _IOR('f', 1, long) @@ -55,6 +56,7 @@ typedef enum { OP_BULKSTAT1, OP_CHOWN, OP_CLONERANGE, + OP_COPYRANGE, OP_CREAT, OP_DEDUPERANGE, OP_DREAD, @@ -163,6 +165,7 @@ void bulkstat_f(int, long); void bulkstat1_f(int, long); void chown_f(int, long); void clonerange_f(int, long); +void copyrange_f(int, long); void creat_f(int, long); void deduperange_f(int, long); void dread_f(int, long); @@ -212,6 +215,7 @@ opdesc_t ops[] = { { OP_BULKSTAT1, "bulkstat1", bulkstat1_f, 1, 0 }, { OP_CHOWN, "chown", chown_f, 3, 1 }, { OP_CLONERANGE, "clonerange", clonerange_f, 4, 1 }, + { OP_COPYRANGE, "copyrange", copyrange_f, 4, 1 }, { OP_CREAT, "creat", creat_f, 4, 1 }, { OP_DEDUPERANGE, "deduperange", deduperange_f, 4, 1}, { OP_DREAD, "dread", dread_f, 4, 0 }, @@ -2333,6 +2337,149 @@ clonerange_f( #endif } +/* copy some arbitrary range of f1 to f2. */ +void +copyrange_f( + int opno, + long r) +{ +#ifdef HAVE_COPY_FILE_RANGE + struct pathname fpath1; + struct pathname fpath2; + struct stat64 stat1; + struct stat64 stat2; + char inoinfo1[1024]; + char inoinfo2[1024]; + loff_t lr; + loff_t off1; + loff_t off2; + loff_t max_off2; + size_t len; + int tries = 0; + int v1; + int v2; + int fd1; + int fd2; + size_t ret; + int e; + + /* Load paths */ + init_pathname(&fpath1); + if (!get_fname(FT_REGm, r, &fpath1, NULL, NULL, &v1)) { + if (v1) + printf("%d/%d: copyrange read - no filename\n", + procid, opno); + goto out_fpath1; + } + + init_pathname(&fpath2); + if (!get_fname(FT_REGm, random(), &fpath2, NULL, NULL, &v2)) { + if (v2) + printf("%d/%d: copyrange write - no filename\n", + procid, opno); + goto out_fpath2; + } + + /* Open files */ + fd1 = open_path(&fpath1, O_RDONLY); + e = fd1 < 0 ? errno : 0; + check_cwd(); + if (fd1 < 0) { + if (v1) + printf("%d/%d: copyrange read - open %s failed %d\n", + procid, opno, fpath1.path, e); + goto out_fpath2; + } + + fd2 = open_path(&fpath2, O_WRONLY); + e = fd2 < 0 ? errno : 0; + check_cwd(); + if (fd2 < 0) { + if (v2) + printf("%d/%d: copyrange write - open %s failed %d\n", + procid, opno, fpath2.path, e); + goto out_fd1; + } + + /* Get file stats */ + if (fstat64(fd1, &stat1) < 0) { + if (v1) + printf("%d/%d: copyrange read - fstat64 %s failed %d\n", + procid, opno, fpath1.path, errno); + goto out_fd2; + } + inode_info(inoinfo1, sizeof(inoinfo1), &stat1, v1); + + if (fstat64(fd2, &stat2) < 0) { + if (v2) + printf("%d/%d: copyrange write - fstat64 %s failed %d\n", + procid, opno, fpath2.path, errno); + goto out_fd2; + } + inode_info(inoinfo2, sizeof(inoinfo2), &stat2, v2); + + /* Calculate offsets */ + len = (random() % FILELEN_MAX) + 1; + if (len == 0) + len = stat1.st_blksize; + if (len > stat1.st_size) + len = stat1.st_size; + + lr = ((int64_t)random() << 32) + random(); + if (stat1.st_size == len) + off1 = 0; + else + off1 = (off64_t)(lr % MIN(stat1.st_size - len, MAXFSIZE)); + off1 %= maxfsize; + + /* + * If srcfile == destfile, randomly generate destination ranges + * until we find one that doesn't overlap the source range. + */ + max_off2 = MIN(stat2.st_size + (1024ULL * stat2.st_blksize), MAXFSIZE); + do { + lr = ((int64_t)random() << 32) + random(); + off2 = (off64_t)(lr % max_off2); + off2 %= maxfsize; + } while (stat1.st_ino == stat2.st_ino && llabs(off2 - off1) < len); + + while (len > 0) { + ret = syscall(__NR_copy_file_range, fd1, &off1, fd2, &off2, + len, 0); + if (ret < 0) { + if (errno != EAGAIN || tries++ >= 300) + break; + } else if (ret > len) + break; + else if (ret > 0) + len -= ret; + } + e = ret < 0 ? errno : 0; + if (v1 || v2) { + printf("%d/%d: copyrange %s%s [%lld,%lld] -> %s%s [%lld,%lld]", + procid, opno, + fpath1.path, inoinfo1, (long long)off1, (long long)len, + fpath2.path, inoinfo2, (long long)off2, (long long)len); + + if (ret < 0) + printf(" error %d", e); + else if (len && ret > len) + printf(" asked for %lld, copied %lld??\n", + (long long)len, (long long)ret); + printf("\n"); + } + +out_fd2: + close(fd2); +out_fd1: + close(fd1); +out_fpath2: + free_pathname(&fpath2); +out_fpath1: + free_pathname(&fpath1); +#endif +} + /* dedupe some arbitrary range of f1 to f2...fn. */ void deduperange_f( diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index fb123f33..14e67e18 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -98,3 +98,22 @@ AC_DEFUN([AC_HAVE_GETMNTINFO], AC_MSG_RESULT(no)) AC_SUBST(have_getmntinfo) ]) + +# +# +# Check if we have a copy_file_range system call (Linux) +# +AC_DEFUN([AC_HAVE_COPY_FILE_RANGE], + [ AC_MSG_CHECKING([for copy_file_range]) + AC_TRY_LINK([ +#define _GNU_SOURCE +#include +#include + ], [ + syscall(__NR_copy_file_range, 0, 0, 0, 0, 0, 0); + ], have_copy_file_range=yes + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no)) + AC_SUBST(have_copy_file_range) + ]) +