Message ID | 20170927191003.10099-1-rgoldwyn@suse.de (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues <rgoldwyn@suse.com> > > Tests the RWF_NOWAIT flag so the I/O returns immediately on > a new file, without any block allocations. > > A new program which includes the pwritev2() call is used. This allows > passing flags for the I/O to be performed. Rather than write a one-off test program for this that effectively replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT flag support to xfs_io. Cheers, Dave.
On 09/27/2017 04:34 PM, Dave Chinner wrote: > On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: >> From: Goldwyn Rodrigues <rgoldwyn@suse.com> >> >> Tests the RWF_NOWAIT flag so the I/O returns immediately on >> a new file, without any block allocations. >> >> A new program which includes the pwritev2() call is used. This allows >> passing flags for the I/O to be performed. > > Rather than write a one-off test program for this that effectively > replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT > flag support to xfs_io. > This one off program is required because xfs_io does not support partial writes. It tries to do that within the loop and does not return the number of bytes written. This is required for test generic/462.
On Wed, Sep 27, 2017 at 04:39:20PM -0500, Goldwyn Rodrigues wrote: > > > On 09/27/2017 04:34 PM, Dave Chinner wrote: > > On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: > >> From: Goldwyn Rodrigues <rgoldwyn@suse.com> > >> > >> Tests the RWF_NOWAIT flag so the I/O returns immediately on > >> a new file, without any block allocations. > >> > >> A new program which includes the pwritev2() call is used. This allows > >> passing flags for the I/O to be performed. > > > > Rather than write a one-off test program for this that effectively > > replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT > > flag support to xfs_io. > > > > This one off program is required because xfs_io does not support partial > writes. It tries to do that within the loop and does not return the > number of bytes written. This is required for test generic/462. Then please also extend xfs_io to support partial reads and writes in the manner you need. Cheers, Dave.
On 09/27/2017 04:51 PM, Dave Chinner wrote: > On Wed, Sep 27, 2017 at 04:39:20PM -0500, Goldwyn Rodrigues wrote: >> >> >> On 09/27/2017 04:34 PM, Dave Chinner wrote: >>> On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: >>>> From: Goldwyn Rodrigues <rgoldwyn@suse.com> >>>> >>>> Tests the RWF_NOWAIT flag so the I/O returns immediately on >>>> a new file, without any block allocations. >>>> >>>> A new program which includes the pwritev2() call is used. This allows >>>> passing flags for the I/O to be performed. >>> >>> Rather than write a one-off test program for this that effectively >>> replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT >>> flag support to xfs_io. >>> >> >> This one off program is required because xfs_io does not support partial >> writes. It tries to do that within the loop and does not return the >> number of bytes written. This is required for test generic/462. > > Then please also extend xfs_io to support partial reads and writes > in the manner you need. > That will break existing tests which rely on nothing but the error returned in case of partial writes. I don't mind fixing xfs_io. It could use some love especially in the documentation.
On Wed, Sep 27, 2017 at 05:24:49PM -0500, Goldwyn Rodrigues wrote: > > > On 09/27/2017 04:51 PM, Dave Chinner wrote: > > On Wed, Sep 27, 2017 at 04:39:20PM -0500, Goldwyn Rodrigues wrote: > >> > >> > >> On 09/27/2017 04:34 PM, Dave Chinner wrote: > >>> On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: > >>>> From: Goldwyn Rodrigues <rgoldwyn@suse.com> > >>>> > >>>> Tests the RWF_NOWAIT flag so the I/O returns immediately on > >>>> a new file, without any block allocations. > >>>> > >>>> A new program which includes the pwritev2() call is used. This allows > >>>> passing flags for the I/O to be performed. > >>> > >>> Rather than write a one-off test program for this that effectively > >>> replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT > >>> flag support to xfs_io. > >>> > >> > >> This one off program is required because xfs_io does not support partial > >> writes. It tries to do that within the loop and does not return the > >> number of bytes written. This is required for test generic/462. > > > > Then please also extend xfs_io to support partial reads and writes > > in the manner you need. > > > > That will break existing tests which rely on nothing but the error > returned in case of partial writes. So trigger necessary partial write behaviour only when the CLI option to use RWF_NOWAIT is present.... Cheers, Dave.
On 09/27/2017 08:51 PM, Dave Chinner wrote: > On Wed, Sep 27, 2017 at 05:24:49PM -0500, Goldwyn Rodrigues wrote: >> >> >> On 09/27/2017 04:51 PM, Dave Chinner wrote: >>> On Wed, Sep 27, 2017 at 04:39:20PM -0500, Goldwyn Rodrigues wrote: >>>> >>>> >>>> On 09/27/2017 04:34 PM, Dave Chinner wrote: >>>>> On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: >>>>>> From: Goldwyn Rodrigues <rgoldwyn@suse.com> >>>>>> >>>>>> Tests the RWF_NOWAIT flag so the I/O returns immediately on >>>>>> a new file, without any block allocations. >>>>>> >>>>>> A new program which includes the pwritev2() call is used. This allows >>>>>> passing flags for the I/O to be performed. >>>>> >>>>> Rather than write a one-off test program for this that effectively >>>>> replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT >>>>> flag support to xfs_io. >>>>> >>>> >>>> This one off program is required because xfs_io does not support partial >>>> writes. It tries to do that within the loop and does not return the >>>> number of bytes written. This is required for test generic/462. >>> >>> Then please also extend xfs_io to support partial reads and writes >>> in the manner you need. >>> >> >> That will break existing tests which rely on nothing but the error >> returned in case of partial writes. > > So trigger necessary partial write behaviour only when the CLI > option to use RWF_NOWAIT is present.... > Partial write test case is not related to RWF_NOWAIT test case. These are two separate test cases. Anyways, I am working on implementing this. Would you prefer pwritev2 be a separate subcommand calling pwritev2() or should I transform pwritev() to pwritev2()? The system call is relatively new and there are overlapping features such as RWF_DSYNC and RWF_SYNC. I am assuming the former.
On Wed, Sep 27, 2017 at 09:09:55PM -0500, Goldwyn Rodrigues wrote: > > > On 09/27/2017 08:51 PM, Dave Chinner wrote: > > On Wed, Sep 27, 2017 at 05:24:49PM -0500, Goldwyn Rodrigues wrote: > >> > >> > >> On 09/27/2017 04:51 PM, Dave Chinner wrote: > >>> On Wed, Sep 27, 2017 at 04:39:20PM -0500, Goldwyn Rodrigues wrote: > >>>> > >>>> > >>>> On 09/27/2017 04:34 PM, Dave Chinner wrote: > >>>>> On Wed, Sep 27, 2017 at 02:10:02PM -0500, Goldwyn Rodrigues wrote: > >>>>>> From: Goldwyn Rodrigues <rgoldwyn@suse.com> > >>>>>> > >>>>>> Tests the RWF_NOWAIT flag so the I/O returns immediately on > >>>>>> a new file, without any block allocations. > >>>>>> > >>>>>> A new program which includes the pwritev2() call is used. This allows > >>>>>> passing flags for the I/O to be performed. > >>>>> > >>>>> Rather than write a one-off test program for this that effectively > >>>>> replicates xfs_io pread/pwrite functionality, please add RWF_NOWAIT > >>>>> flag support to xfs_io. > >>>>> > >>>> > >>>> This one off program is required because xfs_io does not support partial > >>>> writes. It tries to do that within the loop and does not return the > >>>> number of bytes written. This is required for test generic/462. > >>> > >>> Then please also extend xfs_io to support partial reads and writes > >>> in the manner you need. > >>> > >> > >> That will break existing tests which rely on nothing but the error > >> returned in case of partial writes. > > > > So trigger necessary partial write behaviour only when the CLI > > option to use RWF_NOWAIT is present.... > > > > Partial write test case is not related to RWF_NOWAIT test case. These > are two separate test cases. > > Anyways, I am working on implementing this. Would you prefer pwritev2 be > a separate subcommand calling pwritev2() or should I transform pwritev() > to pwritev2()? The system call is relatively new and there are > overlapping features such as RWF_DSYNC and RWF_SYNC. I am assuming the > former. If pwritev2 exists at build time, build in support for it. If it returns ENOSYS or it is not present at build time, fall back to pwritev()... Cheers, Dave.
diff --git a/src/Makefile b/src/Makefile index 7d1306bc..427c561e 100644 --- a/src/Makefile +++ b/src/Makefile @@ -23,7 +23,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \ renameat2 t_getcwd e4compact test-nextquota punch-alternating \ attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \ - dio-invalidate-cache stat_test t_encrypted_d_revalidate + dio-invalidate-cache stat_test t_encrypted_d_revalidate pwritev2 SUBDIRS = log-writes diff --git a/src/pwritev2.c b/src/pwritev2.c new file mode 100644 index 00000000..28413910 --- /dev/null +++ b/src/pwritev2.c @@ -0,0 +1,161 @@ +#define _GNU_SOURCE + +#include <stdio.h> +#include <stdlib.h> +#include <sys/uio.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <linux/fs.h> +#include <fcntl.h> +#include <unistd.h> +#include <errno.h> +#include <string.h> +#include <sys/time.h> + +int openflag = O_RDWR | O_LARGEFILE; +long datasz = 4096; +long iovcnt = 1; +long offset = 0; +int readfile = 0; +int measure_time = 0; +int flags = 0; +char *filename = "pwrite-file"; +unsigned int seed = 0xcdcdcdcd; + +int usage() +{ + fprintf(stderr, "Usage: pwritev2 \n"); + fprintf(stderr, "\t-b [buffer size]: Size of each buffer\n"); + fprintf(stderr, "\t-d: Direct write\n"); + fprintf(stderr, "\t-f: create file if not exists\n"); + fprintf(stderr, "\t-s [data size]: Size of each iovec data buffer\n"); + fprintf(stderr, "\t-S [seed]: seed for the data buffer\n"); + fprintf(stderr, "\t-t: measure time\n"); + fprintf(stderr, "\t-o [offset]: Offset where to perform the pwritev2\n"); + fprintf(stderr, "\t-V [num of iovecs]: Number of vectors\n"); + fprintf(stderr, "\t-W: perform I/O with RWF_NOWAIT\n"); + fprintf(stderr, "\t filename: The filename to perform pwritev2\n"); + exit(1); +} + +/* + * Scale value by kilo, mega, or giga. + */ +long long scale_by_kmg(long long value, char scale) +{ + switch (scale) { + case 'g': + case 'G': + value *= 1024; + case 'm': + case 'M': + value *= 1024; + case 'k': + case 'K': + value *= 1024; + break; + case '\0': + break; + default: + usage(); + break; + } + return value; +} + + +int parse_args(int argc, char *argv[]) +{ + int c; + extern int optind; + while ((c = getopt(argc, argv, "b:dfo:V:S:tW")) != -1) { + char *endp; + switch (c) { + case 'b': + datasz = strtol(optarg, &endp, 0); + datasz = (long)scale_by_kmg((long long)datasz, + *endp); + break; + case 'd': + openflag |= O_DIRECT; + break; + case 'f': + openflag |= O_CREAT; + break; + case 'o': + offset = strtol(optarg, &endp, 0); + break; + case 't': + measure_time = 1; + break; + case 'S': + seed = strtoul(optarg, &endp, 0); + break; + case 'V': + iovcnt = strtol(optarg, &endp, 0); + break; + case 'W': + flags |= RWF_NOWAIT; + break; + default: + usage(); + } + } + filename = argv[optind]; +} + + + +int main(int argc, char *argv[]) +{ + int i, fd; + size_t ret = 0; + struct iovec *iovec; + char *data; + struct timeval before, after; + struct timezone tz; + + parse_args(argc, argv); + + iovec = (struct iovec *)malloc(sizeof(struct iovec) * iovcnt); + if (!iovec) { + fprintf(stderr, "Could not allocate iovec\n"); + exit(1); + } + data = aligned_alloc(datasz, datasz); + if (!data) { + fprintf(stderr, "Could not allocate data\n"); + exit(1); + } + + memset(data, seed, datasz); + + for (i=0; i<iovcnt; i++) { + iovec[i].iov_base = data; + iovec[i].iov_len = datasz; + } + + fd = open(filename, openflag, S_IRUSR | S_IWUSR); + + gettimeofday(&before, &tz); + ret = pwritev2(fd, iovec, iovcnt, offset, flags); + gettimeofday(&after, &tz); + printf("pwritev2() ret = %ld %s(%d)\n", ret, strerror(errno), errno); + + if (measure_time) { + unsigned long seconds, ms; + seconds = after.tv_sec - before.tv_sec; + if (after.tv_usec < before.tv_usec) { + seconds -= 1; + ms = 1000000 + after.tv_usec - before.tv_usec; + } else + ms = after.tv_usec - before.tv_usec; + printf("Time: %lu seconds %lu microseconds\n", seconds, ms); + } + + close(fd); + return 0; + +} + + diff --git a/tests/generic/461 b/tests/generic/461 new file mode 100755 index 00000000..7bbedd5d --- /dev/null +++ b/tests/generic/461 @@ -0,0 +1,80 @@ +#! /bin/bash +# FS QA Test No. 461 +# +# write() to a file opened with O_DIRECT with count > remaining +# bytes. Result should be remaining bytes (to aligned bytes) instead +# of ENOSPC error +# +#----------------------------------------------------------------------- +# Copyright (c) 2015, Oracle and/or its affiliates. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $tmp.* $testdir1 +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/populate +. ./common/filter +. ./common/attr + +# real QA test starts here +_supported_os Linux +_require_odirect + +rm -f $seqres.full + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testdir=$SCRATCH_MNT/test-$seq +mkdir $testdir + +# Create a file with pwrite nowait (will fail with EAGAIN) +out=`src/pwritev2 -f -W -d -V 1 -b 128M -t $testdir/f1 | grep Time` +microsecs=`echo $out | cut -f 4 -d " "` +secs=`echo $out | cut -f 2 -d " "` +if [ $microsecs -lt 50 -a $secs -eq 0 ]; then + echo "RWF_NOWAIT time is within limits." +else + echo "RWF_NOWAIT took $secs seconds and $microsecs microsecs" +fi + +# Write the file without nowait +src/pwritev2 -S 0xaa -d -V 1 -b 128M $testdir/f1 + +# Write the file with nowait +src/pwritev2 -W -d -V 1 -b 128M $testdir/f1 + +# Read the file to check if data is correct +$XFS_IO_PROG -d -c "pread 0 4k -v" $testdir/f1 | _filter_xfs_io_unique + +# success, all done +status=0 +exit diff --git a/tests/generic/461.out b/tests/generic/461.out new file mode 100644 index 00000000..1c331d5b --- /dev/null +++ b/tests/generic/461.out @@ -0,0 +1,9 @@ +QA output created by 461 +Format and mount +RWF_NOWAIT time is within limits. +pwritev2() ret = 134217728 Success(0) +pwritev2() ret = 134217728 Success(0) +00000000: cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd ................ +* +read 4096/4096 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/generic/group b/tests/generic/group index f922b496..6352064a 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -463,3 +463,4 @@ 458 auto quick clone 459 auto dangerous 460 auto quick rw +461 auto quick rw