| Message ID | 1461878218-3844-6-git-send-email-vishal.l.verma@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
> index 79defba..97a1f5f 100644 > --- a/fs/block_dev.c > +++ b/fs/block_dev.c > @@ -167,12 +167,21 @@ blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) > struct file *file = iocb->ki_filp; > struct inode *inode = bdev_file_inode(file); > > - if (IS_DAX(inode)) > + if (iocb_is_direct(iocb)) > + return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, > + offset, blkdev_get_block, NULL, > + NULL, DIO_SKIP_DIO_COUNT); > + else if (iocb_is_dax(iocb)) > return dax_do_io(iocb, inode, iter, offset, blkdev_get_block, > NULL, DIO_SKIP_DIO_COUNT); > - return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, offset, > - blkdev_get_block, NULL, NULL, > - DIO_SKIP_DIO_COUNT); > + else { > + /* > + * If we're in the direct_IO path, either the IOCB_DIRECT or > + * IOCB_DAX flags must be set. > + */ > + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); > + return -ENXIO; > + } DAX should not even end up in ->direct_IO. > --- a/fs/xfs/xfs_file.c > +++ b/fs/xfs/xfs_file.c > @@ -300,7 +300,7 @@ xfs_file_read_iter( > > XFS_STATS_INC(mp, xs_read_calls); > > - if (unlikely(iocb->ki_flags & IOCB_DIRECT)) > + if (unlikely(iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX))) > ioflags |= XFS_IO_ISDIRECT; please also add a XFS_IO_ISDAX flag to propagate the information properly and allow tracing to display the actual I/O type. > +static inline bool iocb_is_dax(struct kiocb *iocb) > { > + return IS_DAX(file_inode(iocb->ki_filp)) && > + (iocb->ki_flags & IOCB_DAX); > +} > + > +static inline bool iocb_is_direct(struct kiocb *iocb) > +{ > + return iocb->ki_flags & IOCB_DIRECT; > } No need for these helpers - especially as IOCB_DAX should never be set if IS_DAX is false.
On 04/29/2016 12:16 AM, Vishal Verma wrote: > All IO in a dax filesystem used to go through dax_do_io, which cannot > handle media errors, and thus cannot provide a recovery path that can > send a write through the driver to clear errors. > > Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO > path for DAX filesystems, use the same direct_IO path for both DAX and > direct_io iocbs, but use the flags to identify when we are in O_DIRECT > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional > direct_IO path instead of DAX. > Really? What are your thinking here? What about all the current users of O_DIRECT, you have just made them 4 times slower and "less concurrent*" then "buffred io" users. Since direct_IO path will queue an IO request and all. (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) I hate it that you overload the semantics of a known and expected O_DIRECT flag, for special pmem quirks. This is an incompatible and unrelated overload of the semantics of O_DIRECT. > This allows us a recovery path in the form of opening the file with > O_DIRECT and writing to it with the usual O_DIRECT semantics (sector > alignment restrictions). > I understand that you want a sector aligned IO, right? for the clear of errors. But I hate it that you forced all O_DIRECT IO to be slow for this. Can you not make dax_do_io handle media errors? At least for the parts of the IO that are aligned. (And your recovery path application above can use only aligned IO to make sure) Please look for another solution. Even a special IOCTL_DAX_CLEAR_ERROR [*"less concurrent" because of the queuing done in bdev. Note how pmem is not even multi-queue, and even if it was it will be much slower then DAX because of the code depth and all the locks and task switches done in the block layer. In DAX the final memcpy is done directly on the user-mode thread] Thanks Boaz > Cc: Matthew Wilcox <matthew@wil.cx> > Cc: Dan Williams <dan.j.williams@intel.com> > Cc: Ross Zwisler <ross.zwisler@linux.intel.com> > Cc: Jeff Moyer <jmoyer@redhat.com> > Cc: Christoph Hellwig <hch@infradead.org> > Cc: Dave Chinner <david@fromorbit.com> > Cc: Jan Kara <jack@suse.cz> > Cc: Jens Axboe <axboe@fb.com> > Cc: Al Viro <viro@zeniv.linux.org.uk> > Signed-off-by: Vishal Verma <vishal.l.verma@intel.com> > --- > drivers/block/loop.c | 2 +- > fs/block_dev.c | 17 +++++++++++++---- > fs/ext2/inode.c | 16 ++++++++++++---- > fs/ext4/file.c | 2 +- > fs/ext4/inode.c | 19 +++++++++++++------ > fs/xfs/xfs_aops.c | 20 +++++++++++++------- > fs/xfs/xfs_file.c | 4 ++-- > include/linux/fs.h | 15 ++++++++++++--- > mm/filemap.c | 4 ++-- > 9 files changed, 69 insertions(+), 30 deletions(-) > > diff --git a/drivers/block/loop.c b/drivers/block/loop.c > index 80cf8ad..c0a24c3 100644 > --- a/drivers/block/loop.c > +++ b/drivers/block/loop.c > @@ -568,7 +568,7 @@ struct switch_request { > > static inline void loop_update_dio(struct loop_device *lo) > { > - __loop_update_dio(lo, io_is_direct(lo->lo_backing_file) | > + __loop_update_dio(lo, (lo->lo_backing_file->f_flags & O_DIRECT) | > lo->use_dio); > } > > diff --git a/fs/block_dev.c b/fs/block_dev.c > index 79defba..97a1f5f 100644 > --- a/fs/block_dev.c > +++ b/fs/block_dev.c > @@ -167,12 +167,21 @@ blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) > struct file *file = iocb->ki_filp; > struct inode *inode = bdev_file_inode(file); > > - if (IS_DAX(inode)) > + if (iocb_is_direct(iocb)) > + return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, > + offset, blkdev_get_block, NULL, > + NULL, DIO_SKIP_DIO_COUNT); > + else if (iocb_is_dax(iocb)) > return dax_do_io(iocb, inode, iter, offset, blkdev_get_block, > NULL, DIO_SKIP_DIO_COUNT); > - return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, offset, > - blkdev_get_block, NULL, NULL, > - DIO_SKIP_DIO_COUNT); > + else { > + /* > + * If we're in the direct_IO path, either the IOCB_DIRECT or > + * IOCB_DAX flags must be set. > + */ > + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); > + return -ENXIO; > + } > } > > int __sync_blockdev(struct block_device *bdev, int wait) > diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c > index 35f2b0bf..45f2b51 100644 > --- a/fs/ext2/inode.c > +++ b/fs/ext2/inode.c > @@ -861,12 +861,20 @@ ext2_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) > size_t count = iov_iter_count(iter); > ssize_t ret; > > - if (IS_DAX(inode)) > - ret = dax_do_io(iocb, inode, iter, offset, ext2_get_block, NULL, > - DIO_LOCKING); > - else > + if (iocb_is_direct(iocb)) > ret = blockdev_direct_IO(iocb, inode, iter, offset, > ext2_get_block); > + else if (iocb_is_dax(iocb)) > + ret = dax_do_io(iocb, inode, iter, offset, ext2_get_block, NULL, > + DIO_LOCKING); > + else { > + /* > + * If we're in the direct_IO path, either the IOCB_DIRECT or > + * IOCB_DAX flags must be set. > + */ > + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); > + return -ENXIO; > + } > if (ret < 0 && iov_iter_rw(iter) == WRITE) > ext2_write_failed(mapping, offset + count); > return ret; > diff --git a/fs/ext4/file.c b/fs/ext4/file.c > index 2e9aa49..165a0b8 100644 > --- a/fs/ext4/file.c > +++ b/fs/ext4/file.c > @@ -94,7 +94,7 @@ ext4_file_write_iter(struct kiocb *iocb, struct iov_iter *from) > struct file *file = iocb->ki_filp; > struct inode *inode = file_inode(iocb->ki_filp); > struct blk_plug plug; > - int o_direct = iocb->ki_flags & IOCB_DIRECT; > + int o_direct = iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX); > int unaligned_aio = 0; > int overwrite = 0; > ssize_t ret; > diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c > index 6d5d5c1..0b6d77a 100644 > --- a/fs/ext4/inode.c > +++ b/fs/ext4/inode.c > @@ -3410,15 +3410,22 @@ static ssize_t ext4_direct_IO_write(struct kiocb *iocb, struct iov_iter *iter, > #ifdef CONFIG_EXT4_FS_ENCRYPTION > BUG_ON(ext4_encrypted_inode(inode) && S_ISREG(inode->i_mode)); > #endif > - if (IS_DAX(inode)) { > - ret = dax_do_io(iocb, inode, iter, offset, get_block_func, > - ext4_end_io_dio, dio_flags); > - } else > + if (iocb_is_direct(iocb)) > ret = __blockdev_direct_IO(iocb, inode, > inode->i_sb->s_bdev, iter, offset, > get_block_func, > ext4_end_io_dio, NULL, dio_flags); > - > + else if (iocb_is_dax(iocb)) > + ret = dax_do_io(iocb, inode, iter, offset, get_block_func, > + ext4_end_io_dio, dio_flags); > + else { > + /* > + * If we're in the direct_IO path, either the IOCB_DIRECT or > + * IOCB_DAX flags must be set. > + */ > + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); > + return -ENXIO; > + } > if (ret > 0 && !overwrite && ext4_test_inode_state(inode, > EXT4_STATE_DIO_UNWRITTEN)) { > int err; > @@ -3503,7 +3510,7 @@ static ssize_t ext4_direct_IO_read(struct kiocb *iocb, struct iov_iter *iter, > else > unlocked = 1; > } > - if (IS_DAX(inode)) { > + if (iocb_is_dax(iocb)) { > ret = dax_do_io(iocb, inode, iter, offset, ext4_dio_get_block, > NULL, unlocked ? 0 : DIO_LOCKING); > } else { > diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c > index e49b240..8134e99 100644 > --- a/fs/xfs/xfs_aops.c > +++ b/fs/xfs/xfs_aops.c > @@ -1412,21 +1412,27 @@ xfs_vm_direct_IO( > struct inode *inode = iocb->ki_filp->f_mapping->host; > dio_iodone_t *endio = NULL; > int flags = 0; > - struct block_device *bdev; > + struct block_device *bdev = xfs_find_bdev_for_inode(inode); > > if (iov_iter_rw(iter) == WRITE) { > endio = xfs_end_io_direct_write; > flags = DIO_ASYNC_EXTEND; > } > > - if (IS_DAX(inode)) { > + if (iocb_is_direct(iocb)) > + return __blockdev_direct_IO(iocb, inode, bdev, iter, offset, > + xfs_get_blocks_direct, endio, NULL, flags); > + else if (iocb_is_dax(iocb)) > return dax_do_io(iocb, inode, iter, offset, > - xfs_get_blocks_direct, endio, 0); > + xfs_get_blocks_direct, endio, 0); > + else { > + /* > + * If we're in the direct_IO path, either the IOCB_DIRECT or > + * IOCB_DAX flags must be set. > + */ > + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); > + return -ENXIO; > } > - > - bdev = xfs_find_bdev_for_inode(inode); > - return __blockdev_direct_IO(iocb, inode, bdev, iter, offset, > - xfs_get_blocks_direct, endio, NULL, flags); > } > > /* > diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c > index c2946f4..3d5d3c2 100644 > --- a/fs/xfs/xfs_file.c > +++ b/fs/xfs/xfs_file.c > @@ -300,7 +300,7 @@ xfs_file_read_iter( > > XFS_STATS_INC(mp, xs_read_calls); > > - if (unlikely(iocb->ki_flags & IOCB_DIRECT)) > + if (unlikely(iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX))) > ioflags |= XFS_IO_ISDIRECT; > if (file->f_mode & FMODE_NOCMTIME) > ioflags |= XFS_IO_INVIS; > @@ -898,7 +898,7 @@ xfs_file_write_iter( > if (XFS_FORCED_SHUTDOWN(ip->i_mount)) > return -EIO; > > - if ((iocb->ki_flags & IOCB_DIRECT) || IS_DAX(inode)) > + if ((iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX))) > ret = xfs_file_dio_aio_write(iocb, from); > else > ret = xfs_file_buffered_aio_write(iocb, from); > diff --git a/include/linux/fs.h b/include/linux/fs.h > index 9f28130..adca1d8 100644 > --- a/include/linux/fs.h > +++ b/include/linux/fs.h > @@ -322,6 +322,7 @@ struct writeback_control; > #define IOCB_APPEND (1 << 1) > #define IOCB_DIRECT (1 << 2) > #define IOCB_HIPRI (1 << 3) > +#define IOCB_DAX (1 << 4) > > struct kiocb { > struct file *ki_filp; > @@ -2930,9 +2931,15 @@ extern int generic_show_options(struct seq_file *m, struct dentry *root); > extern void save_mount_options(struct super_block *sb, char *options); > extern void replace_mount_options(struct super_block *sb, char *options); > > -static inline bool io_is_direct(struct file *filp) > +static inline bool iocb_is_dax(struct kiocb *iocb) > { > - return (filp->f_flags & O_DIRECT) || IS_DAX(filp->f_mapping->host); > + return IS_DAX(file_inode(iocb->ki_filp)) && > + (iocb->ki_flags & IOCB_DAX); > +} > + > +static inline bool iocb_is_direct(struct kiocb *iocb) > +{ > + return iocb->ki_flags & IOCB_DIRECT; > } > > static inline int iocb_flags(struct file *file) > @@ -2940,8 +2947,10 @@ static inline int iocb_flags(struct file *file) > int res = 0; > if (file->f_flags & O_APPEND) > res |= IOCB_APPEND; > - if (io_is_direct(file)) > + if (file->f_flags & O_DIRECT) > res |= IOCB_DIRECT; > + if (IS_DAX(file_inode(file))) > + res |= IOCB_DAX; > return res; > } > > diff --git a/mm/filemap.c b/mm/filemap.c > index 3effd5c..b959acf 100644 > --- a/mm/filemap.c > +++ b/mm/filemap.c > @@ -1849,7 +1849,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) > if (!count) > goto out; /* skip atime */ > > - if (iocb->ki_flags & IOCB_DIRECT) { > + if (iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX)) { > struct address_space *mapping = file->f_mapping; > struct inode *inode = mapping->host; > loff_t size; > @@ -2719,7 +2719,7 @@ ssize_t __generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) > if (err) > goto out; > > - if (iocb->ki_flags & IOCB_DIRECT) { > + if (iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX)) { > loff_t pos, endbyte; > > written = generic_file_direct_write(iocb, from, iocb->ki_pos); >
On Mon, 2016-05-02 at 07:56 -0700, Christoph Hellwig wrote: > > > > index 79defba..97a1f5f 100644 > > --- a/fs/block_dev.c > > +++ b/fs/block_dev.c > > @@ -167,12 +167,21 @@ blkdev_direct_IO(struct kiocb *iocb, struct > > iov_iter *iter, loff_t offset) > > struct file *file = iocb->ki_filp; > > struct inode *inode = bdev_file_inode(file); > > > > - if (IS_DAX(inode)) > > + if (iocb_is_direct(iocb)) > > + return __blockdev_direct_IO(iocb, inode, > > I_BDEV(inode), iter, > > + offset, > > blkdev_get_block, NULL, > > + NULL, > > DIO_SKIP_DIO_COUNT); > > + else if (iocb_is_dax(iocb)) > > return dax_do_io(iocb, inode, iter, offset, > > blkdev_get_block, > > NULL, DIO_SKIP_DIO_COUNT); > > - return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), > > iter, offset, > > - blkdev_get_block, NULL, NULL, > > - DIO_SKIP_DIO_COUNT); > > + else { > > + /* > > + * If we're in the direct_IO path, either the > > IOCB_DIRECT or > > + * IOCB_DAX flags must be set. > > + */ > > + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); > > + return -ENXIO; > > + } > DAX should not even end up in ->direct_IO. Do you mean to say remove the last 'else' clause entirely? I agree that it should never be hit, which is why it is a WARN.. But I'm happy to remove it. > > > > > --- a/fs/xfs/xfs_file.c > > +++ b/fs/xfs/xfs_file.c > > @@ -300,7 +300,7 @@ xfs_file_read_iter( > > > > XFS_STATS_INC(mp, xs_read_calls); > > > > - if (unlikely(iocb->ki_flags & IOCB_DIRECT)) > > + if (unlikely(iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX))) > > ioflags |= XFS_IO_ISDIRECT; > please also add a XFS_IO_ISDAX flag to propagate the information > properly and allow tracing to display the actual I/O type. Will do. > > > > > +static inline bool iocb_is_dax(struct kiocb *iocb) > > { > > + return IS_DAX(file_inode(iocb->ki_filp)) && > > + (iocb->ki_flags & IOCB_DAX); > > +} > > + > > +static inline bool iocb_is_direct(struct kiocb *iocb) > > +{ > > + return iocb->ki_flags & IOCB_DIRECT; > > } > No need for these helpers - especially as IOCB_DAX should never be > set > if IS_DAX is false. Ok. So check the flags directly where needed? > -- > To unsubscribe from this list: send the line "unsubscribe linux- > block" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 2016-05-02 at 18:41 +0300, Boaz Harrosh wrote: > On 04/29/2016 12:16 AM, Vishal Verma wrote: > > > > All IO in a dax filesystem used to go through dax_do_io, which > > cannot > > handle media errors, and thus cannot provide a recovery path that > > can > > send a write through the driver to clear errors. > > > > Add a new iocb flag for DAX, and set it only for DAX mounts. In the > > IO > > path for DAX filesystems, use the same direct_IO path for both DAX > > and > > direct_io iocbs, but use the flags to identify when we are in > > O_DIRECT > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the > > conventional > > direct_IO path instead of DAX. > > > Really? What are your thinking here? > > What about all the current users of O_DIRECT, you have just made them > 4 times slower and "less concurrent*" then "buffred io" users. Since > direct_IO path will queue an IO request and all. > (And if it is not so slow then why do we need dax_do_io at all? > [Rhetorical]) > > I hate it that you overload the semantics of a known and expected > O_DIRECT flag, for special pmem quirks. This is an incompatible > and unrelated overload of the semantics of O_DIRECT. We overloaded O_DIRECT a long time ago when we made DAX piggyback on the same path: static inline bool io_is_direct(struct file *filp) { return (filp->f_flags & O_DIRECT) || IS_DAX(filp->f_mapping->host); } Yes O_DIRECT on a DAX mounted file system will now be slower, but - > > > > > This allows us a recovery path in the form of opening the file with > > O_DIRECT and writing to it with the usual O_DIRECT semantics > > (sector > > alignment restrictions). > > > I understand that you want a sector aligned IO, right? for the > clear of errors. But I hate it that you forced all O_DIRECT IO > to be slow for this. > Can you not make dax_do_io handle media errors? At least for the > parts of the IO that are aligned. > (And your recovery path application above can use only aligned > IO to make sure) > > Please look for another solution. Even a special > IOCTL_DAX_CLEAR_ERROR - see all the versions of this series prior to this one, where we try to do a fallback... > > [*"less concurrent" because of the queuing done in bdev. Note how > pmem is not even multi-queue, and even if it was it will be much > slower then DAX because of the code depth and all the locks and > task > switches done in the block layer. In DAX the final memcpy is done > directly > on the user-mode thread] > > Thanks > Boaz >
On Mon, May 2, 2016 at 8:41 AM, Boaz Harrosh <boaz@plexistor.com> wrote: > On 04/29/2016 12:16 AM, Vishal Verma wrote: >> All IO in a dax filesystem used to go through dax_do_io, which cannot >> handle media errors, and thus cannot provide a recovery path that can >> send a write through the driver to clear errors. >> >> Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO >> path for DAX filesystems, use the same direct_IO path for both DAX and >> direct_io iocbs, but use the flags to identify when we are in O_DIRECT >> mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional >> direct_IO path instead of DAX. >> > > Really? What are your thinking here? > > What about all the current users of O_DIRECT, you have just made them > 4 times slower and "less concurrent*" then "buffred io" users. Since > direct_IO path will queue an IO request and all. > (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) > > I hate it that you overload the semantics of a known and expected > O_DIRECT flag, for special pmem quirks. This is an incompatible > and unrelated overload of the semantics of O_DIRECT. I think it is the opposite situation, it us undoing the premature overloading of O_DIRECT that went in without performance numbers. This implementation clarifies that dax_do_io() handles the lack of a page cache for buffered I/O and O_DIRECT behaves as it nominally would by sending an I/O to the driver. It has the benefit of matching the error semantics of a typical block device where a buffered write could hit an error filling the page cache, but an O_DIRECT write potentially triggers the drive to remap the block.
On 05/02/2016 06:51 PM, Vishal Verma wrote: > On Mon, 2016-05-02 at 18:41 +0300, Boaz Harrosh wrote: >> On 04/29/2016 12:16 AM, Vishal Verma wrote: >>> >>> All IO in a dax filesystem used to go through dax_do_io, which >>> cannot >>> handle media errors, and thus cannot provide a recovery path that >>> can >>> send a write through the driver to clear errors. >>> >>> Add a new iocb flag for DAX, and set it only for DAX mounts. In the >>> IO >>> path for DAX filesystems, use the same direct_IO path for both DAX >>> and >>> direct_io iocbs, but use the flags to identify when we are in >>> O_DIRECT >>> mode vs non O_DIRECT with DAX, and for O_DIRECT, use the >>> conventional >>> direct_IO path instead of DAX. >>> >> Really? What are your thinking here? >> >> What about all the current users of O_DIRECT, you have just made them >> 4 times slower and "less concurrent*" then "buffred io" users. Since >> direct_IO path will queue an IO request and all. >> (And if it is not so slow then why do we need dax_do_io at all? >> [Rhetorical]) >> >> I hate it that you overload the semantics of a known and expected >> O_DIRECT flag, for special pmem quirks. This is an incompatible >> and unrelated overload of the semantics of O_DIRECT. > > We overloaded O_DIRECT a long time ago when we made DAX piggyback on > the same path: > > static inline bool io_is_direct(struct file *filp) > { > return (filp->f_flags & O_DIRECT) || IS_DAX(filp->f_mapping->host); > } > No as far as the user is concerned we have not. The O_DIRECT user is still getting all the semantics he wants, .i.e no syncs no memory cache usage, no copies ... Only with DAX the buffered IO is the same since with pmem it is faster. Then why not? The basic contract with the user did not break. The above was just an implementation detail to easily navigate through the Linux vfs IO stack and make the least amount of changes in every FS that wanted to support DAX.(And since dax_do_io is much more like direct_IO then like page-cache IO) > Yes O_DIRECT on a DAX mounted file system will now be slower, but - > >> >>> >>> This allows us a recovery path in the form of opening the file with >>> O_DIRECT and writing to it with the usual O_DIRECT semantics >>> (sector >>> alignment restrictions). >>> >> I understand that you want a sector aligned IO, right? for the >> clear of errors. But I hate it that you forced all O_DIRECT IO >> to be slow for this. >> Can you not make dax_do_io handle media errors? At least for the >> parts of the IO that are aligned. >> (And your recovery path application above can use only aligned >> IO to make sure) >> >> Please look for another solution. Even a special >> IOCTL_DAX_CLEAR_ERROR > > - see all the versions of this series prior to this one, where we try > to do a fallback... > And? So now all O_DIRECT APPs go 4 times slower. I will have a look but if it is really so bad than please consider an IOCTL or syscall. Or a special O_DAX_ERRORS flag ... Please do not trash all the O_DIRECT users, they are the more important clients, like DBs and VMs. Thanks Boaz >> >> [*"less concurrent" because of the queuing done in bdev. Note how >> pmem is not even multi-queue, and even if it was it will be much >> slower then DAX because of the code depth and all the locks and >> task >> switches done in the block layer. In DAX the final memcpy is done >> directly >> on the user-mode thread] >> >> Thanks >> Boaz >> >
On 05/02/2016 07:01 PM, Dan Williams wrote: > On Mon, May 2, 2016 at 8:41 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >> On 04/29/2016 12:16 AM, Vishal Verma wrote: >>> All IO in a dax filesystem used to go through dax_do_io, which cannot >>> handle media errors, and thus cannot provide a recovery path that can >>> send a write through the driver to clear errors. >>> >>> Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO >>> path for DAX filesystems, use the same direct_IO path for both DAX and >>> direct_io iocbs, but use the flags to identify when we are in O_DIRECT >>> mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional >>> direct_IO path instead of DAX. >>> >> >> Really? What are your thinking here? >> >> What about all the current users of O_DIRECT, you have just made them >> 4 times slower and "less concurrent*" then "buffred io" users. Since >> direct_IO path will queue an IO request and all. >> (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) >> >> I hate it that you overload the semantics of a known and expected >> O_DIRECT flag, for special pmem quirks. This is an incompatible >> and unrelated overload of the semantics of O_DIRECT. > > I think it is the opposite situation, it us undoing the premature > overloading of O_DIRECT that went in without performance numbers. We have tons of measurements. Is not hard to imagine the results though. Specially the 1000 threads case > This implementation clarifies that dax_do_io() handles the lack of a > page cache for buffered I/O and O_DIRECT behaves as it nominally would > by sending an I/O to the driver. > It has the benefit of matching the > error semantics of a typical block device where a buffered write could > hit an error filling the page cache, but an O_DIRECT write potentially > triggers the drive to remap the block. > I fail to see how in writes the device error semantics regarding remapping of blocks is any different between buffered and direct IO. As far as the block device it is the same exact code path. All The big difference is higher in the VFS. And ... So you are willing to sacrifice the 99% hotpath for the sake of the 1% error path? and piggybacking on poor O_DIRECT. Again there are tons of O_DIRECT apps out there, why are you forcing them to change if they want true pmem performance? I still believe dax_do_io() can be made more resilient to errors, and clear errors on writes. Me going digging in old patches ... Cheers Boaz
On Mon, May 2, 2016 at 9:22 AM, Boaz Harrosh <boaz@plexistor.com> wrote: > On 05/02/2016 07:01 PM, Dan Williams wrote: >> On Mon, May 2, 2016 at 8:41 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >>> On 04/29/2016 12:16 AM, Vishal Verma wrote: >>>> All IO in a dax filesystem used to go through dax_do_io, which cannot >>>> handle media errors, and thus cannot provide a recovery path that can >>>> send a write through the driver to clear errors. >>>> >>>> Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO >>>> path for DAX filesystems, use the same direct_IO path for both DAX and >>>> direct_io iocbs, but use the flags to identify when we are in O_DIRECT >>>> mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional >>>> direct_IO path instead of DAX. >>>> >>> >>> Really? What are your thinking here? >>> >>> What about all the current users of O_DIRECT, you have just made them >>> 4 times slower and "less concurrent*" then "buffred io" users. Since >>> direct_IO path will queue an IO request and all. >>> (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) >>> >>> I hate it that you overload the semantics of a known and expected >>> O_DIRECT flag, for special pmem quirks. This is an incompatible >>> and unrelated overload of the semantics of O_DIRECT. >> >> I think it is the opposite situation, it us undoing the premature >> overloading of O_DIRECT that went in without performance numbers. > > We have tons of measurements. Is not hard to imagine the results though. > Specially the 1000 threads case > >> This implementation clarifies that dax_do_io() handles the lack of a >> page cache for buffered I/O and O_DIRECT behaves as it nominally would >> by sending an I/O to the driver. > >> It has the benefit of matching the >> error semantics of a typical block device where a buffered write could >> hit an error filling the page cache, but an O_DIRECT write potentially >> triggers the drive to remap the block. >> > > I fail to see how in writes the device error semantics regarding remapping of > blocks is any different between buffered and direct IO. As far as the block > device it is the same exact code path. All The big difference is higher in the > VFS. > > And ... So you are willing to sacrifice the 99% hotpath for the sake of the > 1% error path? and piggybacking on poor O_DIRECT. > > Again there are tons of O_DIRECT apps out there, why are you forcing them to > change if they want true pmem performance? This isn't forcing them to change. This is the path of least surprise as error semantics are identical to a typical block device. Yes, an application can go faster by switching to the "buffered" / dax_do_io() path it can go even faster to switch to mmap() I/O and use DAX directly. If we can later optimize the O_DIRECT path to bring it's performance more in line with dax_do_io(), great, but the implementation should be correct first and optimized later.
On 05/02/2016 07:49 PM, Dan Williams wrote: > On Mon, May 2, 2016 at 9:22 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >> On 05/02/2016 07:01 PM, Dan Williams wrote: >>> On Mon, May 2, 2016 at 8:41 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >>>> On 04/29/2016 12:16 AM, Vishal Verma wrote: >>>>> All IO in a dax filesystem used to go through dax_do_io, which cannot >>>>> handle media errors, and thus cannot provide a recovery path that can >>>>> send a write through the driver to clear errors. >>>>> >>>>> Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO >>>>> path for DAX filesystems, use the same direct_IO path for both DAX and >>>>> direct_io iocbs, but use the flags to identify when we are in O_DIRECT >>>>> mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional >>>>> direct_IO path instead of DAX. >>>>> >>>> >>>> Really? What are your thinking here? >>>> >>>> What about all the current users of O_DIRECT, you have just made them >>>> 4 times slower and "less concurrent*" then "buffred io" users. Since >>>> direct_IO path will queue an IO request and all. >>>> (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) >>>> >>>> I hate it that you overload the semantics of a known and expected >>>> O_DIRECT flag, for special pmem quirks. This is an incompatible >>>> and unrelated overload of the semantics of O_DIRECT. >>> >>> I think it is the opposite situation, it us undoing the premature >>> overloading of O_DIRECT that went in without performance numbers. >> >> We have tons of measurements. Is not hard to imagine the results though. >> Specially the 1000 threads case >> >>> This implementation clarifies that dax_do_io() handles the lack of a >>> page cache for buffered I/O and O_DIRECT behaves as it nominally would >>> by sending an I/O to the driver. >> >>> It has the benefit of matching the >>> error semantics of a typical block device where a buffered write could >>> hit an error filling the page cache, but an O_DIRECT write potentially >>> triggers the drive to remap the block. >>> >> >> I fail to see how in writes the device error semantics regarding remapping of >> blocks is any different between buffered and direct IO. As far as the block >> device it is the same exact code path. All The big difference is higher in the >> VFS. >> >> And ... So you are willing to sacrifice the 99% hotpath for the sake of the >> 1% error path? and piggybacking on poor O_DIRECT. >> >> Again there are tons of O_DIRECT apps out there, why are you forcing them to >> change if they want true pmem performance? > > This isn't forcing them to change. This is the path of least surprise > as error semantics are identical to a typical block device. Yes, an > application can go faster by switching to the "buffered" / dax_do_io() > path it can go even faster to switch to mmap() I/O and use DAX > directly. If we can later optimize the O_DIRECT path to bring it's > performance more in line with dax_do_io(), great, but the > implementation should be correct first and optimized later. > Why does it need to be either or. Why not both? And also I disagree if you are correct and dax_do_io is bad and needs fixing than you have broken applications. Because in current model: read => -EIO, write-bufferd, sync() gives you the same error semantics as: read => -EIO, write-direct-io In fact this is what the delete, restore from backup model does today. Who said it uses / must direct IO. Actually I think it does not. Two things I can think of which are better: [1] Why not go deeper into the dax io loops, and for any WRITE failed page call bdev_rw_page() to let the pmem.c clear / relocate the error page. So reads return -EIO - is what you wanted no? writes get a memory error and retry with bdev_rw_page() to let the bdev relocate / clear the error - is what you wanted no? In the partial page WRITE case on bad sectors. we can carefully read-modify-write sector-by-sector and zero-out the bad-sectors that could not be read, what else? (Or enhance the bdev_rw_page() API) [2] Only switch to slow O_DIRECT, on presence of errors like you wanted. But I still hate that you overload error semantics with O_DIRECT which does not exist today see above Thanks Boaz
On Mon, May 2, 2016 at 10:44 AM, Boaz Harrosh <boaz@plexistor.com> wrote: > On 05/02/2016 07:49 PM, Dan Williams wrote: >> On Mon, May 2, 2016 at 9:22 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >>> On 05/02/2016 07:01 PM, Dan Williams wrote: >>>> On Mon, May 2, 2016 at 8:41 AM, Boaz Harrosh <boaz@plexistor.com> wrote: >>>>> On 04/29/2016 12:16 AM, Vishal Verma wrote: >>>>>> All IO in a dax filesystem used to go through dax_do_io, which cannot >>>>>> handle media errors, and thus cannot provide a recovery path that can >>>>>> send a write through the driver to clear errors. >>>>>> >>>>>> Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO >>>>>> path for DAX filesystems, use the same direct_IO path for both DAX and >>>>>> direct_io iocbs, but use the flags to identify when we are in O_DIRECT >>>>>> mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional >>>>>> direct_IO path instead of DAX. >>>>>> >>>>> >>>>> Really? What are your thinking here? >>>>> >>>>> What about all the current users of O_DIRECT, you have just made them >>>>> 4 times slower and "less concurrent*" then "buffred io" users. Since >>>>> direct_IO path will queue an IO request and all. >>>>> (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) >>>>> >>>>> I hate it that you overload the semantics of a known and expected >>>>> O_DIRECT flag, for special pmem quirks. This is an incompatible >>>>> and unrelated overload of the semantics of O_DIRECT. >>>> >>>> I think it is the opposite situation, it us undoing the premature >>>> overloading of O_DIRECT that went in without performance numbers. >>> >>> We have tons of measurements. Is not hard to imagine the results though. >>> Specially the 1000 threads case >>> >>>> This implementation clarifies that dax_do_io() handles the lack of a >>>> page cache for buffered I/O and O_DIRECT behaves as it nominally would >>>> by sending an I/O to the driver. >>> >>>> It has the benefit of matching the >>>> error semantics of a typical block device where a buffered write could >>>> hit an error filling the page cache, but an O_DIRECT write potentially >>>> triggers the drive to remap the block. >>>> >>> >>> I fail to see how in writes the device error semantics regarding remapping of >>> blocks is any different between buffered and direct IO. As far as the block >>> device it is the same exact code path. All The big difference is higher in the >>> VFS. >>> >>> And ... So you are willing to sacrifice the 99% hotpath for the sake of the >>> 1% error path? and piggybacking on poor O_DIRECT. >>> >>> Again there are tons of O_DIRECT apps out there, why are you forcing them to >>> change if they want true pmem performance? >> >> This isn't forcing them to change. This is the path of least surprise >> as error semantics are identical to a typical block device. Yes, an >> application can go faster by switching to the "buffered" / dax_do_io() >> path it can go even faster to switch to mmap() I/O and use DAX >> directly. If we can later optimize the O_DIRECT path to bring it's >> performance more in line with dax_do_io(), great, but the >> implementation should be correct first and optimized later. >> > > Why does it need to be either or. Why not both? > And also I disagree if you are correct and dax_do_io is bad and needs fixing > than you have broken applications. Because in current model: > > read => -EIO, write-bufferd, sync() > gives you the same error semantics as: read => -EIO, write-direct-io > In fact this is what the delete, restore from backup model does today. > Who said it uses / must direct IO. Actually I think it does not. The semantic I am talking about preserving is: buffered / unaligned write of a bad sector => -EIO on reading into the page cache ...and that the only guaranteed way to clear an error (assuming the block device supports it) is an O_DIRECT write. > > Two things I can think of which are better: > [1] > Why not go deeper into the dax io loops, and for any WRITE > failed page call bdev_rw_page() to let the pmem.c clear / relocate > the error page. Where do you get the rest of the data to complete a full page write? > So reads return -EIO - is what you wanted no? That's well understood. What we are debating is the method to clear errors / ask the storage device to remap bad blocks. > writes get a memory error and retry with bdev_rw_page() to let the bdev > relocate / clear the error - is what you wanted no? > > In the partial page WRITE case on bad sectors. we can carefully read-modify-write > sector-by-sector and zero-out the bad-sectors that could not be read, what else? > (Or enhance the bdev_rw_page() API) See all the previous discussions on why the fallback path is problematic to implement. > > [2] > Only switch to slow O_DIRECT, on presence of errors like you wanted. But I still > hate that you overload error semantics with O_DIRECT which does not exist today > see above I still think we're talking past each other on this point. This patch set is not overloading error semantics, it's fixing the error handling problem that was introduced in this commit: d475c6346a38 dax,ext2: replace XIP read and write with DAX I/O ...where we started overloading O_DIRECT and dax_do_io() semantics.
On 05/02/2016 09:10 PM, Dan Williams wrote: <> > > The semantic I am talking about preserving is: > > buffered / unaligned write of a bad sector => -EIO on reading into the > page cache > What about aligned buffered write? like write 0-to-eof This still broken? (and is what restore apps do) > ...and that the only guaranteed way to clear an error (assuming the > block device supports it) is an O_DIRECT write. > Sure fixing dax_do_io will guaranty that. <> > I still think we're talking past each other on this point. Yes we are! > This patch > set is not overloading error semantics, it's fixing the error handling > problem that was introduced in this commit: > > d475c6346a38 dax,ext2: replace XIP read and write with DAX I/O > > ...where we started overloading O_DIRECT and dax_do_io() semantics. > But above does not fix them does it? it just completely NULLs DAX for O_DIRECT which is a great pity, why did we do all this work in the first place. And then it keeps broken the aligned buffered writes, which are still broken after this set. I have by now read the v2 patches. And I think you guys did not yet try the proper fix for dax_do_io. I think you need to go deeper into the loops and selectively call bdev_* when error on a specific page copy. No need to go through direct_IO path at all. Do you need that I send you a patch to demonstrate what I mean? But yes I feel too that "we're talking past each other". I did want to come to LSF and talk to you, but was not invited. Should I call you? Thanks Boaz
On Mon, May 2, 2016 at 11:32 AM, Boaz Harrosh <boaz@plexistor.com> wrote: > On 05/02/2016 09:10 PM, Dan Williams wrote: > <> >> >> The semantic I am talking about preserving is: >> >> buffered / unaligned write of a bad sector => -EIO on reading into the >> page cache >> > > What about aligned buffered write? like write 0-to-eof > This still broken? (and is what restore apps do) > >> ...and that the only guaranteed way to clear an error (assuming the >> block device supports it) is an O_DIRECT write. >> > > Sure fixing dax_do_io will guaranty that. > > <> >> I still think we're talking past each other on this point. > > Yes we are! > >> This patch >> set is not overloading error semantics, it's fixing the error handling >> problem that was introduced in this commit: >> >> d475c6346a38 dax,ext2: replace XIP read and write with DAX I/O >> >> ...where we started overloading O_DIRECT and dax_do_io() semantics. >> > > But above does not fix them does it? it just completely NULLs DAX for > O_DIRECT which is a great pity, why did we do all this work in the first > place. This is hyperbole. We don't impact "all the work" we did for the mmap I/O case and the acceleration of the non-direct-I/O case. > And then it keeps broken the aligned buffered writes, which are still > broken after this set. ...identical to the current situation with a traditional disk. > I have by now read the v2 patches. And I think you guys did not yet try > the proper fix for dax_do_io. I think you need to go deeper into the loops > and selectively call bdev_* when error on a specific page copy. No need to > go through direct_IO path at all. We still reach a point where the minimum granularity of bdev_direct_access() is larger than a sector, so you end up still needing to have the application understand how to send a properly aligned I/O. The semantics of how to send a properly aligned direct-I/O are already well understood, so we simply reuse that path. > Do you need that I send you a patch to demonstrate what I mean? I remain skeptical of what you are proposing, but yes, a patch has a better chance to move the discussion forward.
On Mon, 2016-05-02 at 19:03 +0300, Boaz Harrosh wrote: > On 05/02/2016 06:51 PM, Vishal Verma wrote: > > > > On Mon, 2016-05-02 at 18:41 +0300, Boaz Harrosh wrote: > > > > > > On 04/29/2016 12:16 AM, Vishal Verma wrote: > > > > > > > > > > > > All IO in a dax filesystem used to go through dax_do_io, which > > > > cannot > > > > handle media errors, and thus cannot provide a recovery path > > > > that > > > > can > > > > send a write through the driver to clear errors. > > > > > > > > Add a new iocb flag for DAX, and set it only for DAX mounts. In > > > > the > > > > IO > > > > path for DAX filesystems, use the same direct_IO path for both > > > > DAX > > > > and > > > > direct_io iocbs, but use the flags to identify when we are in > > > > O_DIRECT > > > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the > > > > conventional > > > > direct_IO path instead of DAX. > > > > > > > Really? What are your thinking here? > > > > > > What about all the current users of O_DIRECT, you have just made > > > them > > > 4 times slower and "less concurrent*" then "buffred io" users. > > > Since > > > direct_IO path will queue an IO request and all. > > > (And if it is not so slow then why do we need dax_do_io at all? > > > [Rhetorical]) > > > > > > I hate it that you overload the semantics of a known and expected > > > O_DIRECT flag, for special pmem quirks. This is an incompatible > > > and unrelated overload of the semantics of O_DIRECT. > > We overloaded O_DIRECT a long time ago when we made DAX piggyback on > > the same path: > > > > static inline bool io_is_direct(struct file *filp) > > { > > return (filp->f_flags & O_DIRECT) || IS_DAX(filp->f_mapping- > > >host); > > } > > > No as far as the user is concerned we have not. The O_DIRECT user > is still getting all the semantics he wants, .i.e no syncs no > memory cache usage, no copies ... > > Only with DAX the buffered IO is the same since with pmem it is > faster. > Then why not? The basic contract with the user did not break. > > The above was just an implementation detail to easily navigate > through the Linux vfs IO stack and make the least amount of changes > in every FS that wanted to support DAX.(And since dax_do_io is much > more like direct_IO then like page-cache IO) > > > > > Yes O_DIRECT on a DAX mounted file system will now be slower, but - > > > > > > > > > > > > > > > > > > > > This allows us a recovery path in the form of opening the file > > > > with > > > > O_DIRECT and writing to it with the usual O_DIRECT semantics > > > > (sector > > > > alignment restrictions). > > > > > > > I understand that you want a sector aligned IO, right? for the > > > clear of errors. But I hate it that you forced all O_DIRECT IO > > > to be slow for this. > > > Can you not make dax_do_io handle media errors? At least for the > > > parts of the IO that are aligned. > > > (And your recovery path application above can use only aligned > > > IO to make sure) > > > > > > Please look for another solution. Even a special > > > IOCTL_DAX_CLEAR_ERROR > > - see all the versions of this series prior to this one, where we > > try > > to do a fallback... > > > And? > > So now all O_DIRECT APPs go 4 times slower. I will have a look but if > it is really so bad than please consider an IOCTL or syscall. Or a > special > O_DAX_ERRORS flag ... I'm curious where the 4x slower comes from.. The O_DIRECT path is still without page-cache copies, and nor does it go through request queues (since pmem is a bio-based driver). The only overhead is that of submitting a bio - and while I agree it is more overhead than dax_do_io, 4x seems a bit high. > > Please do not trash all the O_DIRECT users, they are the more > important > clients, like DBs and VMs. Shouldn't they be using mmaps and dax faults? I was under the impression that the dax_do_io path is a nice-to-have, but for anyone that will want to use DAX, they will want the mmap/fault path, not the IO path. This is just making the IO path 'more correct' by allowing it a way to deal with errors. > > Thanks > Boaz > > > > > > > > > > > > [*"less concurrent" because of the queuing done in bdev. Note how > > > pmem is not even multi-queue, and even if it was it will be much > > > slower then DAX because of the code depth and all the locks and > > > task > > > switches done in the block layer. In DAX the final memcpy is > > > done > > > directly > > > on the user-mode thread] > > > > > > Thanks > > > Boaz > > >
On 05/02/2016 09:48 PM, Dan Williams wrote: <> >> And then it keeps broken the aligned buffered writes, which are still >> broken after this set. > > ...identical to the current situation with a traditional disk. > Not true!! please see what I wrote "aligned buffered writes" If there are no reads involved then there are no errors returned to application. >> I have by now read the v2 patches. And I think you guys did not yet try >> the proper fix for dax_do_io. I think you need to go deeper into the loops >> and selectively call bdev_* when error on a specific page copy. No need to >> go through direct_IO path at all. > > We still reach a point where the minimum granularity of > bdev_direct_access() is larger than a sector, so you end up still > needing to have the application understand how to send a properly > aligned I/O. The semantics of how to send a properly aligned > direct-I/O are already well understood, so we simply reuse that path. > You are making a mountain out of a mouse. The simple copy of a file from start (offset ZERO) to end-of-file which is the most common usage on earth is perfectly aligned and needs not any O_DIRECT and is what is used everywhere. >> Do you need that I send you a patch to demonstrate what I mean? > > I remain skeptical of what you are proposing, but yes, a patch has a > better chance to move the discussion forward. > Sigh! OK Boaz
On Mon, May 02, 2016 at 06:41:51PM +0300, Boaz Harrosh wrote: > > All IO in a dax filesystem used to go through dax_do_io, which cannot > > handle media errors, and thus cannot provide a recovery path that can > > send a write through the driver to clear errors. > > > > Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO > > path for DAX filesystems, use the same direct_IO path for both DAX and > > direct_io iocbs, but use the flags to identify when we are in O_DIRECT > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional > > direct_IO path instead of DAX. > > > > Really? What are your thinking here? > > What about all the current users of O_DIRECT, you have just made them > 4 times slower and "less concurrent*" then "buffred io" users. Since > direct_IO path will queue an IO request and all. > (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) > > I hate it that you overload the semantics of a known and expected > O_DIRECT flag, for special pmem quirks. This is an incompatible > and unrelated overload of the semantics of O_DIRECT. Agreed - makig O_DIRECT less direct than not having it is plain stupid, and I somehow missed this initially. This whole DAX story turns into a major nightmare, and I fear all our hodge podge tweaks to the semantics aren't helping it. It seems like we simply need an explicit O_DAX for the read/write bypass if can't sort out the semantics (error, writer synchronization) just as we need a special flag for MMAP..
On Thu, May 5, 2016 at 7:24 AM, Christoph Hellwig <hch@infradead.org> wrote: > On Mon, May 02, 2016 at 06:41:51PM +0300, Boaz Harrosh wrote: >> > All IO in a dax filesystem used to go through dax_do_io, which cannot >> > handle media errors, and thus cannot provide a recovery path that can >> > send a write through the driver to clear errors. >> > >> > Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO >> > path for DAX filesystems, use the same direct_IO path for both DAX and >> > direct_io iocbs, but use the flags to identify when we are in O_DIRECT >> > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional >> > direct_IO path instead of DAX. >> > >> >> Really? What are your thinking here? >> >> What about all the current users of O_DIRECT, you have just made them >> 4 times slower and "less concurrent*" then "buffred io" users. Since >> direct_IO path will queue an IO request and all. >> (And if it is not so slow then why do we need dax_do_io at all? [Rhetorical]) >> >> I hate it that you overload the semantics of a known and expected >> O_DIRECT flag, for special pmem quirks. This is an incompatible >> and unrelated overload of the semantics of O_DIRECT. > > Agreed - makig O_DIRECT less direct than not having it is plain stupid, > and I somehow missed this initially. Of course I disagree because like Dave argues in the msync case we should do the correct thing first and make it fast later, but also like Dave this arguing in circles is getting tiresome. > This whole DAX story turns into a major nightmare, and I fear all our > hodge podge tweaks to the semantics aren't helping it. > > It seems like we simply need an explicit O_DAX for the read/write > bypass if can't sort out the semantics (error, writer synchronization) > just as we need a special flag for MMAP. I don't see how O_DAX makes this situation better if the goal is to accelerate unmodified applications... Vishal, at least the "delete a file with a badblock" model will still work for implicitly clearing errors with your changes to stop doing block clearing in fs/dax.c. This combined with a new -EBADBLOCK (as Dave suggests) and explicit logging of I/Os that fail for this reason at least gives a chance to communicate errors in files to suitably aware applications / environments.
On Thu, May 05, 2016 at 08:15:32AM -0700, Dan Williams wrote: > > Agreed - makig O_DIRECT less direct than not having it is plain stupid, > > and I somehow missed this initially. > > Of course I disagree because like Dave argues in the msync case we > should do the correct thing first and make it fast later, but also > like Dave this arguing in circles is getting tiresome. We should do the right thing first, and make it fast later. But this proposal is not getting it right - it still does not handle errors for the fast path, but magically makes it work for direct I/O by in general using a less optional path for O_DIRECT. It's getting the worst of all choices. As far as I can tell the only sensible option is to: - always try dax-like I/O first - have a custom get_user_pages + rw_bytes fallback handles bad blocks when hitting EIO And then we need to sort out the concurrent write synchronization. Again there I think we absolutely have to obey Posix for the !O_DIRECT case and can avoid it for O_DIRECT, similar to the existing non-DAX semantics. If we want any special additional semantics we _will_ need a special O_DAX flag.
On Thu, May 5, 2016 at 8:22 AM, Christoph Hellwig <hch@infradead.org> wrote: > On Thu, May 05, 2016 at 08:15:32AM -0700, Dan Williams wrote: >> > Agreed - makig O_DIRECT less direct than not having it is plain stupid, >> > and I somehow missed this initially. >> >> Of course I disagree because like Dave argues in the msync case we >> should do the correct thing first and make it fast later, but also >> like Dave this arguing in circles is getting tiresome. > > We should do the right thing first, and make it fast later. But this > proposal is not getting it right - it still does not handle errors > for the fast path, but magically makes it work for direct I/O by > in general using a less optional path for O_DIRECT. It's getting the > worst of all choices. > > As far as I can tell the only sensible option is to: > > - always try dax-like I/O first > - have a custom get_user_pages + rw_bytes fallback handles bad blocks > when hitting EIO If you're on board with more special fallbacks for dax-capable block devices that indeed opens up the thinking. The O_DIRECT approach was meant to keep the error clearing model close to the traditional block device case, but yes that does constrain the implementation in sub-optimal ways. However, we still have the alignment problem in the rw_bytes case, how do we communicate to the application that only writes with a certain size/alignment will clear errors? That forced alignment assumption was the other appeal of O_DIRECT. Perhaps we can at least start with hole punching and block reallocation as the error clearing method while we think more about the write-to-clear case? > And then we need to sort out the concurrent write synchronization. > Again there I think we absolutely have to obey Posix for the !O_DIRECT > case and can avoid it for O_DIRECT, similar to the existing non-DAX > semantics. If we want any special additional semantics we _will_ need > a special O_DAX flag. Ok, makes sense.
On Thu, 2016-05-05 at 07:24 -0700, Christoph Hellwig wrote: > On Mon, May 02, 2016 at 06:41:51PM +0300, Boaz Harrosh wrote: > > > > > > > > All IO in a dax filesystem used to go through dax_do_io, which > > > cannot > > > handle media errors, and thus cannot provide a recovery path that > > > can > > > send a write through the driver to clear errors. > > > > > > Add a new iocb flag for DAX, and set it only for DAX mounts. In > > > the IO > > > path for DAX filesystems, use the same direct_IO path for both DAX > > > and > > > direct_io iocbs, but use the flags to identify when we are in > > > O_DIRECT > > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the > > > conventional > > > direct_IO path instead of DAX. > > > > > Really? What are your thinking here? > > > > What about all the current users of O_DIRECT, you have just made > > them > > 4 times slower and "less concurrent*" then "buffred io" users. Since > > direct_IO path will queue an IO request and all. > > (And if it is not so slow then why do we need dax_do_io at all? > > [Rhetorical]) > > > > I hate it that you overload the semantics of a known and expected > > O_DIRECT flag, for special pmem quirks. This is an incompatible > > and unrelated overload of the semantics of O_DIRECT. > Agreed - makig O_DIRECT less direct than not having it is plain > stupid, > and I somehow missed this initially. How is it any 'less direct'? All it does now is follow the blockdev O_DIRECT path. There still isn't any page cache involved.. > > This whole DAX story turns into a major nightmare, and I fear all our > hodge podge tweaks to the semantics aren't helping it. > > It seems like we simply need an explicit O_DAX for the read/write > bypass if can't sort out the semantics (error, writer synchronization) > just as we need a special flag for MMAP..
On Thu, 2016-05-05 at 08:15 -0700, Dan Williams wrote: > On Thu, May 5, 2016 at 7:24 AM, Christoph Hellwig <hch@infradead.org> > wrote: > > > > On Mon, May 02, 2016 at 06:41:51PM +0300, Boaz Harrosh wrote: > > > > > > > > > > > All IO in a dax filesystem used to go through dax_do_io, which > > > > cannot > > > > handle media errors, and thus cannot provide a recovery path > > > > that can > > > > send a write through the driver to clear errors. > > > > > > > > Add a new iocb flag for DAX, and set it only for DAX mounts. In > > > > the IO > > > > path for DAX filesystems, use the same direct_IO path for both > > > > DAX and > > > > direct_io iocbs, but use the flags to identify when we are in > > > > O_DIRECT > > > > mode vs non O_DIRECT with DAX, and for O_DIRECT, use the > > > > conventional > > > > direct_IO path instead of DAX. > > > > > > > Really? What are your thinking here? > > > > > > What about all the current users of O_DIRECT, you have just made > > > them > > > 4 times slower and "less concurrent*" then "buffred io" users. > > > Since > > > direct_IO path will queue an IO request and all. > > > (And if it is not so slow then why do we need dax_do_io at all? > > > [Rhetorical]) > > > > > > I hate it that you overload the semantics of a known and expected > > > O_DIRECT flag, for special pmem quirks. This is an incompatible > > > and unrelated overload of the semantics of O_DIRECT. > > Agreed - makig O_DIRECT less direct than not having it is plain > > stupid, > > and I somehow missed this initially. > Of course I disagree because like Dave argues in the msync case we > should do the correct thing first and make it fast later, but also > like Dave this arguing in circles is getting tiresome. > > > > > This whole DAX story turns into a major nightmare, and I fear all > > our > > hodge podge tweaks to the semantics aren't helping it. > > > > It seems like we simply need an explicit O_DAX for the read/write > > bypass if can't sort out the semantics (error, writer > > synchronization) > > just as we need a special flag for MMAP. > I don't see how O_DAX makes this situation better if the goal is to > accelerate unmodified applications... > > Vishal, at least the "delete a file with a badblock" model will still > work for implicitly clearing errors with your changes to stop doing > block clearing in fs/dax.c. This combined with a new -EBADBLOCK (as > Dave suggests) and explicit logging of I/Os that fail for this reason > at least gives a chance to communicate errors in files to suitably > aware applications / environments. Agreed - I'll send out a series that has just the zeroing changes, and drop the dax_io fallback/O_DIRECT tweak for now while we figure out the right thing to do. That should get us to a place where we still have dax in the presence of errors, and have _a_ path for recovery. > _______________________________________________ > Linux-nvdimm mailing list > Linux-nvdimm@lists.01.org > https://lists.01.org/mailman/listinfo/linux-nvdimm
On Thu, 2016-05-05 at 08:22 -0700, Christoph Hellwig wrote: > On Thu, May 05, 2016 at 08:15:32AM -0700, Dan Williams wrote: > > > > > > > > Agreed - makig O_DIRECT less direct than not having it is plain > > > stupid, > > > and I somehow missed this initially. > > Of course I disagree because like Dave argues in the msync case we > > should do the correct thing first and make it fast later, but also > > like Dave this arguing in circles is getting tiresome. > We should do the right thing first, and make it fast later. But this > proposal is not getting it right - it still does not handle errors > for the fast path, but magically makes it work for direct I/O by > in general using a less optional path for O_DIRECT. It's getting the > worst of all choices. > > As far as I can tell the only sensible option is to: > > - always try dax-like I/O first > - have a custom get_user_pages + rw_bytes fallback handles bad blocks > when hitting EIO I'm not sure I completely understand how this will work? Can you explain a bit? Would we have to export rw_bytes up to layers above the pmem driver? Where does get_user_pages come in? > > And then we need to sort out the concurrent write synchronization. > Again there I think we absolutely have to obey Posix for the !O_DIRECT > case and can avoid it for O_DIRECT, similar to the existing non-DAX > semantics. If we want any special additional semantics we _will_ need > a special O_DAX flag. > _______________________________________________ > Linux-nvdimm mailing list > Linux-nvdimm@lists.01.org > https://lists.01.org/mailman/listinfo/linux-nvdimm
On Thu, May 05, 2016 at 09:45:07PM +0000, Verma, Vishal L wrote: > I'm not sure I completely understand how this will work? Can you explain > a bit? Would we have to export rw_bytes up to layers above the pmem > driver? Where does get_user_pages come in? A DAX filesystem can directly use the nvdimm layer the same way btt doe,s what's the problem? Re get_user_pages my idea was to simply use that to lock down the user pages so that we can call rw_bytes on it. How else would you do it? Do a kmalloc, copy_from_user and then another memcpy?
On Thu, May 05, 2016 at 09:39:14PM +0000, Verma, Vishal L wrote: > How is it any 'less direct'? All it does now is follow the blockdev > O_DIRECT path. There still isn't any page cache involved.. It's still more overhead than the play DAX I/O path.
On Sun, 2016-05-08 at 02:01 -0700, hch@infradead.org wrote: > On Thu, May 05, 2016 at 09:45:07PM +0000, Verma, Vishal L wrote: > > > > I'm not sure I completely understand how this will work? Can you > > explain > > a bit? Would we have to export rw_bytes up to layers above the pmem > > driver? Where does get_user_pages come in? > A DAX filesystem can directly use the nvdimm layer the same way btt > doe,s what's the problem? The BTT does rw_bytes through an internal-to-libnvdimm mechanism, but rw_bytes isn't exported to the filesystem, currently.. To do this we'd have to either add an rw_bytes to block device operations...or something. Another thing is rw_bytes currently doesn't do error clearing either. We store badblocks at sector granularity, and like Dan said earlier, that hides the clear_error alignment requirements and upper layers don't have to be aware of it. To make rw_bytes clear sub-sector errors, we'd have to change the granularity of bad-blocks, and make upper layers aware of the clearing alignment requirements. Using a block-write semantic for clearing hides all this away. > > Re get_user_pages my idea was to simply use that to lock down the > user > pages so that we can call rw_bytes on it. How else would you do > it? Do > a kmalloc, copy_from_user and then another memcpy?
diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 80cf8ad..c0a24c3 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -568,7 +568,7 @@ struct switch_request { static inline void loop_update_dio(struct loop_device *lo) { - __loop_update_dio(lo, io_is_direct(lo->lo_backing_file) | + __loop_update_dio(lo, (lo->lo_backing_file->f_flags & O_DIRECT) | lo->use_dio); } diff --git a/fs/block_dev.c b/fs/block_dev.c index 79defba..97a1f5f 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -167,12 +167,21 @@ blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) struct file *file = iocb->ki_filp; struct inode *inode = bdev_file_inode(file); - if (IS_DAX(inode)) + if (iocb_is_direct(iocb)) + return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, + offset, blkdev_get_block, NULL, + NULL, DIO_SKIP_DIO_COUNT); + else if (iocb_is_dax(iocb)) return dax_do_io(iocb, inode, iter, offset, blkdev_get_block, NULL, DIO_SKIP_DIO_COUNT); - return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, offset, - blkdev_get_block, NULL, NULL, - DIO_SKIP_DIO_COUNT); + else { + /* + * If we're in the direct_IO path, either the IOCB_DIRECT or + * IOCB_DAX flags must be set. + */ + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); + return -ENXIO; + } } int __sync_blockdev(struct block_device *bdev, int wait) diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c index 35f2b0bf..45f2b51 100644 --- a/fs/ext2/inode.c +++ b/fs/ext2/inode.c @@ -861,12 +861,20 @@ ext2_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) size_t count = iov_iter_count(iter); ssize_t ret; - if (IS_DAX(inode)) - ret = dax_do_io(iocb, inode, iter, offset, ext2_get_block, NULL, - DIO_LOCKING); - else + if (iocb_is_direct(iocb)) ret = blockdev_direct_IO(iocb, inode, iter, offset, ext2_get_block); + else if (iocb_is_dax(iocb)) + ret = dax_do_io(iocb, inode, iter, offset, ext2_get_block, NULL, + DIO_LOCKING); + else { + /* + * If we're in the direct_IO path, either the IOCB_DIRECT or + * IOCB_DAX flags must be set. + */ + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); + return -ENXIO; + } if (ret < 0 && iov_iter_rw(iter) == WRITE) ext2_write_failed(mapping, offset + count); return ret; diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 2e9aa49..165a0b8 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -94,7 +94,7 @@ ext4_file_write_iter(struct kiocb *iocb, struct iov_iter *from) struct file *file = iocb->ki_filp; struct inode *inode = file_inode(iocb->ki_filp); struct blk_plug plug; - int o_direct = iocb->ki_flags & IOCB_DIRECT; + int o_direct = iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX); int unaligned_aio = 0; int overwrite = 0; ssize_t ret; diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 6d5d5c1..0b6d77a 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3410,15 +3410,22 @@ static ssize_t ext4_direct_IO_write(struct kiocb *iocb, struct iov_iter *iter, #ifdef CONFIG_EXT4_FS_ENCRYPTION BUG_ON(ext4_encrypted_inode(inode) && S_ISREG(inode->i_mode)); #endif - if (IS_DAX(inode)) { - ret = dax_do_io(iocb, inode, iter, offset, get_block_func, - ext4_end_io_dio, dio_flags); - } else + if (iocb_is_direct(iocb)) ret = __blockdev_direct_IO(iocb, inode, inode->i_sb->s_bdev, iter, offset, get_block_func, ext4_end_io_dio, NULL, dio_flags); - + else if (iocb_is_dax(iocb)) + ret = dax_do_io(iocb, inode, iter, offset, get_block_func, + ext4_end_io_dio, dio_flags); + else { + /* + * If we're in the direct_IO path, either the IOCB_DIRECT or + * IOCB_DAX flags must be set. + */ + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); + return -ENXIO; + } if (ret > 0 && !overwrite && ext4_test_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN)) { int err; @@ -3503,7 +3510,7 @@ static ssize_t ext4_direct_IO_read(struct kiocb *iocb, struct iov_iter *iter, else unlocked = 1; } - if (IS_DAX(inode)) { + if (iocb_is_dax(iocb)) { ret = dax_do_io(iocb, inode, iter, offset, ext4_dio_get_block, NULL, unlocked ? 0 : DIO_LOCKING); } else { diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index e49b240..8134e99 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -1412,21 +1412,27 @@ xfs_vm_direct_IO( struct inode *inode = iocb->ki_filp->f_mapping->host; dio_iodone_t *endio = NULL; int flags = 0; - struct block_device *bdev; + struct block_device *bdev = xfs_find_bdev_for_inode(inode); if (iov_iter_rw(iter) == WRITE) { endio = xfs_end_io_direct_write; flags = DIO_ASYNC_EXTEND; } - if (IS_DAX(inode)) { + if (iocb_is_direct(iocb)) + return __blockdev_direct_IO(iocb, inode, bdev, iter, offset, + xfs_get_blocks_direct, endio, NULL, flags); + else if (iocb_is_dax(iocb)) return dax_do_io(iocb, inode, iter, offset, - xfs_get_blocks_direct, endio, 0); + xfs_get_blocks_direct, endio, 0); + else { + /* + * If we're in the direct_IO path, either the IOCB_DIRECT or + * IOCB_DAX flags must be set. + */ + WARN_ONCE(1, "Kernel Bug with iocb flags\n"); + return -ENXIO; } - - bdev = xfs_find_bdev_for_inode(inode); - return __blockdev_direct_IO(iocb, inode, bdev, iter, offset, - xfs_get_blocks_direct, endio, NULL, flags); } /* diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index c2946f4..3d5d3c2 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -300,7 +300,7 @@ xfs_file_read_iter( XFS_STATS_INC(mp, xs_read_calls); - if (unlikely(iocb->ki_flags & IOCB_DIRECT)) + if (unlikely(iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX))) ioflags |= XFS_IO_ISDIRECT; if (file->f_mode & FMODE_NOCMTIME) ioflags |= XFS_IO_INVIS; @@ -898,7 +898,7 @@ xfs_file_write_iter( if (XFS_FORCED_SHUTDOWN(ip->i_mount)) return -EIO; - if ((iocb->ki_flags & IOCB_DIRECT) || IS_DAX(inode)) + if ((iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX))) ret = xfs_file_dio_aio_write(iocb, from); else ret = xfs_file_buffered_aio_write(iocb, from); diff --git a/include/linux/fs.h b/include/linux/fs.h index 9f28130..adca1d8 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -322,6 +322,7 @@ struct writeback_control; #define IOCB_APPEND (1 << 1) #define IOCB_DIRECT (1 << 2) #define IOCB_HIPRI (1 << 3) +#define IOCB_DAX (1 << 4) struct kiocb { struct file *ki_filp; @@ -2930,9 +2931,15 @@ extern int generic_show_options(struct seq_file *m, struct dentry *root); extern void save_mount_options(struct super_block *sb, char *options); extern void replace_mount_options(struct super_block *sb, char *options); -static inline bool io_is_direct(struct file *filp) +static inline bool iocb_is_dax(struct kiocb *iocb) { - return (filp->f_flags & O_DIRECT) || IS_DAX(filp->f_mapping->host); + return IS_DAX(file_inode(iocb->ki_filp)) && + (iocb->ki_flags & IOCB_DAX); +} + +static inline bool iocb_is_direct(struct kiocb *iocb) +{ + return iocb->ki_flags & IOCB_DIRECT; } static inline int iocb_flags(struct file *file) @@ -2940,8 +2947,10 @@ static inline int iocb_flags(struct file *file) int res = 0; if (file->f_flags & O_APPEND) res |= IOCB_APPEND; - if (io_is_direct(file)) + if (file->f_flags & O_DIRECT) res |= IOCB_DIRECT; + if (IS_DAX(file_inode(file))) + res |= IOCB_DAX; return res; } diff --git a/mm/filemap.c b/mm/filemap.c index 3effd5c..b959acf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1849,7 +1849,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) if (!count) goto out; /* skip atime */ - if (iocb->ki_flags & IOCB_DIRECT) { + if (iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX)) { struct address_space *mapping = file->f_mapping; struct inode *inode = mapping->host; loff_t size; @@ -2719,7 +2719,7 @@ ssize_t __generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) if (err) goto out; - if (iocb->ki_flags & IOCB_DIRECT) { + if (iocb->ki_flags & (IOCB_DIRECT | IOCB_DAX)) { loff_t pos, endbyte; written = generic_file_direct_write(iocb, from, iocb->ki_pos);
All IO in a dax filesystem used to go through dax_do_io, which cannot handle media errors, and thus cannot provide a recovery path that can send a write through the driver to clear errors. Add a new iocb flag for DAX, and set it only for DAX mounts. In the IO path for DAX filesystems, use the same direct_IO path for both DAX and direct_io iocbs, but use the flags to identify when we are in O_DIRECT mode vs non O_DIRECT with DAX, and for O_DIRECT, use the conventional direct_IO path instead of DAX. This allows us a recovery path in the form of opening the file with O_DIRECT and writing to it with the usual O_DIRECT semantics (sector alignment restrictions). Cc: Matthew Wilcox <matthew@wil.cx> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Ross Zwisler <ross.zwisler@linux.intel.com> Cc: Jeff Moyer <jmoyer@redhat.com> Cc: Christoph Hellwig <hch@infradead.org> Cc: Dave Chinner <david@fromorbit.com> Cc: Jan Kara <jack@suse.cz> Cc: Jens Axboe <axboe@fb.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com> --- drivers/block/loop.c | 2 +- fs/block_dev.c | 17 +++++++++++++---- fs/ext2/inode.c | 16 ++++++++++++---- fs/ext4/file.c | 2 +- fs/ext4/inode.c | 19 +++++++++++++------ fs/xfs/xfs_aops.c | 20 +++++++++++++------- fs/xfs/xfs_file.c | 4 ++-- include/linux/fs.h | 15 ++++++++++++--- mm/filemap.c | 4 ++-- 9 files changed, 69 insertions(+), 30 deletions(-)