From patchwork Thu Jan 21 08:58:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25E0AC433DB for ; Thu, 21 Jan 2021 09:06:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8F1F22B51 for ; Thu, 21 Jan 2021 09:06:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728507AbhAUJG0 (ORCPT ); Thu, 21 Jan 2021 04:06:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728458AbhAUJEs (ORCPT ); Thu, 21 Jan 2021 04:04:48 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37C47C061575; Thu, 21 Jan 2021 01:04:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=88lk0T+xwMLbkob9NIGxStlrrZWA3V58cXPReaPHcJQ=; b=fwkCHBwoj5wYP12+P6mGVzzq6l TQ4MMVBRqOFu88rkpAbSMeaqVi6raPo+ti3vL0rxBcQyPaXupVL1Fkp9Xu4xMXdwyHEs3MaV1KeSw okZ4ft3vcc9RaHodZ85jocjahO3GsGu4IlkITyCsUtgaRcrzg3Th4AT8qQ0W3xWNB4cfIX5AlgIfi PQflKcRTXkAbs5T9T04UmyhSgCF8tlXZ3UXg0XrNUZNADaLqiDune89QOs/Eskg5Ayz8Lr018QZuZ h2FxJv4GeyHOO61j3pOP5UDryegWGxyVTAFGFGqEOl84JN8dGQm4OWWIEi270FwwUbWMNo8p8csUE LKe7036A==; Received: from 089144206130.atnat0015.highway.bob.at ([89.144.206.130] helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2VsZ-00Gq7R-0w; Thu, 21 Jan 2021 09:03:58 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 01/11] xfs: factor out a xfs_ilock_iocb helper Date: Thu, 21 Jan 2021 09:58:56 +0100 Message-Id: <20210121085906.322712-2-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a helper to factor out the nowait locking logical for the read/write helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 55 +++++++++++++++++++++++++---------------------- 1 file changed, 29 insertions(+), 26 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 5b0f93f738372d..c441cddfa4acbc 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -197,6 +197,23 @@ xfs_file_fsync( return error; } +static int +xfs_ilock_iocb( + struct kiocb *iocb, + unsigned int lock_mode) +{ + struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); + + if (iocb->ki_flags & IOCB_NOWAIT) { + if (!xfs_ilock_nowait(ip, lock_mode)) + return -EAGAIN; + } else { + xfs_ilock(ip, lock_mode); + } + + return 0; +} + STATIC ssize_t xfs_file_dio_aio_read( struct kiocb *iocb, @@ -213,12 +230,9 @@ xfs_file_dio_aio_read( file_accessed(iocb->ki_filp); - if (iocb->ki_flags & IOCB_NOWAIT) { - if (!xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED)) - return -EAGAIN; - } else { - xfs_ilock(ip, XFS_IOLOCK_SHARED); - } + ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + if (ret) + return ret; ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, is_sync_kiocb(iocb)); xfs_iunlock(ip, XFS_IOLOCK_SHARED); @@ -240,13 +254,9 @@ xfs_file_dax_read( if (!count) return 0; /* skip atime */ - if (iocb->ki_flags & IOCB_NOWAIT) { - if (!xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED)) - return -EAGAIN; - } else { - xfs_ilock(ip, XFS_IOLOCK_SHARED); - } - + ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + if (ret) + return ret; ret = dax_iomap_rw(iocb, to, &xfs_read_iomap_ops); xfs_iunlock(ip, XFS_IOLOCK_SHARED); @@ -264,12 +274,9 @@ xfs_file_buffered_aio_read( trace_xfs_file_buffered_read(ip, iov_iter_count(to), iocb->ki_pos); - if (iocb->ki_flags & IOCB_NOWAIT) { - if (!xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED)) - return -EAGAIN; - } else { - xfs_ilock(ip, XFS_IOLOCK_SHARED); - } + ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + if (ret) + return ret; ret = generic_file_read_iter(iocb, to); xfs_iunlock(ip, XFS_IOLOCK_SHARED); @@ -608,13 +615,9 @@ xfs_file_dax_write( size_t count; loff_t pos; - if (iocb->ki_flags & IOCB_NOWAIT) { - if (!xfs_ilock_nowait(ip, iolock)) - return -EAGAIN; - } else { - xfs_ilock(ip, iolock); - } - + ret = xfs_ilock_iocb(iocb, iolock); + if (ret) + return ret; ret = xfs_file_aio_write_checks(iocb, from, &iolock); if (ret) goto out; From patchwork Thu Jan 21 08:58:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E9C1C433E0 for ; Thu, 21 Jan 2021 09:07:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E41522343E for ; Thu, 21 Jan 2021 09:07:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725829AbhAUJHq (ORCPT ); Thu, 21 Jan 2021 04:07:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727249AbhAUJHH (ORCPT ); Thu, 21 Jan 2021 04:07:07 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86891C061757; Thu, 21 Jan 2021 01:06:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=yMXBD3uXS7Xr+YNrdm80QvI+Gbfd0g98bpw+sBpq1p8=; b=W5lizb3TrKzA16PTK2jeAA9iFj DT/5IESercQb23bN0lea+YIgGzZ0AT3wqRTWvS4tDQcFkVr7duG6X5yFrLb+ip8JdvvRjOT0lH6y+ 5lELhpnWNuX4lVBd0KasgQugkzSmzA4DZPyiZjccRJH8/vQ6bRnIW5xA8T0SlnYESVQH7eCbmgWSL CnfiVlNGGnA7D5ImfPpYxvm2a8khBORh3IYJEgTHvkZVPuPYTqUZQKGqRHEMPEoBnv8yi027UJk7O yRwxJAsB9XYOZS/Mb8vGhTaPtuSG6g83iePn5AduqidvEtkzCjEz2kxXYiOCwPr0t0FI7nCEVhHOx hueqo5Ng==; Received: from 089144206130.atnat0015.highway.bob.at ([89.144.206.130] helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2Vuy-00GqLe-J0; Thu, 21 Jan 2021 09:06:22 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 02/11] xfs: make xfs_file_aio_write_checks IOCB_NOWAIT-aware Date: Thu, 21 Jan 2021 09:58:57 +0100 Message-Id: <20210121085906.322712-3-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Ensure we don't block on the iolock, or waiting for I/O in xfs_file_aio_write_checks if the caller asked to avoid that. Fixes: 29a5d29ec181 ("xfs: nowait aio support") Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index c441cddfa4acbc..fb4e6f2852bb8b 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -335,7 +335,14 @@ xfs_file_aio_write_checks( if (error <= 0) return error; - error = xfs_break_layouts(inode, iolock, BREAK_WRITE); + if (iocb->ki_flags & IOCB_NOWAIT) { + error = break_layout(inode, false); + if (error == -EWOULDBLOCK) + error = -EAGAIN; + } else { + error = xfs_break_layouts(inode, iolock, BREAK_WRITE); + } + if (error) return error; @@ -346,7 +353,11 @@ xfs_file_aio_write_checks( if (*iolock == XFS_IOLOCK_SHARED && !IS_NOSEC(inode)) { xfs_iunlock(ip, *iolock); *iolock = XFS_IOLOCK_EXCL; - xfs_ilock(ip, *iolock); + error = xfs_ilock_iocb(iocb, *iolock); + if (error) { + *iolock = 0; + return error; + } goto restart; } /* @@ -368,6 +379,10 @@ xfs_file_aio_write_checks( isize = i_size_read(inode); if (iocb->ki_pos > isize) { spin_unlock(&ip->i_flags_lock); + + if (iocb->ki_flags & IOCB_NOWAIT) + return -EAGAIN; + if (!drained_dio) { if (*iolock == XFS_IOLOCK_SHARED) { xfs_iunlock(ip, *iolock); @@ -593,7 +608,8 @@ xfs_file_dio_aio_write( &xfs_dio_write_ops, is_sync_kiocb(iocb) || unaligned_io); out: - xfs_iunlock(ip, iolock); + if (iolock) + xfs_iunlock(ip, iolock); /* * No fallback to buffered IO after short writes for XFS, direct I/O @@ -632,7 +648,8 @@ xfs_file_dax_write( error = xfs_setfilesize(ip, pos, ret); } out: - xfs_iunlock(ip, iolock); + if (iolock) + xfs_iunlock(ip, iolock); if (error) return error; From patchwork Thu Jan 21 08:58:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C932EC433E6 for ; Thu, 21 Jan 2021 09:27:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 718212395A for ; Thu, 21 Jan 2021 09:27:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726427AbhAUJKJ (ORCPT ); Thu, 21 Jan 2021 04:10:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728150AbhAUJJw (ORCPT ); Thu, 21 Jan 2021 04:09:52 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A07C4C061575; Thu, 21 Jan 2021 01:09:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=7yDv+i0eIg8ZvTjuDWGz32b4qM8yWB/b7IST0cofvso=; b=FZ1kFHbfoe6Eijy1a6AQSl0HSk PiUtI+lXZh1Xa03Neap3pBwAPPUIAYM3tkbjjTkYWEAXnu6O2rq1SUrAtN1alefsjj3TU0Lk9ZLSo ASTa5IO8JsH/JvrHIzwogE8ClZednMGKMG+W/Adyd0pzOtwzHKghvdM3+6bgyY0sYypEHmyw6dMo+ v0YYr6pJfry15Af1CvSc0DpUVsvup9ZKTAKtP7+CRO9hdsjQaD3u4iL2+30F2bf0pc9y8f2w7qgT1 kUrAAyeF44xRrM2BEP1c2OkxELH1rmKwklaUbHLDDaK7KCiJYcqrIHeeO61NPNV/mIFiYpipNiD21 mocMK3mQ==; Received: from 089144206130.atnat0015.highway.bob.at ([89.144.206.130] helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2VxD-00GqVK-Q8; Thu, 21 Jan 2021 09:08:51 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 03/11] xfs: cleanup the read/write helper naming Date: Thu, 21 Jan 2021 09:58:58 +0100 Message-Id: <20210121085906.322712-4-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Drop a few pointless aio_ prefixes. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index fb4e6f2852bb8b..ae7313ccaa11ed 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -215,7 +215,7 @@ xfs_ilock_iocb( } STATIC ssize_t -xfs_file_dio_aio_read( +xfs_file_dio_read( struct kiocb *iocb, struct iov_iter *to) { @@ -265,7 +265,7 @@ xfs_file_dax_read( } STATIC ssize_t -xfs_file_buffered_aio_read( +xfs_file_buffered_read( struct kiocb *iocb, struct iov_iter *to) { @@ -300,9 +300,9 @@ xfs_file_read_iter( if (IS_DAX(inode)) ret = xfs_file_dax_read(iocb, to); else if (iocb->ki_flags & IOCB_DIRECT) - ret = xfs_file_dio_aio_read(iocb, to); + ret = xfs_file_dio_read(iocb, to); else - ret = xfs_file_buffered_aio_read(iocb, to); + ret = xfs_file_buffered_read(iocb, to); if (ret > 0) XFS_STATS_ADD(mp, xs_read_bytes, ret); @@ -317,7 +317,7 @@ xfs_file_read_iter( * if called for a direct write beyond i_size. */ STATIC ssize_t -xfs_file_aio_write_checks( +xfs_file_write_checks( struct kiocb *iocb, struct iov_iter *from, int *iolock) @@ -502,7 +502,7 @@ static const struct iomap_dio_ops xfs_dio_write_ops = { }; /* - * xfs_file_dio_aio_write - handle direct IO writes + * xfs_file_dio_write - handle direct IO writes * * Lock the inode appropriately to prepare for and issue a direct IO write. * By separating it from the buffered write path we remove all the tricky to @@ -527,7 +527,7 @@ static const struct iomap_dio_ops xfs_dio_write_ops = { * negative return values. */ STATIC ssize_t -xfs_file_dio_aio_write( +xfs_file_dio_write( struct kiocb *iocb, struct iov_iter *from) { @@ -549,7 +549,7 @@ xfs_file_dio_aio_write( /* * Don't take the exclusive iolock here unless the I/O is unaligned to * the file system block size. We don't need to consider the EOF - * extension case here because xfs_file_aio_write_checks() will relock + * extension case here because xfs_file_write_checks() will relock * the inode as necessary for EOF zeroing cases and fill out the new * inode size as appropriate. */ @@ -580,7 +580,7 @@ xfs_file_dio_aio_write( xfs_ilock(ip, iolock); } - ret = xfs_file_aio_write_checks(iocb, from, &iolock); + ret = xfs_file_write_checks(iocb, from, &iolock); if (ret) goto out; count = iov_iter_count(from); @@ -590,7 +590,7 @@ xfs_file_dio_aio_write( * in-flight at the same time or we risk data corruption. Wait for all * other IO to drain before we submit. If the IO is aligned, demote the * iolock if we had to take the exclusive lock in - * xfs_file_aio_write_checks() for other reasons. + * xfs_file_write_checks() for other reasons. */ if (unaligned_io) { inode_dio_wait(inode); @@ -634,7 +634,7 @@ xfs_file_dax_write( ret = xfs_ilock_iocb(iocb, iolock); if (ret) return ret; - ret = xfs_file_aio_write_checks(iocb, from, &iolock); + ret = xfs_file_write_checks(iocb, from, &iolock); if (ret) goto out; @@ -663,7 +663,7 @@ xfs_file_dax_write( } STATIC ssize_t -xfs_file_buffered_aio_write( +xfs_file_buffered_write( struct kiocb *iocb, struct iov_iter *from) { @@ -682,7 +682,7 @@ xfs_file_buffered_aio_write( iolock = XFS_IOLOCK_EXCL; xfs_ilock(ip, iolock); - ret = xfs_file_aio_write_checks(iocb, from, &iolock); + ret = xfs_file_write_checks(iocb, from, &iolock); if (ret) goto out; @@ -769,12 +769,12 @@ xfs_file_write_iter( * CoW. In all other directio scenarios we do not * allow an operation to fall back to buffered mode. */ - ret = xfs_file_dio_aio_write(iocb, from); + ret = xfs_file_dio_write(iocb, from); if (ret != -ENOTBLK) return ret; } - return xfs_file_buffered_aio_write(iocb, from); + return xfs_file_buffered_write(iocb, from); } static void From patchwork Thu Jan 21 08:58:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 227C8C433E0 for ; Thu, 21 Jan 2021 09:10:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D47D9238E7 for ; Thu, 21 Jan 2021 09:10:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728224AbhAUJKS (ORCPT ); Thu, 21 Jan 2021 04:10:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728171AbhAUJKA (ORCPT ); Thu, 21 Jan 2021 04:10:00 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B0DDC061757; Thu, 21 Jan 2021 01:09:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ym1CuL3+wqw+X8HU/55ILNdm6HWcfbAaWLvIFZ0mv1o=; b=nAqS2uGSPWn5NU7UYNNs4o9TK9 i6LhP8Wa5mlAM3xsbIt1PXTbH3LcXUgrfGe+TkHb+6z/O8ErPgievz6s0mSX8fVq8vcx2zXACNi2L NAz3QFdwewAWEo9pKNHxSU7eFugg/Xsno7DqC67U9QLff6PvZHxae/H1CnCmYCHUku8j0HIClEtpn AjQL/35yS4hjUAnuuxaiy9ZJk+Xn4Cd/BDHXUSCcB3v0543nC4kExuSQs6U9ABNz0E7NAgu9MjYED b07bCtOvMvSbqRuNPmIWABpbgafpUVIN8CGZ98hK4INAwl5CcdhSuPbSMsgzv7nAeDKz1dfAdJja5 phNeJIhw==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2Vxg-00GqWX-0J; Thu, 21 Jan 2021 09:09:05 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 04/11] xfs: remove the buffered I/O fallback assert Date: Thu, 21 Jan 2021 09:58:59 +0100 Message-Id: <20210121085906.322712-5-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The iomap code has been designed from the start not to do magic fallback, so remove the assert in preparation for further code cleanups. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index ae7313ccaa11ed..97836ec53397d4 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -610,12 +610,6 @@ xfs_file_dio_write( out: if (iolock) xfs_iunlock(ip, iolock); - - /* - * No fallback to buffered IO after short writes for XFS, direct I/O - * will either complete fully or return an error. - */ - ASSERT(ret < 0 || ret == count); return ret; } From patchwork Thu Jan 21 08:59:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B771C433DB for ; Thu, 21 Jan 2021 09:10:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3A365235E4 for ; Thu, 21 Jan 2021 09:10:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728341AbhAUJKh (ORCPT ); Thu, 21 Jan 2021 04:10:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728176AbhAUJKE (ORCPT ); Thu, 21 Jan 2021 04:10:04 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66753C0613C1; Thu, 21 Jan 2021 01:09:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=t4QPBONQe1EUoY6MBK77KtrA9WQl5HRRm2WEDJnTyyU=; b=DPH1s9BMnJwQ8JQnM83otOZfsw i+FKhXSUYC3TTNe0I6eKUalUF0wbOYzMteCS8+jresknFXfSxJPjqoVagGTKmfYlqLfK1O8YgkeQx DGaRPUNNxW3/3cXs3GOGfrKmWqHXEnpgGwbuuPs2aPTUJFKdPxLfln5OfxYEsRLE7QDe+aIYEyjjG tJvHlLrD9K1kpcrGT/FtDaKRk41IosnhpDiLV7MCuZ9j78zFm9iOSbn9Lc7mI1GmNQkVWgkUm4Kin nFM8kcPT42bvaBzQZ5ttRasBRRZ9gLcaV3uCmHzB4AVCkh2dlJ5+p4/hep2VGnSU0mRP77rR3QQP7 mGQePQ8A==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2Vxs-00GqWw-73; Thu, 21 Jan 2021 09:09:18 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 05/11] xfs: simplify the read/write tracepoints Date: Thu, 21 Jan 2021 09:59:00 +0100 Message-Id: <20210121085906.322712-6-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Pass the iocb and iov_iter to the tracepoints and leave decoding of actual arguments to the code only run when tracing is enabled. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 20 ++++++++------------ fs/xfs/xfs_trace.h | 18 +++++++++--------- 2 files changed, 17 insertions(+), 21 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 97836ec53397d4..aa64e78fc3c467 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -220,12 +220,11 @@ xfs_file_dio_read( struct iov_iter *to) { struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); - size_t count = iov_iter_count(to); ssize_t ret; - trace_xfs_file_direct_read(ip, count, iocb->ki_pos); + trace_xfs_file_direct_read(iocb, to); - if (!count) + if (!iov_iter_count(to)) return 0; /* skip atime */ file_accessed(iocb->ki_filp); @@ -246,12 +245,11 @@ xfs_file_dax_read( struct iov_iter *to) { struct xfs_inode *ip = XFS_I(iocb->ki_filp->f_mapping->host); - size_t count = iov_iter_count(to); ssize_t ret = 0; - trace_xfs_file_dax_read(ip, count, iocb->ki_pos); + trace_xfs_file_dax_read(iocb, to); - if (!count) + if (!iov_iter_count(to)) return 0; /* skip atime */ ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); @@ -272,7 +270,7 @@ xfs_file_buffered_read( struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); ssize_t ret; - trace_xfs_file_buffered_read(ip, iov_iter_count(to), iocb->ki_pos); + trace_xfs_file_buffered_read(iocb, to); ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); if (ret) @@ -599,7 +597,7 @@ xfs_file_dio_write( iolock = XFS_IOLOCK_SHARED; } - trace_xfs_file_direct_write(ip, count, iocb->ki_pos); + trace_xfs_file_direct_write(iocb, from); /* * If unaligned, this is the only IO in-flight. Wait on it before we * release the iolock to prevent subsequent overlapping IO. @@ -622,7 +620,6 @@ xfs_file_dax_write( struct xfs_inode *ip = XFS_I(inode); int iolock = XFS_IOLOCK_EXCL; ssize_t ret, error = 0; - size_t count; loff_t pos; ret = xfs_ilock_iocb(iocb, iolock); @@ -633,9 +630,8 @@ xfs_file_dax_write( goto out; pos = iocb->ki_pos; - count = iov_iter_count(from); - trace_xfs_file_dax_write(ip, count, pos); + trace_xfs_file_dax_write(iocb, from); ret = dax_iomap_rw(iocb, from, &xfs_direct_write_iomap_ops); if (ret > 0 && iocb->ki_pos > i_size_read(inode)) { i_size_write(inode, iocb->ki_pos); @@ -683,7 +679,7 @@ xfs_file_buffered_write( /* We can write back this queue in page reclaim */ current->backing_dev_info = inode_to_bdi(inode); - trace_xfs_file_buffered_write(ip, iov_iter_count(from), iocb->ki_pos); + trace_xfs_file_buffered_write(iocb, from); ret = iomap_file_buffered_write(iocb, from, &xfs_buffered_write_iomap_ops); if (likely(ret >= 0)) diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index 5a263ae3d4f008..a6d04d860a565e 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -1287,8 +1287,8 @@ TRACE_EVENT(xfs_log_assign_tail_lsn, ) DECLARE_EVENT_CLASS(xfs_file_class, - TP_PROTO(struct xfs_inode *ip, size_t count, loff_t offset), - TP_ARGS(ip, count, offset), + TP_PROTO(struct kiocb *iocb, struct iov_iter *iter), + TP_ARGS(iocb, iter), TP_STRUCT__entry( __field(dev_t, dev) __field(xfs_ino_t, ino) @@ -1297,11 +1297,11 @@ DECLARE_EVENT_CLASS(xfs_file_class, __field(size_t, count) ), TP_fast_assign( - __entry->dev = VFS_I(ip)->i_sb->s_dev; - __entry->ino = ip->i_ino; - __entry->size = ip->i_d.di_size; - __entry->offset = offset; - __entry->count = count; + __entry->dev = file_inode(iocb->ki_filp)->i_sb->s_dev; + __entry->ino = XFS_I(file_inode(iocb->ki_filp))->i_ino; + __entry->size = XFS_I(file_inode(iocb->ki_filp))->i_d.di_size; + __entry->offset = iocb->ki_pos; + __entry->count = iov_iter_count(iter); ), TP_printk("dev %d:%d ino 0x%llx size 0x%llx offset 0x%llx count 0x%zx", MAJOR(__entry->dev), MINOR(__entry->dev), @@ -1313,8 +1313,8 @@ DECLARE_EVENT_CLASS(xfs_file_class, #define DEFINE_RW_EVENT(name) \ DEFINE_EVENT(xfs_file_class, name, \ - TP_PROTO(struct xfs_inode *ip, size_t count, loff_t offset), \ - TP_ARGS(ip, count, offset)) + TP_PROTO(struct kiocb *iocb, struct iov_iter *iter), \ + TP_ARGS(iocb, iter)) DEFINE_RW_EVENT(xfs_file_buffered_read); DEFINE_RW_EVENT(xfs_file_direct_read); DEFINE_RW_EVENT(xfs_file_dax_read); From patchwork Thu Jan 21 08:59:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FD6CC433DB for ; Thu, 21 Jan 2021 09:21:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA465239A1 for ; Thu, 21 Jan 2021 09:21:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728269AbhAUJVR (ORCPT ); Thu, 21 Jan 2021 04:21:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728222AbhAUJKQ (ORCPT ); Thu, 21 Jan 2021 04:10:16 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E20D8C0613CF; Thu, 21 Jan 2021 01:09:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=irLuJu46wFlvKBuIGQmEQWUaIHSxZIK3WfK+r5wo/RA=; b=bz0N5nERLIkMD78jT17ei6m69e FbhWn87c1XEz5f9a5bOs5m2Z6MP/P9agrkF5y4LujB46Omzr1GKR5rL3OoA9a6bNCIj0g4z/NU4k7 mD61nNUO2+bJqJJkDXiI3EOAdZ++LK5QIh1P6ess+tj9dWWl8WPJOAUQjhG+dLcQSdGzPUStwgfGe ZJ08Lx9qHFjO5l1J/ZAINWysSE+g1mQC8TTYWZ2LfjMI//ApjFxPm/AreUuUcC0RG+JbpwEAaPat4 y8g7MtgjQkb8keCV3k9j13YYjtFV+yhRBrXUKZxIQFEosfwWWJr67wMotLmsfMmNeNvTE1Nm/QZ3I 9rY4Akkw==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2Vy0-00GqX9-1a; Thu, 21 Jan 2021 09:09:26 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 06/11] xfs: improve the reflink_bounce_dio_write tracepoint Date: Thu, 21 Jan 2021 09:59:01 +0100 Message-Id: <20210121085906.322712-7-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Use a more suitable event class. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 2 +- fs/xfs/xfs_trace.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index aa64e78fc3c467..a696bd34f71d21 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -560,7 +560,7 @@ xfs_file_dio_write( * files yet, as we can't unshare a partial block. */ if (xfs_is_cow_inode(ip)) { - trace_xfs_reflink_bounce_dio_write(ip, iocb->ki_pos, count); + trace_xfs_reflink_bounce_dio_write(iocb, from); return -ENOTBLK; } iolock = XFS_IOLOCK_EXCL; diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index a6d04d860a565e..0cfd65cd67c190 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -1321,6 +1321,8 @@ DEFINE_RW_EVENT(xfs_file_dax_read); DEFINE_RW_EVENT(xfs_file_buffered_write); DEFINE_RW_EVENT(xfs_file_direct_write); DEFINE_RW_EVENT(xfs_file_dax_write); +DEFINE_RW_EVENT(xfs_reflink_bounce_dio_write); + DECLARE_EVENT_CLASS(xfs_imap_class, TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count, @@ -3294,8 +3296,6 @@ DEFINE_INODE_IREC_EVENT(xfs_reflink_cow_found); DEFINE_INODE_IREC_EVENT(xfs_reflink_cow_enospc); DEFINE_INODE_IREC_EVENT(xfs_reflink_convert_cow); -DEFINE_SIMPLE_IO_EVENT(xfs_reflink_bounce_dio_write); - DEFINE_SIMPLE_IO_EVENT(xfs_reflink_cancel_cow_range); DEFINE_SIMPLE_IO_EVENT(xfs_reflink_end_cow); DEFINE_INODE_IREC_EVENT(xfs_reflink_cow_remap); From patchwork Thu Jan 21 08:59:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FE5FC433E0 for ; Thu, 21 Jan 2021 09:11:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 56CD023602 for ; Thu, 21 Jan 2021 09:11:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727047AbhAUJLC (ORCPT ); Thu, 21 Jan 2021 04:11:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728128AbhAUJKj (ORCPT ); Thu, 21 Jan 2021 04:10:39 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C8DFC0613D3; Thu, 21 Jan 2021 01:09:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=RohE3O4jp6xBmHfxCrVqtW9l+skkoRowNmJukDDdVSI=; b=tnbO/J1xElXjuoHAEuAHgUCnZv p0v7V12zDLGMfbMHqfiwNUbQDswOr4DKmLATgw8QhBUe/2RsHYruvzoR4v0/EB9R/LPEtLLD5cudC wvGmgV+GXfOQfaa6skpKvC/zWdeGMkmkMlfMxBDtpMf1zKpYtJs3O4DsCP0cSAKsjIbSEvEcg+2rK /xpO1XI17+M8qpm6qnqIjm4QKAeJ57+5P2iKI/TLS6IVHETRpQ8lL4grsJXekZTARhK6SinWYibLt PbOV5J7Xto5Pnm4hsj0ySxwU5g7ZgkryfCio1OknPmoT/YRdX9yZfFMxcxZDGEyxnyOyqD7mzoKWM Jb4R9uVg==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2VyB-00GqYE-Gk; Thu, 21 Jan 2021 09:09:38 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 07/11] xfs: split unaligned DIO write code out Date: Thu, 21 Jan 2021 09:59:02 +0100 Message-Id: <20210121085906.322712-8-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Dave Chinner The unaligned DIO write path is more convolted than the normal path, and we are about to make it more complex. Keep the block aligned fast path dio write code trim and simple by splitting out the unaligned DIO code from it. Signed-off-by: Dave Chinner [hch: rebased, fixed a few minor nits] Signed-off-by: Christoph Hellwig Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/xfs/xfs_file.c | 168 +++++++++++++++++++++++++--------------------- 1 file changed, 92 insertions(+), 76 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index a696bd34f71d21..bffd7240cefb7f 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -500,117 +500,133 @@ static const struct iomap_dio_ops xfs_dio_write_ops = { }; /* - * xfs_file_dio_write - handle direct IO writes + * Handle block aligned direct IO writes * * Lock the inode appropriately to prepare for and issue a direct IO write. - * By separating it from the buffered write path we remove all the tricky to - * follow locking changes and looping. * * If there are cached pages or we're extending the file, we need IOLOCK_EXCL * until we're sure the bytes at the new EOF have been zeroed and/or the cached * pages are flushed out. + */ +static noinline ssize_t +xfs_file_dio_write_aligned( + struct xfs_inode *ip, + struct kiocb *iocb, + struct iov_iter *from) +{ + int iolock = XFS_IOLOCK_SHARED; + ssize_t ret; + + ret = xfs_ilock_iocb(iocb, iolock); + if (ret) + return ret; + ret = xfs_file_write_checks(iocb, from, &iolock); + if (ret) + goto out_unlock; + + /* + * We don't need to hold the IOLOCK exclusively across the IO, so demote + * the iolock back to shared if we had to take the exclusive lock in + * xfs_file_write_checks() for other reasons. + */ + if (iolock == XFS_IOLOCK_EXCL) { + xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); + iolock = XFS_IOLOCK_SHARED; + } + trace_xfs_file_direct_write(iocb, from); + ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, + &xfs_dio_write_ops, is_sync_kiocb(iocb)); +out_unlock: + if (iolock) + xfs_iunlock(ip, iolock); + return ret; +} + +/* + * Handle block unaligned direct IO writes + * + * In most cases direct IO writes will be done holding IOLOCK_SHARED, allowing + * them to be done in parallel with reads and other direct IO writes. However, + * if the I/O is not aligned to filesystem blocks, the direct I/O layer may + * need to do sub-block zeroing and that requires serialisation against other + * direct I/Os to the same block. In this case we need to serialise the + * submission of the unaligned I/Os so that we don't get racing block zeroing in + * the dio layer. * - * In most cases the direct IO writes will be done holding IOLOCK_SHARED - * allowing them to be done in parallel with reads and other direct IO writes. - * However, if the IO is not aligned to filesystem blocks, the direct IO layer - * needs to do sub-block zeroing and that requires serialisation against other - * direct IOs to the same block. In this case we need to serialise the - * submission of the unaligned IOs so that we don't get racing block zeroing in - * the dio layer. To avoid the problem with aio, we also need to wait for + * To provide the same serialisation for AIO, we also need to wait for * outstanding IOs to complete so that unwritten extent conversion is completed * before we try to map the overlapping block. This is currently implemented by * hitting it with a big hammer (i.e. inode_dio_wait()). * - * Returns with locks held indicated by @iolock and errors indicated by - * negative return values. + * This means that unaligned dio writes always block. There is no "nowait" fast + * path in this code - if IOCB_NOWAIT is set we simply return -EAGAIN up front + * and we don't have to worry about that anymore. */ -STATIC ssize_t -xfs_file_dio_write( +static noinline ssize_t +xfs_file_dio_write_unaligned( + struct xfs_inode *ip, struct kiocb *iocb, struct iov_iter *from) { - struct file *file = iocb->ki_filp; - struct address_space *mapping = file->f_mapping; - struct inode *inode = mapping->host; - struct xfs_inode *ip = XFS_I(inode); - struct xfs_mount *mp = ip->i_mount; - ssize_t ret = 0; - int unaligned_io = 0; - int iolock; - size_t count = iov_iter_count(from); - struct xfs_buftarg *target = xfs_inode_buftarg(ip); + int iolock = XFS_IOLOCK_EXCL; + ssize_t ret; - /* DIO must be aligned to device logical sector size */ - if ((iocb->ki_pos | count) & target->bt_logical_sectormask) - return -EINVAL; + /* unaligned dio always waits, bail */ + if (iocb->ki_flags & IOCB_NOWAIT) + return -EAGAIN; + xfs_ilock(ip, iolock); /* - * Don't take the exclusive iolock here unless the I/O is unaligned to - * the file system block size. We don't need to consider the EOF - * extension case here because xfs_file_write_checks() will relock - * the inode as necessary for EOF zeroing cases and fill out the new - * inode size as appropriate. + * We can't properly handle unaligned direct I/O to reflink files yet, + * as we can't unshare a partial block. */ - if ((iocb->ki_pos & mp->m_blockmask) || - ((iocb->ki_pos + count) & mp->m_blockmask)) { - unaligned_io = 1; - - /* - * We can't properly handle unaligned direct I/O to reflink - * files yet, as we can't unshare a partial block. - */ - if (xfs_is_cow_inode(ip)) { - trace_xfs_reflink_bounce_dio_write(iocb, from); - return -ENOTBLK; - } - iolock = XFS_IOLOCK_EXCL; - } else { - iolock = XFS_IOLOCK_SHARED; - } - - if (iocb->ki_flags & IOCB_NOWAIT) { - /* unaligned dio always waits, bail */ - if (unaligned_io) - return -EAGAIN; - if (!xfs_ilock_nowait(ip, iolock)) - return -EAGAIN; - } else { - xfs_ilock(ip, iolock); + if (xfs_is_cow_inode(ip)) { + trace_xfs_reflink_bounce_dio_write(iocb, from); + ret = -ENOTBLK; + goto out_unlock; } ret = xfs_file_write_checks(iocb, from, &iolock); if (ret) - goto out; - count = iov_iter_count(from); + goto out_unlock; /* - * If we are doing unaligned IO, we can't allow any other overlapping IO - * in-flight at the same time or we risk data corruption. Wait for all - * other IO to drain before we submit. If the IO is aligned, demote the - * iolock if we had to take the exclusive lock in - * xfs_file_write_checks() for other reasons. + * If we are doing unaligned I/O, we can't allow any other overlapping + * I/O in-flight at the same time or we risk data corruption. Wait for + * all other I/O to drain before we submit. */ - if (unaligned_io) { - inode_dio_wait(inode); - } else if (iolock == XFS_IOLOCK_EXCL) { - xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); - iolock = XFS_IOLOCK_SHARED; - } + inode_dio_wait(VFS_I(ip)); - trace_xfs_file_direct_write(iocb, from); /* - * If unaligned, this is the only IO in-flight. Wait on it before we - * release the iolock to prevent subsequent overlapping IO. + * This must be the only I/O in-flight. Wait on it before we release the + * iolock to prevent subsequent overlapping I/O. */ + trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, - is_sync_kiocb(iocb) || unaligned_io); -out: + &xfs_dio_write_ops, true); +out_unlock: if (iolock) xfs_iunlock(ip, iolock); return ret; } +static ssize_t +xfs_file_dio_write( + struct kiocb *iocb, + struct iov_iter *from) +{ + struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); + struct xfs_buftarg *target = xfs_inode_buftarg(ip); + size_t count = iov_iter_count(from); + + /* DIO must be aligned to device logical sector size */ + if ((iocb->ki_pos | count) & target->bt_logical_sectormask) + return -EINVAL; + if ((iocb->ki_pos | count) & ip->i_mount->m_blockmask) + return xfs_file_dio_write_unaligned(ip, iocb, from); + return xfs_file_dio_write_aligned(ip, iocb, from); +} + static noinline ssize_t xfs_file_dax_write( struct kiocb *iocb, From patchwork Thu Jan 21 08:59:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAB31C433DB for ; Thu, 21 Jan 2021 09:12:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6399F2395A for ; Thu, 21 Jan 2021 09:12:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728295AbhAUJMB (ORCPT ); Thu, 21 Jan 2021 04:12:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728354AbhAUJLY (ORCPT ); Thu, 21 Jan 2021 04:11:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55722C061793; Thu, 21 Jan 2021 01:10:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=i6Dde36ggmoorkkDA8BX+rEq6/brHc3OpzN1LILZN5k=; b=JKIL0Z5NZve0hPKuRlGDntJk75 +cYDthyvh3XVQ/JMF4HpGw891LUyPPodc+1qpujwjLl/8+e0FrLFyKcxi5nSIsBYroE08OX8GbZ5f L3AfD/Fm0rv0T4NJ4RiqIXstnKIzUnJzr6heqSxw1WWGB3N5XEHgDBp//47XaL22z/EeNIukOTSEv DiMqPPEvdDEeG9dM41EkjJlAuCsKWP+m0UmMk1UUmz1oqwppGtXYZvUYc2PwceOIxg8vchzmHUmtQ 5uuwfn377J5acAhu/iY9cr2zhlR9z2UqFhLdhRVh4y4iOvYaO8tF5/wmu7GbjrKteOVPyjM1tU6a/ 1y1Lun+A==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2VyV-00GqZr-4U; Thu, 21 Jan 2021 09:10:15 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner , Brian Foster , "Darrick J . Wong" Subject: [PATCH 08/11] iomap: rename the flags variable in __iomap_dio_rw Date: Thu, 21 Jan 2021 09:59:03 +0100 Message-Id: <20210121085906.322712-9-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Rename flags to iomap_flags to make the usage a little more clear. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/iomap/direct-io.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 933f234d5becd0..604103ab76f9c5 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -427,7 +427,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, size_t count = iov_iter_count(iter); loff_t pos = iocb->ki_pos; loff_t end = iocb->ki_pos + count - 1, ret = 0; - unsigned int flags = IOMAP_DIRECT; + unsigned int iomap_flags = IOMAP_DIRECT; struct blk_plug plug; struct iomap_dio *dio; @@ -461,7 +461,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, if (iter_is_iovec(iter)) dio->flags |= IOMAP_DIO_DIRTY; } else { - flags |= IOMAP_WRITE; + iomap_flags |= IOMAP_WRITE; dio->flags |= IOMAP_DIO_WRITE; /* for data sync or sync, we need sync completion processing */ @@ -483,7 +483,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, ret = -EAGAIN; goto out_free_dio; } - flags |= IOMAP_NOWAIT; + iomap_flags |= IOMAP_NOWAIT; } ret = filemap_write_and_wait_range(mapping, pos, end); @@ -514,7 +514,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, blk_start_plug(&plug); do { - ret = iomap_apply(inode, pos, count, flags, ops, dio, + ret = iomap_apply(inode, pos, count, iomap_flags, ops, dio, iomap_dio_actor); if (ret <= 0) { /* magic error code to fall back to buffered I/O */ From patchwork Thu Jan 21 08:59:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12289C433E0 for ; Thu, 21 Jan 2021 09:14:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8518238EC for ; Thu, 21 Jan 2021 09:14:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728394AbhAUJNm (ORCPT ); Thu, 21 Jan 2021 04:13:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728413AbhAUJLw (ORCPT ); Thu, 21 Jan 2021 04:11:52 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6774C0613D3; Thu, 21 Jan 2021 01:11:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=RPXWIQ1ILqPDeZnJGfxTXl3MvEeRfc/WF99yNxToSQU=; b=eEizG3+/DBlnm6Jx+kNIIg67fx JkDNOrJI6eNRRJ4KFvZ9aBqQOrjGLdhoaVMFbKncj3AFgOTw+LBUMIyhOk2Eye8Z6TEgHEIZLi+Xi lVb8D+La+1nDchrO77uL82MKFrjCk3o9QP+vH7SZ+vRXzvHVELiugAAy7gWbY7zQedMNgje+/nI0V o0H0WfmUEhTZndamr2oZzbrpr3UAWNwft2yX1gvrUxAdcx0GX21IrDIeAiWzlhXdOV6LzREtwSh/2 Wk9XuJ82XYosqQl55xh3kW5oP7o28Jcg8/JlijS1RyzaIPY7ni2l7lnggTbqyhNdcqaI8DlX/UUzz +5heMeUA==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2VzH-00GqgR-Uv; Thu, 21 Jan 2021 09:10:51 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Brian Foster Subject: [PATCH 09/11] iomap: pass a flags argument to iomap_dio_rw Date: Thu, 21 Jan 2021 09:59:04 +0100 Message-Id: <20210121085906.322712-10-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Pass a set of flags to iomap_dio_rw instead of the boolean wait_for_completion argument. The IOMAP_DIO_FORCE_WAIT flag replaces the wait_for_completion, but only needs to be passed when the iocb isn't synchronous to start with to simplify the callers. Signed-off-by: Christoph Hellwig Reviewed-by: Brian Foster Reviewed-by: Dave Chinner Reviewed-by: Darrick J. Wong --- fs/btrfs/file.c | 7 +++---- fs/ext4/file.c | 5 ++--- fs/gfs2/file.c | 7 ++----- fs/iomap/direct-io.c | 11 +++++------ fs/xfs/xfs_file.c | 7 +++---- fs/zonefs/super.c | 4 ++-- include/linux/iomap.h | 10 ++++++++-- 7 files changed, 25 insertions(+), 26 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 0e41459b8de667..ddfd2e2adedf58 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1949,8 +1949,8 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) goto buffered; } - dio = __iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, - &btrfs_dio_ops, is_sync_kiocb(iocb)); + dio = __iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, + 0); btrfs_inode_unlock(inode, ilock_flags); @@ -3622,8 +3622,7 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to) return 0; btrfs_inode_lock(inode, BTRFS_ILOCK_SHARED); - ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - is_sync_kiocb(iocb)); + ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, 0); btrfs_inode_unlock(inode, BTRFS_ILOCK_SHARED); return ret; } diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 349b27f0dda0cb..194f5d00fa3267 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -74,8 +74,7 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) return generic_file_read_iter(iocb, to); } - ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, - is_sync_kiocb(iocb)); + ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0); inode_unlock_shared(inode); file_accessed(iocb->ki_filp); @@ -550,7 +549,7 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) if (ilock_shared) iomap_ops = &ext4_iomap_overwrite_ops; ret = iomap_dio_rw(iocb, from, iomap_ops, &ext4_dio_write_ops, - is_sync_kiocb(iocb) || unaligned_io || extend); + (unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0); if (ret == -ENOTBLK) ret = 0; diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index b39b339feddc93..89609c2997177a 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -797,9 +797,7 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to, if (ret) goto out_uninit; - ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, - is_sync_kiocb(iocb)); - + ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, 0); gfs2_glock_dq(gh); out_uninit: gfs2_holder_uninit(gh); @@ -833,8 +831,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, if (offset + len > i_size_read(&ip->i_inode)) goto out; - ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, - is_sync_kiocb(iocb)); + ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, 0); if (ret == -ENOTBLK) ret = 0; out: diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 604103ab76f9c5..947343730e2c93 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -420,13 +420,15 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, struct iomap_dio * __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - bool wait_for_completion) + unsigned int dio_flags) { struct address_space *mapping = iocb->ki_filp->f_mapping; struct inode *inode = file_inode(iocb->ki_filp); size_t count = iov_iter_count(iter); loff_t pos = iocb->ki_pos; loff_t end = iocb->ki_pos + count - 1, ret = 0; + bool wait_for_completion = + is_sync_kiocb(iocb) || (dio_flags & IOMAP_DIO_FORCE_WAIT); unsigned int iomap_flags = IOMAP_DIRECT; struct blk_plug plug; struct iomap_dio *dio; @@ -434,9 +436,6 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, if (!count) return NULL; - if (WARN_ON(is_sync_kiocb(iocb) && !wait_for_completion)) - return ERR_PTR(-EIO); - dio = kmalloc(sizeof(*dio), GFP_KERNEL); if (!dio) return ERR_PTR(-ENOMEM); @@ -598,11 +597,11 @@ EXPORT_SYMBOL_GPL(__iomap_dio_rw); ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - bool wait_for_completion) + unsigned int dio_flags) { struct iomap_dio *dio; - dio = __iomap_dio_rw(iocb, iter, ops, dops, wait_for_completion); + dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags); if (IS_ERR_OR_NULL(dio)) return PTR_ERR_OR_ZERO(dio); return iomap_dio_complete(dio); diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index bffd7240cefb7f..b181db42f2f32f 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -232,8 +232,7 @@ xfs_file_dio_read( ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); if (ret) return ret; - ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, - is_sync_kiocb(iocb)); + ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0); xfs_iunlock(ip, XFS_IOLOCK_SHARED); return ret; @@ -535,7 +534,7 @@ xfs_file_dio_write_aligned( } trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, is_sync_kiocb(iocb)); + &xfs_dio_write_ops, 0); out_unlock: if (iolock) xfs_iunlock(ip, iolock); @@ -603,7 +602,7 @@ xfs_file_dio_write_unaligned( */ trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, true); + &xfs_dio_write_ops, IOMAP_DIO_FORCE_WAIT); out_unlock: if (iolock) xfs_iunlock(ip, iolock); diff --git a/fs/zonefs/super.c b/fs/zonefs/super.c index bec47f2d074beb..0e7ab0bc00ae8e 100644 --- a/fs/zonefs/super.c +++ b/fs/zonefs/super.c @@ -780,7 +780,7 @@ static ssize_t zonefs_file_dio_write(struct kiocb *iocb, struct iov_iter *from) ret = zonefs_file_dio_append(iocb, from); else ret = iomap_dio_rw(iocb, from, &zonefs_iomap_ops, - &zonefs_write_dio_ops, sync); + &zonefs_write_dio_ops, 0); if (zi->i_ztype == ZONEFS_ZTYPE_SEQ && (ret > 0 || ret == -EIOCBQUEUED)) { if (ret > 0) @@ -917,7 +917,7 @@ static ssize_t zonefs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) } file_accessed(iocb->ki_filp); ret = iomap_dio_rw(iocb, to, &zonefs_iomap_ops, - &zonefs_read_dio_ops, is_sync_kiocb(iocb)); + &zonefs_read_dio_ops, 0); } else { ret = generic_file_read_iter(iocb, to); if (ret == -EIO) diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 5bd3cac4df9cb4..be4e1e1e01e801 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -256,12 +256,18 @@ struct iomap_dio_ops { struct bio *bio, loff_t file_offset); }; +/* + * Wait for the I/O to complete in iomap_dio_rw even if the kiocb is not + * synchronous. + */ +#define IOMAP_DIO_FORCE_WAIT (1 << 0) + ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - bool wait_for_completion); + unsigned int dio_flags); struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - bool wait_for_completion); + unsigned int dio_flags); ssize_t iomap_dio_complete(struct iomap_dio *dio); int iomap_dio_iopoll(struct kiocb *kiocb, bool spin); From patchwork Thu Jan 21 08:59:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87AD3C433E0 for ; Thu, 21 Jan 2021 09:12:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 42F5C2388E for ; Thu, 21 Jan 2021 09:12:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728429AbhAUJMj (ORCPT ); Thu, 21 Jan 2021 04:12:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728393AbhAUJMc (ORCPT ); Thu, 21 Jan 2021 04:12:32 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D1172C061575; Thu, 21 Jan 2021 01:11:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9o83V42vu6yUi+o2Wb0ZeVDwO25X9PTxXcCtlcoTFTU=; b=Iw8d5ugekNLJpkoQuGlh61OR9M velsn9W270MQY2pED2ZJWoostwyn9DMNIuUvgF/eulNd4NVQVgIJ2pM1MDsOddZCZtbU+qihjIDVh 9fRhshEgfFwdiLIKsJ9OgEnGvFq5M1rCM5bK+ehqwVLwxKj4K4PmuRfo3gVjA0n9IhirrBLtzXaLp KOAw3z0dq9+QzJf43IzyGJDmc6uYgvtVhDZkzVIptScgh8ClI5LquvMqRnj+7WPFtGXQrMNr3HJ15 CLV9FStwGQGpM7UUAnRJuT5cpT64y5/mTCMkoeN6TmniLqeX/Jn5GkHmcu5W8hbf6BWSN7Fn974e/ ABdIE7Tw==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2Vzj-00Gqhw-6R; Thu, 21 Jan 2021 09:11:28 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com Subject: [PATCH 10/11] iomap: add a IOMAP_DIO_OVERWRITE_ONLY flag Date: Thu, 21 Jan 2021 09:59:05 +0100 Message-Id: <20210121085906.322712-11-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a flag to signal an only pure overwrites are allowed. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster Reviewed-by: Darrick J. Wong --- fs/iomap/direct-io.c | 7 +++++++ include/linux/iomap.h | 8 ++++++++ 2 files changed, 15 insertions(+) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 947343730e2c93..65d32364345d22 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -485,6 +485,13 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, iomap_flags |= IOMAP_NOWAIT; } + if (dio_flags & IOMAP_DIO_OVERWRITE_ONLY) { + ret = -EAGAIN; + if (pos >= dio->i_size || pos + count > dio->i_size) + goto out_free_dio; + iomap_flags |= IOMAP_OVERWRITE_ONLY; + } + ret = filemap_write_and_wait_range(mapping, pos, end); if (ret) goto out_free_dio; diff --git a/include/linux/iomap.h b/include/linux/iomap.h index be4e1e1e01e801..cfa20afd7d5b87 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -122,6 +122,7 @@ struct iomap_page_ops { #define IOMAP_FAULT (1 << 3) /* mapping for page fault */ #define IOMAP_DIRECT (1 << 4) /* direct I/O */ #define IOMAP_NOWAIT (1 << 5) /* do not block */ +#define IOMAP_OVERWRITE_ONLY (1 << 6) /* purely overwrites allowed */ struct iomap_ops { /* @@ -262,6 +263,13 @@ struct iomap_dio_ops { */ #define IOMAP_DIO_FORCE_WAIT (1 << 0) +/* + * Do not allocate blocks or zero partial blocks, but instead fall back to + * the caller by returning -EAGAIN. Used to optimize direct I/O writes that + * are not aligned to the file system block size. + */ +#define IOMAP_DIO_OVERWRITE_ONLY (1 << 1) + ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, unsigned int dio_flags); From patchwork Thu Jan 21 08:59:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12035319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9818C433E0 for ; Thu, 21 Jan 2021 09:13:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B272E2388E for ; Thu, 21 Jan 2021 09:13:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728150AbhAUJM7 (ORCPT ); Thu, 21 Jan 2021 04:12:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728428AbhAUJMi (ORCPT ); Thu, 21 Jan 2021 04:12:38 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FE34C061757; Thu, 21 Jan 2021 01:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ucAXD5ulRNLTJu2me6wvJotPaN8/O/Jmprk/oFDZemY=; b=OX7xwDJ/7IVDiS9Bamz8WPSGhs hE7/Wq0mzThzsvqDHztbs7cvbARAXwpOgCEhixPEe0EODJxSJqcbWCRAp1M0vbdp01/kpodwe1SU/ xzjHQFYbL9ioBmWAvNeIoCue9FcgZCWfNitbikfTgcUHq576Y08sokQw1XbKtNmZZb3J1vQigESsE JF5g0oFC5RGgqn2IQuDmpBVDMMIj+dDtgck1hLgJ++3HI8bIGbKis462JD1x7WJ2Xhoxjb/5ARJsk +Vvsesr5nw/D9gu7wVX9M4tqaARdb7oKi8DLyjyYldu7TyBlDA/Ish8sof5KF92quCLkSzufWOH9k Gg4iJlXw==; Received: from [2001:4bb8:188:1954:d5b3:2657:287:e45f] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.94 #2 (Red Hat Linux)) id 1l2W0K-00Gqjb-G5; Thu, 21 Jan 2021 09:11:51 +0000 From: Christoph Hellwig To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, avi@scylladb.com, Dave Chinner Subject: [PATCH 11/11] xfs: reduce exclusive locking on unaligned dio Date: Thu, 21 Jan 2021 09:59:06 +0100 Message-Id: <20210121085906.322712-12-hch@lst.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121085906.322712-1-hch@lst.de> References: <20210121085906.322712-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Dave Chinner Attempt shared locking for unaligned DIO, but only if the the underlying extent is already allocated and in written state. On failure, retry with the existing exclusive locking. Test case is fio randrw of 512 byte IOs using AIO and an iodepth of 32 IOs. Vanilla: READ: bw=4560KiB/s (4670kB/s), 4560KiB/s-4560KiB/s (4670kB/s-4670kB/s), io=134MiB (140MB), run=30001-30001msec WRITE: bw=4567KiB/s (4676kB/s), 4567KiB/s-4567KiB/s (4676kB/s-4676kB/s), io=134MiB (140MB), run=30001-30001msec Patched: READ: bw=37.6MiB/s (39.4MB/s), 37.6MiB/s-37.6MiB/s (39.4MB/s-39.4MB/s), io=1127MiB (1182MB), run=30002-30002msec WRITE: bw=37.6MiB/s (39.4MB/s), 37.6MiB/s-37.6MiB/s (39.4MB/s-39.4MB/s), io=1128MiB (1183MB), run=30002-30002msec That's an improvement from ~18k IOPS to a ~150k IOPS, which is about the IOPS limit of the VM block device setup I'm testing on. 4kB block IO comparison: READ: bw=296MiB/s (310MB/s), 296MiB/s-296MiB/s (310MB/s-310MB/s), io=8868MiB (9299MB), run=30002-30002msec WRITE: bw=296MiB/s (310MB/s), 296MiB/s-296MiB/s (310MB/s-310MB/s), io=8878MiB (9309MB), run=30002-30002msec Which is ~150k IOPS, same as what the test gets for sub-block AIO+DIO writes with this patch. Signed-off-by: Dave Chinner [hch: rebased, split unaligned from nowait] Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Brian Foster --- fs/xfs/xfs_file.c | 87 ++++++++++++++++++++++++++++++++-------------- fs/xfs/xfs_iomap.c | 30 +++++++++++----- 2 files changed, 83 insertions(+), 34 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index b181db42f2f32f..33899a5cca53f9 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -544,22 +544,35 @@ xfs_file_dio_write_aligned( /* * Handle block unaligned direct IO writes * - * In most cases direct IO writes will be done holding IOLOCK_SHARED, allowing - * them to be done in parallel with reads and other direct IO writes. However, - * if the I/O is not aligned to filesystem blocks, the direct I/O layer may - * need to do sub-block zeroing and that requires serialisation against other - * direct I/Os to the same block. In this case we need to serialise the - * submission of the unaligned I/Os so that we don't get racing block zeroing in - * the dio layer. + * In most cases direct IO writes will be done holding IOLOCK_SHARED + * allowing them to be done in parallel with reads and other direct IO writes. + * However, if the IO is not aligned to filesystem blocks, the direct IO layer + * may need to do sub-block zeroing and that requires serialisation against other + * direct IOs to the same block. In the case where sub-block zeroing is not + * required, we can do concurrent sub-block dios to the same block successfully. * - * To provide the same serialisation for AIO, we also need to wait for + * Hence we have two cases here - the shared, optimisitic fast path for written + * extents, and everything else that needs exclusive IO path access across the + * entire IO. + * + * For the first case, we do all the checks we need at the mapping layer in the + * DIO code as part of the existing NOWAIT infrastructure. Hence all we need to + * do to support concurrent subblock dio is first try a non-blocking submission. + * If that returns -EAGAIN, then we simply repeat the IO submission with full + * IO exclusivity guaranteed so that we avoid racing sub-block zeroing. + * + * The only wrinkle in this case is that the iomap DIO code always does + * partial tail sub-block zeroing for post-EOF writes. Hence for any IO that + * _ends_ past the current EOF we need to run with full exclusivity. Note that + * we also check for the start of IO being beyond EOF because then zeroing + * between the old EOF and the start of the IO is required and that also + * requires exclusivity. Hence we avoid lock cycles and blocking under + * IOCB_NOWAIT for this situation, too. + * + * To provide the exclusivity required when using AIO, we also need to wait for * outstanding IOs to complete so that unwritten extent conversion is completed * before we try to map the overlapping block. This is currently implemented by * hitting it with a big hammer (i.e. inode_dio_wait()). - * - * This means that unaligned dio writes always block. There is no "nowait" fast - * path in this code - if IOCB_NOWAIT is set we simply return -EAGAIN up front - * and we don't have to worry about that anymore. */ static noinline ssize_t xfs_file_dio_write_unaligned( @@ -567,13 +580,27 @@ xfs_file_dio_write_unaligned( struct kiocb *iocb, struct iov_iter *from) { - int iolock = XFS_IOLOCK_EXCL; + size_t isize = i_size_read(VFS_I(ip)); + size_t count = iov_iter_count(from); + int iolock = XFS_IOLOCK_SHARED; + unsigned int flags = IOMAP_DIO_OVERWRITE_ONLY; ssize_t ret; - /* unaligned dio always waits, bail */ - if (iocb->ki_flags & IOCB_NOWAIT) - return -EAGAIN; - xfs_ilock(ip, iolock); + /* + * Extending writes need exclusivity because of the sub-block zeroing + * that the DIO code always does for partial tail blocks beyond EOF. + */ + if (iocb->ki_pos > isize || iocb->ki_pos + count >= isize) { +retry_exclusive: + if (iocb->ki_flags & IOCB_NOWAIT) + return -EAGAIN; + iolock = XFS_IOLOCK_EXCL; + flags = IOMAP_DIO_FORCE_WAIT; + } + + ret = xfs_ilock_iocb(iocb, iolock); + if (ret) + return ret; /* * We can't properly handle unaligned direct I/O to reflink files yet, @@ -590,19 +617,27 @@ xfs_file_dio_write_unaligned( goto out_unlock; /* - * If we are doing unaligned I/O, we can't allow any other overlapping - * I/O in-flight at the same time or we risk data corruption. Wait for - * all other I/O to drain before we submit. + * If we are doing exclusive unaligned IO, we can't allow any other + * overlapping IO in-flight at the same time or we risk data corruption. + * Wait for all other IO to drain before we submit. */ - inode_dio_wait(VFS_I(ip)); + if (flags & IOMAP_DIO_FORCE_WAIT) + inode_dio_wait(VFS_I(ip)); - /* - * This must be the only I/O in-flight. Wait on it before we release the - * iolock to prevent subsequent overlapping I/O. - */ trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, IOMAP_DIO_FORCE_WAIT); + &xfs_dio_write_ops, flags); + /* + * Retry unaligned IO with exclusive blocking semantics if the DIO + * layer rejected it for mapping or locking reasons. If we are doing + * nonblocking user IO, propagate the error. + */ + if (ret == -EAGAIN && !(iocb->ki_flags & IOCB_NOWAIT)) { + ASSERT(flags & IOMAP_DIO_OVERWRITE_ONLY); + xfs_iunlock(ip, iolock); + goto retry_exclusive; + } + out_unlock: if (iolock) xfs_iunlock(ip, iolock); diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 7b9ff824e82d48..596af78f910596 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -784,15 +784,29 @@ xfs_direct_write_iomap_begin( goto allocate_blocks; /* - * NOWAIT IO needs to span the entire requested IO with a single map so - * that we avoid partial IO failures due to the rest of the IO range not - * covered by this map triggering an EAGAIN condition when it is - * subsequently mapped and aborting the IO. + * NOWAIT and OVERWRITE needs to span the entire requested IO with a + * single map so that we avoid partial IO failures due to the rest of + * the IO range not covered by this map triggering an EAGAIN condition + * when it is subsequently mapped and aborting the IO. */ - if ((flags & IOMAP_NOWAIT) && - !imap_spans_range(&imap, offset_fsb, end_fsb)) { + if (flags & (IOMAP_NOWAIT | IOMAP_OVERWRITE_ONLY)) { error = -EAGAIN; - goto out_unlock; + if (!imap_spans_range(&imap, offset_fsb, end_fsb)) + goto out_unlock; + } + + /* + * For overwrite only I/O, we cannot convert unwritten extents without + * requiring sub-block zeroing. This can only be done under an + * exclusive IOLOCK, hence return -EAGAIN if this is not a written + * extent to tell the caller to try again. + */ + if (flags & IOMAP_OVERWRITE_ONLY) { + error = -EAGAIN; + if (imap.br_state != XFS_EXT_NORM && + ((offset & mp->m_blockmask) || + ((offset + length) & mp->m_blockmask))) + goto out_unlock; } xfs_iunlock(ip, lockmode); @@ -801,7 +815,7 @@ xfs_direct_write_iomap_begin( allocate_blocks: error = -EAGAIN; - if (flags & IOMAP_NOWAIT) + if (flags & (IOMAP_NOWAIT | IOMAP_OVERWRITE_ONLY)) goto out_unlock; /*