From patchwork Fri Dec 13 15:05:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Foster X-Patchwork-Id: 13907326 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DBE81E231D for ; Fri, 13 Dec 2024 15:03:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102229; cv=none; b=Vt3j4IWuo6ii5d/YecG2Kd+KlB13aFZ9a+TWLDZG8/54vqfnkUupkuOn+FxbDondWwrUm7eO0jckKtf9hxv20rnGjj7MWhX5OqpjKOmHzlQ4l+dxPl1lU+mvUwyOOyn71D1VYiH7HAf5L20/fRblGHGtGa8vWIY3o2AEuByAQJc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102229; c=relaxed/simple; bh=KaiRfniO8ypmC9LnN8mkDxYIOrzwWRP4c6dpRh53d/Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fW4WlM37geXwj+vEeHcNMP6Ec/I1CuHnlheGExarYCp8PaxNWBjmfuudMSDIV8RavY2mAr9bEiHniPpu0DWJOKRBScu1RZfzi7KrGYDmz81zJFHquZufy9HFuFAjMPV97jPW34iogBa1b14V0Dw1e42mys4ObersBDTkwhN30PA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LvcGqtPS; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LvcGqtPS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734102226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lqFX0oyHRDEmE94Jp5u6aCubudExYzqG6Xpe0M6XTQQ=; b=LvcGqtPSg68EJQS6GQbLwzd8ZpwTmPZxQEkg5Irnk8U7boTQoXLKiw2XVLoJCI/oXlYPq9 iqbzt4Dsr0dfIaHRPiYKX1JmMdZ9zD+cHYVj81vG4/msFaYTrlm7s5qHtvuDKCHWAuCLIU +bIY4Azl2NfUs7AdlEXlwYK/7sy18a8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-480-HZsnzgIdOmqrq_GrixwtdA-1; Fri, 13 Dec 2024 10:03:43 -0500 X-MC-Unique: HZsnzgIdOmqrq_GrixwtdA-1 X-Mimecast-MFC-AGG-ID: HZsnzgIdOmqrq_GrixwtdA Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A0A7C1955E91; Fri, 13 Dec 2024 15:03:42 +0000 (UTC) Received: from bfoster.redhat.com (unknown [10.22.90.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 09E1F1953951; Fri, 13 Dec 2024 15:03:41 +0000 (UTC) From: Brian Foster To: linux-fsdevel@vger.kernel.org Cc: linux-xfs@vger.kernel.org Subject: [PATCH RFCv2 1/4] iomap: prep work for folio_batch support Date: Fri, 13 Dec 2024 10:05:25 -0500 Message-ID: <20241213150528.1003662-2-bfoster@redhat.com> In-Reply-To: <20241213150528.1003662-1-bfoster@redhat.com> References: <20241213150528.1003662-1-bfoster@redhat.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Squash of misc. prep work for folio_batch support. Not-Signed-off-by: Brian Foster --- fs/iomap/buffered-io.c | 100 +++++++++++++++++++++++------------------ 1 file changed, 57 insertions(+), 43 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index e0ae46b11413..7fdf593b58b1 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -743,10 +743,13 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, return 0; } -static struct folio *__iomap_get_folio(struct iomap_iter *iter, loff_t pos, - size_t len) +static struct folio *__iomap_get_folio(struct iomap_iter *iter, size_t len) { const struct iomap_folio_ops *folio_ops = iter->iomap.folio_ops; + loff_t pos = iter->pos; + + if (!mapping_large_folio_support(iter->inode->i_mapping)) + len = min_t(size_t, len, PAGE_SIZE - offset_in_page(pos)); if (folio_ops && folio_ops->get_folio) return folio_ops->get_folio(iter, pos, len); @@ -754,10 +757,11 @@ static struct folio *__iomap_get_folio(struct iomap_iter *iter, loff_t pos, return iomap_get_folio(iter, pos, len); } -static void __iomap_put_folio(struct iomap_iter *iter, loff_t pos, size_t ret, +static void __iomap_put_folio(struct iomap_iter *iter, size_t ret, struct folio *folio) { const struct iomap_folio_ops *folio_ops = iter->iomap.folio_ops; + loff_t pos = iter->pos; if (folio_ops && folio_ops->put_folio) { folio_ops->put_folio(iter->inode, pos, ret, folio); @@ -767,6 +771,21 @@ static void __iomap_put_folio(struct iomap_iter *iter, loff_t pos, size_t ret, } } +/* trim pos and bytes to within a given folio */ +static loff_t iomap_trim_folio_range(struct iomap_iter *iter, + struct folio *folio, size_t *offset, size_t *bytes) +{ + loff_t pos = iter->pos; + size_t fsize = folio_size(folio); + + WARN_ON_ONCE(pos < folio_pos(folio) || pos >= folio_pos(folio) + fsize); + + *offset = offset_in_folio(folio, pos); + if (*bytes > fsize - *offset) + *bytes = fsize - *offset; + return pos; +} + static int iomap_write_begin_inline(const struct iomap_iter *iter, struct folio *folio) { @@ -776,25 +795,27 @@ static int iomap_write_begin_inline(const struct iomap_iter *iter, return iomap_read_inline_data(iter, folio); } -static int iomap_write_begin(struct iomap_iter *iter, loff_t pos, - size_t len, struct folio **foliop) +/* + * Grab and prepare a folio for write based on iter state. Returns the folio, + * offset, and length. Callers can optionally pass a max length *plen, + * otherwise init to zero. + */ +static int iomap_write_begin(struct iomap_iter *iter, struct folio **foliop, + size_t *poffset, size_t *plen) { const struct iomap_folio_ops *folio_ops = iter->iomap.folio_ops; const struct iomap *srcmap = iomap_iter_srcmap(iter); + loff_t pos; + size_t len = min_t(u64, SIZE_MAX, iomap_length(iter)); struct folio *folio; int status = 0; - BUG_ON(pos + len > iter->iomap.offset + iter->iomap.length); - if (srcmap != &iter->iomap) - BUG_ON(pos + len > srcmap->offset + srcmap->length); + len = *plen > 0 ? min_t(u64, len, *plen) : len; if (fatal_signal_pending(current)) return -EINTR; - if (!mapping_large_folio_support(iter->inode->i_mapping)) - len = min_t(size_t, len, PAGE_SIZE - offset_in_page(pos)); - - folio = __iomap_get_folio(iter, pos, len); + folio = __iomap_get_folio(iter, len); if (IS_ERR(folio)) return PTR_ERR(folio); @@ -818,8 +839,10 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos, } } - if (pos + len > folio_pos(folio) + folio_size(folio)) - len = folio_pos(folio) + folio_size(folio) - pos; + pos = iomap_trim_folio_range(iter, folio, poffset, &len); + BUG_ON(pos + len > iter->iomap.offset + iter->iomap.length); + if (srcmap != &iter->iomap) + BUG_ON(pos + len > srcmap->offset + srcmap->length); if (srcmap->type == IOMAP_INLINE) status = iomap_write_begin_inline(iter, folio); @@ -832,10 +855,11 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos, goto out_unlock; *foliop = folio; + *plen = len; return 0; out_unlock: - __iomap_put_folio(iter, pos, 0, folio); + __iomap_put_folio(iter, 0, folio); return status; } @@ -885,10 +909,11 @@ static void iomap_write_end_inline(const struct iomap_iter *iter, * Returns true if all copied bytes have been written to the pagecache, * otherwise return false. */ -static bool iomap_write_end(struct iomap_iter *iter, loff_t pos, size_t len, - size_t copied, struct folio *folio) +static bool iomap_write_end(struct iomap_iter *iter, size_t len, size_t copied, + struct folio *folio) { const struct iomap *srcmap = iomap_iter_srcmap(iter); + loff_t pos = iter->pos; if (srcmap->type == IOMAP_INLINE) { iomap_write_end_inline(iter, folio, pos, copied); @@ -922,12 +947,12 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ size_t written; /* Bytes have been written */ - loff_t pos = iter->pos; + loff_t pos; loff_t length = iomap_length(iter); bytes = iov_iter_count(i); retry: - offset = pos & (chunk - 1); + offset = iter->pos & (chunk - 1); bytes = min(chunk - offset, bytes); status = balance_dirty_pages_ratelimited_flags(mapping, bdp_flags); @@ -952,23 +977,21 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) break; } - status = iomap_write_begin(iter, pos, bytes, &folio); + status = iomap_write_begin(iter, &folio, &offset, &bytes); if (unlikely(status)) { - iomap_write_failed(iter->inode, pos, bytes); + iomap_write_failed(iter->inode, iter->pos, bytes); break; } if (iter->iomap.flags & IOMAP_F_STALE) break; - offset = offset_in_folio(folio, pos); - if (bytes > folio_size(folio) - offset) - bytes = folio_size(folio) - offset; + pos = iter->pos; if (mapping_writably_mapped(mapping)) flush_dcache_folio(folio); copied = copy_folio_from_iter_atomic(folio, offset, bytes, i); - written = iomap_write_end(iter, pos, bytes, copied, folio) ? + written = iomap_write_end(iter, bytes, copied, folio) ? copied : 0; /* @@ -983,7 +1006,7 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) i_size_write(iter->inode, pos + written); iter->iomap.flags |= IOMAP_F_SIZE_CHANGED; } - __iomap_put_folio(iter, pos, written, folio); + __iomap_put_folio(iter, written, folio); if (old_size < pos) pagecache_isize_extended(iter->inode, old_size, pos); @@ -1276,22 +1299,17 @@ static loff_t iomap_unshare_iter(struct iomap_iter *iter) struct folio *folio; int status; size_t offset; - size_t bytes = min_t(u64, SIZE_MAX, iomap_length(iter)); - loff_t pos = iter->pos; + size_t bytes = 0; bool ret; - status = iomap_write_begin(iter, pos, bytes, &folio); + status = iomap_write_begin(iter, &folio, &offset, &bytes); if (unlikely(status)) return status; if (iomap->flags & IOMAP_F_STALE) break; - offset = offset_in_folio(folio, pos); - if (bytes > folio_size(folio) - offset) - bytes = folio_size(folio) - offset; - - ret = iomap_write_end(iter, pos, bytes, bytes, folio); - __iomap_put_folio(iter, pos, bytes, folio); + ret = iomap_write_end(iter, bytes, bytes, folio); + __iomap_put_folio(iter, bytes, folio); if (WARN_ON_ONCE(!ret)) return -EIO; @@ -1347,11 +1365,10 @@ static loff_t iomap_zero_iter(struct iomap_iter *iter, bool *did_zero) struct folio *folio; int status; size_t offset; - size_t bytes = min_t(u64, SIZE_MAX, iomap_length(iter)); - loff_t pos = iter->pos; + size_t bytes = 0; bool ret; - status = iomap_write_begin(iter, pos, bytes, &folio); + status = iomap_write_begin(iter, &folio, &offset, &bytes); if (status) return status; if (iter->iomap.flags & IOMAP_F_STALE) @@ -1359,15 +1376,12 @@ static loff_t iomap_zero_iter(struct iomap_iter *iter, bool *did_zero) /* warn about zeroing folios beyond eof that won't write back */ WARN_ON_ONCE(folio_pos(folio) > iter->inode->i_size); - offset = offset_in_folio(folio, pos); - if (bytes > folio_size(folio) - offset) - bytes = folio_size(folio) - offset; folio_zero_range(folio, offset, bytes); folio_mark_accessed(folio); - ret = iomap_write_end(iter, pos, bytes, bytes, folio); - __iomap_put_folio(iter, pos, bytes, folio); + ret = iomap_write_end(iter, bytes, bytes, folio); + __iomap_put_folio(iter, bytes, folio); if (WARN_ON_ONCE(!ret)) return -EIO; From patchwork Fri Dec 13 15:05:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Foster X-Patchwork-Id: 13907323 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4FF11E22E6 for ; Fri, 13 Dec 2024 15:03:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102228; cv=none; b=A1YfCkmYunbfVTAPCmc8Fw41rnLKCD8Flj7hqyfOCrR1Uk8y7owSS0LdWtD5qPntBQt5hP2ppzE/p8uDiMYwGrLn5/jCQRMx+YeYzLvb/4qyQmtXNB7TY7P07F5GlZMO9qzmS+8yvUPHddBK+KZadzmuaLOJBYr1alR+8GZ8ols= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102228; c=relaxed/simple; bh=XyOAmvfe21viuZJY78oI6BQo/3Z13/h8azFdPRy7YMI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HFJjB5nexZY4Uc32AS3P8ge0DvZweZFR4Z1dvk27Uhhn4zqrzGDRLD7eid/0JxLf80Jbkl0j7jk/AeXHHdySIQEYTnTL1LCSoVbCCqLYWqH6kVOSyu3pxIyaXrwQ/7k3fNaY0+d2rUq8dSt0tx4rS3fTooH0oNYf5wUSBsmwWA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=JF+otIVd; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JF+otIVd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734102225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r9V+GpZZTi3GLiT0j5cISAdQq8//hsNV1rtZciA7bZI=; b=JF+otIVd2kZEx5XY2FORNXq3YHhuon1ZL160zEESQLhCO79B5AdCKeSxaJw6GK1MDwKauD 1kTw9aMW75LQ2J+gf3qmGEO5FxAoXQyW1/7KAOb2JNSM5rn3c0n2mK6J5cjQhiABpQxvqY pm/7e6On8HPNgKl7el3wjPv9T01cvkE= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-173-CIEjrz_SM6OuegUm98NSVw-1; Fri, 13 Dec 2024 10:03:44 -0500 X-MC-Unique: CIEjrz_SM6OuegUm98NSVw-1 X-Mimecast-MFC-AGG-ID: CIEjrz_SM6OuegUm98NSVw Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 87B5519560B4; Fri, 13 Dec 2024 15:03:43 +0000 (UTC) Received: from bfoster.redhat.com (unknown [10.22.90.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E3835195394B; Fri, 13 Dec 2024 15:03:42 +0000 (UTC) From: Brian Foster To: linux-fsdevel@vger.kernel.org Cc: linux-xfs@vger.kernel.org Subject: [PATCH RFCv2 2/4] iomap: optional zero range dirty folio processing Date: Fri, 13 Dec 2024 10:05:26 -0500 Message-ID: <20241213150528.1003662-3-bfoster@redhat.com> In-Reply-To: <20241213150528.1003662-1-bfoster@redhat.com> References: <20241213150528.1003662-1-bfoster@redhat.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 The only way zero range can currently process unwritten mappings with dirty pagecache is to check whether the range is dirty before mapping lookup and then flush when at least one underlying mapping is unwritten. This ordering is required to prevent iomap lookup from racing with folio writeback and reclaim. Since zero range can skip ranges of unwritten mappings that are clean in cache, this operation can be improved by allowing the filesystem to provide the set of folios backed by such mappings that require zeroing up. In turn, rather than flush or iterate file offsets, zero range can process each folio as normal and skip any clean or uncached ranges in between. As a first pass prototype solution, stuff a folio_batch in struct iomap, provide a helper that the fs can use to populate the batch at lookup time, and define a flag to indicate the mapping was checked. Note that since the helper is intended for use under internal fs locks, it trylocks folios in order to filter out clean folios. This loosely follows the logic from filemap_range_has_writeback(). Signed-off-by: Brian Foster --- fs/iomap/buffered-io.c | 77 ++++++++++++++++++++++++++++++++++++++++-- fs/iomap/iter.c | 6 ++++ include/linux/iomap.h | 4 +++ 3 files changed, 84 insertions(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 7fdf593b58b1..5492dc7fe963 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -751,6 +751,15 @@ static struct folio *__iomap_get_folio(struct iomap_iter *iter, size_t len) if (!mapping_large_folio_support(iter->inode->i_mapping)) len = min_t(size_t, len, PAGE_SIZE - offset_in_page(pos)); + if (iter->fbatch) { + struct folio *folio = folio_batch_next(iter->fbatch); + if (folio) { + folio_get(folio); + folio_lock(folio); + } + return folio; + } + if (folio_ops && folio_ops->get_folio) return folio_ops->get_folio(iter, pos, len); else @@ -839,6 +848,15 @@ static int iomap_write_begin(struct iomap_iter *iter, struct folio **foliop, } } + if (!folio) { + WARN_ON_ONCE(!iter->fbatch); + len = 0; + goto out; + } else if (folio_pos(folio) > iter->pos) { + BUG_ON(folio_pos(folio) - iter->pos >= iomap_length(iter)); + iomap_iter_advance(iter, folio_pos(folio) - iter->pos); + } + pos = iomap_trim_folio_range(iter, folio, poffset, &len); BUG_ON(pos + len > iter->iomap.offset + iter->iomap.length); if (srcmap != &iter->iomap) @@ -854,6 +872,7 @@ static int iomap_write_begin(struct iomap_iter *iter, struct folio **foliop, if (unlikely(status)) goto out_unlock; +out: *foliop = folio; *plen = len; return 0; @@ -1374,6 +1393,11 @@ static loff_t iomap_zero_iter(struct iomap_iter *iter, bool *did_zero) if (iter->iomap.flags & IOMAP_F_STALE) break; + if (!folio) { + iomap_iter_advance(iter, iomap_length(iter)); + break; + } + /* warn about zeroing folios beyond eof that won't write back */ WARN_ON_ONCE(folio_pos(folio) > iter->inode->i_size); @@ -1393,6 +1417,49 @@ static loff_t iomap_zero_iter(struct iomap_iter *iter, bool *did_zero) return 0; } +loff_t +iomap_fill_dirty_folios( + struct iomap_iter *iter, + loff_t offset, + loff_t length) +{ + struct address_space *mapping = iter->inode->i_mapping; + struct folio_batch fbatch; + loff_t end_pos = offset + length; + pgoff_t start = offset >> PAGE_SHIFT; + pgoff_t end = (end_pos - 1) >> PAGE_SHIFT; + + folio_batch_init(&fbatch); + iter->fbatch = kmalloc(sizeof(struct folio_batch), GFP_KERNEL); + if (!iter->fbatch) + return end_pos; + folio_batch_init(iter->fbatch); + + while (filemap_get_folios(mapping, &start, end, &fbatch) && + folio_batch_space(iter->fbatch)) { + struct folio *folio; + while ((folio = folio_batch_next(&fbatch))) { + if (folio_trylock(folio)) { + bool clean = !folio_test_dirty(folio) && + !folio_test_writeback(folio); + folio_unlock(folio); + if (clean) + continue; + } + + folio_get(folio); + if (!folio_batch_add(iter->fbatch, folio)) { + end_pos = folio_pos(folio) + folio_size(folio); + break; + } + } + folio_batch_release(&fbatch); + } + + return end_pos; +} +EXPORT_SYMBOL_GPL(iomap_fill_dirty_folios); + int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, const struct iomap_ops *ops) @@ -1420,7 +1487,7 @@ iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, * flushing on partial eof zeroing, special case it to zero the * unaligned start portion if already dirty in pagecache. */ - if (off && + if (!iter.fbatch && off && filemap_range_needs_writeback(mapping, pos, pos + plen - 1)) { iter.len = plen; while ((ret = iomap_iter(&iter, ops)) > 0) @@ -1441,8 +1508,12 @@ iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, while ((ret = iomap_iter(&iter, ops)) > 0) { const struct iomap *srcmap = iomap_iter_srcmap(&iter); - if (srcmap->type == IOMAP_HOLE || - srcmap->type == IOMAP_UNWRITTEN) { + if (WARN_ON_ONCE(iter.fbatch && srcmap->type != IOMAP_UNWRITTEN)) + return -EIO; + + if (!iter.fbatch && + (srcmap->type == IOMAP_HOLE || + srcmap->type == IOMAP_UNWRITTEN)) { loff_t proc = iomap_length(&iter); if (range_dirty) { diff --git a/fs/iomap/iter.c b/fs/iomap/iter.c index 5fe0edb51fe5..911846d7386c 100644 --- a/fs/iomap/iter.c +++ b/fs/iomap/iter.c @@ -9,6 +9,12 @@ static inline void iomap_iter_reset_iomap(struct iomap_iter *iter) { + if (iter->fbatch) { + folio_batch_release(iter->fbatch); + kfree(iter->fbatch); + iter->fbatch = NULL; + } + iter->processed = 0; memset(&iter->iomap, 0, sizeof(iter->iomap)); memset(&iter->srcmap, 0, sizeof(iter->srcmap)); diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 704ed98159f7..d01e5265de27 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -9,6 +9,7 @@ #include #include #include +#include struct address_space; struct fiemap_extent_info; @@ -228,6 +229,7 @@ struct iomap_iter { unsigned flags; struct iomap iomap; struct iomap srcmap; + struct folio_batch *fbatch; void *private; }; @@ -315,6 +317,8 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len); bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio); int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len, const struct iomap_ops *ops); +loff_t iomap_fill_dirty_folios(struct iomap_iter *iter, loff_t offset, + loff_t length); int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, const struct iomap_ops *ops); int iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero, From patchwork Fri Dec 13 15:05:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Foster X-Patchwork-Id: 13907325 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7F371E22FC for ; Fri, 13 Dec 2024 15:03:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102229; cv=none; b=tajU39v5y/oquEgeI7AV+OYfJWJ84kNjNSoyMjzgzZCoUGeGsU65z+KHoB6lp3E1wIrV0P08fWAjbydJdvzRhiCiDaOP5jVE4G6dxjB2PmJAZxqYilvWwhjh8h5Ekscyl4K/J4iSwoeljYiJEtMPleZTQMZCMw56uTUi203PLWc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102229; c=relaxed/simple; bh=xRGmHdpsTiIU+Z1zrdTOxwCPBqCUeP8NWJzQZCO7uBM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=XE86+Nz4gBkZzDQhW8P1b7XV6A4z97s7IdQMUMsiMYN+XMKgctKGR9W3/GkBOgpzyw+EAADCTiYunI9/5eBMPfRIqnUASz2uc2nY6bzcRU1e61HEkTMoPobqeL7nAnhlOS6LV9GUsFdhrWdQ+hZOLwZ2VHBujQubr5PQH6/MOKc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=N+xggxrv; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N+xggxrv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734102226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hYSD/z+zYJavLMYA5D03SkykgbMAFv2ge+5Q5GT8i74=; b=N+xggxrvYWkyGLUT9ONPTLE+TYoEUnz/zEi6UlfSFb756XZuZ8hMk/hiGNXipwpdTO9SPo opAvtPGS1Y391TV33Gqh2swksHNo8LhY3OYDEj5rtzUeCcCRmnO7pRL2+cxGtN7OpuQf4R kvD9ZZF51HLcoZMbENbbibejqDg/D3o= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-395-vgXCyxLXNoa0esIZ_wDVBw-1; Fri, 13 Dec 2024 10:03:45 -0500 X-MC-Unique: vgXCyxLXNoa0esIZ_wDVBw-1 X-Mimecast-MFC-AGG-ID: vgXCyxLXNoa0esIZ_wDVBw Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6BD77195609F; Fri, 13 Dec 2024 15:03:44 +0000 (UTC) Received: from bfoster.redhat.com (unknown [10.22.90.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CA4E6195394B; Fri, 13 Dec 2024 15:03:43 +0000 (UTC) From: Brian Foster To: linux-fsdevel@vger.kernel.org Cc: linux-xfs@vger.kernel.org Subject: [PATCH RFCv2 3/4] xfs: always trim mapping to requested range for zero range Date: Fri, 13 Dec 2024 10:05:27 -0500 Message-ID: <20241213150528.1003662-4-bfoster@redhat.com> In-Reply-To: <20241213150528.1003662-1-bfoster@redhat.com> References: <20241213150528.1003662-1-bfoster@redhat.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Refactor and tweak the IOMAP_ZERO logic in preparation to support filling the folio batch for unwritten mappings. Drop the superfluous imap offset check since the hole case has already been filtered out. Split the the delalloc case handling into a sub-branch, and always trim the imap to the requested offset/count so it can be more easily used to bound the range to lookup in pagecache. Signed-off-by: Brian Foster --- fs/xfs/xfs_iomap.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 50fa3ef89f6c..97fa860a6401 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -1059,21 +1059,20 @@ xfs_buffered_write_iomap_begin( } /* - * For zeroing, trim a delalloc extent that extends beyond the EOF - * block. If it starts beyond the EOF block, convert it to an + * For zeroing, trim extents that extend beyond the EOF block. If a + * delalloc extent starts beyond the EOF block, convert it to an * unwritten extent. */ - if ((flags & IOMAP_ZERO) && imap.br_startoff <= offset_fsb && - isnullstartblock(imap.br_startblock)) { + if (flags & IOMAP_ZERO) { xfs_fileoff_t eof_fsb = XFS_B_TO_FSB(mp, XFS_ISIZE(ip)); - if (offset_fsb >= eof_fsb) + if (isnullstartblock(imap.br_startblock) && + offset_fsb >= eof_fsb) goto convert_delay; - if (end_fsb > eof_fsb) { + if (offset_fsb < eof_fsb && end_fsb > eof_fsb) end_fsb = eof_fsb; - xfs_trim_extent(&imap, offset_fsb, - end_fsb - offset_fsb); - } + + xfs_trim_extent(&imap, offset_fsb, end_fsb - offset_fsb); } /* From patchwork Fri Dec 13 15:05:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Foster X-Patchwork-Id: 13907327 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6C811E231F for ; Fri, 13 Dec 2024 15:03:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102230; cv=none; b=OQ/nOjzfkdDcOEJG/eIvQhzlY5CLyZOJePwoPSEMnQ/NRrHEKF+Wk3If9JUTYgEkyyxizJjmw5mdE2zTdROKGPhjgvjOu+G0jgWwAbadkCAvberSW+lAQw1MAtKalu2CXdc7N26rG68Pg1B1UUN+q4MznDlQ8zerJe2WKfpjDUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734102230; c=relaxed/simple; bh=YRJkWpVrsK5tWAkj3h7O0KKz9hSJzbc1riBkCq6zkRk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CT4EbJ6ZExweyRxkvPGL6I0qVHYFfDvfnzk7qOoftX/BQpvnrY2unKcXIv9SNOi9xsb9xOdRhQs+zrvNvi1a+MWLkEomJ3DGhA2KsjuRLUi18GQuK4zdRoZooL9PMCxPY4RbonpHHEnr0pix8YpqvEEJs1DkkYWkpsHAyOm1R3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GTMb4BZO; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GTMb4BZO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734102227; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gtlfk7brtiOHWfKLJSeT8G4FA2trYaZJpMkIqijZqRc=; b=GTMb4BZO3ALfxnI3WM3qVrbgU+8/V36xKmkcPhfBF2nZripcMgZ572CxFef69BB2IsxYor 6p3BSI0hevrZdRMZnALCCYK0ExCI9/Zxy3q94yerYsoRiDFnBhr+6jIl/F4Le7wFtHEtYv 2gZ4KBzD2GUZmGKn39AKCXES1lSaxt8= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-130-RbX2NjB4MFWGeghGE-D9EA-1; Fri, 13 Dec 2024 10:03:46 -0500 X-MC-Unique: RbX2NjB4MFWGeghGE-D9EA-1 X-Mimecast-MFC-AGG-ID: RbX2NjB4MFWGeghGE-D9EA Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4F95919560A3; Fri, 13 Dec 2024 15:03:45 +0000 (UTC) Received: from bfoster.redhat.com (unknown [10.22.90.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ADEA9195394B; Fri, 13 Dec 2024 15:03:44 +0000 (UTC) From: Brian Foster To: linux-fsdevel@vger.kernel.org Cc: linux-xfs@vger.kernel.org Subject: [PATCH RFCv2 4/4] xfs: fill dirty folios on zero range of unwritten mappings Date: Fri, 13 Dec 2024 10:05:28 -0500 Message-ID: <20241213150528.1003662-5-bfoster@redhat.com> In-Reply-To: <20241213150528.1003662-1-bfoster@redhat.com> References: <20241213150528.1003662-1-bfoster@redhat.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Use the iomap folio batch mechanism to identify which folios to zero on zero range of unwritten mappings. Trim the resulting mapping if the batch is filled (unlikely) and set the HAS_FOLIOS flag to inform iomap that pagecache has been checked for dirty folios. Signed-off-by: Brian Foster --- fs/xfs/xfs_iomap.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 97fa860a6401..b7dbd34fc02f 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -998,6 +998,7 @@ xfs_buffered_write_iomap_begin( struct iomap *iomap, struct iomap *srcmap) { + struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap); struct xfs_inode *ip = XFS_I(inode); struct xfs_mount *mp = ip->i_mount; xfs_fileoff_t offset_fsb = XFS_B_TO_FSBT(mp, offset); @@ -1065,12 +1066,21 @@ xfs_buffered_write_iomap_begin( */ if (flags & IOMAP_ZERO) { xfs_fileoff_t eof_fsb = XFS_B_TO_FSB(mp, XFS_ISIZE(ip)); + u64 end; if (isnullstartblock(imap.br_startblock) && offset_fsb >= eof_fsb) goto convert_delay; if (offset_fsb < eof_fsb && end_fsb > eof_fsb) end_fsb = eof_fsb; + if (imap.br_state == XFS_EXT_UNWRITTEN && + offset_fsb < eof_fsb) { + xfs_trim_extent(&imap, offset_fsb, end_fsb - offset_fsb); + end = iomap_fill_dirty_folios(iter, + XFS_FSB_TO_B(mp, imap.br_startoff), + XFS_FSB_TO_B(mp, imap.br_blockcount)); + end_fsb = min_t(xfs_fileoff_t, end_fsb, XFS_B_TO_FSB(mp, end)); + } xfs_trim_extent(&imap, offset_fsb, end_fsb - offset_fsb); }