From patchwork Wed Dec 27 13:37:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508383 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2B151362 for ; Mon, 1 Jan 2024 00:37:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Xvk2OJA0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B36EDC433C8; Mon, 1 Jan 2024 00:37:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704069442; bh=LGAKg4ItIUvTiglH6RA8uTvgIgqgDQ/xp0MIE1PphFY=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Xvk2OJA0JoNgVwL/j7b23AjVf0xfDaY8LMkwknsC2BDg257CZaMJQ/zjK7VXPkGVs b8uBz4qmtOQOXTrGHDi6iz7UJHqJaRjnv3kBYqlUPwFW8THKas9eXaRbsTW5eQ3LKt zfbULtehQPnt6OSQ4aZ0RvJBNmP9l5fhH+h4jYOV/Yb/FWDJ7NiInJzqQLZGseX5cq AKixG/O0q/ZmvIcQrW5g0MC5eZag+bN9y7aVVutRO7MizgzJa55yN/Hkbhv5yVMofZ GKBdaLnDqF4EgB5NdnKWFT3xYZWkHvnZzlDOVVXj4JhDcCHMRRup0Aky2li8cGGsBN nnFK3VxhYD9MQ== Date: Sun, 31 Dec 2023 16:37:22 +9900 Subject: [PATCH 1/3] xfs: enable extent size hints for CoW when rtextsize > 1 From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170405018025.1818169.13143459343794388781.stgit@frogsfrogsfrogs> In-Reply-To: <170405018010.1818169.15531409874864543325.stgit@frogsfrogsfrogs> References: <170405018010.1818169.15531409874864543325.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong CoW extent size hints are not allowed on filesystems that have large realtime extents because we only want to perform the minimum required amount of write-around (aka write amplification) for shared extents. On filesystems where rtextsize > 1, allocations can only be done in units of full rt extents, which means that we can only map an entire rt extent's worth of blocks into the data fork. Hole punch requests become conversions to unwritten if the request isn't aligned properly. Because a copy-write fundamentally requires remapping, this means that we also can only do copy-writes of a full rt extent. This is too expensive for large hint sizes, since it's all or nothing. Signed-off-by: Darrick J. Wong --- libxfs/xfs_bmap.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/libxfs/xfs_bmap.c b/libxfs/xfs_bmap.c index bc703b13b7e..13bcf146d08 100644 --- a/libxfs/xfs_bmap.c +++ b/libxfs/xfs_bmap.c @@ -6402,6 +6402,28 @@ xfs_get_cowextsz_hint( if (ip->i_diflags2 & XFS_DIFLAG2_COWEXTSIZE) a = ip->i_cowextsize; if (XFS_IS_REALTIME_INODE(ip)) { + /* + * For realtime files, the realtime extent is the fundamental + * unit of allocation. This means that data sharing and CoW + * remapping can only be done in those units. For filesystems + * where the extent size is larger than one block, write + * requests that are not aligned to an extent boundary employ + * an unshare-around strategy to ensure that all pages for a + * shared extent are fully dirtied. + * + * Because the remapping alignment requirement applies equally + * to all CoW writes, any regular overwrites that could be + * turned (by a speculative CoW preallocation) into a CoW write + * must either employ this dirty-around strategy, or be smart + * enough to ignore the CoW fork mapping unless the entire + * extent is dirty or becomes shared by writeback time. Doing + * the first would dramatically increase write amplification, + * and the second would require deeper insight into the state + * of the page cache during a writeback request. For now, we + * ignore the hint. + */ + if (ip->i_mount->m_sb.sb_rextsize > 1) + return ip->i_mount->m_sb.sb_rextsize; b = 0; if (ip->i_diflags & XFS_DIFLAG_EXTSIZE) b = ip->i_extsize; From patchwork Wed Dec 27 13:37:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508389 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 757931370 for ; Mon, 1 Jan 2024 00:37:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Nqt2lQ2W" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49082C433C7; Mon, 1 Jan 2024 00:37:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704069458; bh=5UvRm7wgt+nEud3CUVbju/HTMwq5CftDTN/qCgstk/w=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Nqt2lQ2WkdChtq3tbseS5saKXUEbgea6hSZXIEy09Ma4AZSX9vtXZ+A3cjmJrO2U4 jTEJxOLzbc3HZAAi4maIPz9ItUBbrEc7O1qlDa2cAUE5QF+cCTnisSQ8C6DbHOuejd vwnLguCwmxygCQV4NOFQrjVOaBiM+LLnUEJWqJdSi3EurX/oVPrsSI9pfCsTqh8c36 MPE5efulngTy+nv8l8Nk7eR9cWLJiiCiWp6HaJusGM8No8Kthw6+wwjaRUPNXh71+8 571o6NLFvdMF2oeAxBiApznxDmtj+7uOBHD0NKqlY2L8KdsmMhtP2+5ZhwNr3xiOkH UxqqvxMJZqCSg== Date: Sun, 31 Dec 2023 16:37:37 +9900 Subject: [PATCH 2/3] xfs: fix integer overflow when validating extent size hints From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170405018038.1818169.12212197712076555126.stgit@frogsfrogsfrogs> In-Reply-To: <170405018010.1818169.15531409874864543325.stgit@frogsfrogsfrogs> References: <170405018010.1818169.15531409874864543325.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Both file extent size hints are stored as 32-bit quantities, in units of filesystem blocks. As part of validating the hints, we convert these quantities to bytes to ensure that the hint is congruent with the file's allocation size. The maximum possible hint value is 2097151 (aka XFS_MAX_BMBT_EXTLEN). If the file allocation unit is larger than 2048, the unit conversion will exceed 32 bits in size, which overflows the uint32_t used to store the value used in the comparison. This isn't a problem for files on the data device since the hint will always be a multiple of the block size. However, this is a problem for realtime files because the rtextent size can be any integer number of fs blocks, and truncation of upper bits changes the outcome of division. Eliminate the overflow by performing the congruency check in units of blocks, not bytes. Otherwise, we get errors like this: $ truncate -s 500T /tmp/a $ mkfs.xfs -f -N /tmp/a -d extszinherit=2097151,rtinherit=1 -r extsize=28k illegal extent size hint 2097151, must be less than 2097151 and a multiple of 7. Signed-off-by: Darrick J. Wong --- libxfs/xfs_inode_buf.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/libxfs/xfs_inode_buf.c b/libxfs/xfs_inode_buf.c index d9fc254b354..085c128c542 100644 --- a/libxfs/xfs_inode_buf.c +++ b/libxfs/xfs_inode_buf.c @@ -767,13 +767,11 @@ xfs_inode_validate_extsize( bool rt_flag; bool hint_flag; bool inherit_flag; - uint32_t extsize_bytes; - uint32_t blocksize_bytes; + uint32_t alloc_unit = 1; rt_flag = (flags & XFS_DIFLAG_REALTIME); hint_flag = (flags & XFS_DIFLAG_EXTSIZE); inherit_flag = (flags & XFS_DIFLAG_EXTSZINHERIT); - extsize_bytes = XFS_FSB_TO_B(mp, extsize); /* * This comment describes a historic gap in this verifier function. @@ -802,9 +800,7 @@ xfs_inode_validate_extsize( */ if (rt_flag) - blocksize_bytes = XFS_FSB_TO_B(mp, mp->m_sb.sb_rextsize); - else - blocksize_bytes = mp->m_sb.sb_blocksize; + alloc_unit = mp->m_sb.sb_rextsize; if ((hint_flag || inherit_flag) && !(S_ISDIR(mode) || S_ISREG(mode))) return __this_address; @@ -822,7 +818,7 @@ xfs_inode_validate_extsize( if (mode && !(hint_flag || inherit_flag) && extsize != 0) return __this_address; - if (extsize_bytes % blocksize_bytes) + if (extsize % alloc_unit) return __this_address; if (extsize > XFS_MAX_BMBT_EXTLEN) @@ -857,12 +853,10 @@ xfs_inode_validate_cowextsize( { bool rt_flag; bool hint_flag; - uint32_t cowextsize_bytes; - uint32_t blocksize_bytes; + uint32_t alloc_unit = 1; rt_flag = (flags & XFS_DIFLAG_REALTIME); hint_flag = (flags2 & XFS_DIFLAG2_COWEXTSIZE); - cowextsize_bytes = XFS_FSB_TO_B(mp, cowextsize); /* * Similar to extent size hints, a directory can be configured to @@ -877,9 +871,7 @@ xfs_inode_validate_cowextsize( */ if (rt_flag) - blocksize_bytes = XFS_FSB_TO_B(mp, mp->m_sb.sb_rextsize); - else - blocksize_bytes = mp->m_sb.sb_blocksize; + alloc_unit = mp->m_sb.sb_rextsize; if (hint_flag && !xfs_has_reflink(mp)) return __this_address; @@ -894,7 +886,7 @@ xfs_inode_validate_cowextsize( if (mode && !hint_flag && cowextsize != 0) return __this_address; - if (cowextsize_bytes % blocksize_bytes) + if (cowextsize % alloc_unit) return __this_address; if (cowextsize > XFS_MAX_BMBT_EXTLEN) From patchwork Wed Dec 27 13:37:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13508390 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 743451368 for ; Mon, 1 Jan 2024 00:37:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JLlWPFk0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC598C433C7; Mon, 1 Jan 2024 00:37:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704069474; bh=ZNxv2CSYubuLNIYsl7/i9xfp9phA7MzfBm/s86nAaBU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=JLlWPFk0AVKrVGk741b9cuHoOuTUWJa14ejj7oARVEvVCD/oBSyj22LgzGgVxpha2 FNUYkgdhdrEFUbgJZpFArerIOy//ykc4vdIAGccXHKCfGtlfjTaT+JkEh4f3DYR1D5 p5Igzw3Cx2r5/rYv0g/YNVlqYTTrvEBOYlHVQWWjmApgAxSI5OWeRi5rBwfPHwe1GJ bcdJv+U1mSFJ2Ds3HDcaSTzjDLRqB/tV0oMxUiqcBJIc+srZhwJDiF35RHGbPw2beX YxLSXp/aiSEIH5zjPQe/P79Gc/ru10OKREHRSaM7tgwCOWcAPc0FUw/jerD7SiFoO7 lDfjO9rzWxzAA== Date: Sun, 31 Dec 2023 16:37:53 +9900 Subject: [PATCH 3/3] mkfs: enable reflink with realtime extent sizes > 1 From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: linux-xfs@vger.kernel.org Message-ID: <170405018052.1818169.15071829414225680751.stgit@frogsfrogsfrogs> In-Reply-To: <170405018010.1818169.15531409874864543325.stgit@frogsfrogsfrogs> References: <170405018010.1818169.15531409874864543325.stgit@frogsfrogsfrogs> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Allow creation of filesystems with reflink enabled and realtime extent size larger than 1 block. Signed-off-by: Darrick J. Wong --- libxfs/init.c | 7 ------- mkfs/xfs_mkfs.c | 37 ------------------------------------- 2 files changed, 44 deletions(-) diff --git a/libxfs/init.c b/libxfs/init.c index 2e0d3b1ce94..0f626784902 100644 --- a/libxfs/init.c +++ b/libxfs/init.c @@ -300,13 +300,6 @@ rtmount_init( if (mp->m_sb.sb_rblocks == 0) return 0; - if (xfs_has_reflink(mp) && mp->m_sb.sb_rextsize > 1) { - fprintf(stderr, - _("%s: Reflink not compatible with realtime extent size > 1. Please try a newer xfsprogs.\n"), - progname); - return -1; - } - if (mp->m_rtdev_targp->bt_bdev == 0 && !xfs_is_debugger(mp)) { fprintf(stderr, _("%s: filesystem has a realtime subvolume\n"), progname); diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index 83ab4a6c316..f2961c4d134 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -2467,24 +2467,6 @@ _("parent pointers not supported on v4 filesystems\n")); } if (cli->xi->rt.name) { - if (cli->rtextsize && cli->sb_feat.reflink) { - if (cli_opt_set(&mopts, M_REFLINK)) { - fprintf(stderr, -_("reflink not supported on realtime devices with rt extent size specified\n")); - usage(); - } - cli->sb_feat.reflink = false; - } - if (cli->blocksize < XFS_MIN_RTEXTSIZE && cli->sb_feat.reflink) { - if (cli_opt_set(&mopts, M_REFLINK)) { - fprintf(stderr, -_("reflink not supported on realtime devices with blocksize %d < %d\n"), - cli->blocksize, - XFS_MIN_RTEXTSIZE); - usage(); - } - cli->sb_feat.reflink = false; - } if (!cli->sb_feat.rtgroups && cli->sb_feat.reflink) { if (cli_opt_set(&mopts, M_REFLINK) && cli_opt_set(&ropts, R_RTGROUPS)) { @@ -2664,19 +2646,6 @@ validate_rtextsize( usage(); } cfg->rtextblocks = (xfs_extlen_t)(rtextbytes >> cfg->blocklog); - } else if (cli->sb_feat.reflink && cli->xi->rt.name) { - /* - * reflink doesn't support rt extent size > 1FSB yet, so set - * an extent size of 1FSB. Make sure we still satisfy the - * minimum rt extent size. - */ - if (cfg->blocksize < XFS_MIN_RTEXTSIZE) { - fprintf(stderr, - _("reflink not supported on rt volume with blocksize %d\n"), - cfg->blocksize); - usage(); - } - cfg->rtextblocks = 1; } else { /* * If realtime extsize has not been specified by the user, @@ -2708,12 +2677,6 @@ validate_rtextsize( } } ASSERT(cfg->rtextblocks); - - if (cli->sb_feat.reflink && cfg->rtblocks > 0 && cfg->rtextblocks > 1) { - fprintf(stderr, -_("reflink not supported on realtime with extent sizes > 1\n")); - usage(); - } } /* Validate the incoming extsize hint. */