From patchwork Mon Dec 23 23:17:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13919544 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EECC318A6D7; Mon, 23 Dec 2024 23:17:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995866; cv=none; b=m39SroKpy83XgKmMk1TXOdbOLR15xsjuHYEnG77P/HX61qlw1wPKLh/XUhQmzTL8NkU3RTIsqATQaPZJ1/GKwOybjFzvOdEieWIZcfFcXcUyFSwcDTjVLLRjkncrciLKS6Hp7OVHMYkKR5oUfxm8WFmlIj2LqD9qKgHfVNxahRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995866; c=relaxed/simple; bh=9F6E7B/+UpnV8oVtP3agx7yhISfWGR0CYE6CInjQsfk=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=uXO/A5+diEAyNWOwLuZnibqfF25OhICuyOzxvq2wgUVMTyAUI2zjm7EcuK6zu0na0ayBh9dcyDsF5tbROI0WzEmdSncbSg28NjDVNMr9HcRYrp9cr838KmvqTFlEkXYrKXlOnAEBoqtwF1MHcnVnY8kolIwJJHTWnr3lTGgVqcY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=k6t4vUNz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="k6t4vUNz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B9EFC4CED3; Mon, 23 Dec 2024 23:17:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734995865; bh=9F6E7B/+UpnV8oVtP3agx7yhISfWGR0CYE6CInjQsfk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=k6t4vUNz1Ng9wpDNTuqlyYK9VhAdZ7mI7v8djP6Y/cT9J9hqBp3JeI/ispzsSpeQV vKChh0Ds1LlEOPTUmrYdz7oPHJaJL+1QH0WdVTqQJNP9//q59gLs5svxOXFhtRjt1t 4xrBbpiZBQU1lA0NX4fLoG3uMCgFU1pKTvJGZ/4S8tGb7IO9ZQ8BB0F7MjC/c+MbMO peqwvzeb0MB+HovdNgkb4nX1PIgks6kH1VFZlFV9vG4p0IUAcBzGmY46Fxdhxcvaon SwnwY5HcI3gm8WbJxeLFdTgJtJVA5DZ1Yg71e9SlssTaif0RcDmn8+cS6yyzZO0Ec9 gufi6dakKCcdw== Date: Mon, 23 Dec 2024 15:17:44 -0800 Subject: [GIT PULL 1/5] xfs: bug fixes for 6.13 From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: eflorac@intellique.com, hch@lst.de, linux-xfs@vger.kernel.org, stable@vger.kernel.org, syzbot+3126ab3db03db42e7a31@syzkaller.appspotmail.com Message-ID: <173499428661.2382820.13397448738755922887.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20241223224906.GS6174@frogsfrogsfrogs> References: <20241223224906.GS6174@frogsfrogsfrogs> Hi Carlos, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 4bbf9020becbfd8fc2c3da790855b7042fad455b: Linux 6.13-rc4 (2024-12-22 13:22:21 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/xfs-6.13-fixes_2024-12-23 for you to fetch changes up to 1aacd3fac248902ea1f7607f2d12b93929a4833b: xfs: release the dquot buf outside of qli_lock (2024-12-23 13:06:01 -0800) ---------------------------------------------------------------- xfs: bug fixes for 6.13 [01/14] Bug fixes for 6.13. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: "Darrick J. Wong" ---------------------------------------------------------------- Darrick J. Wong (2): xfs: don't over-report free space or inodes in statvfs xfs: release the dquot buf outside of qli_lock fs/xfs/xfs_dquot.c | 12 ++++++++---- fs/xfs/xfs_qm_bhv.c | 27 +++++++++++++++++---------- 2 files changed, 25 insertions(+), 14 deletions(-) From patchwork Mon Dec 23 23:18:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13919545 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1EBDB13FEE for ; Mon, 23 Dec 2024 23:18:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995881; cv=none; b=S3XjwOqV9mjcJtYdaXPis7m3lwoPfYfJSEYWZKO7qJpmxDgYiGocvmvuGKjB4NXWS8dr2uiOHptIf+WEedKDXYH795TQ7FX3AXPwAiMIovon1C1iL1dFLx8ahhaBy30EdkTBedPIRw0BDvQ9/6B9ekW3Cb6R6L+MUgeufx6VFCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995881; c=relaxed/simple; bh=ChjQ6Rs93Ss6unlj3ktZUL93JsDBJXOFNPGf20RqMA8=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=DwwrId669LQsrziwkQHtN75Uj2opSDLHzRXnO8fIqf/zJXnh5e1IMlP92bwus4uGgdGph+RU5oX4nGwXMGvrfsVm1YZMYJwmZ4GPQYZYdwpy8R6JcqdOETZCsXVDMuZpPy9xwZR9oQUxCbpjfVLcUleMD9jWWHpC5kjFPrcWen0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AChWrYhz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AChWrYhz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E53BFC4CED3; Mon, 23 Dec 2024 23:18:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734995881; bh=ChjQ6Rs93Ss6unlj3ktZUL93JsDBJXOFNPGf20RqMA8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=AChWrYhzLq9X6fHJivmAFBeTBSlYibJQMmhnJYM6UT6PyqC2Ii1g2ebLrDB1LufVJ r8pfUXb+lxttVLaFXYoANnnpkDfRi1mPlXn9Gzk1rYZXm7mmMut75jSpsaRQb2RM7o MT7YFVCxknbRbCb0zUA3PDZiZEa+eStZ0gWXIL4dfXF7joWqbuxNMrUjmWAC+46KEX dfg2C8pG3rxpfSn1ODmey2wSJTzOmef969wd1XDsVLFrwqboNu44RoilgqVi8SMGbk TMrAGQG9jps/gp5wENdqbcyXyqhr3tH6QQcEa1abwI79YJYgt7dumNJfAMavyZpVne XFKi0HV7vDhTQ== Date: Mon, 23 Dec 2024 15:18:00 -0800 Subject: [GIT PULL 2/5] xfs: refactor btrees to support records in inode root From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <173499428759.2382820.11756798556084282447.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20241223224906.GS6174@frogsfrogsfrogs> References: <20241223224906.GS6174@frogsfrogsfrogs> Hi Carlos, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 1aacd3fac248902ea1f7607f2d12b93929a4833b: xfs: release the dquot buf outside of qli_lock (2024-12-23 13:06:01 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/btree-ifork-records_2024-12-23 for you to fetch changes up to 2f63b20b7a26c9a7c76ea5a6565ca38cd9e31282: xfs: support storing records in the inode core root (2024-12-23 13:06:03 -0800) ---------------------------------------------------------------- xfs: refactor btrees to support records in inode root [v6.2 02/14] Amend the btree code to support storing btree rcords in the inode root, because the current bmbt code does not support this. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: "Darrick J. Wong" ---------------------------------------------------------------- Darrick J. Wong (8): xfs: tidy up xfs_iroot_realloc xfs: refactor the inode fork memory allocation functions xfs: make xfs_iroot_realloc take the new numrecs instead of deltas xfs: make xfs_iroot_realloc a bmap btree function xfs: tidy up xfs_bmap_broot_realloc a bit xfs: hoist the node iroot update code out of xfs_btree_new_iroot xfs: hoist the node iroot update code out of xfs_btree_kill_iroot xfs: support storing records in the inode core root fs/xfs/libxfs/xfs_bmap.c | 7 +- fs/xfs/libxfs/xfs_bmap_btree.c | 111 +++++++++++++ fs/xfs/libxfs/xfs_bmap_btree.h | 3 + fs/xfs/libxfs/xfs_btree.c | 333 ++++++++++++++++++++++++++++---------- fs/xfs/libxfs/xfs_btree.h | 18 ++- fs/xfs/libxfs/xfs_btree_staging.c | 9 +- fs/xfs/libxfs/xfs_inode_fork.c | 170 ++++++------------- fs/xfs/libxfs/xfs_inode_fork.h | 6 +- 8 files changed, 445 insertions(+), 212 deletions(-) From patchwork Mon Dec 23 23:18:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13919546 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0E8E7462 for ; Mon, 23 Dec 2024 23:18:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995896; cv=none; b=qwag5GW1hwLXh+WwUBIdgdKYlH2+YYcICSClpMa+IsNNF4aqUzznnPwVJrIiDXFbzJ4Jlt+tw3A8JaF2E1aTdIE8H4li4ZliuVuKUOU8lBtHk8Y4/V6/CaJlh7s2gQWlkfWptNJfV4oGUi1SkuG6kM682yrm3cgFD1tv6LyDcHY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995896; c=relaxed/simple; bh=IrGnXSFLU0ic85NWVsEzRJzFfHoLIUvZn8lwso30CY0=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=U/M69i4cUK8TRlTKWduZSJyz22icJWlKoZdCXMWPPhi+rVzzDOiawZ+odePDZu3mqjdt7nkUkmO4o4Unjhb5zJfvWboNxpcdqt/oFW8yfyntJoUECpgOWc3Scqn/4e1v1sGckfQfnZIG/QspqsoqBXa6izM+3cv2r7zvu3zL5aE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YxDFDQBB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YxDFDQBB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86E27C4CED3; Mon, 23 Dec 2024 23:18:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734995896; bh=IrGnXSFLU0ic85NWVsEzRJzFfHoLIUvZn8lwso30CY0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=YxDFDQBBa5TbDW5E02xG7exxrSrczYmEv1ATbSZpNfEbnBA9HfnwTxbHDsqfnj7VH qC/9g+tAcMhGVgl4BEhXGsrMouJD+vgGE5c9hORu2nc7Tyj/7VO3ZjSd9E4JymVnTG UKydddLZGTN821lenuBsph2A8LORv4uqVtVokRQN2Gof3fUrQYXUnt+EjlfJsrDpVn muXZmyiyzMAcubNQ2clu3/FJQjT0m0KC61N9DVT1cf6gpi5OA5n4ghOoyWghrm5Vkt UsjWWSmvYsBipxazrgqYaR8+ZGAEgU+8hOfvzIwlp1l9GrPOIh2wKvHxi+uctcvOhj 8BRW/3zufwqyg== Date: Mon, 23 Dec 2024 15:18:16 -0800 Subject: [GIT PULL 3/5] xfs: enable in-core block reservation for rt metadata From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <173499428862.2382820.11178785771990555823.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20241223224906.GS6174@frogsfrogsfrogs> References: <20241223224906.GS6174@frogsfrogsfrogs> Hi Carlos, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 2f63b20b7a26c9a7c76ea5a6565ca38cd9e31282: xfs: support storing records in the inode core root (2024-12-23 13:06:03 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/reserve-rt-metadata-space_2024-12-23 for you to fetch changes up to 05290bd5c6236b8ad659157edb36bd2d38f46d3e: xfs: allow inode-based btrees to reserve space in the data device (2024-12-23 13:06:03 -0800) ---------------------------------------------------------------- xfs: enable in-core block reservation for rt metadata [v6.2 03/14] In preparation for adding reverse mapping and refcounting to the realtime device, enhance the metadir code to reserve free space for btree shape changes as delayed allocation blocks. This enables us to pre-allocate space for the rmap and refcount btrees in the same manner as we do for the data device counterparts, which is how we avoid ENOSPC failures when space is low but we've already committed to a COW operation. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: "Darrick J. Wong" ---------------------------------------------------------------- Darrick J. Wong (2): xfs: prepare to reuse the dquot pointer space in struct xfs_inode xfs: allow inode-based btrees to reserve space in the data device fs/xfs/libxfs/xfs_ag_resv.c | 3 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_bmap.c | 4 +- fs/xfs/libxfs/xfs_errortag.h | 4 +- fs/xfs/libxfs/xfs_metadir.c | 4 + fs/xfs/libxfs/xfs_metafile.c | 205 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_metafile.h | 11 +++ fs/xfs/libxfs/xfs_types.h | 7 ++ fs/xfs/scrub/tempfile.c | 1 + fs/xfs/xfs_dquot.h | 3 + fs/xfs/xfs_error.c | 3 + fs/xfs/xfs_exchrange.c | 3 + fs/xfs/xfs_fsops.c | 17 ++++ fs/xfs/xfs_inode.h | 16 +++- fs/xfs/xfs_mount.c | 10 +++ fs/xfs/xfs_mount.h | 1 + fs/xfs/xfs_qm.c | 2 + fs/xfs/xfs_quota.h | 5 -- fs/xfs/xfs_rtalloc.c | 21 +++++ fs/xfs/xfs_rtalloc.h | 5 ++ fs/xfs/xfs_trace.h | 45 ++++++++++ fs/xfs/xfs_trans.c | 4 + fs/xfs/xfs_trans_dquot.c | 8 +- 23 files changed, 367 insertions(+), 19 deletions(-) From patchwork Mon Dec 23 23:18:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13919547 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B033718A6D7; Mon, 23 Dec 2024 23:18:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995912; cv=none; b=kDzwM14LBlzHMekIDhHylq8Mzji5SyiEmyb8KHAf2FrTQuhNVK8kuZRCWYNcBcLqMckSYWf4BIYX2IYp/MGJdvmhJyKjgN2Qq+nMbu1sD3tr+0Sdh2E+RX2vJD0wbckbith8DV0dvsQcrPY+0Pso1L94KeY4DU3nZhAn7hoFiRY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995912; c=relaxed/simple; bh=NcoRf7TwfIHicJvtCwxq4k871Edw73uuPcAVKmAb41g=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=X+x5Eg/cBDiL3wjqIsPgL4A6Ef7hfcjRjs2tS5/Ey5GhbXpXHlMwUnhBvAjres0p2qLi8rQ1qPlIC5x/DWPx7yQ1vc/IEm1dtqH157+SCBSuIp9gTuAI0b52qy99U1BvdEe5e45V/DxuU44hz6ZRiRH31amU3il/qzB8q3AmeOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=V4NoSeeB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V4NoSeeB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 321C1C4CED3; Mon, 23 Dec 2024 23:18:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734995912; bh=NcoRf7TwfIHicJvtCwxq4k871Edw73uuPcAVKmAb41g=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=V4NoSeeBiPWVNq4NIYDfdKiHECENkTECdoL0DGzqJ56a9dNhV0W3tJoMOtpUHekHi 5AgxRwmmMUtdLoHLLOL2cWEmUwSdXWlOkVzr7r9pJbHjy+D2PNuTN+cR0UGEXDfP8b smpZDmwkY1WJPFGlnEJGLZzHo06N2IQpbe++4fa1I9wNM1Ce/rJ7EPJAt4zkcBWULa SsUGwcviKE1ryJnsd5l+bncuL8nLnwieevZY15DQEpxIKVIC0dV5CvAw8V2Xi5BdlR qvLIf++TFzROMsIHHc6s/RSeFKzR9qxAJV3vqHK3J2sKIC8MJgkBgdLIUd0ZlHHV4j 4zO5oQlEVfOlg== Date: Mon, 23 Dec 2024 15:18:31 -0800 Subject: [GIT PULL 4/5] xfs: realtime reverse-mapping support From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org, stable@vger.kernel.org Message-ID: <173499428975.2382820.17644608915723809135.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20241223224906.GS6174@frogsfrogsfrogs> References: <20241223224906.GS6174@frogsfrogsfrogs> Hi Carlos, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 05290bd5c6236b8ad659157edb36bd2d38f46d3e: xfs: allow inode-based btrees to reserve space in the data device (2024-12-23 13:06:03 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/realtime-rmap_2024-12-23 for you to fetch changes up to c2358439af374cad47f771797875d0beb8256738: xfs: enable realtime rmap btree (2024-12-23 13:06:09 -0800) ---------------------------------------------------------------- xfs: realtime reverse-mapping support [v6.2 04/14] This is the latest revision of a patchset that adds to XFS kernel support for reverse mapping for the realtime device. This time around I've fixed some of the bitrot that I've noticed over the past few months, and most notably have converted rtrmapbt to use the metadata inode directory feature instead of burning more space in the superblock. At the beginning of the set are patches to implement storing B+tree leaves in an inode root, since the realtime rmapbt is rooted in an inode, unlike the regular rmapbt which is rooted in an AG block. Prior to this, the only btree that could be rooted in the inode fork was the block mapping btree; if all the extent records fit in the inode, format would be switched from 'btree' to 'extents'. The next few patches enhance the reverse mapping routines to handle the parts that are specific to rtgroups -- adding the new btree type, adding a new log intent item type, and wiring up the metadata directory tree entries. Finally, implement GETFSMAP with the rtrmapbt and scrub functionality for the rtrmapbt and rtbitmap and online fsck functionality. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: "Darrick J. Wong" ---------------------------------------------------------------- Darrick J. Wong (37): xfs: add some rtgroup inode helpers xfs: prepare rmap btree cursor tracepoints for realtime xfs: simplify the xfs_rmap_{alloc,free}_extent calling conventions xfs: introduce realtime rmap btree ondisk definitions xfs: realtime rmap btree transaction reservations xfs: add realtime rmap btree operations xfs: prepare rmap functions to deal with rtrmapbt xfs: add a realtime flag to the rmap update log redo items xfs: support recovering rmap intent items targetting realtime extents xfs: pretty print metadata file types in error messages xfs: support file data forks containing metadata btrees xfs: add realtime reverse map inode to metadata directory xfs: add metadata reservations for realtime rmap btrees xfs: wire up a new metafile type for the realtime rmap xfs: wire up rmap map and unmap to the realtime rmapbt xfs: create routine to allocate and initialize a realtime rmap btree inode xfs: wire up getfsmap to the realtime reverse mapping btree xfs: check that the rtrmapbt maxlevels doesn't increase when growing fs xfs: report realtime rmap btree corruption errors to the health system xfs: allow queued realtime intents to drain before scrubbing xfs: scrub the realtime rmapbt xfs: cross-reference realtime bitmap to realtime rmapbt scrubber xfs: cross-reference the realtime rmapbt xfs: scan rt rmap when we're doing an intense rmap check of bmbt mappings xfs: scrub the metadir path of rt rmap btree files xfs: walk the rt reverse mapping tree when rebuilding rmap xfs: online repair of realtime file bmaps xfs: repair inodes that have realtime extents xfs: repair rmap btree inodes xfs: online repair of realtime bitmaps for a realtime group xfs: support repairing metadata btrees rooted in metadir inodes xfs: online repair of the realtime rmap btree xfs: create a shadow rmap btree during realtime rmap repair xfs: hook live realtime rmap operations during a repair operation xfs: don't shut down the filesystem for media failures beyond end of log xfs: react to fsdax failure notifications on the rt device xfs: enable realtime rmap btree fs/xfs/Makefile | 3 + fs/xfs/libxfs/xfs_btree.c | 73 +++ fs/xfs/libxfs/xfs_btree.h | 8 +- fs/xfs/libxfs/xfs_btree_mem.c | 1 + fs/xfs/libxfs/xfs_btree_staging.c | 1 + fs/xfs/libxfs/xfs_defer.h | 1 + fs/xfs/libxfs/xfs_exchmaps.c | 4 +- fs/xfs/libxfs/xfs_format.h | 28 +- fs/xfs/libxfs/xfs_fs.h | 7 +- fs/xfs/libxfs/xfs_health.h | 4 +- fs/xfs/libxfs/xfs_inode_buf.c | 32 +- fs/xfs/libxfs/xfs_inode_fork.c | 25 + fs/xfs/libxfs/xfs_log_format.h | 6 +- fs/xfs/libxfs/xfs_log_recover.h | 2 + fs/xfs/libxfs/xfs_metafile.c | 18 + fs/xfs/libxfs/xfs_metafile.h | 2 + fs/xfs/libxfs/xfs_ondisk.h | 2 + fs/xfs/libxfs/xfs_refcount.c | 6 +- fs/xfs/libxfs/xfs_rmap.c | 171 +++++-- fs/xfs/libxfs/xfs_rmap.h | 12 +- fs/xfs/libxfs/xfs_rtbitmap.c | 2 +- fs/xfs/libxfs/xfs_rtbitmap.h | 9 + fs/xfs/libxfs/xfs_rtgroup.c | 53 +- fs/xfs/libxfs/xfs_rtgroup.h | 49 +- fs/xfs/libxfs/xfs_rtrmap_btree.c | 1011 +++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_rtrmap_btree.h | 210 ++++++++ fs/xfs/libxfs/xfs_sb.c | 6 + fs/xfs/libxfs/xfs_shared.h | 14 + fs/xfs/libxfs/xfs_trans_resv.c | 12 +- fs/xfs/libxfs/xfs_trans_space.h | 13 + fs/xfs/scrub/alloc_repair.c | 5 +- fs/xfs/scrub/bmap.c | 108 +++- fs/xfs/scrub/bmap_repair.c | 129 ++++- fs/xfs/scrub/common.c | 160 +++++- fs/xfs/scrub/common.h | 23 +- fs/xfs/scrub/health.c | 1 + fs/xfs/scrub/inode.c | 10 +- fs/xfs/scrub/inode_repair.c | 136 ++++- fs/xfs/scrub/metapath.c | 3 + fs/xfs/scrub/newbt.c | 42 ++ fs/xfs/scrub/newbt.h | 1 + fs/xfs/scrub/reap.c | 41 ++ fs/xfs/scrub/reap.h | 2 + fs/xfs/scrub/repair.c | 191 +++++++ fs/xfs/scrub/repair.h | 17 + fs/xfs/scrub/rgsuper.c | 6 +- fs/xfs/scrub/rmap_repair.c | 84 ++- fs/xfs/scrub/rtbitmap.c | 75 ++- fs/xfs/scrub/rtbitmap.h | 55 ++ fs/xfs/scrub/rtbitmap_repair.c | 429 +++++++++++++++- fs/xfs/scrub/rtrmap.c | 271 ++++++++++ fs/xfs/scrub/rtrmap_repair.c | 903 +++++++++++++++++++++++++++++++++ fs/xfs/scrub/rtsummary.c | 17 +- fs/xfs/scrub/rtsummary_repair.c | 3 +- fs/xfs/scrub/scrub.c | 11 +- fs/xfs/scrub/scrub.h | 14 + fs/xfs/scrub/stats.c | 1 + fs/xfs/scrub/tempexch.h | 2 +- fs/xfs/scrub/tempfile.c | 20 +- fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 228 ++++++++- fs/xfs/xfs_buf.c | 1 + fs/xfs/xfs_buf_item_recover.c | 4 + fs/xfs/xfs_drain.c | 20 +- fs/xfs/xfs_drain.h | 7 +- fs/xfs/xfs_fsmap.c | 174 ++++++- fs/xfs/xfs_fsops.c | 11 + fs/xfs/xfs_health.c | 1 + fs/xfs/xfs_inode.c | 19 +- fs/xfs/xfs_inode_item.c | 2 + fs/xfs/xfs_inode_item_recover.c | 44 +- fs/xfs/xfs_log_recover.c | 2 + fs/xfs/xfs_mount.c | 5 +- fs/xfs/xfs_mount.h | 9 + fs/xfs/xfs_notify_failure.c | 230 +++++---- fs/xfs/xfs_notify_failure.h | 11 + fs/xfs/xfs_qm.c | 8 +- fs/xfs/xfs_rmap_item.c | 216 +++++++- fs/xfs/xfs_rtalloc.c | 82 ++- fs/xfs/xfs_rtalloc.h | 10 + fs/xfs/xfs_stats.c | 4 +- fs/xfs/xfs_stats.h | 2 + fs/xfs/xfs_super.c | 6 - fs/xfs/xfs_super.h | 1 - fs/xfs/xfs_trace.h | 104 ++-- 85 files changed, 5381 insertions(+), 366 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_rtrmap_btree.c create mode 100644 fs/xfs/libxfs/xfs_rtrmap_btree.h create mode 100644 fs/xfs/scrub/rtrmap.c create mode 100644 fs/xfs/scrub/rtrmap_repair.c create mode 100644 fs/xfs/xfs_notify_failure.h From patchwork Mon Dec 23 23:18:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13919548 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A2CE198E60 for ; Mon, 23 Dec 2024 23:18:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995928; cv=none; b=bTA3PHh0VkP6cL4tsdiXlO9SEfzPo7ZGljKo8gWod/0HPTStrRxrXw24gZtR7zbzE2iimG6SY5gXt3dfWRS/QiWoJrigUWy11VkDT4lUOyT/PKoGyZZCLW7FHtfDuz4I2F8DZV22p4HCuT0Hwx6dG1+zfxuRV6dicKTaBVGFkT0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734995928; c=relaxed/simple; bh=AJMLK0sYMard/kzxos44iQgxJhVTACmQkaSHvphdZZs=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=DUXhDV7+V+Sr+f09m/0+Ig78ssNCuoMJwIx0LdfJqofrYXUOaG2FuOspdjGXkBsIfOXcMOeF4jGVuNfrQ3UeWRVUI5aP00q/PAx0K74/H4OCYAREPqqi83CXIYpAPKQKGSisH0+Y3bTpmUNchLFoC9Hb9w7StTscITRKErUZmgs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SMtQBcJA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SMtQBcJA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C6C6DC4CED3; Mon, 23 Dec 2024 23:18:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734995927; bh=AJMLK0sYMard/kzxos44iQgxJhVTACmQkaSHvphdZZs=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=SMtQBcJArEeF510slx1vk55oQb68/EW8eg8dx+9dWhm8Yyf7iJuXUSc3Xbu+07R5j 23KK/obGWASXLQL7OjWJbsJf6Sv7AmLNm53oHZJprfr3OXkFNfViFnE7BWdPXPys5I aZsTEy3mEJLjWGsl7zNtJOvuPCiekv6wqKoSERYlfhagDQNpHaCSbcUsDMH0wrwWrp 7qNapRD0mDatILYPLfOVLhq+1jND8EKi0fN0zo8YlF2MmstecIamrrkZIpHsy9NWuo 4lEA5x6N/d/MFMC3S5GBEeLToIpDf7JI5cF3iQRrGs77r5b5cs7Bgfo6TvngGCrkzT Vh5rtNurL6ygA== Date: Mon, 23 Dec 2024 15:18:47 -0800 Subject: [GIT PULL 5/5] xfs: reflink on the realtime device From: "Darrick J. Wong" To: cem@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <173499429082.2382820.13008491658242175877.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20241223224906.GS6174@frogsfrogsfrogs> References: <20241223224906.GS6174@frogsfrogsfrogs> Hi Carlos, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit c2358439af374cad47f771797875d0beb8256738: xfs: enable realtime rmap btree (2024-12-23 13:06:09 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/realtime-reflink_2024-12-23 for you to fetch changes up to 155debbe7e627baf6439e75f15856d40ec115c5d: xfs: enable realtime reflink (2024-12-23 13:06:17 -0800) ---------------------------------------------------------------- xfs: reflink on the realtime device [v6.2 05/14] This patchset enables use of the file data block sharing feature (i.e. reflink) on the realtime device. It follows the same basic sequence as the realtime rmap series -- first a few cleanups; then introduction of the new btree format and inode fork format. Next comes enabling CoW and remapping for the rt device; new scrub, repair, and health reporting code; and at the end we implement some code to lengthen write requests so that rt extents are always CoWed fully. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: "Darrick J. Wong" ---------------------------------------------------------------- Christoph Hellwig (1): xfs: refactor xfs_reflink_find_shared Darrick J. Wong (42): xfs: prepare refcount btree cursor tracepoints for realtime xfs: namespace the maximum length/refcount symbols xfs: introduce realtime refcount btree ondisk definitions xfs: realtime refcount btree transaction reservations xfs: add realtime refcount btree operations xfs: prepare refcount functions to deal with rtrefcountbt xfs: add a realtime flag to the refcount update log redo items xfs: support recovering refcount intent items targetting realtime extents xfs: add realtime refcount btree block detection to log recovery xfs: add realtime refcount btree inode to metadata directory xfs: add metadata reservations for realtime refcount btree xfs: wire up a new metafile type for the realtime refcount xfs: wire up realtime refcount btree cursors xfs: create routine to allocate and initialize a realtime refcount btree inode xfs: update rmap to allow cow staging extents in the rt rmap xfs: compute rtrmap btree max levels when reflink enabled xfs: refactor reflink quota updates xfs: enable CoW for realtime data xfs: enable sharing of realtime file blocks xfs: allow inodes to have the realtime and reflink flags xfs: recover CoW leftovers in the realtime volume xfs: fix xfs_get_extsz_hint behavior with realtime alwayscow files xfs: apply rt extent alignment constraints to CoW extsize hint xfs: enable extent size hints for CoW operations xfs: check that the rtrefcount maxlevels doesn't increase when growing fs xfs: report realtime refcount btree corruption errors to the health system xfs: scrub the realtime refcount btree xfs: cross-reference checks with the rt refcount btree xfs: allow overlapping rtrmapbt records for shared data extents xfs: check reference counts of gaps between rt refcount records xfs: allow dquot rt block count to exceed rt blocks on reflink fs xfs: detect and repair misaligned rtinherit directory cowextsize hints xfs: scrub the metadir path of rt refcount btree files xfs: don't flag quota rt block usage on rtreflink filesystems xfs: check new rtbitmap records against rt refcount btree xfs: walk the rt reference count tree when rebuilding rmap xfs: capture realtime CoW staging extents when rebuilding rt rmapbt xfs: online repair of the realtime refcount btree xfs: repair inodes that have a refcount btree in the data fork xfs: check for shared rt extents when rebuilding rt file's data fork xfs: fix CoW forks for realtime files xfs: enable realtime reflink fs/xfs/Makefile | 3 + fs/xfs/libxfs/xfs_bmap.c | 23 +- fs/xfs/libxfs/xfs_btree.c | 5 + fs/xfs/libxfs/xfs_btree.h | 2 +- fs/xfs/libxfs/xfs_defer.h | 1 + fs/xfs/libxfs/xfs_format.h | 25 +- fs/xfs/libxfs/xfs_fs.h | 7 +- fs/xfs/libxfs/xfs_health.h | 4 +- fs/xfs/libxfs/xfs_inode_buf.c | 33 +- fs/xfs/libxfs/xfs_inode_fork.c | 6 + fs/xfs/libxfs/xfs_log_format.h | 6 +- fs/xfs/libxfs/xfs_log_recover.h | 2 + fs/xfs/libxfs/xfs_ondisk.h | 2 + fs/xfs/libxfs/xfs_refcount.c | 276 +++++++++--- fs/xfs/libxfs/xfs_refcount.h | 23 +- fs/xfs/libxfs/xfs_rmap.c | 7 + fs/xfs/libxfs/xfs_rtgroup.c | 19 + fs/xfs/libxfs/xfs_rtgroup.h | 11 +- fs/xfs/libxfs/xfs_rtrefcount_btree.c | 757 +++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_rtrefcount_btree.h | 189 +++++++++ fs/xfs/libxfs/xfs_rtrmap_btree.c | 28 +- fs/xfs/libxfs/xfs_sb.c | 8 + fs/xfs/libxfs/xfs_shared.h | 7 + fs/xfs/libxfs/xfs_trans_resv.c | 25 +- fs/xfs/scrub/agheader_repair.c | 2 +- fs/xfs/scrub/bmap.c | 30 +- fs/xfs/scrub/bmap_repair.c | 21 +- fs/xfs/scrub/common.c | 10 + fs/xfs/scrub/common.h | 5 +- fs/xfs/scrub/cow_repair.c | 180 +++++++- fs/xfs/scrub/health.c | 1 + fs/xfs/scrub/inode.c | 31 +- fs/xfs/scrub/inode_repair.c | 57 ++- fs/xfs/scrub/metapath.c | 3 + fs/xfs/scrub/quota.c | 8 +- fs/xfs/scrub/quota_repair.c | 2 +- fs/xfs/scrub/reap.c | 247 ++++++++++- fs/xfs/scrub/reap.h | 7 + fs/xfs/scrub/refcount.c | 2 +- fs/xfs/scrub/refcount_repair.c | 6 +- fs/xfs/scrub/repair.c | 6 + fs/xfs/scrub/repair.h | 7 + fs/xfs/scrub/rgb_bitmap.h | 37 ++ fs/xfs/scrub/rmap_repair.c | 7 + fs/xfs/scrub/rtb_bitmap.h | 37 ++ fs/xfs/scrub/rtbitmap.c | 2 + fs/xfs/scrub/rtbitmap_repair.c | 24 +- fs/xfs/scrub/rtrefcount.c | 661 +++++++++++++++++++++++++++++ fs/xfs/scrub/rtrefcount_repair.c | 783 +++++++++++++++++++++++++++++++++++ fs/xfs/scrub/rtrmap.c | 54 ++- fs/xfs/scrub/rtrmap_repair.c | 103 +++++ fs/xfs/scrub/scrub.c | 7 + fs/xfs/scrub/scrub.h | 12 + fs/xfs/scrub/stats.c | 1 + fs/xfs/scrub/trace.h | 54 ++- fs/xfs/xfs_buf_item_recover.c | 4 + fs/xfs/xfs_fsmap.c | 25 +- fs/xfs/xfs_fsops.c | 2 + fs/xfs/xfs_health.c | 1 + fs/xfs/xfs_inode_item.c | 14 + fs/xfs/xfs_inode_item_recover.c | 4 + fs/xfs/xfs_ioctl.c | 21 +- fs/xfs/xfs_log_recover.c | 2 + fs/xfs/xfs_mount.c | 5 +- fs/xfs/xfs_mount.h | 9 + fs/xfs/xfs_refcount_item.c | 240 ++++++++++- fs/xfs/xfs_reflink.c | 325 +++++++++++---- fs/xfs/xfs_reflink.h | 4 +- fs/xfs/xfs_rtalloc.c | 24 +- fs/xfs/xfs_rtalloc.h | 5 + fs/xfs/xfs_stats.c | 3 +- fs/xfs/xfs_stats.h | 1 + fs/xfs/xfs_super.c | 6 +- fs/xfs/xfs_trace.h | 111 +++-- 74 files changed, 4330 insertions(+), 352 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_rtrefcount_btree.c create mode 100644 fs/xfs/libxfs/xfs_rtrefcount_btree.h create mode 100644 fs/xfs/scrub/rgb_bitmap.h create mode 100644 fs/xfs/scrub/rtb_bitmap.h create mode 100644 fs/xfs/scrub/rtrefcount.c create mode 100644 fs/xfs/scrub/rtrefcount_repair.c