From patchwork Wed Apr 24 03:33:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641123 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 219C613B5A6 for ; Wed, 24 Apr 2024 03:33:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929635; cv=none; b=qIIPvt47DElOSX/eu8GQ9MhxD4j4ydDOcUntWgLz+VxCgO6hw4HYetvtn86Tfo8HOzaudQdle924IHzqQf7QLIeO+qcH/u15AvnbXoqxcjwKzNT6TGlYJvkjZaEG88DjOz+W6ezbhlkbMDMRuMQUAJ8zQFRBKcIswyI/cwqqt/o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929635; c=relaxed/simple; bh=TLA/+unjMvUmp5HFpTqyyeXW+mCJHEnbikOxG2rfRZM=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=pY0RkIHWerDe96NUeHRn7z4/jO+vUEDwuEhRTj4My+bmn3UgA6h3HiQmLzteF/woA6me2b6ftxW+cOxG1OJNeJpryQOWmtGTXGLdWumUpdcRsFLfeHp4SSQmj1+XTYqQStjlYQcfDxBz+ggs/2e3gaV++2YjSDFmZNcSBJ1mP0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZvvjWvCT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZvvjWvCT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5206C32781; Wed, 24 Apr 2024 03:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929634; bh=TLA/+unjMvUmp5HFpTqyyeXW+mCJHEnbikOxG2rfRZM=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ZvvjWvCTdv+AHq4NsSHgCs0Vo93ogD8HHUNKaFHAvL1+XkHSDjcYVn5pL2X0D3b7n WmfuoKo/IkUUVu/1dI3l+lZex2MGL/vWCevCd7PM1iiEbmS3MCEtIwR6LX2q3sd/fS jXhAkTqbpXprGRG/H172kfb9CIqNWojcSxFR12bN6pGS4YinZZcBuJ/Ot57a3bWs75 2FWzmSQZHP/rtWoW9r2yqh5lQ2Y6P9NRyCve57mmZq6oQgQYZ0ko2NHhPtPHNhdBw9 w2g3apbYZU2wTFm+pNJTOU13boJG//73/qSmU4K1gxgDJZj2z+K/nurUwuYqllgROU HlkgEJHgC/62w== Date: Tue, 23 Apr 2024 20:33:54 -0700 Subject: [GIT PULL 1/9] xfs: shrink struct xfs_da_args From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392949873.1941278.1022229329837312067.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 6a94b1acda7e7262418e23f906c12a2b08b69d12: xfs: reinstate delalloc for RT inodes (if sb_rextsize == 1) (2024-04-22 18:00:50 +0530) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/shrink-dirattr-args-6.10_2024-04-23 for you to fetch changes up to cda60317ac57add7a0a2865aa29afbc6caad3e9a: xfs: rearrange xfs_da_args a bit to use less space (2024-04-23 07:46:51 -0700) ---------------------------------------------------------------- xfs: shrink struct xfs_da_args [v13.4 1/9] Let's clean out some unused flags and fields from struct xfs_da_args. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (5): xfs: remove XFS_DA_OP_REMOVE xfs: remove XFS_DA_OP_NOTIME xfs: remove xfs_da_args.attr_flags xfs: make attr removal an explicit operation xfs: rearrange xfs_da_args a bit to use less space fs/xfs/libxfs/xfs_attr.c | 31 ++++++++++++++++--------------- fs/xfs/libxfs/xfs_attr.h | 11 +++++++++-- fs/xfs/libxfs/xfs_da_btree.h | 29 +++++++++++++---------------- fs/xfs/scrub/attr.c | 1 - fs/xfs/scrub/attr_repair.c | 3 +-- fs/xfs/xfs_acl.c | 17 +++++++++-------- fs/xfs/xfs_ioctl.c | 19 ++++++++++--------- fs/xfs/xfs_iops.c | 2 +- fs/xfs/xfs_trace.h | 7 +------ fs/xfs/xfs_xattr.c | 22 ++++++++++++++++++---- fs/xfs/xfs_xattr.h | 3 ++- 11 files changed, 80 insertions(+), 65 deletions(-) From patchwork Wed Apr 24 03:34:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641124 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E16513340B for ; Wed, 24 Apr 2024 03:34:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929651; cv=none; b=a5hei4rFHDYhSvRGmS0Bw7I5650tOdGqn8fDDcTmde5fAVpnjml6v8rsXHr63cv22/GmaNP8qSv6o/ioLnnES3UURPkxAj7r+0WuLlkQyWEOjulkYYWuxwsesxPev79G8vfcC9x5lzxTfej122/liCQQ80X9cPWOfzP2QLCTIE4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929651; c=relaxed/simple; bh=o9OZw06xLAozHMYR6wbHSkW33oziX3ra3evpo/Q8NOE=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=r3uHpL9R4N4C0wtUXLAbH1FquNJ5pjwtwxVDGjuUGKaFbjK+vQNk4uS3Yu7uhqZR+NKmcFvTlwVsjTyDzKkXwea6jM1OCjdLuZtwll0zdKuWW0RtoqWFnpWw1CClg6W6siFdhCg15oyZM0ArnEnChrW8/qfgy9ZGHv8Ovni7wWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EkV9F8ei; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EkV9F8ei" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 94E6BC116B1; Wed, 24 Apr 2024 03:34:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929650; bh=o9OZw06xLAozHMYR6wbHSkW33oziX3ra3evpo/Q8NOE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=EkV9F8eixbmkLejvVwhF+M3VP5BKxh/YHIuuKrrxZaOKr8wawMpIyyXuRT0T1fyYj w1wZYztLTrYtX6vQ0ZM+29HeHvjNsJbchlTtRyw6IxZ4FFxJwkZfaWHKnGxjz1fJ6m Y1GaCt5hN8eaWO21otLcRdRNrNADgrOW9qUQf0N6Qti57/qRZzdecyTjwrnNPJAMg8 lCaVCF3vYU8yDoeqnaBnoSpXFWCVJ1ReKEP7CWpfcAoARw/09ZlglzDj7qXo69NiNx iKTtV7xNGSDXAr8e+/D9G4QwrCpziwWJe6fGgzNdKiOQ+aNwCAFsMeoiLuub1uxVuX JV4JG757uNzKA== Date: Tue, 23 Apr 2024 20:34:10 -0700 Subject: [GIT PULL 2/9] xfs: improve extended attribute validation From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392950368.1941278.17612824185696231303.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit cda60317ac57add7a0a2865aa29afbc6caad3e9a: xfs: rearrange xfs_da_args a bit to use less space (2024-04-23 07:46:51 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/improve-attr-validation-6.10_2024-04-23 for you to fetch changes up to ea0b3e814741fb64e7785b564ea619578058e0b0: xfs: enforce one namespace per attribute (2024-04-23 07:46:54 -0700) ---------------------------------------------------------------- xfs: improve extended attribute validation [v13.4 2/9] Prior to introducing parent pointer extended attributes, let's spend some time cleaning up the attr code and strengthening the validation that it performs on attrs coming in from the disk. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (14): xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf xfs: require XFS_SB_FEAT_INCOMPAT_LOG_XATTRS for attr log intent item recovery xfs: use an XFS_OPSTATE_ flag for detecting if logged xattrs are available xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2 xfs: fix missing check for invalid attr flags xfs: check shortform attr entry flags specifically xfs: restructure xfs_attr_complete_op a bit xfs: use helpers to extract xattr op from opflags xfs: validate recovered name buffers when recovering xattr items xfs: always set args->value in xfs_attri_item_recover xfs: use local variables for name and value length in _attri_commit_pass2 xfs: refactor name/length checks in xfs_attri_validate xfs: refactor name/value iovec validation in xlog_recover_attri_commit_pass2 xfs: enforce one namespace per attribute fs/xfs/libxfs/xfs_attr.c | 37 ++++++- fs/xfs/libxfs/xfs_attr.h | 9 +- fs/xfs/libxfs/xfs_attr_leaf.c | 7 +- fs/xfs/libxfs/xfs_da_format.h | 5 + fs/xfs/scrub/attr.c | 34 ++++-- fs/xfs/scrub/attr_repair.c | 4 +- fs/xfs/xfs_attr_item.c | 242 +++++++++++++++++++++++++++++++++--------- fs/xfs/xfs_attr_list.c | 18 +++- fs/xfs/xfs_mount.c | 16 +++ fs/xfs/xfs_mount.h | 6 +- fs/xfs/xfs_xattr.c | 3 +- 11 files changed, 304 insertions(+), 77 deletions(-) From patchwork Wed Apr 24 03:34:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641125 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7713E13C3C1 for ; Wed, 24 Apr 2024 03:34:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929666; cv=none; b=c8SewnxbFub14vNgQ+jt8Fbw8VwFM/RKbQBnFpqkQvpq1u9oblkK5jD50iYRgEXTmV9Z2+aaFz35ARb7/rs7xcVexPmmJ2GoH8tXn7NqZDtmcREj6mcHytVY5d+uuhJPLogtolvdKuZLYLfwA8eNwCRv5V3PtNWtWU2zxvUGNaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929666; c=relaxed/simple; bh=yjkeVMSAW7G7K9INTwkfonNXlol4Fxmx2dYjzXvx6jc=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=cckRvBsr5pcQXrvmnEJujCKTBvQhWm76h1NEoir1dcrNUmjP01LvZGM1jMrNPGKr+2nL0nEKUzVK5hwDSobRrBxbdZ4HWmw9/fFuwrrbXIsFTlvnIvgjsBCg9Lx/H7SVnTOin0URgykErGwlmP5GQaEggZun/LVgmJ/4aWA0XZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rQExiCSx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rQExiCSx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 48DF6C2BD11; Wed, 24 Apr 2024 03:34:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929666; bh=yjkeVMSAW7G7K9INTwkfonNXlol4Fxmx2dYjzXvx6jc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=rQExiCSx0gw9F74OOYSaKszTZvPifOMyz4LGYAA8TJeMIjGKA16ZGRSwIagHkD62M bRxoS6soyzPbNf+V3u9dPZn10Vz0LZJhfkgsF3HgXPHQBjmbnncd6KjfVRg0z9uh5V Z/xkymLWDwjmeE7EhLc5R3dUBBw3dEO3OGV9tclGPZVfgIyIOFs0/gefgiuDb5nV6E CpRbvuWk+iiFoPnrAIlSY7G9SSsTVBd1pamxQhQgWH1Wm3pF9JSH/Ht07EpAMRXcxb bgp+W7LXvyeZ7YdXexHs39/kCEmZlRNfW8pK76UuallBy3ubxXdXSRiGgYjU6Md5Xk rxVgDulry9onA== Date: Tue, 23 Apr 2024 20:34:25 -0700 Subject: [GIT PULL 3/9] xfs: Parent Pointers From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: allison.henderson@oracle.com, catherine.hoang@oracle.com, darrick.wong@oracle.com, dchinner@redhat.com, hch@lst.de, linux-xfs@vger.kernel.org, mark.tinguely@oracle.com Message-ID: <171392950848.1941278.17369768789928295836.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit ea0b3e814741fb64e7785b564ea619578058e0b0: xfs: enforce one namespace per attribute (2024-04-23 07:46:54 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/pptrs-6.10_2024-04-23 for you to fetch changes up to 67ac7091e35bd34b75c0ec77331b53ca052e0cb3: xfs: enable parent pointers (2024-04-23 07:47:01 -0700) ---------------------------------------------------------------- xfs: Parent Pointers [v13.4 3/9] This is the latest parent pointer attributes for xfs. The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point. Directory parent pointers are stored as namespaced extended attributes of a file. Because parent pointers are an indivisible tuple of (dirent_name, parent_ino, parent_gen) we cannot use the usual attr name lookup functions to find a parent pointer. This is solvable by introducing a new lookup mode that checks both the name and the value of the xattr. Therefore, introduce this new name-value lookup mode that's gated on the XFS_ATTR_PARENT namespace. This requires the introduction of new opcodes for the extended attribute update log intent items, which actually means that parent pointers (itself an INCOMPAT feature) does not depend on the LOGGED_XATTRS log incompat feature bit. To reduce collisions on the dirent names of parent pointers, introduce a new attr hash mode that is the dir2 namehash of the dirent name xor'd with the parent inode number. At this point, Allison has moved on to other things, so I've merged her patchset into djwong-dev for merging. Updates since v12 [djwong]: Rebase on 6.9-rc and update the online fsck design document. Redesign the ondisk format to use the name-value lookups to get us back to the point where the attr is (dirent_name -> parent_ino/gen). Updates since v11 [djwong]: Rebase on 6.4-rc and make some tweaks and bugfixes to enable the repair prototypes. Merge with djwong-dev and make online repair actually work. Updates since v10 [djwong]: Merge in the ondisk format changes to get rid of the diroffset conflicts with the parent pointer repair code, rebase the entire series with the attr vlookup changes first, and merge all the other random fixes. Updates since v9: Reordered patches 2 and 3 to be 6 and 7 xfs: Add xfs_verify_pptr moved parent pointer validators to xfs_parent xfs: Add parent pointer ioctl Extra validation checks for fs id added missing release for the inode use GFP_KERNEL flags for malloc/realloc reworked ioctl to use pptr listenty and flex array NEW xfs: don't remove the attr fork when parent pointers are enabled NEW directory lookups should return diroffsets too NEW xfs: move/add parent pointer validators to xfs_parent Updates since v8: xfs: parent pointer attribute creation Fix xfs_parent_init to release log assist on alloc fail Add slab cache for xfs_parent_defer Fix xfs_create to release after unlock Add xfs_parent_start and xfs_parent_finish wrappers removed unused xfs_parent_name_irec and xfs_init_parent_name_irec xfs: add parent attributes to link Start/finish wrapper updates Fix xfs_link to disallow reservationless quotas xfs: add parent attributes to symlink Fix xfs_symlink to release after unlock Start/finish wrapper updates xfs: remove parent pointers in unlink Start/finish wrapper updates Add missing parent free xfs: Add parent pointers to rename Start/finish wrapper updates Fix rename to only grab logged xattr once Fix xfs_rename to disallow reservationless quotas Fix double unlock on dqattach fail Move parent frees to out_release_wip xfs: Add parent pointers to xfs_cross_rename Hoist parent pointers into rename Questions comments and feedback appreciated! Thanks all! Allison This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Allison Henderson (15): xfs: add parent pointer support to attribute code xfs: define parent pointer ondisk extended attribute format xfs: Expose init_xattrs in xfs_create_tmpfile xfs: add parent pointer validator functions xfs: extend transaction reservations for parent attributes xfs: parent pointer attribute creation xfs: add parent attributes to link xfs: add parent attributes to symlink xfs: remove parent pointers in unlink xfs: Add parent pointers to rename xfs: Add parent pointers to xfs_cross_rename xfs: don't return XFS_ATTR_PARENT attributes via listxattr xfs: pass the attr value to put_listent when possible xfs: don't remove the attr fork when parent pointers are enabled xfs: add a incompat feature bit for parent pointers Christoph Hellwig (1): xfs: check the flags earlier in xfs_attr_match Darrick J. Wong (14): xfs: rearrange xfs_attr_match parameters xfs: move xfs_attr_defer_add to xfs_attr_item.c xfs: create a separate hashname function for extended attributes xfs: allow xattr matching on name and value for parent pointers xfs: refactor xfs_is_using_logged_xattrs checks in attr item recovery xfs: create attr log item opcodes and formats for parent pointers xfs: record inode generation in xattr update log intent items xfs: create a hashname function for parent pointers xfs: move handle ioctl code to xfs_handle.c xfs: split out handle management helpers a bit xfs: add parent pointer ioctls xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res xfs: drop compatibility minimum log size computations for reflink xfs: enable parent pointers fs/xfs/Makefile | 3 + fs/xfs/libxfs/xfs_attr.c | 92 ++-- fs/xfs/libxfs/xfs_attr.h | 23 +- fs/xfs/libxfs/xfs_attr_leaf.c | 87 +++- fs/xfs/libxfs/xfs_attr_sf.h | 1 + fs/xfs/libxfs/xfs_da_btree.h | 4 + fs/xfs/libxfs/xfs_da_format.h | 25 +- fs/xfs/libxfs/xfs_format.h | 4 +- fs/xfs/libxfs/xfs_fs.h | 79 +++- fs/xfs/libxfs/xfs_log_format.h | 25 +- fs/xfs/libxfs/xfs_log_rlimit.c | 46 ++ fs/xfs/libxfs/xfs_ondisk.h | 6 + fs/xfs/libxfs/xfs_parent.c | 293 +++++++++++++ fs/xfs/libxfs/xfs_parent.h | 99 +++++ fs/xfs/libxfs/xfs_sb.c | 4 + fs/xfs/libxfs/xfs_trans_resv.c | 326 +++++++++++--- fs/xfs/libxfs/xfs_trans_space.c | 121 +++++ fs/xfs/libxfs/xfs_trans_space.h | 25 +- fs/xfs/scrub/attr.c | 15 +- fs/xfs/scrub/dir_repair.c | 2 +- fs/xfs/scrub/orphanage.c | 5 +- fs/xfs/scrub/parent_repair.c | 3 +- fs/xfs/scrub/symlink_repair.c | 2 +- fs/xfs/scrub/tempfile.c | 2 +- fs/xfs/xfs_attr_item.c | 331 ++++++++++++-- fs/xfs/xfs_attr_item.h | 10 + fs/xfs/xfs_attr_list.c | 13 +- fs/xfs/xfs_export.c | 2 +- fs/xfs/xfs_export.h | 2 + fs/xfs/xfs_handle.c | 952 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_handle.h | 33 ++ fs/xfs/xfs_inode.c | 218 +++++++-- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_ioctl.c | 595 +------------------------ fs/xfs/xfs_ioctl.h | 28 -- fs/xfs/xfs_ioctl32.c | 1 + fs/xfs/xfs_iops.c | 15 +- fs/xfs/xfs_super.c | 14 + fs/xfs/xfs_symlink.c | 30 +- fs/xfs/xfs_trace.c | 1 + fs/xfs/xfs_trace.h | 95 +++- fs/xfs/xfs_xattr.c | 5 + 42 files changed, 2807 insertions(+), 832 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/libxfs/xfs_trans_space.c create mode 100644 fs/xfs/xfs_handle.c create mode 100644 fs/xfs/xfs_handle.h From patchwork Wed Apr 24 03:34:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641126 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A04B13340B for ; Wed, 24 Apr 2024 03:34:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929682; cv=none; b=G2qqOSf7pEM98b/YdeYIaK7YI6eXtnyItLBzoQFNAj4BnJyZst2JNtS2uwzJ0Xg67RcEkLfWH+VySxJfDu7WilNahFyq6++b7DayR3Lpi3Dyl4yFR35iaSyzvEc0GQzcEoNmaPiuKYtfc2mym+/Sdkh60uZe+BOZzDCKJheo98M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929682; c=relaxed/simple; bh=Oyt3BaF223uFAOKojHp++gOTw9S1IZzhCHt98yjkiaE=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=IDVGJxgej8vem6+kZ2+Ux0B+dETS4z2WWNwY89dMuVD90SZs8ixovd4YVa7YAySanmeus3WcRZYOFwMpDhViwp25WT1mRRMYApay3GJsKBYBznRLQWCuoBxdLioSQeOBHbLFPbw1erqBIV3XKVSCkS/ponKzC8fAscfDgOIDrxU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=I7+jIRJ2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="I7+jIRJ2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E24A8C116B1; Wed, 24 Apr 2024 03:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929682; bh=Oyt3BaF223uFAOKojHp++gOTw9S1IZzhCHt98yjkiaE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=I7+jIRJ2a3jMheR9e+eojNcye6DxABS7X019NKwMyvJ/KM2AoFYXmxc5zZFp35cYm BQ8R738YWG6nTpTaFwTcqbBp350yn0nofKXA/ioF/KntVTZocaFfeaKEgKGGGPutdo 3E0w7Z46XyW3rbHdgm7xMyMe19kW5xnVyPowR4DzUxKHVYj2DAxX/LFB8pzT1Ss96d qw/fhfKDRPotvisiaEVsLD/2rTVvlb4GZZ+E2rnQj4DbA/vtDFZ9490r+PBtxg6ziy lTs3cH0IHswyd61Iu4sUkg05SwfCIWY44DHVUMjhkEdY/6qlMAd03KLgryTgBSRUI4 nQxP+yezqa//A== Date: Tue, 23 Apr 2024 20:34:41 -0700 Subject: [GIT PULL 4/9] xfs: scrubbing for parent pointers From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: allison.henderson@oracle.com, catherine.hoang@oracle.com, hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392951311.1941278.4476150123059380764.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 67ac7091e35bd34b75c0ec77331b53ca052e0cb3: xfs: enable parent pointers (2024-04-23 07:47:01 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-pptrs-6.10_2024-04-23 for you to fetch changes up to 59a2af9086f0d60fc8de7346da67db7d764c7221: xfs: check parent pointer xattrs when scrubbing (2024-04-23 07:47:03 -0700) ---------------------------------------------------------------- xfs: scrubbing for parent pointers [v13.4 4/9] Teach online fsck to use parent pointers to assist in checking directories, parent pointers, extended attributes, and link counts. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (7): xfs: revert commit 44af6c7e59b12 xfs: check dirents have parent pointers xfs: deferred scrub of dirents xfs: scrub parent pointers xfs: deferred scrub of parent pointers xfs: walk directory parent pointers to determine backref count xfs: check parent pointer xattrs when scrubbing fs/xfs/Makefile | 2 +- fs/xfs/libxfs/xfs_parent.c | 22 ++ fs/xfs/libxfs/xfs_parent.h | 5 + fs/xfs/scrub/attr.c | 27 +- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/dir.c | 342 ++++++++++++++++++++- fs/xfs/scrub/nlinks.c | 85 +++++- fs/xfs/scrub/nlinks_repair.c | 2 + fs/xfs/scrub/parent.c | 685 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/readdir.c | 78 +++++ fs/xfs/scrub/readdir.h | 3 + fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 65 ++++ 13 files changed, 1307 insertions(+), 11 deletions(-) From patchwork Wed Apr 24 03:34:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641127 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7B4413340B for ; Wed, 24 Apr 2024 03:34:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929697; cv=none; b=Xexlo/TVaEaJR/z3iUXBO6cfzuK9+nQ5w+6zNZPwQH0DhKHrFTKT4h560rtRc7BP1xKNlSHOQMsrevCwxUZV+zfcdLtXKkIziT5WnSQ1Au1zsbD9MQj7xW+J7/FVhGYUIMdIOKCZfC3vCRwUQCjMwG+/bUE+lcAJZAkUBkbjBaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929697; c=relaxed/simple; bh=I/hDE0wvXKvfg9CKDlO3S7C/S2bXNMviAikDbe3z0kY=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=FGr1gaxMUin2X0/g01RQcqe4gYGAi3NvaWaCanFhiXc9dLAAkCdFNbMIcsNR04h+aMFCW+gECqF4NVYRV2TUg58BEoDDsmgeWrys/VwmMTENU/DuVaTYq4o1mDHkEjZYd+wMq/VIe/oR2vco0Qnk3cCv6YHVsGRqUQL9e0z880o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ouK/LhsK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ouK/LhsK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 911CCC116B1; Wed, 24 Apr 2024 03:34:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929697; bh=I/hDE0wvXKvfg9CKDlO3S7C/S2bXNMviAikDbe3z0kY=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ouK/LhsKqHiZX2DyS8e2u0HMVPLX+/yyZvJBMXyNNl5jJGeLQ/vNBAal2Ur7BZHPt ngu05rmISiWeizTGV+FsbC0+btj+0uu/c9VEJDpHOhR5SsQWgMn7P5lj2iZAScZZkt 3zhIUKp2smOvsiKpegGmSYc+N4/q1GyeLRKnJvDZNz23RQcEDpMNyxb4QCpS5OF5gM qx0SO3rAPNyP/eyjTBwjmh97YQH/Vlw3P3FDixSx1VBTUnMFbn/3DdrSfFmthMF9S6 HWzNyXoM0NQMXwQ6ThKEFLXxBZGnuXYtJFu6zzoBUiAXfB0ssdnCs4RrX+oZhUyJWP JlEMU4Sg9rfkw== Date: Tue, 23 Apr 2024 20:34:57 -0700 Subject: [GIT PULL 5/9] xfs: online repair for parent pointers From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: allison.henderson@oracle.com, catherine.hoang@oracle.com, hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392951775.1941278.427384803477410943.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 59a2af9086f0d60fc8de7346da67db7d764c7221: xfs: check parent pointer xattrs when scrubbing (2024-04-23 07:47:03 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-pptrs-6.10_2024-04-23 for you to fetch changes up to 327ed702d84034879572812f580cb769848af7ae: xfs: inode repair should ensure there's an attr fork to store parent pointers (2024-04-23 16:55:16 -0700) ---------------------------------------------------------------- xfs: online repair for parent pointers [v13.4 5/9] This series implements online repair for directory parent pointer metadata. The checking half is fairly straightforward -- for each outgoing directory link (forward or backwards), grab the inode at the other end, and confirm that there's a corresponding link. If we can't grab an inode or lock it, we'll save that link for a slower loop that cycles all the locks, confirms the continued existence of the link, and rechecks the link if it's actually still there. Repairs are a bit more involved -- for directories, we walk the entire filesystem to rebuild the dirents from parent pointer information. Parent pointer repairs do the same walk but rebuild the pptrs from the dirent information, but with the added twist that it duplicates all the xattrs so that it can use the atomic extent swapping code to commit the repairs atomically. This introduces an added twist to the xattr repair code -- we use dirent hooks to detect a colliding update to the pptr data while we're not holding the ILOCKs; if one is detected, we restart the xattr salvaging process but this time hold all the ILOCKs until the end of the scan. For offline repair, the phase6 directory connectivity scan generates an index of all the expected parent pointers in the filesystem. Then it walks each file and compares the parent pointers attached to that file against the index generated, and resyncs the results as necessary. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (16): xfs: remove some boilerplate from xfs_attr_set xfs: make the reserved block permission flag explicit in xfs_attr_set xfs: salvage parent pointers when rebuilding xattr structures xfs: add raw parent pointer apis to support repair xfs: repair directories by scanning directory parent pointers xfs: implement live updates for directory repairs xfs: replay unlocked parent pointer updates that accrue during xattr repair xfs: repair directory parent pointers by scanning for dirents xfs: implement live updates for parent pointer repairs xfs: remove pointless unlocked assertion xfs: split xfs_bmap_add_attrfork into two pieces xfs: add a per-leaf block callback to xchk_xattr_walk xfs: actually rebuild the parent pointer xattrs xfs: adapt the orphanage code to handle parent pointers xfs: repair link count of nondirectories after rebuilding parent pointers xfs: inode repair should ensure there's an attr fork to store parent pointers fs/xfs/libxfs/xfs_attr.c | 76 ++- fs/xfs/libxfs/xfs_attr.h | 3 +- fs/xfs/libxfs/xfs_bmap.c | 38 +- fs/xfs/libxfs/xfs_bmap.h | 3 +- fs/xfs/libxfs/xfs_dir2.c | 2 +- fs/xfs/libxfs/xfs_dir2.h | 2 +- fs/xfs/libxfs/xfs_parent.c | 64 +++ fs/xfs/libxfs/xfs_parent.h | 6 + fs/xfs/scrub/attr.c | 2 +- fs/xfs/scrub/attr_repair.c | 484 +++++++++++++++- fs/xfs/scrub/attr_repair.h | 4 + fs/xfs/scrub/dir_repair.c | 567 +++++++++++++++++- fs/xfs/scrub/findparent.c | 12 +- fs/xfs/scrub/findparent.h | 10 +- fs/xfs/scrub/inode_repair.c | 41 ++ fs/xfs/scrub/listxattr.c | 10 +- fs/xfs/scrub/listxattr.h | 4 +- fs/xfs/scrub/nlinks.c | 3 +- fs/xfs/scrub/orphanage.c | 38 ++ fs/xfs/scrub/orphanage.h | 3 + fs/xfs/scrub/parent.c | 7 +- fs/xfs/scrub/parent_repair.c | 1307 +++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/scrub.c | 2 + fs/xfs/scrub/trace.h | 153 +++++ fs/xfs/xfs_xattr.c | 26 +- 25 files changed, 2743 insertions(+), 124 deletions(-) From patchwork Wed Apr 24 03:35:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641128 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F54113340B for ; Wed, 24 Apr 2024 03:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929713; cv=none; b=o4QmaeOwLOMHJcJLDkQw84qFN7cZh2RymIBq6cP467WlpyfLy3+PMtrdJUQySZ5PGdLL0mOj47iUPCZrh36Cb78sc739QsXMfR2Dl4jxGvAlYAxrWf76qvyshjy0b7BqcEWYR0dY5wmPK9+g7dlir14/L5/64i/l7C7+DbT9BIc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929713; c=relaxed/simple; bh=B+Xj8HGakEKyOOzCB35nTqDlD8aYzx2Gr5g2sgddbco=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=l0lYdOdZX6pXe3hSKAFeO6Vax0z6P4LSllhXaC9IlhJ9IyFsr+UKE+GqXDZbSNhCQO+cXgAmXu9lUKzttf7INWkUfyFMyiikt9VDcXLC19Pxan23CZptDyu4KPrm9asOmv9im6xcWsR225Ql3rjVqxX9uQhqMN+WvmQXOdgsBZg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MtQdozPc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MtQdozPc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30959C116B1; Wed, 24 Apr 2024 03:35:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929713; bh=B+Xj8HGakEKyOOzCB35nTqDlD8aYzx2Gr5g2sgddbco=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=MtQdozPcHSeLimbxKhGWVaj7NrqJJAGH7KVFaLvhtXNwWt05/yiTdf+wEjiS1hEjM 0VwBYo4l1XJfBurBDOP386B6+u8d4I1XxhRc5/ufuSVjnXdhqFzKhOgodXMUmMbc+L 48Jd7UQUMOp6AB4dUrxdhUYcSQOjVBMq9wqCEdpgvNzjPJaS9YUtzY+anEcwzdproi YdvmQKk0Io7VXiAyqWGLoMTJIofN+hu/pdF1tbHH1TQPL/LgPMBi3Bf/ADqbEqblVY wY41Ue0QZ9ma+jFDppp+rwB/W9rRxdse3HbddQ2RJ4tp05K0Qp4AEWKSLhMM1/BcHW lgCTmUAHba2wQ== Date: Tue, 23 Apr 2024 20:35:12 -0700 Subject: [GIT PULL 6/9] xfs: detect and correct directory tree problems From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392952262.1941278.14474287012383303820.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 327ed702d84034879572812f580cb769848af7ae: xfs: inode repair should ensure there's an attr fork to store parent pointers (2024-04-23 16:55:16 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-directory-tree-6.10_2024-04-23 for you to fetch changes up to 3f31406aef493b3f19020909d29974e28253f91c: xfs: fix corruptions in the directory tree (2024-04-23 16:55:17 -0700) ---------------------------------------------------------------- xfs: detect and correct directory tree problems [v13.4 6/9] Historically, checking the tree-ness of the directory tree structure has not been complete. Cycles of subdirectories break the tree properties, as do subdirectories with multiple parents. It's easy enough for DFS to detect problems as long as one of the participants is reachable from the root, but this technique cannot find unconnected cycles. Directory parent pointers change that, because we can discover all of these problems from a simple walk from a subdirectory towards the root. For each child we start with, if the walk terminates without reaching the root, we know the path is disconnected and ought to be attached to the lost and found. If we find ourselves, we know this is a cycle and can delete an incoming edge. If we find multiple paths to the root, we know to delete an incoming edge. Even better, once we've finished walking paths, we've identified the good ones and know which other path(s) to remove. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (4): xfs: teach online scrub to find directory tree structure problems xfs: invalidate dirloop scrub path data when concurrent updates happen xfs: report directory tree corruption in the health information xfs: fix corruptions in the directory tree fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_fs.h | 4 +- fs/xfs/libxfs/xfs_health.h | 4 +- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/dirtree.c | 985 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/dirtree.h | 178 ++++++++ fs/xfs/scrub/dirtree_repair.c | 821 +++++++++++++++++++++++++++++++++++ fs/xfs/scrub/health.c | 1 + fs/xfs/scrub/ino_bitmap.h | 37 ++ fs/xfs/scrub/orphanage.c | 6 + fs/xfs/scrub/orphanage.h | 8 + fs/xfs/scrub/repair.h | 4 + fs/xfs/scrub/scrub.c | 7 + fs/xfs/scrub/scrub.h | 1 + fs/xfs/scrub/stats.c | 1 + fs/xfs/scrub/trace.c | 4 + fs/xfs/scrub/trace.h | 272 +++++++++++- fs/xfs/scrub/xfarray.h | 1 + fs/xfs/xfs_health.c | 1 + fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode.h | 1 + 21 files changed, 2337 insertions(+), 4 deletions(-) create mode 100644 fs/xfs/scrub/dirtree.c create mode 100644 fs/xfs/scrub/dirtree.h create mode 100644 fs/xfs/scrub/dirtree_repair.c create mode 100644 fs/xfs/scrub/ino_bitmap.h From patchwork Wed Apr 24 03:35:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641129 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0657713340B for ; Wed, 24 Apr 2024 03:35:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929729; cv=none; b=NkZJA4o8ZjSuWIxdL7vfoV7WhWIDwLMeQD0zjCHjmZ4XuRlRFNYH5hHyjh/476dDAaRTt8DPFh/Au961PIm2atvhNHAR9Np3wEWW8uNxUQ2IWl4hVKOXNt34N+QkUtRsvvsaB5zBeN9DG8uxviSNMa/vqWp851OIPgzAeR3M/bI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929729; c=relaxed/simple; bh=CvEnNRnytyKFebqY1qbfnPzXC7ujgFIM1zvfdhMZ8RU=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=DO8aPxtLRB2nc+vAQmNMTdFWxBI2LY9AAgV2C4+ZWbueg1gD2zNA3H6AkX1ygNtgLrtzlwA47UxiqODnOujXtkxehL4AGehrQXluhJmtKFQKPhXB6GiJl7jWn3zcCR+Nvo9jPsR/ngWmbRt2JB+wBnZ4PCEodXPfFoSTr3kwgF8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nLjzhS6u; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nLjzhS6u" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D55BBC2BD11; Wed, 24 Apr 2024 03:35:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929728; bh=CvEnNRnytyKFebqY1qbfnPzXC7ujgFIM1zvfdhMZ8RU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=nLjzhS6uifYSzltkLYWUWcxvH4pVtFB7GRW6Bv671iy6tnv4zepkcsx2BBWuZMBVf 7qOfq4Od1/TsAVFOv1gE+x1KZ2MpDy0gFRsYP4uvIbk0YHS+a6usjQJ1druGvg0MvZ eRgsb64cbe98H7LKrCUuwV3jqlwaV2uaQPb9w5ru/eG15So4v588HhGgFGvK3DPRYh 807+HEMg6fPwaB+V7aomuVLDeMLcnTow8NlMT5/s66A7mol3gffpL5cjZdCTJ/OFa4 28arJXguD5xGevvSjeu5TjV1OE//bces1BC3kKWHbiiBhTZX/RDsrGfd7c/KzL81XW p/yZnUVx8PbWg== Date: Tue, 23 Apr 2024 20:35:28 -0700 Subject: [GIT PULL 7/9] xfs: vectorize scrub kernel calls From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392952737.1941278.15338406430048213468.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 3f31406aef493b3f19020909d29974e28253f91c: xfs: fix corruptions in the directory tree (2024-04-23 16:55:17 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/vectorized-scrub-6.10_2024-04-23 for you to fetch changes up to c77b37584c2d1054452853e47e42c7350b8fe687: xfs: introduce vectored scrub mode (2024-04-23 16:55:18 -0700) ---------------------------------------------------------------- xfs: vectorize scrub kernel calls [v13.4 7/9] Create a vectorized version of the metadata scrub and repair ioctl, and adapt xfs_scrub to use that. This mitigates the impact of system call overhead on xfs_scrub runtime. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (3): xfs: reduce the rate of cond_resched calls inside scrub xfs: move xfs_ioc_scrub_metadata to scrub.c xfs: introduce vectored scrub mode fs/xfs/libxfs/xfs_fs.h | 33 +++++++++ fs/xfs/scrub/common.h | 25 ------- fs/xfs/scrub/scrub.c | 177 ++++++++++++++++++++++++++++++++++++++++++++++- fs/xfs/scrub/scrub.h | 64 +++++++++++++++++ fs/xfs/scrub/trace.h | 79 ++++++++++++++++++++- fs/xfs/scrub/xfarray.c | 10 +-- fs/xfs/scrub/xfarray.h | 3 + fs/xfs/scrub/xfile.c | 2 +- fs/xfs/scrub/xfs_scrub.h | 6 +- fs/xfs/xfs_ioctl.c | 26 +------ 10 files changed, 366 insertions(+), 59 deletions(-) From patchwork Wed Apr 24 03:35:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641130 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96210947E for ; Wed, 24 Apr 2024 03:35:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929744; cv=none; b=r22fpH5FAI7xByG/z7J44Zndhz/2d9nZFgnCjGafqauaX0wyYuk30a3kIVe9I4qkOPgZPySLGgoIA8oaoy6iRfe5iuimeIIrh+IoHv2wN4dutlkiZIFvv4H6oBqm7tag0M3W9whZ/oRWEnLOLpTHZ4N+sxj8JzBLGz8SMLLttnE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929744; c=relaxed/simple; bh=cvUpwSY3GYjNdEe1Opr0v5SvE01dAg4c3mo02QVHxZI=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=psH6t7gcYLASbmTQt5AGjAIeeoN++767SSwV5AfrUbE3ygfTG5oOXPwZGaAuFF1td4OrSY5L83SjhvuQ/5+PoPYaTzgCLP68KZggkPBTpchjMjwil7I74hy8S4BEcbFq1bRhgGZTX8mhDjOTfgwCIxiDxDfine/DeU8QRESYHmY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Lsg3I0qv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Lsg3I0qv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B335C116B1; Wed, 24 Apr 2024 03:35:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929744; bh=cvUpwSY3GYjNdEe1Opr0v5SvE01dAg4c3mo02QVHxZI=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Lsg3I0qv2bHpD+1+Nu6h2GqMb7qkVkehXFcgYv/fqMnAs0kCAeMVUjL7H83PTo1Yv JTJuXj4auDJ6prOXNleZz2Apze1C8SREvPkGrFGa59lAHhD2wOZnr763twSNTAy/H9 xIgxOHpmgNrXGziAhKFLagSQREPQxH6usPGn9n6HTowaKGuj/mgHHIrCFsaptI5U2H mtXPDkigcNiyOn3hfc5xxBRD+xscdl5QhmBiFu6uJMdhpkdTSS1+ybjCdEUTvPNkc+ DLVv9I118ui016xrha8GxJPojhESdwgD8mWEPeTDWixeI2aO/DvH6lx79Jk8rxNzJs CBAf00mpQgNQQ== Date: Tue, 23 Apr 2024 20:35:44 -0700 Subject: [GIT PULL 8/9] xfs: reduce iget overhead in scrub From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392953196.1941278.2628285121167396370.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit c77b37584c2d1054452853e47e42c7350b8fe687: xfs: introduce vectored scrub mode (2024-04-23 16:55:18 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/reduce-scrub-iget-overhead-6.10_2024-04-23 for you to fetch changes up to 4ad350ac58627bfe81f71f43f6738e36b4eb75c6: xfs: only iget the file once when doing vectored scrub-by-handle (2024-04-23 16:55:18 -0700) ---------------------------------------------------------------- xfs: reduce iget overhead in scrub [v13.4 8/9] This patchset looks to reduce iget overhead in two ways: First, a previous patch conditionally set DONTCACHE on inodes during xchk_irele on the grounds that we knew better at irele time if an inode should be dropped. Unfortunately, over time that patch morphed into a call to d_mark_dontcache, which resulted in inodes being dropped even if they were referenced by the dcache. This actually caused *more* recycle overhead than if we'd simply called xfs_iget to set DONTCACHE only on misses. The second patch reduces the cost of untrusted iget for a vectored scrub call by having the scrubv code maintain a separate refcount to the inode so that the cache will always hit. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (2): xfs: use dontcache for grabbing inodes during scrub xfs: only iget the file once when doing vectored scrub-by-handle fs/xfs/scrub/common.c | 12 +++--------- fs/xfs/scrub/iscan.c | 13 +++++++++++-- fs/xfs/scrub/scrub.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/scrub.h | 7 +++++++ 4 files changed, 66 insertions(+), 11 deletions(-) From patchwork Wed Apr 24 03:35:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13641131 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C1FB947E for ; Wed, 24 Apr 2024 03:36:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929760; cv=none; b=eYxY2zNNUO6yMvc8VQcI+uRn4hdtMez+yGgEk54mJKD+2pIQupZ5lkI8dpAw1AbWwhG3tJnAcseBwUFspnsd/nVKW1V7Kf/+cmWrIsSghJUJJjCHu4hE5d5N92E356t4dXuq7DHw+t9/LMBIX8pMRD8wmIZoPJtM/Tb+yoMHxcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713929760; c=relaxed/simple; bh=DL4cmgT6BJqL6crLOow1UsRJoKVDBrMayO8kOGuJQDU=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=N0Ib2MR/m5uZqKAlrmFxYsOlXcqaZXALGgQpXFsip/7xk7eaAUboLyyE7yPvN19jBNhnXJpuu1JlKNZQdv4UqjcyN+LeIIC6tiKJp58otRj6sqUsWNtIuvfx99jtk6uu7z/7ScBL9rn6z2yoDOvEoLul4yCJnDqoiCXp8y8E9yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MJgXnCWt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MJgXnCWt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11062C116B1; Wed, 24 Apr 2024 03:36:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1713929760; bh=DL4cmgT6BJqL6crLOow1UsRJoKVDBrMayO8kOGuJQDU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=MJgXnCWtnuQmamCwQvxaAEmUIV5EF5xh+plTafW5QTxIOGyWcWWuhfqcUn3KRcpMy GP0CiStC8OWQD0bjnZ9Q3kge8h+VJXIC1kypduyxmq7T3GwkSkIRDe/utErazhE3yj GvpNrQqEut4ttgDEgJBGId4j61FMT+GgtmHo4cIQXFRamsATRvrGLOqz6P5pxO5ueB 5qYFArbURlLvFcm7pDri+twvnVusM+faiqvunCGTq7yFM/mxnbdvL6XVw5+Nnc6ZSu Ga8RowgUUrTYGuw9VsjcyZF5UpoEX3NYbDiFgD6tzX2RScNqGSr/lfNVlrRitdIqEY kQfxbo+cGcG7w== Date: Tue, 23 Apr 2024 20:35:59 -0700 Subject: [GIT PULL 9/9] xfs: minor fixes to online repair From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <171392953650.1941278.18020942589466688173.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240424033018.GB360940@frogsfrogsfrogs> References: <20240424033018.GB360940@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.10-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 4ad350ac58627bfe81f71f43f6738e36b4eb75c6: xfs: only iget the file once when doing vectored scrub-by-handle (2024-04-23 16:55:18 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-fixes-6.10_2024-04-23 for you to fetch changes up to 5e1c7d0b29f7e05b01e448d2579a469cf3a0d350: xfs: invalidate dentries for a file before moving it to the orphanage (2024-04-23 16:55:19 -0700) ---------------------------------------------------------------- xfs: minor fixes to online repair [v13.4 9/9] Here are some miscellaneous bug fixes for the online repair code. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (4): xfs: drop the scrub file's iolock when transaction allocation fails xfs: fix iunlock calls in xrep_adoption_trans_alloc xfs: exchange-range for repairs is no longer dynamic xfs: invalidate dentries for a file before moving it to the orphanage fs/xfs/scrub/attr_repair.c | 3 +++ fs/xfs/scrub/dir_repair.c | 3 +++ fs/xfs/scrub/nlinks_repair.c | 4 +++- fs/xfs/scrub/orphanage.c | 49 ++++++++++++++++++----------------------- fs/xfs/scrub/parent_repair.c | 10 ++++++--- fs/xfs/scrub/rtsummary_repair.c | 10 ++++----- fs/xfs/scrub/scrub.c | 8 +++---- fs/xfs/scrub/scrub.h | 7 ------ fs/xfs/scrub/symlink_repair.c | 3 +++ fs/xfs/scrub/tempexch.h | 1 - fs/xfs/scrub/tempfile.c | 24 ++------------------ fs/xfs/scrub/trace.h | 3 --- 12 files changed, 49 insertions(+), 76 deletions(-)