From patchwork Fri Dec 15 21:55:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13495078 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BB8456778 for ; Fri, 15 Dec 2023 21:55:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MRRwH2OA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 96310C433C7; Fri, 15 Dec 2023 21:55:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702677312; bh=yQiZLv7CHrS0se9lS/YvEFTHBctWx+Rk6WfYWvbwpic=; h=Date:Subject:From:To:Cc:From; b=MRRwH2OAzNdMhS94iO3SlCR0SP0hB34y4c7puag85enJNDtuWwqcKq7KKw0NTDQg2 ijH/lY1SgoOYccZXg/0FYoWLppJgw9s3Xiwofm4EqKEgWWQEI2SRZn3q/IJT4bAZwF 0RjUHSxmHXd5ab8JIRR52QUxj5L6ZxpFIvoUJ0goBvdUHU+Ka4Lp6S/TTP5DDxIhuH RZ4RkrATftkX0FcmxGBRj9p0yOqLNIHGvb3NEzwVGM0uGovOX1PUAKSHaUjCDoy82D QRWKYWk8hjhKaDjDLkVeUn4/gTiHSj7Cs2QHCnd0av2msedtql3ToqcH+ASXnKK7O4 brS5zZNfIjlzA== Date: Fri, 15 Dec 2023 13:55:11 -0800 Subject: [GIT PULL 1/6] xfs: prepare repair for bulk loading From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: linux-xfs@vger.kernel.org Message-ID: <170267713236.2577253.8439413037561169584.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 0573676fdde7ce3829ee6a42a8e5a56355234712: xfs: initialise di_crc in xfs_log_dinode (2023-12-15 09:33:29 +0530) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-prep-for-bulk-loading-6.8_2023-12-15 for you to fetch changes up to e069d549705e49841247acf9b3176744e27d5425: xfs: constrain dirty buffers while formatting a staged btree (2023-12-15 10:03:29 -0800) ---------------------------------------------------------------- xfs: prepare repair for bulk loading [v28.3] Before we start merging the online repair functions, let's improve the bulk loading code a bit. First, we need to fix a misinteraction between the AIL and the btree bulkloader wherein the delwri at the end of the bulk load fails to queue a buffer for writeback if it happens to be on the AIL list. Second, we introduce a defer ops barrier object so that the process of reaping blocks after a repair cannot queue more than two extents per EFI log item. This increases our exposure to leaking blocks if the system goes down during a reap, but also should prevent transaction overflows, which result in the system going down. Third, we change the bulkloader itself to copy multiple records into a block if possible, and add some debugging knobs so that developers can control the slack factors, just like they can do for xfs_repair. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (6): xfs: force all buffers to be written during btree bulk load xfs: set XBF_DONE on newly formatted btree block that are ready for writing xfs: read leaf blocks when computing keys for bulkloading into node blocks xfs: add debug knobs to control btree bulk load slack factors xfs: move btree bulkload record initialization to ->get_record implementations xfs: constrain dirty buffers while formatting a staged btree fs/xfs/libxfs/xfs_btree.c | 2 +- fs/xfs/libxfs/xfs_btree.h | 3 ++ fs/xfs/libxfs/xfs_btree_staging.c | 78 ++++++++++++++++++++++++++++----------- fs/xfs/libxfs/xfs_btree_staging.h | 25 ++++++++++--- fs/xfs/scrub/newbt.c | 12 ++++-- fs/xfs/xfs_buf.c | 44 ++++++++++++++++++++-- fs/xfs/xfs_buf.h | 1 + fs/xfs/xfs_globals.c | 12 ++++++ fs/xfs/xfs_sysctl.h | 2 + fs/xfs/xfs_sysfs.c | 54 +++++++++++++++++++++++++++ 10 files changed, 198 insertions(+), 35 deletions(-) From patchwork Fri Dec 15 21:55:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13495079 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8660563A3 for ; Fri, 15 Dec 2023 21:55:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JEisZa5/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E0A8C433C8; Fri, 15 Dec 2023 21:55:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702677328; bh=yMjtb7YIbCAYZ2tGwsWpmxTromdtine8+QLvwreOlL0=; h=Date:Subject:From:To:Cc:From; b=JEisZa5/BKKuZM18kNQ23ZApvpu2G+UGV3gZonAwITuEDNlU5COULy3Qx2TuyxEuf uASFbXFxFlu9ieugu/lGSD79X8Dc3tYhJnjFf64TOjMYKEO+Vau1ToQqMLQ197GuaT Zh8U/IB1jm+dM6iLpHvw7NKjF81LoQA58BY63LIMKcOheTsy/+b9SMBYq+KtV0m1s5 aaeIl/jUKKMgX9JgpEs9weuRPznC7v7uWc53k+cWxz3xPu7HFUBOrdpEELtMNL8qto 8gAk7GhgZkSvrSrUWyV/+7935RE3RIFpjTpmuibmNdywY0A+XNTf7z9W1NtMJD0bbv u11VoZHMLIurQ== Date: Fri, 15 Dec 2023 13:55:27 -0800 Subject: [GIT PULL 2/6] xfs: online repair of AG btrees From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: dchinner@redhat.com, linux-xfs@vger.kernel.org Message-ID: <170267713343.2577253.11790987121430782895.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit e069d549705e49841247acf9b3176744e27d5425: xfs: constrain dirty buffers while formatting a staged btree (2023-12-15 10:03:29 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-ag-btrees-6.8_2023-12-15 for you to fetch changes up to 9099cd38002f8029c9a1da08e6832d1cd18e8451: xfs: repair refcount btrees (2023-12-15 10:03:33 -0800) ---------------------------------------------------------------- xfs: online repair of AG btrees [v28.3] Now that we've spent a lot of time reworking common code in online fsck, we're ready to start rebuilding the AG space btrees. This series implements repair functions for the free space, inode, and refcount btrees. Rebuilding the reverse mapping btree is much more intense and is left for a subsequent patchset. The fstests counterpart of this patchset implements stress testing of repair. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (7): xfs: create separate structures and code for u32 bitmaps xfs: move the per-AG datatype bitmaps to separate files xfs: roll the scrub transaction after completing a repair xfs: remove trivial bnobt/inobt scrub helpers xfs: repair free space btrees xfs: repair inode btrees xfs: repair refcount btrees fs/xfs/Makefile | 4 + fs/xfs/libxfs/xfs_ag.h | 10 + fs/xfs/libxfs/xfs_ag_resv.c | 2 + fs/xfs/libxfs/xfs_alloc.c | 10 +- fs/xfs/libxfs/xfs_alloc.h | 2 +- fs/xfs/libxfs/xfs_alloc_btree.c | 13 +- fs/xfs/libxfs/xfs_btree.c | 26 ++ fs/xfs/libxfs/xfs_btree.h | 2 + fs/xfs/libxfs/xfs_ialloc.c | 31 +- fs/xfs/libxfs/xfs_ialloc.h | 3 +- fs/xfs/libxfs/xfs_refcount.c | 8 +- fs/xfs/libxfs/xfs_refcount.h | 2 +- fs/xfs/libxfs/xfs_refcount_btree.c | 13 +- fs/xfs/libxfs/xfs_types.h | 7 + fs/xfs/scrub/agb_bitmap.c | 103 ++++ fs/xfs/scrub/agb_bitmap.h | 68 +++ fs/xfs/scrub/agheader_repair.c | 19 +- fs/xfs/scrub/alloc.c | 52 ++- fs/xfs/scrub/alloc_repair.c | 934 +++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/bitmap.c | 509 +++++++++++++------- fs/xfs/scrub/bitmap.h | 111 ++--- fs/xfs/scrub/common.c | 1 + fs/xfs/scrub/common.h | 19 + fs/xfs/scrub/ialloc.c | 39 +- fs/xfs/scrub/ialloc_repair.c | 884 +++++++++++++++++++++++++++++++++++ fs/xfs/scrub/newbt.c | 48 +- fs/xfs/scrub/newbt.h | 3 + fs/xfs/scrub/reap.c | 6 +- fs/xfs/scrub/refcount.c | 2 +- fs/xfs/scrub/refcount_repair.c | 794 +++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.c | 131 ++++++ fs/xfs/scrub/repair.h | 43 ++ fs/xfs/scrub/rmap.c | 1 + fs/xfs/scrub/scrub.c | 30 +- fs/xfs/scrub/scrub.h | 15 +- fs/xfs/scrub/trace.h | 112 +++-- fs/xfs/scrub/xfarray.h | 22 + fs/xfs/xfs_extent_busy.c | 13 + fs/xfs/xfs_extent_busy.h | 2 + 39 files changed, 3709 insertions(+), 385 deletions(-) create mode 100644 fs/xfs/scrub/agb_bitmap.c create mode 100644 fs/xfs/scrub/agb_bitmap.h create mode 100644 fs/xfs/scrub/alloc_repair.c create mode 100644 fs/xfs/scrub/ialloc_repair.c create mode 100644 fs/xfs/scrub/refcount_repair.c From patchwork Fri Dec 15 21:55:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13495080 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA85C5D910 for ; Fri, 15 Dec 2023 21:55:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YWl9cncv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2DD9BC433C7; Fri, 15 Dec 2023 21:55:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702677344; bh=In0aolk83/YXRfY8xw6Yn1hz8/+cW7I0RWsby00ovo4=; h=Date:Subject:From:To:Cc:From; b=YWl9cncvyVDWv1wNmawBTcH+s4CFcf0QgEIECllKGhLB/gjHNAmPk8G++6awxVN3b idK+5jpXO2QXRyaqaleAHseV/g6LycmVZ618Q5hsgwZcA/XQb2+kKrsoPyxOUFPTIx CNZVRRW0X8EDZgNH70Bq+EqXzQLIP6FboVzmS4OIQJWU5/ggxDu42muuVMZ7ZIszCb WD8qMPvHpeWj/zE6It85ghAa7M6VwFowro9n4VFZ0Q7b70Bke9Kc5yO+hbUko0gCGx gi24ya2dZs72rt/gWaBDuPZGxzAj5Bs75S/xlRSw2PvrO3htA+aNH5UeBgDJcfnAX4 SQWw9FZR1dxMQ== Date: Fri, 15 Dec 2023 13:55:43 -0800 Subject: [GIT PULL 3/6] xfs: online repair of inodes and forks From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: linux-xfs@vger.kernel.org Message-ID: <170267713452.2577253.5573004281850246886.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 9099cd38002f8029c9a1da08e6832d1cd18e8451: xfs: repair refcount btrees (2023-12-15 10:03:33 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-inodes-6.8_2023-12-15 for you to fetch changes up to c3a22c2e4b45fcf3184e7dd1c755e6b45dc9f499: xfs: skip the rmapbt search on an empty attr fork unless we know it was zapped (2023-12-15 10:03:38 -0800) ---------------------------------------------------------------- xfs: online repair of inodes and forks [v28.3] In this series, online repair gains the ability to repair inode records. To do this, we must repair the ondisk inode and fork information enough to pass the iget verifiers and hence make the inode igettable again. Once that's done, we can perform higher level repairs on the incore inode. The fstests counterpart of this patchset implements stress testing of repair. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (9): xfs: disable online repair quota helpers when quota not enabled xfs: try to attach dquots to files before repairing them xfs: add missing nrext64 inode flag check to scrub xfs: dont cast to char * for XFS_DFORK_*PTR macros xfs: set inode sick state flags when we zap either ondisk fork xfs: repair inode records xfs: zap broken inode forks xfs: abort directory parent scrub scans if we encounter a zapped directory xfs: skip the rmapbt search on an empty attr fork unless we know it was zapped fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_attr_leaf.c | 13 +- fs/xfs/libxfs/xfs_attr_leaf.h | 3 +- fs/xfs/libxfs/xfs_bmap.c | 22 +- fs/xfs/libxfs/xfs_bmap.h | 2 + fs/xfs/libxfs/xfs_dir2_priv.h | 3 +- fs/xfs/libxfs/xfs_dir2_sf.c | 13 +- fs/xfs/libxfs/xfs_format.h | 2 +- fs/xfs/libxfs/xfs_health.h | 10 + fs/xfs/libxfs/xfs_inode_fork.c | 33 +- fs/xfs/libxfs/xfs_shared.h | 2 +- fs/xfs/libxfs/xfs_symlink_remote.c | 8 +- fs/xfs/scrub/bmap.c | 144 +++- fs/xfs/scrub/common.c | 28 + fs/xfs/scrub/common.h | 8 + fs/xfs/scrub/dir.c | 42 +- fs/xfs/scrub/health.c | 32 + fs/xfs/scrub/health.h | 2 + fs/xfs/scrub/inode.c | 16 +- fs/xfs/scrub/inode_repair.c | 1525 ++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/parent.c | 17 + fs/xfs/scrub/repair.c | 57 +- fs/xfs/scrub/repair.h | 29 + fs/xfs/scrub/rtbitmap.c | 4 + fs/xfs/scrub/rtsummary.c | 4 + fs/xfs/scrub/scrub.c | 2 +- fs/xfs/scrub/symlink.c | 20 +- fs/xfs/scrub/trace.h | 171 ++++ fs/xfs/xfs_dir2_readdir.c | 3 + fs/xfs/xfs_health.c | 8 +- fs/xfs/xfs_inode.c | 35 + fs/xfs/xfs_inode.h | 2 + fs/xfs/xfs_symlink.c | 3 + fs/xfs/xfs_xattr.c | 6 + 34 files changed, 2185 insertions(+), 85 deletions(-) create mode 100644 fs/xfs/scrub/inode_repair.c From patchwork Fri Dec 15 21:55:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13495081 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80E005F84E for ; Fri, 15 Dec 2023 21:56:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nrXSJVMG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E33B5C433C7; Fri, 15 Dec 2023 21:55:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702677360; bh=F/srrVY6oISwzdBfqABZN6x10zD0huKhsT/U4Uu6Pxk=; h=Date:Subject:From:To:Cc:From; b=nrXSJVMG3PEP5nTF64SSwT5MR2U+83JvBb03X8GeOHzWKi4HKu1AfrQoXhcwx+sSv jxTmvbawwc0TE3OXF286xije4qQkYEIMJxtBDa8R6BVwnsfUOvaQ72WHlg5zfa5dXx yKKt8nKfIq3N/otrM+5zy2gI3uNmgQqKKXF/B1Z9GFiN/jEW53pGDZVIygMH5vvdI2 HJeq+/ohmXJjEZs6ELeOyAsvD6/QT9I5upB+e1VS+/tOqLFhuScG2loaSoJS2kUqrs tcC//9jxxTI9nYOecMg6K1J92lndPKUMaoPV3sxFo4ntC7TaQXMtTA9KRg8CvdGl// GL3ht3MBXQmAw== Date: Fri, 15 Dec 2023 13:55:59 -0800 Subject: [GIT PULL 4/6] xfs: online repair of file fork mappings From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: linux-xfs@vger.kernel.org Message-ID: <170267713559.2577253.17958854624353416929.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit c3a22c2e4b45fcf3184e7dd1c755e6b45dc9f499: xfs: skip the rmapbt search on an empty attr fork unless we know it was zapped (2023-12-15 10:03:38 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-file-mappings-6.8_2023-12-15 for you to fetch changes up to dbbdbd0086320a026903ca34efedb6abf55230ed: xfs: repair problems in CoW forks (2023-12-15 10:03:40 -0800) ---------------------------------------------------------------- xfs: online repair of file fork mappings [v28.3] In this series, online repair gains the ability to rebuild data and attr fork mappings from the reverse mapping information. It is at this point where we reintroduce the ability to reap file extents. Repair of CoW forks is a little different -- on disk, CoW staging extents are owned by the refcount btree and cannot be mapped back to individual files. Hence we can only detect staging extents that don't quite look right (missing reverse mappings, shared staging extents) and replace them with fresh allocations. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (5): xfs: reintroduce reaping of file metadata blocks to xrep_reap_extents xfs: repair inode fork block mapping data structures xfs: refactor repair forcing tests into a repair.c helper xfs: create a ranged query function for refcount btrees xfs: repair problems in CoW forks fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_bmap_btree.c | 121 +++++- fs/xfs/libxfs/xfs_bmap_btree.h | 5 + fs/xfs/libxfs/xfs_btree_staging.c | 11 +- fs/xfs/libxfs/xfs_btree_staging.h | 2 +- fs/xfs/libxfs/xfs_iext_tree.c | 23 +- fs/xfs/libxfs/xfs_inode_fork.c | 1 + fs/xfs/libxfs/xfs_inode_fork.h | 3 + fs/xfs/libxfs/xfs_refcount.c | 41 ++ fs/xfs/libxfs/xfs_refcount.h | 10 + fs/xfs/scrub/bmap.c | 18 + fs/xfs/scrub/bmap_repair.c | 858 ++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 6 +- fs/xfs/scrub/cow_repair.c | 614 +++++++++++++++++++++++++++ fs/xfs/scrub/fsb_bitmap.h | 37 ++ fs/xfs/scrub/off_bitmap.h | 37 ++ fs/xfs/scrub/reap.c | 153 ++++++- fs/xfs/scrub/reap.h | 5 + fs/xfs/scrub/repair.c | 50 +++ fs/xfs/scrub/repair.h | 11 + fs/xfs/scrub/scrub.c | 20 +- fs/xfs/scrub/trace.h | 118 +++++- fs/xfs/xfs_trans.c | 62 +++ fs/xfs/xfs_trans.h | 4 + 24 files changed, 2160 insertions(+), 52 deletions(-) create mode 100644 fs/xfs/scrub/bmap_repair.c create mode 100644 fs/xfs/scrub/cow_repair.c create mode 100644 fs/xfs/scrub/fsb_bitmap.h create mode 100644 fs/xfs/scrub/off_bitmap.h From patchwork Fri Dec 15 21:56:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13495082 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D327A5D910 for ; Fri, 15 Dec 2023 21:56:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="daeJ1HgT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A0438C433C8; Fri, 15 Dec 2023 21:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702677375; bh=A6JNK9tQFi0gXvNGDmHvLkFwsaRNbALYqz4ZFZPEyQc=; h=Date:Subject:From:To:Cc:From; b=daeJ1HgT8hORQLvYpJ4+zGHWvy3vjkhqPZf/+jvTu0QgkW9+0VVpZ8WaPfA2XXDRH cuI1XGH9CN2Kv6jQ7nkSUkQ5p1Tlal45pcl2hgz/lbZFX7meVMfm4VN/F85pPGhXJB QZxGyzrYkPemYQeGhi/Zh6B5DfkMCpPRjZWmzljL8EG/Tjzbym+bqIgU44Yg5r9TG8 64kZAGKZTGbTY6tVTbbU2Gjo26BbqL2tS87PtvSxGiztw4AVJgvRDXBsZ+1+AjpQbz 0pKPCuoKC1GL6sm5mVOPI58ubP1Kj0mfzMfo9ByyxlBr3xQdWcgHbmYGYuFNXBzHhU lFGWrGLyquGsQ== Date: Fri, 15 Dec 2023 13:56:15 -0800 Subject: [GIT PULL 5/6] xfs: online repair of rt bitmap file From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: linux-xfs@vger.kernel.org Message-ID: <170267713665.2577253.13608710440348653318.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit dbbdbd0086320a026903ca34efedb6abf55230ed: xfs: repair problems in CoW forks (2023-12-15 10:03:40 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-rtbitmap-6.8_2023-12-15 for you to fetch changes up to ffd37b22bd2b7cca7749c85a0a08268158903e55: xfs: online repair of realtime bitmaps (2023-12-15 10:03:43 -0800) ---------------------------------------------------------------- xfs: online repair of rt bitmap file [v28.3] Add in the necessary infrastructure to check the inode and data forks of metadata files, then apply that to the realtime bitmap file. We won't be able to reconstruct the contents of the rtbitmap file until rmapbt is added for realtime volumes, but we can at least get the basics started. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (6): xfs: check rt bitmap file geometry more thoroughly xfs: check rt summary file geometry more thoroughly xfs: always check the rtbitmap and rtsummary files xfs: repair the inode core and forks of a metadata inode xfs: create a new inode fork block unmap helper xfs: online repair of realtime bitmaps fs/xfs/Makefile | 4 + fs/xfs/libxfs/xfs_bmap.c | 41 ++++++++- fs/xfs/libxfs/xfs_bmap.h | 5 +- fs/xfs/scrub/bmap_repair.c | 17 +++- fs/xfs/scrub/repair.c | 153 +++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 9 ++ fs/xfs/scrub/rtbitmap.c | 103 +++++++++++++++++---- fs/xfs/scrub/rtbitmap.h | 22 +++++ fs/xfs/scrub/rtbitmap_repair.c | 202 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/rtsummary.c | 137 ++++++++++++++++++++++------ fs/xfs/scrub/scrub.c | 4 +- fs/xfs/xfs_inode.c | 24 +---- 12 files changed, 647 insertions(+), 74 deletions(-) create mode 100644 fs/xfs/scrub/rtbitmap.h create mode 100644 fs/xfs/scrub/rtbitmap_repair.c From patchwork Fri Dec 15 21:56:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13495083 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA9F48CE6 for ; Fri, 15 Dec 2023 21:56:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BiCzeCiL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A257C433C7; Fri, 15 Dec 2023 21:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702677391; bh=PV/0qorKdvjhKxU7oF0yoVosQQkWt9ydTne6BEY1GmM=; h=Date:Subject:From:To:Cc:From; b=BiCzeCiLaQdXDfYJwDZvvFP9LBr6DNQsorhfqOLlKG/U9tmPoeBQbDLmmzMQUpgOy A3/J9WgsGuO3+ASGtl5kUJahIcSW6jhIF7QxknIhVWN9gvrHXXx5z2DVDgitgu6q3q Vkmr/9Qqwmbv51TKgDfQswINhGbRYattVSyoX8fdXzd6uMPXPfdR0vL6xwuZ8n4FD/ OFEE+j4YxZUo/STKZOFZ1NRL//R1NFt9B9m4QN2xMA9Vv+TFVfpNsZHiIYkFInksbh n1I608SNKvRMJByl5bZbKtZlCIRJSN3huvSIxR9br9EHbA+tgVPn7l2iSH6NcqUfe+ MPOVTI4YKX/AA== Date: Fri, 15 Dec 2023 13:56:30 -0800 Subject: [GIT PULL 6/6] xfs: online repair of quota and rt metadata files From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: linux-xfs@vger.kernel.org Message-ID: <170267713764.2577253.13285937294329651179.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit ffd37b22bd2b7cca7749c85a0a08268158903e55: xfs: online repair of realtime bitmaps (2023-12-15 10:03:43 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-quota-6.8_2023-12-15 for you to fetch changes up to a5b91555403e3a09ae00bed85fc78b60801dda24: xfs: repair quotas (2023-12-15 10:03:45 -0800) ---------------------------------------------------------------- xfs: online repair of quota and rt metadata files [v28.3] XFS stores quota records and free space bitmap information in files. Add the necessary infrastructure to enable repairing metadata inodes and their forks, and then make it so that we can repair the file metadata for the rtbitmap. Repairing the bitmap contents (and the summary file) is left for subsequent patchsets. We also add the ability to repair file metadata the quota files. As part of these repairs, we also reinitialize the ondisk dquot records as necessary to get the incore dquots working. We can also correct obviously bad dquot record attributes, but we leave checking the resource usage counts for the next patchsets. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (4): xfs: check the ondisk space mapping behind a dquot xfs: check dquot resource timers xfs: improve dquot iteration for scrub xfs: repair quotas fs/xfs/Makefile | 9 +- fs/xfs/libxfs/xfs_format.h | 3 + fs/xfs/scrub/dqiterate.c | 211 ++++++++++++++++ fs/xfs/scrub/quota.c | 107 ++++++++- fs/xfs/scrub/quota.h | 36 +++ fs/xfs/scrub/quota_repair.c | 575 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 7 + fs/xfs/scrub/scrub.c | 6 +- fs/xfs/scrub/trace.c | 3 + fs/xfs/scrub/trace.h | 78 ++++++ fs/xfs/xfs_dquot.c | 37 +-- fs/xfs/xfs_dquot.h | 8 +- 12 files changed, 1026 insertions(+), 54 deletions(-) create mode 100644 fs/xfs/scrub/dqiterate.c create mode 100644 fs/xfs/scrub/quota.h create mode 100644 fs/xfs/scrub/quota_repair.c