From patchwork Thu Dec 7 02:48:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13482634 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09FE87097F for ; Thu, 7 Dec 2023 02:48:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Kgl0u6KW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 78327C433C7; Thu, 7 Dec 2023 02:48:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1701917316; bh=IZFN1NhSm9D+eDoXK3lh+5R2AGFwWVIzKiZmX+spmFs=; h=Date:Subject:From:To:Cc:From; b=Kgl0u6KWvP5NxOXTeEa5HP9ir7HyxNVnAKf5MPW9lZ7RZdG4LYhICZD11bCPJ2SQJ 0TmkJ7mMBbcEhYgcfzGurwEt78cRYrjwXOCU8SNmCMbGSiHN3McIZWCiKP9Dk+a2U/ dCrzJOgemXnsS9WhfzN8R4Fcn8Wpkm51BXNyNcFn6QZzmN40EHchBfHTlw6jqubJCm Nes98Ve+rF45nuQDB1nwUcsMfaqUid9swQW5h7uVpGe1yz0d9Wvoa1yaPkFU5swFMm v64EDhejaIyrhUTUY+yrGP56PYK4OKnWdjtV5If7lKZb6CBPq4kOumiel6j55vrZCS l45zvtqcxyMEg== Date: Wed, 06 Dec 2023 18:48:36 -0800 Subject: [GIT PULL 1/3] xfs: elide defer work ->create_done if no intent From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org, hch@lst.de Cc: linux-xfs@vger.kernel.org Message-ID: <170191727770.1193846.6148843866019229192.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 33cc938e65a98f1d29d0a18403dbbee050dcad9a: Linux 6.7-rc4 (2023-12-03 18:52:56 +0900) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/defer-elide-create-done-6.8_2023-12-06 for you to fetch changes up to 9c07bca793b4ff9f0b7871e2a928a1b28b8fa4e3: xfs: elide ->create_done calls for unlogged deferred work (2023-12-06 18:45:17 -0800) ---------------------------------------------------------------- xfs: elide defer work ->create_done if no intent [v2] Christoph pointed out that the defer ops machinery doesn't need to call ->create_done if the deferred work item didn't generate a log intent item in the first place. Let's clean that up and save an indirect call in the non-logged xattr update call path. v2: pick up rvb tags This has been lightly tested with fstests. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (22): xfs: don't leak recovered attri intent items xfs: use xfs_defer_pending objects to recover intent items xfs: pass the xfs_defer_pending object to iop_recover xfs: transfer recovered intent item ownership in ->iop_recover xfs: recreate work items when recovering intent items xfs: dump the recovered xattri log item if corruption happens xfs: use xfs_defer_finish_one to finish recovered work items xfs: move ->iop_recover to xfs_defer_op_type xfs: don't set XFS_TRANS_HAS_INTENT_DONE when there's no ATTRD log item xfs: hoist intent done flag setting to ->finish_item callsite xfs: collapse the ->finish_item helpers xfs: hoist ->create_intent boilerplate to its callsite xfs: use xfs_defer_create_done for the relogging operation xfs: clean out XFS_LI_DIRTY setting boilerplate from ->iop_relog xfs: hoist xfs_trans_add_item calls to defer ops functions xfs: collapse the ->create_done functions xfs: move ->iop_relog to struct xfs_defer_op_type xfs: make rextslog computation consistent with mkfs xfs: fix 32-bit truncation in xfs_compute_rextslog xfs: don't allow overly small or large realtime volumes xfs: document what LARP means xfs: elide ->create_done calls for unlogged deferred work fs/xfs/libxfs/xfs_defer.c | 184 ++++++++++++++++++---- fs/xfs/libxfs/xfs_defer.h | 22 +++ fs/xfs/libxfs/xfs_log_recover.h | 7 + fs/xfs/libxfs/xfs_rtbitmap.c | 14 ++ fs/xfs/libxfs/xfs_rtbitmap.h | 16 ++ fs/xfs/libxfs/xfs_sb.c | 6 +- fs/xfs/xfs_attr_item.c | 252 ++++++++++++------------------- fs/xfs/xfs_bmap_item.c | 199 ++++++++++-------------- fs/xfs/xfs_extfree_item.c | 327 +++++++++++++++------------------------- fs/xfs/xfs_log.c | 1 + fs/xfs/xfs_log_priv.h | 1 + fs/xfs/xfs_log_recover.c | 129 ++++++++-------- fs/xfs/xfs_refcount_item.c | 233 +++++++++------------------- fs/xfs/xfs_rmap_item.c | 256 +++++++++++++------------------ fs/xfs/xfs_rtalloc.c | 6 +- fs/xfs/xfs_sysfs.c | 9 ++ fs/xfs/xfs_trans.h | 12 -- 17 files changed, 768 insertions(+), 906 deletions(-) From patchwork Thu Dec 7 02:48:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13482635 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7CD21841 for ; Thu, 7 Dec 2023 02:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RDqGgR0e" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21D02C433C7; Thu, 7 Dec 2023 02:48:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1701917332; bh=9OqoJuAf3Ip8gTWDIQLjLpCJniRbFaBhHnsVoQD+fUk=; h=Date:Subject:From:To:Cc:From; b=RDqGgR0eFWIDOQ/XFkyOCIn9r7QIks9/ojwFzqty16f9fg8mhaIAOYuW8Q/5cd+KI YJgFENSEwwBfwtkkSIDz8Ce4h53ZkJ0dE5nbLHjT0pdbWwW4PCXcqa2H1WaG1LiwzD YmwG0X4CAhhO6yv7v3y0HzMdTfeshN1v+rMOYffZarZfpHFBMUxai4NaZV5AvuwGsZ CpxUh6bt26nhh/LhwAzmOukiX21fcWXjJBkOr1f7VFtihh5ed9Bx9YF1XE4scSAgfO Jwnxyf8bsYxUOsMJdyrchjAn8DvRIc2Di9sFVJSHtRHLY/Yupgevl/4j8moLm4Zd0V BCu3fUa0wbFGw== Date: Wed, 06 Dec 2023 18:48:51 -0800 Subject: [GIT PULL 2/3] xfs: prevent livelocks in xchk_iget From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: dchinner@redhat.com, hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170191728214.1193846.17364904847492247344.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 9c07bca793b4ff9f0b7871e2a928a1b28b8fa4e3: xfs: elide ->create_done calls for unlogged deferred work (2023-12-06 18:45:17 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-livelock-prevention-6.8_2023-12-06 for you to fetch changes up to 3f113c2739b1b068854c7ffed635c2bd790d1492: xfs: make xchk_iget safer in the presence of corrupt inode btrees (2023-12-06 18:45:17 -0800) ---------------------------------------------------------------- xfs: prevent livelocks in xchk_iget [v28.1] Prevent scrub from live locking in xchk_iget if there's a cycle in the inobt by allocating an empty transaction. This has been lightly tested with fstests. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (1): xfs: make xchk_iget safer in the presence of corrupt inode btrees fs/xfs/scrub/common.c | 6 ++++-- fs/xfs/scrub/common.h | 25 +++++++++++++++++++++++++ fs/xfs/scrub/inode.c | 4 ++-- 3 files changed, 31 insertions(+), 4 deletions(-) From patchwork Thu Dec 7 02:49:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13482636 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03D1A1841 for ; Thu, 7 Dec 2023 02:49:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="M5wlwg0n" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C59BAC433C8; Thu, 7 Dec 2023 02:49:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1701917347; bh=4dqp09gG7Gnq4MboeU7E0ofbf52Dry/hMkGgrPM79XM=; h=Date:Subject:From:To:Cc:From; b=M5wlwg0nlCyd+0HlibiwmNsNgMLh8yRzAXiQDszqXbObjnwI45+29TwknnDgD/cCI 9n0w+X6eqQoJBaofEQOhU84Doi995qGZeB5dc+kSmAc4xt+BdRYtCFzPdssDRPvWfI mQRl/03br52RTBhuyI1Ao26Bnrgl9K0TtsWxarP6rBAqa2fb5cq/OU+PRRNJIH/7bS iMgEkywbosgyDlEggIotzS5KLgeGWddd2bs62c5LtTMKvAoTIKj9woj0rYIT/VY9S/ x9qb9MuuVeZJ0IpU55G+cTlsHamG61SW26R5EPDWawHvoDvMdpW+fsc1PZ7afYDGHw libsEFFXgA7Rw== Date: Wed, 06 Dec 2023 18:49:07 -0800 Subject: [GIT PULL 3/3] xfs: reserve disk space for online repairs From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: dchinner@redhat.com, hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170191728637.1193846.13594475500319328405.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi Chandan, Please pull this branch with changes for xfs for 6.8-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 3f113c2739b1b068854c7ffed635c2bd790d1492: xfs: make xchk_iget safer in the presence of corrupt inode btrees (2023-12-06 18:45:17 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-auto-reap-space-reservations-6.8_2023-12-06 for you to fetch changes up to 3f3cec031099c37513727efc978a12b6346e326d: xfs: force small EFIs for reaping btree extents (2023-12-06 18:45:19 -0800) ---------------------------------------------------------------- xfs: reserve disk space for online repairs [v28.1] Online repair fixes metadata structures by writing a new copy out to disk and atomically committing the new structure into the filesystem. For this to work, we need to reserve all the space we're going to need ahead of time so that the atomic commit transaction is as small as possible. We also require the reserved space to be freed if the system goes down, or if we decide not to commit the repair, or if we reserve too much space. To keep the atomic commit transaction as small as possible, we would like to allocate some space and simultaneously schedule automatic reaping of the reserved space, even on log recovery. EFIs are the mechanism to get us there, but we need to use them in a novel manner. Once we allocate the space, we want to hold on to the EFI (relogging as necessary) until we can commit or cancel the repair. EFIs for written committed blocks need to go away, but unwritten or uncommitted blocks can be freed like normal. Earlier versions of this patchset directly manipulated the log items, but Dave thought that to be a layering violation. For v27, I've modified the defer ops handling code to be capable of pausing a deferred work item. Log intent items are created as they always have been, but paused items are pushed onto a side list when finishing deferred work items, and pushed back onto the transaction after that. Log intent done item are not created for paused work. The second part adds a "stale" flag to the EFI so that the repair reservation code can dispose of an EFI the normal way, but without the space actually being freed. This has been lightly tested with fstests. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (8): xfs: don't append work items to logged xfs_defer_pending objects xfs: allow pausing of pending deferred work items xfs: remove __xfs_free_extent_later xfs: automatic freeing of freshly allocated unwritten space xfs: remove unused fields from struct xbtree_ifakeroot xfs: implement block reservation accounting for btrees we're staging xfs: log EFIs for all btree blocks being used to stage a btree xfs: force small EFIs for reaping btree extents fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_ag.c | 2 +- fs/xfs/libxfs/xfs_alloc.c | 104 +++++++- fs/xfs/libxfs/xfs_alloc.h | 22 +- fs/xfs/libxfs/xfs_bmap.c | 4 +- fs/xfs/libxfs/xfs_bmap_btree.c | 2 +- fs/xfs/libxfs/xfs_btree_staging.h | 6 - fs/xfs/libxfs/xfs_defer.c | 261 ++++++++++++++++--- fs/xfs/libxfs/xfs_defer.h | 20 +- fs/xfs/libxfs/xfs_ialloc.c | 5 +- fs/xfs/libxfs/xfs_ialloc_btree.c | 2 +- fs/xfs/libxfs/xfs_refcount.c | 6 +- fs/xfs/libxfs/xfs_refcount_btree.c | 2 +- fs/xfs/scrub/newbt.c | 513 +++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/newbt.h | 65 +++++ fs/xfs/scrub/reap.c | 7 +- fs/xfs/scrub/trace.h | 37 +++ fs/xfs/xfs_extfree_item.c | 9 +- fs/xfs/xfs_reflink.c | 2 +- fs/xfs/xfs_trace.h | 13 +- 20 files changed, 1007 insertions(+), 76 deletions(-) create mode 100644 fs/xfs/scrub/newbt.c create mode 100644 fs/xfs/scrub/newbt.h