From patchwork Sat Feb 24 01:29:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570210 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBF8E1870 for ; Sat, 24 Feb 2024 01:29:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738153; cv=none; b=QFN/RP/LL5wvo9r5YbNGjWECjNECBDXxcmvKVg4Ii8d1bXYFgce0WvonNM8GZLUfg5IyyFY5wFSPipQsEEaqhqQbx2MnthrVYc+778d6M8ZguxUr8KFDpQJHs+8HOAkvuoY/F6Xm40aKB5xcmXDhdvgQdB+YTp/7/6nA3pFRCPM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738153; c=relaxed/simple; bh=7XBbzJKPNo+qPb9XHGP29LmYlgkqzuorzm9KaCQTsp0=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=oB4dWM39jUFndJXd0cc3vEtOaLWWJfNm5R4oiARYXJsrjeTSqeyXxXjAEV82lpAbt3VFnY+F4dsOfMerHZuTFQtK0aABz+OI7iydJNQtLMxA7+2U5sVLjY8mb2QzGhw0qJRvGJJqTADjJ5uzNeclSHVZN6feiYHHEAl7Mg3QGt4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nV9FP3As; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nV9FP3As" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A4D24C433F1; Sat, 24 Feb 2024 01:29:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738153; bh=7XBbzJKPNo+qPb9XHGP29LmYlgkqzuorzm9KaCQTsp0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=nV9FP3AsJ44+wQWWWG4zBh/Hm6TRSp2AsCPB5q+eXHJnFzWDtONY/qyllRChpX8M0 eA5ttwk+8y4d5lGWogcc6KBNf8e/BkfRDV0/o/16nLovKdgAUmzO7qrmJNoTBDRjOA BT8HJFrQlSFgGmseufUJWHe2xQaS4T0tGNVn/zZbBvf1BYVQRS0IWqa9Ms0+a6obsU O5lHXBmrb0hbBZi6CYaS6pJeiLiIJmRv0pp8q/pIofDwCaw2d3VGHnrAjHPmkOhQX5 7TPAloJ0YpnTLasONso5JhUsx/iWrmJNqsrLbt23ml463mPY0Kqd86i2WGYl7tIkZw By0IoqySVdnPA== Date: Fri, 23 Feb 2024 17:29:13 -0800 Subject: [GIT PULL 1/18] xfs: repair inode mode by scanning dirs From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873800188.1891722.3806756019608757588.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 4b2f459d86252619448455013f581836c8b1b7da: xfs: fix SEEK_HOLE/DATA for regions with active COW extents (2024-02-21 12:31:12 +0530) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-inode-mode-6.9_2024-02-23 for you to fetch changes up to 5385f1a60d4e5b73e8ecd2757865352b68f54fb9: xfs: repair file modes by scanning for a dirent pointing to us (2024-02-22 12:30:51 -0800) ---------------------------------------------------------------- xfs: repair inode mode by scanning dirs [v29.3 01/18] One missing piece of functionality in the inode record repair code is figuring out what to do with a file whose mode is so corrupt that we cannot tell us the type of the file. Originally this was done by guessing the mode from the ondisk inode contents, but Christoph didn't like that because it read from data fork block 0, which could be user controlled data. Therefore, I've replaced all that with a directory scanner that looks for any dirents that point to the file with the garbage mode. If so, the ftype in the dirent will tell us exactly what mode to set on the file. Since users cannot directly write to the ftype field of a dirent, this should be safe. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (10): xfs: speed up xfs_iwalk_adjust_start a little bit xfs: implement live inode scan for scrub xfs: allow scrub to hook metadata updates in other writers xfs: stagger the starting AG of scrub iscans to reduce contention xfs: cache a bunch of inodes for repair scans xfs: iscan batching should handle unallocated inodes too xfs: create a static name for the dot entry too xfs: create a predicate to determine if two xfs_names are the same xfs: create a macro for decoding ftypes in tracepoints xfs: repair file modes by scanning for a dirent pointing to us fs/xfs/Kconfig | 5 + fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_da_format.h | 11 + fs/xfs/libxfs/xfs_dir2.c | 6 + fs/xfs/libxfs/xfs_dir2.h | 13 + fs/xfs/scrub/dir.c | 4 +- fs/xfs/scrub/inode_repair.c | 236 ++++++++++++- fs/xfs/scrub/iscan.c | 767 ++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/iscan.h | 84 +++++ fs/xfs/scrub/trace.c | 2 + fs/xfs/scrub/trace.h | 194 +++++++++++ fs/xfs/xfs_hooks.c | 52 +++ fs/xfs/xfs_hooks.h | 65 ++++ fs/xfs/xfs_iwalk.c | 13 +- fs/xfs/xfs_linux.h | 1 + 15 files changed, 1436 insertions(+), 19 deletions(-) create mode 100644 fs/xfs/scrub/iscan.c create mode 100644 fs/xfs/scrub/iscan.h create mode 100644 fs/xfs/xfs_hooks.c create mode 100644 fs/xfs/xfs_hooks.h From patchwork Sat Feb 24 01:29:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570211 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD76C4A05 for ; Sat, 24 Feb 2024 01:29:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738169; cv=none; b=IgX4roafJIDFdQRglVHEVu7f4W6De4HWO5h+ut8jza8j7uGHXPCjQpQfhx4c2EHrdlvsjC8MyZAOaQZWfdirB1ch0UHSK+am44IQ87nlcs/6mClCXB7SlmnQUpdzP4011NbfLony/5J1YY1RXjfjxDnKxn+IRKYLAFACiQ40wLg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738169; c=relaxed/simple; bh=4TyjXnNymTmFnOEaPPoZKWYl0BDCh3Vz0zvBeg9e9cM=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=jXUbnM1mphf0NRK0iySI8qF+pPr3xjtlpjlGAmYfUBBxo8rSa1i84ZZtHhioKRdiLY+f8bX06CPdzSfecFnQFVIr4SMra6rcp2nVKx86rhcQM3b4ez0cJguujuoOup4nwjw6emBa0R4RDn8LAiPJhnox5Et53MdHZPdLtuJmJv0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uT5zDmOs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uT5zDmOs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3CF45C433C7; Sat, 24 Feb 2024 01:29:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738169; bh=4TyjXnNymTmFnOEaPPoZKWYl0BDCh3Vz0zvBeg9e9cM=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=uT5zDmOs2mjL8oqBPlEkKHK3TQgFvEK3yMGusErvLU1yEgzPbC3N23/+WlFCOm3I4 BlHItfA5buz83c4UbYwFhmmS7nzOrj72ZEKYxL4Fu7AMDKUNvstO9lFwPI2crlTz6W 3hFooCj6igowjv5KMQ2Xecbp/ZKS2JB+dEAROC+09VH6mW4dZKSAfWlzJKm6r4YqtO TVGbVbhPcSNRi/Ol1kUjQ6PND9/6q5FuDLGM7OIpzkmQvIab0zBDHeJns9vW/gW651 zjmtu2xXxL9cRFV1lSV17QLN+pu+Plqd2+l5vZ8q6WyxvAIfkA0thdHeHBXU/q2TDY M/QEY5FXTXHsw== Date: Fri, 23 Feb 2024 17:29:28 -0800 Subject: [GIT PULL 2/18] xfs: online repair of quota counters From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873800654.1891722.9189225009504434462.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 5385f1a60d4e5b73e8ecd2757865352b68f54fb9: xfs: repair file modes by scanning for a dirent pointing to us (2024-02-22 12:30:51 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-quotacheck-6.9_2024-02-23 for you to fetch changes up to 96ed2ae4a9b06b417e1c20c086c77755a43284bf: xfs: repair dquots based on live quotacheck results (2024-02-22 12:30:57 -0800) ---------------------------------------------------------------- xfs: online repair of quota counters [v29.3 02/18] This series uses the inode scanner and live update hook functionality introduced in the last patchset to implement quotacheck on a live filesystem. The quotacheck scrubber builds an incore copy of the dquot resource usage counters and compares it to the live dquots to report discrepancies. If the user chooses to repair the quota counters, the repair function visits each incore dquot to update the counts from the live information. The live update hooks are key to keeping the incore copy up to date. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (8): xfs: report the health of quota counts xfs: create a xchk_trans_alloc_empty helper for scrub xfs: create a helper to count per-device inode block usage xfs: create a sparse load xfarray function xfs: implement live quotacheck inode scan xfs: track quota updates during live quotacheck xfs: repair cannot update the summary counters when logging quota flags xfs: repair dquots based on live quotacheck results fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_fs.h | 4 +- fs/xfs/libxfs/xfs_health.h | 4 +- fs/xfs/scrub/common.c | 49 ++- fs/xfs/scrub/common.h | 11 + fs/xfs/scrub/fscounters.c | 2 +- fs/xfs/scrub/health.c | 1 + fs/xfs/scrub/quotacheck.c | 867 +++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/quotacheck.h | 76 ++++ fs/xfs/scrub/quotacheck_repair.c | 261 ++++++++++++ fs/xfs/scrub/repair.c | 46 ++- fs/xfs/scrub/repair.h | 5 + fs/xfs/scrub/scrub.c | 9 + fs/xfs/scrub/scrub.h | 10 +- fs/xfs/scrub/stats.c | 1 + fs/xfs/scrub/trace.h | 30 +- fs/xfs/scrub/xfarray.h | 19 + fs/xfs/xfs_health.c | 1 + fs/xfs/xfs_inode.c | 16 + fs/xfs/xfs_inode.h | 2 + fs/xfs/xfs_qm.c | 23 +- fs/xfs/xfs_qm.h | 16 + fs/xfs/xfs_qm_bhv.c | 1 + fs/xfs/xfs_quota.h | 46 +++ fs/xfs/xfs_trans_dquot.c | 169 +++++++- 25 files changed, 1645 insertions(+), 26 deletions(-) create mode 100644 fs/xfs/scrub/quotacheck.c create mode 100644 fs/xfs/scrub/quotacheck.h create mode 100644 fs/xfs/scrub/quotacheck_repair.c From patchwork Sat Feb 24 01:29:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570212 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F25AD625 for ; Sat, 24 Feb 2024 01:29:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738185; cv=none; b=nWrIkmo1TA/wRmtKFXm2jdPS0S64n5EGiRSkXtTeedQl3OZLoMZWuhoqaVhM68+yFjDKuzZLNqcPwQE260mnFOKI8a+VzhPk9QfYX8yAZ7rQlDT8OQ5zEDqRNVYEEEZ7EkmnQMd/D61mewlLag7mcm7mnvolCJn/9X8RwhB4LGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738185; c=relaxed/simple; bh=QZo2fZjDosVuMKI3QmLbvhw5aGQVZDFBPGL9qLIcbz8=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=Og93JR17cVT8unpL3xHp27HQO8LSykBeF0B+C6oW7VmuU/rEdTLL8y9OkHMP+jLvnnlNo6AuH2Vj3sTZLNNuAfXz5uFzTu+n8+tgejaB5k2u9gW4MF3wr3C5Rkc4PwXhDHHNHcd4AK/eLLqcjWzzenJzP6nFxe1GC5sHTeKviFk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Xj9f63S3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Xj9f63S3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CF170C433F1; Sat, 24 Feb 2024 01:29:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738184; bh=QZo2fZjDosVuMKI3QmLbvhw5aGQVZDFBPGL9qLIcbz8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Xj9f63S3n9Fyx/wwk/dEOcH9dtgKIAfRHgR3mXGNPgkXxb9IcTt8Qx36RvDPvsaW0 H4Aok8+t7EzUU5CRB/al1H67OsWx/lOAnKGZBdPUkp7UDt/juU3EStdJBhL9yNtWWP AWSNsZBW50KlFNFusXIQu8iufBskazUEFBmCPCgCU7m4ql9PhmOIcHD+XdLXu+C3yq uk+mpw+05C/bU6V4EFzPq504HxbPe20IjcVNZM0rZZ/03+eul4KO9wsgSR9FH70p7s 80/VoVloPxDeQvCN6oGwiCR6e/VscBcWkkq1hzxkWuRasLkBUlXDLk9cfo16Xc1GGT qvgZ5TWh3fu4Q== Date: Fri, 23 Feb 2024 17:29:44 -0800 Subject: [GIT PULL 3/18] xfs: online repair of file link counts From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873801105.1891722.3134162492855847646.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 96ed2ae4a9b06b417e1c20c086c77755a43284bf: xfs: repair dquots based on live quotacheck results (2024-02-22 12:30:57 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-nlinks-6.9_2024-02-23 for you to fetch changes up to 6b631c60c90a1e5264bc9bcdca2c0adb492d7a62: xfs: teach repair to fix file nlinks (2024-02-22 12:31:00 -0800) ---------------------------------------------------------------- xfs: online repair of file link counts [v29.3 03/18] Now that we've created the infrastructure to perform live scans of every file in the filesystem and the necessary hook infrastructure to observe live updates, use it to scan directories to compute the correct link counts for files in the filesystem, and reset those link counts. This patchset creates a tailored readdir implementation for scrub because the regular version has to cycle ILOCKs to copy information to userspace. We can't cycle the ILOCK during the nlink scan and we don't need all the other VFS support code (maintaining a readdir cursor and translating XFS structures to VFS structures and back) so it was easier to duplicate the code. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (4): xfs: report health of inode link counts xfs: teach scrub to check file nlinks xfs: track directory entry updates during live nlinks fsck xfs: teach repair to fix file nlinks fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_fs.h | 4 +- fs/xfs/libxfs/xfs_health.h | 4 +- fs/xfs/scrub/common.c | 3 + fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/health.c | 1 + fs/xfs/scrub/nlinks.c | 930 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/nlinks.h | 102 +++++ fs/xfs/scrub/nlinks_repair.c | 223 +++++++++++ fs/xfs/scrub/repair.h | 2 + fs/xfs/scrub/scrub.c | 9 + fs/xfs/scrub/scrub.h | 5 +- fs/xfs/scrub/stats.c | 1 + fs/xfs/scrub/trace.c | 2 + fs/xfs/scrub/trace.h | 183 ++++++++- fs/xfs/xfs_health.c | 1 + fs/xfs/xfs_inode.c | 117 ++++++ fs/xfs/xfs_inode.h | 31 ++ fs/xfs/xfs_mount.h | 3 + fs/xfs/xfs_super.c | 2 + fs/xfs/xfs_symlink.c | 1 + 21 files changed, 1623 insertions(+), 4 deletions(-) create mode 100644 fs/xfs/scrub/nlinks.c create mode 100644 fs/xfs/scrub/nlinks.h create mode 100644 fs/xfs/scrub/nlinks_repair.c From patchwork Sat Feb 24 01:29:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570213 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AD4F4688 for ; Sat, 24 Feb 2024 01:30:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738200; cv=none; b=aAZ97oFY/cOx03wrSNTiJ1kCIGa5G3SikBDAGEDfKFrTA5SKmX1ZSvC6Vfl/TGuaF0B/XmN8AtgGd6g4YMrmczc3a2cIeiYppwsYdqH1KnvASHR04thNy+p55DGW2QJAQv57z0uL8ZXui269IHxUfqBjYuFLh26jE6G7quu3mYA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738200; c=relaxed/simple; bh=C3jpxJdzpFlEl9JwjsTus9fT2zx53bmYojH4CdJqYLA=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=YqVFSzyUtAKFzEmYP8aRfHLpfeZtqX2nWHbeT5Ud7k3rkbma0zp3F8+ebFHfvYVEo7CPNiV1euIBvM2L5SQBpgLyHN3LCAM5bMtPdmh5riRCO5whkkFq/3AnFrKRmS3ozLST/iQDVpHkqj04NP6kxvxUe60hAGyRztPBFx5NJy0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qUlDzwsB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qUlDzwsB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 769ACC433C7; Sat, 24 Feb 2024 01:30:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738200; bh=C3jpxJdzpFlEl9JwjsTus9fT2zx53bmYojH4CdJqYLA=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=qUlDzwsBBjgdsmjyhj8owPG9snwssDJDy0D6W4mWJW5QQ151nEJMu56i9dqFhNsrt FRPg/pxWfTGy7qvniRahOkXVKyiLNLLGm/zbsALgxk8SfNk7llN5V8kv+jgEn+nCBG tCqsG3kDO16ilbrwDsC4HC/pmHoipthszE3EiBp3TCA9Efvg7sDwl86PGHzHHBmcHV GcyPROKoaQIVKhRxLw6sh4bSHrZjxH4W1aTsQdlYkJxU2hpWurOP59lTSsQ5kkD7yB 23w3UrPtTbNw09kEwwyG49jXHmFPXL4m8ZxyL+ejDdsJnkt3K66zXPwr6stx/A1ymt YIg4Uf7vyMddA== Date: Fri, 23 Feb 2024 17:29:59 -0800 Subject: [GIT PULL 4/18] xfs: report corruption to the health trackers From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873801577.1891722.17337684732117119668.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 6b631c60c90a1e5264bc9bcdca2c0adb492d7a62: xfs: teach repair to fix file nlinks (2024-02-22 12:31:00 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/corruption-health-reports-6.9_2024-02-23 for you to fetch changes up to 989d5ec3175be7c0012d7744c667ae6a266fab06: xfs: report XFS_IS_CORRUPT errors to the health system (2024-02-22 12:32:55 -0800) ---------------------------------------------------------------- xfs: report corruption to the health trackers [v29.3 04/18] Any time that the runtime code thinks it has found corrupt metadata, it should tell the health tracking subsystem that the corresponding part of the filesystem is sick. These reports come primarily from two places -- code that is reading a buffer that fails validation, and higher level pieces that observe a conflict involving multiple buffers. This patchset uses automated scanning to update all such callsites with a mark_sick call. Doing this enables the health system to record problem observed at runtime, which (for now) can prompt the sysadmin to run xfs_scrub, and (later) may enable more targetted fixing of the filesystem. Note: Earlier reviewers of this patchset suggested that the verifier functions themselves should be responsible for calling _mark_sick. In a higher level language this would be easily accomplished with lambda functions and closures. For the kernel, however, we'd have to create the necessary closures by hand, pass them to the buf_read calls, and then implement necessary state tracking to detach the xfs_buf from the closure at the necessary time. This is far too much work and complexity and will not be pursued further. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (11): xfs: separate the marking of sick and checked metadata xfs: report fs corruption errors to the health tracking system xfs: report ag header corruption errors to the health tracking system xfs: report block map corruption errors to the health tracking system xfs: report btree block corruption errors to the health system xfs: report dir/attr block corruption errors to the health system xfs: report symlink block corruption errors to the health system xfs: report inode corruption errors to the health system xfs: report quota block corruption errors to the health system xfs: report realtime metadata corruption errors to the health system xfs: report XFS_IS_CORRUPT errors to the health system fs/xfs/libxfs/xfs_ag.c | 5 +- fs/xfs/libxfs/xfs_alloc.c | 105 +++++++++++++++++++---- fs/xfs/libxfs/xfs_attr_leaf.c | 4 + fs/xfs/libxfs/xfs_attr_remote.c | 35 +++++--- fs/xfs/libxfs/xfs_bmap.c | 135 +++++++++++++++++++++++++---- fs/xfs/libxfs/xfs_btree.c | 39 ++++++++- fs/xfs/libxfs/xfs_da_btree.c | 37 ++++++-- fs/xfs/libxfs/xfs_dir2.c | 5 +- fs/xfs/libxfs/xfs_dir2_block.c | 2 + fs/xfs/libxfs/xfs_dir2_data.c | 3 + fs/xfs/libxfs/xfs_dir2_leaf.c | 3 + fs/xfs/libxfs/xfs_dir2_node.c | 7 ++ fs/xfs/libxfs/xfs_health.h | 42 ++++++++- fs/xfs/libxfs/xfs_ialloc.c | 57 ++++++++++-- fs/xfs/libxfs/xfs_inode_buf.c | 12 ++- fs/xfs/libxfs/xfs_inode_fork.c | 8 ++ fs/xfs/libxfs/xfs_refcount.c | 43 +++++++++- fs/xfs/libxfs/xfs_rmap.c | 83 ++++++++++++++++-- fs/xfs/libxfs/xfs_rtbitmap.c | 9 +- fs/xfs/libxfs/xfs_sb.c | 2 + fs/xfs/scrub/health.c | 8 +- fs/xfs/scrub/refcount_repair.c | 9 +- fs/xfs/xfs_attr_inactive.c | 4 + fs/xfs/xfs_attr_list.c | 18 +++- fs/xfs/xfs_dir2_readdir.c | 6 +- fs/xfs/xfs_discard.c | 2 + fs/xfs/xfs_dquot.c | 30 +++++++ fs/xfs/xfs_health.c | 186 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_icache.c | 9 ++ fs/xfs/xfs_inode.c | 16 +++- fs/xfs/xfs_iomap.c | 15 +++- fs/xfs/xfs_iwalk.c | 5 +- fs/xfs/xfs_qm.c | 8 +- fs/xfs/xfs_reflink.c | 6 +- fs/xfs/xfs_rtalloc.c | 6 ++ fs/xfs/xfs_symlink.c | 17 ++-- fs/xfs/xfs_trace.h | 4 + 37 files changed, 881 insertions(+), 104 deletions(-) From patchwork Sat Feb 24 01:30:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570214 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41FDB4A05 for ; Sat, 24 Feb 2024 01:30:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738216; cv=none; b=NwRgFZadIysip4Af4e8LwA4H510sjMKNJrzl/SLPHKayunvnhjzk3POz/neRGMoRacFpny6g6x5E5XZXmW/Ro+RJNs2TaLBL0EAwhK81Qjm0nhJJlVx0sW691vvdVLlhB9D8qePbnxrz0QPLiFwjvEeq9JqARHFX7Pwk6f4gQVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738216; c=relaxed/simple; bh=suktxaHdtmDAE3cxTGbaMcpl3bCKByt5FBAl7HfoSvM=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=DMMieb2CBsk+fqBTjT28ycnbSkOBkyyr5kLiXzZvY1qBBeZZat2wDIDgwu2vVowK1HVcVggoCqapSFr1g4CBnoy3vhVJ627kMgCb0a6XoA0jaBFQz3BJF2xifVrh65dV3TnhBr+FtyyujcbQL4c096fJxlDeyVCC7KCxvzA7PSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LmTN25OX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LmTN25OX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 19EF5C433C7; Sat, 24 Feb 2024 01:30:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738216; bh=suktxaHdtmDAE3cxTGbaMcpl3bCKByt5FBAl7HfoSvM=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=LmTN25OXfa1YvtThQuhKi9akqpjcUGuP8P8eDmCFUMlwNzglNDIk+CzzuwtM6TyvN LD7RYvx/+iK3OYBmjCGCyEN37fXG0e16zvvoTN1CpNlb1WtQGnlxT+0FoDquMkX618 gxdA6GjQ8fkQfGWMUGln8ncsqn7UQEEZF7NGauTKZKz0V6bdSwZoOZZy3LVKAoijUH IEXW+769OZAPrExMImg0zGeCHTIETLhGikicCQo2uAp3Uicyr1TwFeSbnTrgOlrbVC 06CL87Ddmwi+MYh94duc/L8wBcAuO+1uzrgOujqDyD1Wx83Wp/cZu/FTsMMU6Yh9ib E5UYBtxLobmGQ== Date: Fri, 23 Feb 2024 17:30:15 -0800 Subject: [GIT PULL 5/18] xfs: indirect health reporting From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873802029.1891722.8679783946344381114.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 989d5ec3175be7c0012d7744c667ae6a266fab06: xfs: report XFS_IS_CORRUPT errors to the health system (2024-02-22 12:32:55 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/indirect-health-reporting-6.9_2024-02-23 for you to fetch changes up to a1f3e0cca41036c3c66abb6a2ed8fedc214e9a4c: xfs: update health status if we get a clean bill of health (2024-02-22 12:33:04 -0800) ---------------------------------------------------------------- xfs: indirect health reporting [v29.3 05/18] This series enables the XFS health reporting infrastructure to remember indirect health concerns when resources are scarce. For example, if a scrub notices that there's something wrong with an inode's metadata but memory reclaim needs to free the incore inode, we want to record in the perag data the fact that there was some inode somewhere with an error. The perag structures never go away. The first two patches in this series set that up, and the third one provides a means for xfs_scrub to tell the kernel that it can forget the indirect problem report. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (3): xfs: add secondary and indirect classes to the health tracking system xfs: remember sick inodes that get inactivated xfs: update health status if we get a clean bill of health fs/xfs/libxfs/xfs_fs.h | 4 ++- fs/xfs/libxfs/xfs_health.h | 47 +++++++++++++++++++++++++ fs/xfs/libxfs/xfs_inode_buf.c | 2 +- fs/xfs/scrub/health.c | 80 +++++++++++++++++++++++++++++++++++++++++-- fs/xfs/scrub/health.h | 1 + fs/xfs/scrub/repair.c | 1 + fs/xfs/scrub/scrub.c | 6 ++++ fs/xfs/scrub/trace.h | 4 ++- fs/xfs/xfs_health.c | 33 +++++++++++------- fs/xfs/xfs_inode.c | 35 +++++++++++++++++++ fs/xfs/xfs_trace.h | 1 + 11 files changed, 196 insertions(+), 18 deletions(-) From patchwork Sat Feb 24 01:30:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570215 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D84321870 for ; Sat, 24 Feb 2024 01:30:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738231; cv=none; b=JCdpbTEC0rDC4acJQt/AePQE+rTpe0/31QNiy5oa0VEkzYsW+yG8A3ljnMiNbT9cFMocOMm6PP8XB/MHoo2zll87jhEth0TL9ZGwEpqAmOL8QOArz5b/9kgtBP63NXIrBEBgUImow7/9Lc4cWOCN5AWSg1XaPgQs51dNjQJFsX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738231; c=relaxed/simple; bh=OQG7RsKBnTatW66q/BKNCY4aBCB578bMo13ZMk6COEc=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=N9qfkeaaXItQPhx5tzqJmlwPc4MMiEdYZ6ugCSZ1m9aX0zaMj8AjLD2rfUYyTfT/8T/IFyMaWWvcyKxgCaXOY2hkM0omeW8kssmmtUyHzHRvvsQnG89ScGFnr7rdK92Ar7Rghe41EgSK5HGCr2v7BKk+3AS7UopC2yzhYv658BI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NU3+NN6C; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NU3+NN6C" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6795C433C7; Sat, 24 Feb 2024 01:30:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738231; bh=OQG7RsKBnTatW66q/BKNCY4aBCB578bMo13ZMk6COEc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=NU3+NN6C9tZgRDW1rEbZsRWj9gVaGdVkSnOR8wHFltc5YNHsG2CWpDiyNJUHp12fR dYs/N2wqO04lKFDOuq/itqfWrcc8AM48+9Nvwld9ve/wgmO0ckwBJgmutFmLcjNU+x yPCbga+gb7LAolwmYXtc5Mqwk8z78hWFlTUu/NVFaa3C19xzIZI4tGp2tlYlNEODK+ Jz3TrORD/IJJXtYRmhJkMHMBFuVEDzFYcVicoZ4XxfWkUCcwln+bTou5NHq9785VX/ 8IHMTtf1IM8l+NjzXSXir/rJxcBHbqcDIf/uw2FHy2v/Ij/1emKtJZ85Io1kMMbKw1 /JxrR9ZZpv0KA== Date: Fri, 23 Feb 2024 17:30:31 -0800 Subject: [GIT PULL 6/18] xfs: online repair for fs summary counters From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873802501.1891722.17210275939815856680.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit a1f3e0cca41036c3c66abb6a2ed8fedc214e9a4c: xfs: update health status if we get a clean bill of health (2024-02-22 12:33:04 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-fscounters-6.9_2024-02-23 for you to fetch changes up to 4ed080cd7cb077bbb4b64f0712be1618c9d55a0d: xfs: repair summary counters (2024-02-22 12:33:05 -0800) ---------------------------------------------------------------- xfs: online repair for fs summary counters [v29.3 06/18] A longstanding deficiency in the online fs summary counter scrubbing code is that it hasn't any means to quiesce the incore percpu counters while it's running. There is no way to coordinate with other threads are reserving or freeing free space simultaneously, which leads to false error reports. Right now, if the discrepancy is large, we just sort of shrug and bail out with an incomplete flag, but this is lame. For repair activity, we actually /do/ need to stabilize the counters to get an accurate reading and install it in the percpu counter. To improve the former and enable the latter, allow the fscounters online fsck code to perform an exclusive mini-freeze on the filesystem. The exclusivity prevents userspace from thawing while we're running, and the mini-freeze means that we don't wait for the log to quiesce, which will make both speedier. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (1): xfs: repair summary counters fs/xfs/Makefile | 1 + fs/xfs/scrub/fscounters.c | 27 +++++++-------- fs/xfs/scrub/fscounters.h | 20 +++++++++++ fs/xfs/scrub/fscounters_repair.c | 72 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 2 ++ fs/xfs/scrub/scrub.c | 2 +- fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 21 +++++++++--- 8 files changed, 128 insertions(+), 18 deletions(-) create mode 100644 fs/xfs/scrub/fscounters.h create mode 100644 fs/xfs/scrub/fscounters_repair.c From patchwork Sat Feb 24 01:30:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570216 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F6704688 for ; Sat, 24 Feb 2024 01:30:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738247; cv=none; b=ZRciSawSo9FXhn/bZnWqsO+xAAZKwC9xmXxkTGq/qKVRnNp2pnVmMPnpijp0+N+zr6Pcbna4rIc+9AjiRWV8d06o1rDqKrFoa14ufugCOGqe4bpyL0HYL+6OmJ9YYlQYVTn9aWOu+WkjNNdx/RuZwcHAbmd0AEdyG5v3+z031Yo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738247; c=relaxed/simple; bh=KO+GKY+7UqFG9w/cP94mtZtMiUjHLxVG9EAmyRlMdJc=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=dLMZTS28Uq2xxrlcNUTprAuNLMjQmjEW5zXHdVa6vYPX2KxmBMJo3MX39e1kauWjmu+SoI2cnK/zP2lZ3HJ5L/KiumlZ5jjNAQGkT09IpS0/mUwzBlPXHNGfScdf7swsDklkl0nFHQXSEK2r2tD+bJ+hVrp/wvyoI/XzgzdV8AI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=k4ayKDRl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="k4ayKDRl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5A517C433F1; Sat, 24 Feb 2024 01:30:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738247; bh=KO+GKY+7UqFG9w/cP94mtZtMiUjHLxVG9EAmyRlMdJc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=k4ayKDRl12iAdCQ9g3BKOudF7JuxgphpapvYc7j9HP8w8612MOzcbzlfNKPZYay2V 32dfx7uGI2qRdjLgu3cud4LLOkGnCL/Oe51wuP8mnnLRMgdBwBdOGCkahWx3GCSF8A 2gIytu7ZuPwJoZ9em7/vvZIcQ+6q5Le9z/eE+e5RDlWHcAkRsQmCdbNjep5Q4He9Na LMyuJ3+BrLkrDF9rku594pcSf/2ryEHWKg9i8C9mPtkh//mMoircC+pRn8uvgOy7Dh zD/+I6AhWnG8W/KJKupXXf2ubDhzDo33VWoBGJI0EkYqRCuzRzXe6IV7R/5ERcBpBq PIU9fAQXrICuw== Date: Fri, 23 Feb 2024 17:30:46 -0800 Subject: [GIT PULL 7/18] xfs: move btree geometry to ops struct From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873802963.1891722.16861837496958424107.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 4ed080cd7cb077bbb4b64f0712be1618c9d55a0d: xfs: repair summary counters (2024-02-22 12:33:05 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/btree-geometry-in-ops-6.9_2024-02-23 for you to fetch changes up to f73def90a7cd24a32a42f689efba6a7a35edeb7b: xfs: create predicate to determine if cursor is at inode root level (2024-02-22 12:37:24 -0800) ---------------------------------------------------------------- xfs: move btree geometry to ops struct [v29.3 07/18] This patchset prepares the generic btree code to allow for the creation of new btree types outside of libxfs. The end goal here is for online fsck to be able to create its own in-memory btrees that will be used to improve the performance (and reduce the memory requirements of) the refcount btree. To enable this, I decided that the btree ops structure is the ideal place to encode all of the geometry information about a btree. The btree ops struture already contains the buffer ops (and hence the btree block magic numbers) as well as the key and record sizes, so it doesn't seem all that farfetched to encode the XFS_BTREE_ flags that determine the geometry (ROOT_IN_INODE, LONG_PTRS, etc). The rest of the patchset cleans up the btree functions that initialize btree blocks and btree buffers. The bulk of this work is to replace btree geometry related function call arguments with a single pointer to the ops structure, and then clean up everything else around that. As a side effect, we rename the functions. Later, Christoph Hellwig and I merged together a bunch more cleanups that he wanted to do for a while. All the btree geometry information is now in the btree ops structure, we've created an explicit btree type (ag, inode, mem) and moved the per-btree type information to a separate union. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Christoph Hellwig (6): xfs: remove bc_ino.flags xfs: consolidate the xfs_alloc_lookup_* helpers xfs: turn the allocbt cursor active field into a btree flag xfs: move the btree stats offset into struct btree_ops xfs: split out a btree type from the btree ops geometry flags xfs: split the per-btree union in struct xfs_btree_cur Darrick J. Wong (17): xfs: consolidate btree block freeing tracepoints xfs: consolidate btree block allocation tracepoints xfs: set the btree cursor bc_ops in xfs_btree_alloc_cursor xfs: drop XFS_BTREE_CRC_BLOCKS xfs: fix imprecise logic in xchk_btree_check_block_owner xfs: encode the btree geometry flags in the btree ops structure xfs: extern some btree ops structures xfs: initialize btree blocks using btree_ops structure xfs: rename btree block/buffer init functions xfs: btree convert xfs_btree_init_block to xfs_btree_init_buf calls xfs: remove the unnecessary daddr paramter to _init_block xfs: set btree block buffer ops in _init_buf xfs: move lru refs to the btree ops structure xfs: factor out a xfs_btree_owner helper xfs: factor out a btree block owner check xfs: store the btree pointer length in struct xfs_btree_ops xfs: create predicate to determine if cursor is at inode root level fs/xfs/libxfs/xfs_ag.c | 33 ++-- fs/xfs/libxfs/xfs_ag.h | 2 +- fs/xfs/libxfs/xfs_alloc.c | 54 +++--- fs/xfs/libxfs/xfs_alloc_btree.c | 39 ++-- fs/xfs/libxfs/xfs_bmap.c | 58 +++--- fs/xfs/libxfs/xfs_bmap_btree.c | 59 +++--- fs/xfs/libxfs/xfs_bmap_btree.h | 3 + fs/xfs/libxfs/xfs_btree.c | 365 +++++++++++++++++++------------------ fs/xfs/libxfs/xfs_btree.h | 165 +++++++++-------- fs/xfs/libxfs/xfs_btree_staging.c | 20 +- fs/xfs/libxfs/xfs_btree_staging.h | 3 +- fs/xfs/libxfs/xfs_ialloc_btree.c | 35 ++-- fs/xfs/libxfs/xfs_refcount.c | 24 +-- fs/xfs/libxfs/xfs_refcount_btree.c | 24 ++- fs/xfs/libxfs/xfs_rmap_btree.c | 19 +- fs/xfs/libxfs/xfs_shared.h | 9 + fs/xfs/scrub/btree.c | 29 +-- fs/xfs/scrub/newbt.c | 2 +- fs/xfs/scrub/trace.c | 2 +- fs/xfs/xfs_trace.h | 83 ++++++++- 20 files changed, 566 insertions(+), 462 deletions(-) From patchwork Sat Feb 24 01:31:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570217 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 513BE1870 for ; Sat, 24 Feb 2024 01:31:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738263; cv=none; b=LqiWK3huPi3ESYtunTYw+/aq3QdGUgn8LEEgNGcUEKHOIn8jFlvwURhQGAthfuOJPb6QOe0s6ywqxQt+TnSkrsGsR9a3cZefalhbXEt4b1uTWciQLbJDZVIqegLDI3va2h5ziQJeT9VZiSosy7TnAEMAIT4DSIH6BM9lgeQM1s4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738263; c=relaxed/simple; bh=zjLMWac6yhFyPvccjPrQAm/1HWYu8rlm5WU6739KsFU=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=Qj2SHTQC4jp+MvVbNst5gAx9g6sXTPxoS4MYvnsNejk3yc3upUZtsHAZwVLzKPZMie3HawCvsPUsoL0f0gktFAQmlm0xce7pMLFyZuz1vzM9HqaVbv8+cylKxk5kksqHPeKb9hiTiQGx9fig/MYRUnaPyvlQowQ2Q/TzPdB1Xc0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=V3THODWb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V3THODWb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 08C19C433C7; Sat, 24 Feb 2024 01:31:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738263; bh=zjLMWac6yhFyPvccjPrQAm/1HWYu8rlm5WU6739KsFU=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=V3THODWbkGSJGqBPqn5npa/Amdk44vpwjiMR3HK4UE3n1QjV6gTizPy3I9ynchLFY oMmG+n6Gt5xiHnRfub1OM6FVVJEqDGVKmawy6VKo3ZRjk/i/srxror16j8/83cWsmd jkV2sX7MBnyhMO+bWvXgtF+RBNMPBOH4TVMoTkSVtSJvNPpO5+/3f+F3GIaNPvunYJ PvSA+00jMz30Sp8Z4idHReBOtpAZ0ujCnwfjl1TC0j8Gtl3+CCFa8U2QA2ZCTmeqA3 5/LVnRpVV83tUjoYw24r3URC9+pye9TOixPyjs6GEX503/uxFmIQGX4NCUj0QgZjLf cD7k0CJdmn5/Q== Date: Fri, 23 Feb 2024 17:31:02 -0800 Subject: [GIT PULL 8/18] xfs: remove bc_btnum from btree cursors From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873803411.1891722.4886429032847981179.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit f73def90a7cd24a32a42f689efba6a7a35edeb7b: xfs: create predicate to determine if cursor is at inode root level (2024-02-22 12:37:24 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/btree-remove-btnum-6.9_2024-02-23 for you to fetch changes up to ec793e690f801d97a7ae2a0d429fea1fee4d44aa: xfs: remove xfs_btnum_t (2024-02-22 12:40:51 -0800) ---------------------------------------------------------------- xfs: remove bc_btnum from btree cursors [v29.3 08/18] From Christoph Hellwig, This series continues the migration of btree geometry information out of the cursor structure and into the ops structure. This time around, we replace the btree type enumeration (btnum) with an explicit name string in the btree ops structure. This enables easy creation of /any/ new btree type without having to mess with libxfs. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Christoph Hellwig (26): xfs: move comment about two 2 keys per pointer in the rmap btree xfs: add a xfs_btree_init_ptr_from_cur xfs: don't override bc_ops for staging btrees xfs: fold xfs_allocbt_init_common into xfs_allocbt_init_cursor xfs: remove xfs_allocbt_stage_cursor xfs: fold xfs_inobt_init_common into xfs_inobt_init_cursor xfs: remove xfs_inobt_stage_cursor xfs: fold xfs_refcountbt_init_common into xfs_refcountbt_init_cursor xfs: remove xfs_refcountbt_stage_cursor xfs: fold xfs_rmapbt_init_common into xfs_rmapbt_init_cursor xfs: remove xfs_rmapbt_stage_cursor xfs: make full use of xfs_btree_stage_ifakeroot in xfs_bmbt_stage_cursor xfs: fold xfs_bmbt_init_common into xfs_bmbt_init_cursor xfs: remove xfs_bmbt_stage_cursor xfs: split the agf_roots and agf_levels arrays xfs: add a name field to struct xfs_btree_ops xfs: add a sick_mask to struct xfs_btree_ops xfs: refactor the btree cursor allocation logic in xchk_ag_btcur_init xfs: split xfs_allocbt_init_cursor xfs: remove xfs_inobt_cur xfs: remove the btnum argument to xfs_inobt_count_blocks xfs: remove the which variable in xchk_iallocbt xfs: split xfs_inobt_insert_sprec xfs: split xfs_inobt_init_cursor xfs: pass a 'bool is_finobt' to xfs_inobt_insert xfs: remove xfs_btnum_t Darrick J. Wong (1): xfs: make staging file forks explicit fs/xfs/libxfs/xfs_ag.c | 13 ++- fs/xfs/libxfs/xfs_ag.h | 8 +- fs/xfs/libxfs/xfs_alloc.c | 99 +++++++++----------- fs/xfs/libxfs/xfs_alloc_btree.c | 156 ++++++++++++++++++-------------- fs/xfs/libxfs/xfs_alloc_btree.h | 10 +- fs/xfs/libxfs/xfs_bmap_btree.c | 89 ++++++------------ fs/xfs/libxfs/xfs_bmap_btree.h | 2 - fs/xfs/libxfs/xfs_btree.c | 114 +++++++++++++++++------ fs/xfs/libxfs/xfs_btree.h | 18 ++-- fs/xfs/libxfs/xfs_btree_staging.c | 111 +---------------------- fs/xfs/libxfs/xfs_btree_staging.h | 7 +- fs/xfs/libxfs/xfs_format.h | 21 ++--- fs/xfs/libxfs/xfs_ialloc.c | 181 +++++++++++++++++++++++-------------- fs/xfs/libxfs/xfs_ialloc_btree.c | 144 +++++++++++++---------------- fs/xfs/libxfs/xfs_ialloc_btree.h | 11 +-- fs/xfs/libxfs/xfs_refcount_btree.c | 52 ++++------- fs/xfs/libxfs/xfs_refcount_btree.h | 2 - fs/xfs/libxfs/xfs_rmap_btree.c | 65 +++++-------- fs/xfs/libxfs/xfs_rmap_btree.h | 2 - fs/xfs/libxfs/xfs_shared.h | 35 +++++++ fs/xfs/libxfs/xfs_types.h | 26 +----- fs/xfs/scrub/agheader.c | 12 +-- fs/xfs/scrub/agheader_repair.c | 47 ++++------ fs/xfs/scrub/alloc_repair.c | 27 +++--- fs/xfs/scrub/bmap_repair.c | 8 +- fs/xfs/scrub/btree.c | 12 +-- fs/xfs/scrub/common.c | 72 ++++++++------- fs/xfs/scrub/health.c | 54 ++--------- fs/xfs/scrub/health.h | 4 +- fs/xfs/scrub/ialloc.c | 20 ++-- fs/xfs/scrub/ialloc_repair.c | 8 +- fs/xfs/scrub/iscan.c | 2 +- fs/xfs/scrub/refcount_repair.c | 4 +- fs/xfs/scrub/repair.c | 14 +-- fs/xfs/scrub/rmap.c | 15 ++- fs/xfs/scrub/trace.h | 48 ++++------ fs/xfs/xfs_discard.c | 2 +- fs/xfs/xfs_fsmap.c | 4 +- fs/xfs/xfs_health.c | 36 +++----- fs/xfs/xfs_iwalk.c | 8 +- fs/xfs/xfs_trace.h | 89 ++++++++---------- 41 files changed, 748 insertions(+), 904 deletions(-) From patchwork Sat Feb 24 01:31:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570218 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3D0833D5 for ; Sat, 24 Feb 2024 01:31:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738278; cv=none; b=a0ueXo1mXoJtEAXVDJxwwjunDPkdzsT6PrKJjWIQtoqnNZo8b5Tu8ss6O9z0DFrtxOJConWjuxwmWDbgDaD0dMWyULQtI/LoHTMUQh3Enxpk/s7Xsr5xr0h3q+pWh04h2V78dP/0eLRX99oFEAgO+VNz0k7vR4MQFXf4jyAJu5U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738278; c=relaxed/simple; bh=SUxXk/d6Di34HzOP+BqlV+rya6+1jiTshokluM59l9k=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=batjZeiutFbmKef1lCTej6Ph0rH1ttkdKGwoWuKdgGNgianHm/UwKOYKk7YUYnajbbmVMu0qN6x8PpS8NZwIs2AOPzptT8zy2T3JKglm8hKVXfz+LAHRr/7pNnU7O9CPEErp9L5w6SCrslg2cfN0Y9L+AyrxU9ExMTbzQvIjoC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=o7sROnav; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="o7sROnav" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9A0EBC433F1; Sat, 24 Feb 2024 01:31:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738278; bh=SUxXk/d6Di34HzOP+BqlV+rya6+1jiTshokluM59l9k=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=o7sROnav25IeN3sApW7vENlMI5LXx+77GDVZRrCk1Wv+m2RF5fyvXTd4XhRGr5QLU DAk5PeTdDMairJoc1CFmeuaHPq0ttlv14JaKjQoc4S3kfSkDwXJ4TqCL/O5puzou+Z O+/Cspnnt6VPHVKMS7QG+0xhq6JWyNKCGg+Is8g4PU2+hcEOBoknjbhzqPBNCxI3fJ qy1noBPlwthyP/Zxr1rDT+I62HXrDnudpdJkeBMfrtcmgVomAfYlfpie1nZTt6XC6T BexZPz2FyFiLeXFnrQEpqf4EKboMSX+cDOavLiHHg2YjQdFyZW6TW/Nys3DPxxTiX2 ubEZRDvbsRmTw== Date: Fri, 23 Feb 2024 17:31:18 -0800 Subject: [GIT PULL 9/18] xfs: btree check cleanups From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873803840.1891722.16794150368300526439.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit ec793e690f801d97a7ae2a0d429fea1fee4d44aa: xfs: remove xfs_btnum_t (2024-02-22 12:40:51 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/btree-check-cleanups-6.9_2024-02-23 for you to fetch changes up to 79e72304dcba471e5c0dea2f3c67fe1a0558c140: xfs: factor out a __xfs_btree_check_lblock_hdr helper (2024-02-22 12:40:59 -0800) ---------------------------------------------------------------- xfs: btree check cleanups [v29.3 09/18] Minor cleanups for the btree block pointer checking code. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Christoph Hellwig (10): xfs: simplify xfs_btree_check_sblock_siblings xfs: simplify xfs_btree_check_lblock_siblings xfs: open code xfs_btree_check_lptr in xfs_bmap_btree_to_extents xfs: consolidate btree ptr checking xfs: misc cleanups for __xfs_btree_check_sblock xfs: remove the crc variable in __xfs_btree_check_lblock xfs: tighten up validation of root block in inode forks xfs: consolidate btree block verification xfs: rename btree helpers that depends on the block number representation xfs: factor out a __xfs_btree_check_lblock_hdr helper fs/xfs/libxfs/xfs_alloc_btree.c | 8 +- fs/xfs/libxfs/xfs_bmap.c | 2 +- fs/xfs/libxfs/xfs_bmap_btree.c | 8 +- fs/xfs/libxfs/xfs_btree.c | 252 ++++++++++++++++++------------------- fs/xfs/libxfs/xfs_btree.h | 44 ++----- fs/xfs/libxfs/xfs_ialloc_btree.c | 8 +- fs/xfs/libxfs/xfs_refcount_btree.c | 8 +- fs/xfs/libxfs/xfs_rmap_btree.c | 8 +- fs/xfs/scrub/btree.c | 21 +--- 9 files changed, 159 insertions(+), 200 deletions(-) From patchwork Sat Feb 24 01:31:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570219 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2ED64A07 for ; Sat, 24 Feb 2024 01:31:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738294; cv=none; b=POZH4XzsPu+ZrTd0oR4d+SL1Du+zTYsDawLSGdWPT3M++luQKGqGhHILNtDz5Mtiyd0o5x94Zpb6pH1CW7dVmFe5ckqguQ5DQx0q/0mLBRsJMPv8k4+g5Vmtp3c+GvGmjbrlb3RBuxK7tEmLNii4v/Fa4wBSL53LJG4ba6AWsew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738294; c=relaxed/simple; bh=U9olQcFxe+kYmPDtYLriwQWo3oAL9r+FtBkX3sr0Ws8=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=Q1T0C/Jr+Myb8darjqxFDRZZvYlb6rROx8zwEihVTwjFLC4XuEQ3fmkhfh/5B9gc7MIQ3TqRHJKtL09shavsZ6Jda9A5az6RYVVNGMIlKzyrr/f1uPGicIRdJV1YqEGFf7/6xN+etOoCsFbNx7X0MbE0gPrFAA7ePNvS2+oIzP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hDTLmgvm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hDTLmgvm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 480B9C433F1; Sat, 24 Feb 2024 01:31:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738294; bh=U9olQcFxe+kYmPDtYLriwQWo3oAL9r+FtBkX3sr0Ws8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=hDTLmgvmOVixpRp4SM45DG3JokibzIDx5MROlQP0QtaY296DwSnQdmWpJUYB5GbzV 0fBU4TG9SnL6tSVdGUOrnoJruR8SJA9HWxm6MQQKYeMhj+cVY+dykL8qCcCaif/mXh 1PdSbsdl3nVt7UsyOdFld2eeThqz02IbB+ufhtniFR+mU2wUn4VZXnRVXWocIZntb5 k6jgr6kHr1wuGn4SbKWl/K0k2TgX2REnjGbaIJNbwjOmBcVpsjkzRXABe0XhFys3gP wNRYs7jiDu6mCV1lNzKFQQF/Bmdsy0xnbWcwFXCd5Y8MLljfE0aeBmNV2Lwp2Yg84x sg9MItC9tap+Q== Date: Fri, 23 Feb 2024 17:31:33 -0800 Subject: [GIT PULL 10/18] xfs: btree readahead cleanups From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873804279.1891722.10129509039878020958.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 79e72304dcba471e5c0dea2f3c67fe1a0558c140: xfs: factor out a __xfs_btree_check_lblock_hdr helper (2024-02-22 12:40:59 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/btree-readahead-cleanups-6.9_2024-02-23 for you to fetch changes up to 24f755e4854e0fddb78d18f610bf1b5cb61db520: xfs: split xfs_buf_rele for cached vs uncached buffers (2024-02-22 12:41:02 -0800) ---------------------------------------------------------------- xfs: btree readahead cleanups [v29.3 10/18] Minor cleanups for the btree block readahead code. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Christoph Hellwig (4): xfs: remove xfs_btree_reada_bufl xfs: remove xfs_btree_reada_bufs xfs: move and rename xfs_btree_read_bufl xfs: split xfs_buf_rele for cached vs uncached buffers fs/xfs/libxfs/xfs_bmap.c | 33 ++++++++++++---- fs/xfs/libxfs/xfs_btree.c | 98 ++++++++--------------------------------------- fs/xfs/libxfs/xfs_btree.h | 36 ----------------- fs/xfs/xfs_buf.c | 46 ++++++++++++++-------- fs/xfs/xfs_iwalk.c | 6 ++- 5 files changed, 76 insertions(+), 143 deletions(-) From patchwork Sat Feb 24 01:31:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570220 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AD25625 for ; Sat, 24 Feb 2024 01:31:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738310; cv=none; b=tW3yTK7kbsShp7g5vtiXWljzkIDuGqzf2bbYqGiQaOm3+zMjVu/8sEmxWYenm9/m/fzr481p64MB1ikooHf+2SHPTO20wm+G8ceMqaeet1Q/tUekA5Iz1ZnfHvJ+MEQfTbMrvYD672n05NpAruzXzLKnjZQe8Pb7D1OH9VBEzXQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738310; c=relaxed/simple; bh=pResZIT9N+Z3yPRkJ9mHr6/wyCdmZyUvyRAtV0JUd8E=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=U4/IzyQ6li5Zo/sVSUDfKmwCofheSbcRV38ri/3bvGeuG6TFpf1rusalJRELBE1EUxNfaQyfoBpUMS0JW9MzkqDFl/jfB1puxwaQj5gmQvQdL9GARb0bVnvzPDOt3QCZoqNREp9fsLm3+arXlTNd+EPNhSe+RqRoDigl6KrRhQE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FiDyMGsz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FiDyMGsz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E17E8C433F1; Sat, 24 Feb 2024 01:31:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738309; bh=pResZIT9N+Z3yPRkJ9mHr6/wyCdmZyUvyRAtV0JUd8E=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=FiDyMGszo/v7FN1w4UjqY14Vgm2jgXBG6dthbUtGM5+UPskEi/bvbFTNjc0sW9mqA PxbWWKb4wpWoxaEUVL95r0y7U6ZW7HTtKrNYsEScFURp8wfd6BLKpuI25B2Lb4Kq3o ghrxd7zQkRltUKksrGLYVhkmI7P7ASqYACbqqyoAbrLbfEneSnkOBF+sPiP17uHdZz BA2gPPGWMCm7EHggUVBr+N62bzS191YM7VfEFpxcrkabHLz4D2U3I0GsEXdvCl0xpS hPeoX2EQbytoPa8WU2l7jS2xT6p9DNhgU9xFEjunkLsGQKG3ITJu/iyPifeMKrilHd bgdCCvpE2zWXA== Date: Fri, 23 Feb 2024 17:31:49 -0800 Subject: [GIT PULL 11/18] xfs: buftarg cleanups From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873804706.1891722.8290634043484810651.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 24f755e4854e0fddb78d18f610bf1b5cb61db520: xfs: split xfs_buf_rele for cached vs uncached buffers (2024-02-22 12:41:02 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/buftarg-cleanups-6.9_2024-02-23 for you to fetch changes up to 1c51ac0998ed9baaca3ac75c0083b4c3b4d993ef: xfs: move setting bt_logical_sectorsize out of xfs_setsize_buftarg (2024-02-22 12:42:45 -0800) ---------------------------------------------------------------- xfs: buftarg cleanups [v29.3 11/18] Clean up the buffer target code in preparation for adding the ability to target tmpfs files. That will enable the creation of in memory btrees. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Christoph Hellwig (3): xfs: remove the xfs_buftarg_t typedef xfs: remove xfs_setsize_buftarg_early xfs: move setting bt_logical_sectorsize out of xfs_setsize_buftarg fs/xfs/xfs_buf.c | 34 +++++++++++++--------------------- fs/xfs/xfs_buf.h | 4 ++-- fs/xfs/xfs_log.c | 14 +++++++------- fs/xfs/xfs_mount.h | 6 +++--- 4 files changed, 25 insertions(+), 33 deletions(-) From patchwork Sat Feb 24 01:32:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570221 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FB0D33D5 for ; Sat, 24 Feb 2024 01:32:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738326; cv=none; b=m9tTLS3wmbtExoOkLnZrlppwbnnwteDt0f6jImPnFOIC7pEIGqaAhyUnd26FutkTjJN9cdjQqyGd7TBZVXlOj+DBbAk62hqQQ8oYUyyjsQSKKXZcBERARbciQF3S4uTZNzYoeBcqadZThFz3hTm1dy7+7f6+5bPV8xM/wU8102o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738326; c=relaxed/simple; bh=6678nJzm9H/zU2bvB48AQIFZf4Lz07TwYQOOJf1LyeA=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=aGousxadq71tlfh8IsMQbEaX3hYnmf4BLJwwdR7u5I06POfLaGzE50ngoSvnDu3vmFb+mMvb7yOSd/DUtMKAUpt79+ZYcbQFpiuzEMdhhPPqZPJH+dRzus3XsAdnQZIA4fnefsbUUJQ6t56ctcQ2KuHtkeYjmqNB3QkShXtagwg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Vd7++OJP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Vd7++OJP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9AE97C433C7; Sat, 24 Feb 2024 01:32:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738325; bh=6678nJzm9H/zU2bvB48AQIFZf4Lz07TwYQOOJf1LyeA=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Vd7++OJP9de6NQTQElOkr9n+znHEOVC4fYnDCi/V/gBlplVhJKBe9LBDaxi28x6rU b6FF5Crmuc/srv3d1P+v8lovgzS1HxkTBu5UquO4UE/ax9KvugVznTSJSs8rKNaSvO tUOfwC0aXZJnKLi673yihaRXZSfV16NkC57lDptCpzciPPQ3GuAoFEOASicqYU7OQ3 bnPN3YXi3V4fTDGNJuQ2bxkd6g6ejrlHqoO8ZM4dNOvRiY9J0wruxo2zTKrbSSnH+Z OkmL3+U7KWCXAJ9Ijr8Wn4td0mFktwWuZebrxqWgMjFRpNv/PDb/RBwU1P5LErI+Aa HFzTu4ODI6byw== Date: Fri, 23 Feb 2024 17:32:05 -0800 Subject: [GIT PULL 12/18] xfs: support in-memory btrees From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org, willy@infradead.org Message-ID: <170873805154.1891722.14629140541264696458.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 1c51ac0998ed9baaca3ac75c0083b4c3b4d993ef: xfs: move setting bt_logical_sectorsize out of xfs_setsize_buftarg (2024-02-22 12:42:45 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/in-memory-btrees-6.9_2024-02-23 for you to fetch changes up to 0dc63c8a1ce39c1ac7da536ee9174cdc714afae2: xfs: launder in-memory btree buffers before transaction commit (2024-02-22 12:43:36 -0800) ---------------------------------------------------------------- xfs: support in-memory btrees [v29.3 12/18] Online repair of the reverse-mapping btrees presens some unique challenges. To construct a new reverse mapping btree, we must scan the entire filesystem, but we cannot afford to quiesce the entire filesystem for the potentially lengthy scan. For rmap btrees, therefore, we relax our requirements of totally atomic repairs. Instead, repairs will scan all inodes, construct a new reverse mapping dataset, format a new btree, and commit it before anyone trips over the corruption. This is exactly the same strategy as was used in the quotacheck and nlink scanners. Unfortunately, the xfarray cannot perform key-based lookups and is therefore unsuitable for supporting live updates. Luckily, we already a data structure that maintains an indexed rmap recordset -- the existing rmap btree code! Hence we port the existing btree and buffer target code to be able to create a btree using the xfile we developed earlier. Live hooks keep the in-memory btree up to date for any resources that have already been scanned. This approach is not maximally memory efficient, but we can use the same rmap code that we do everywhere else, which provides improved stability without growing the code base even more. Note that in-memory btree blocks are always page sized. This patchset modifies the kernel xfs buffer cache to be capable of using a xfile (aka a shmem file) as a backing device. It then augments the btree code to support creating btree cursors with buffers that come from a buftarg other than the data device (namely an xfile-backed buftarg). For the userspace xfs buffer cache, we instead use a memfd or an O_TMPFILE file as a backing device. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Christoph Hellwig (1): xfs: add a xfs_btree_ptrs_equal helper Darrick J. Wong (4): xfs: teach buftargs to maintain their own buffer hashtable xfs: support in-memory buffer cache targets xfs: support in-memory btrees xfs: launder in-memory btree buffers before transaction commit .../filesystems/xfs/xfs-online-fsck-design.rst | 5 +- fs/xfs/Kconfig | 8 + fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_ag.c | 6 +- fs/xfs/libxfs/xfs_ag.h | 4 +- fs/xfs/libxfs/xfs_btree.c | 286 ++++++++++++++--- fs/xfs/libxfs/xfs_btree.h | 7 + fs/xfs/libxfs/xfs_btree_mem.c | 347 +++++++++++++++++++++ fs/xfs/libxfs/xfs_btree_mem.h | 75 +++++ fs/xfs/scrub/scrub.c | 5 + fs/xfs/scrub/scrub.h | 3 + fs/xfs/xfs_buf.c | 214 ++++++++----- fs/xfs/xfs_buf.h | 17 + fs/xfs/xfs_buf_mem.c | 270 ++++++++++++++++ fs/xfs/xfs_buf_mem.h | 34 ++ fs/xfs/xfs_health.c | 3 + fs/xfs/xfs_mount.h | 3 - fs/xfs/xfs_trace.c | 2 + fs/xfs/xfs_trace.h | 167 +++++++++- fs/xfs/xfs_trans.h | 1 + fs/xfs/xfs_trans_buf.c | 42 +++ 21 files changed, 1363 insertions(+), 138 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_btree_mem.c create mode 100644 fs/xfs/libxfs/xfs_btree_mem.h create mode 100644 fs/xfs/xfs_buf_mem.c create mode 100644 fs/xfs/xfs_buf_mem.h From patchwork Sat Feb 24 01:32:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570222 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D27B625 for ; Sat, 24 Feb 2024 01:32:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738341; cv=none; b=GhH8qPY6YbFuPx+nNM5/lx0bc+L9idbhZquI55BXKZNom0CvGQ98xT1WcCkv/w2YocW6TbAPHtvC4ZVrJn7AZHab1y/cbA+zUrRDYuLotrK85yeRh9h7fIXFAeGxAiWZI2kxlEFfv8ymOztHucjgltTI6jfS4QRV5ygqdsm38Kk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738341; c=relaxed/simple; bh=zdli9LFL/uqffAQ7xj8g7uM+DK5FlT30fWtkV4A4jN4=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=O10wKHIVqyL9dP0qCmCVxRy1NbnBV3oYpVcqiztjtWYXlxTBetms5ruDgCEKpDDnzfgaDpc4Z4MfMC+p9sVdfhBi8YvBgacouLmNgv13BqEFqJaimQBW+rXF/VrWwlVjlpd1eEa7GYW0LvEgYQ1EdpEYHAGYVZDXmYOw2IEM21o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NtWI7TnJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NtWI7TnJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B52FC433C7; Sat, 24 Feb 2024 01:32:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738341; bh=zdli9LFL/uqffAQ7xj8g7uM+DK5FlT30fWtkV4A4jN4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=NtWI7TnJNiJpBpWYKkBvad56HUGUa8ee6/jXZGRZdI4lP1IfHEeorvCHmG7H59J4K 7Z3tGizCxY45TzAqBL0ByLozlkN3IEnhaZv71uhdxrMVje7pc37+5RFJrud7uIUz/j v9u1EFkYeTEe+mYUkw0W7EBkMEcCrAblIX9FTTpSzlyYjcArgQlP1owiogFdYWMcGp Tv7l76KS4QBklromTIeQyzkzbmLAcRe1IQT91BOFbBoCRgjVxdg4xsykdFUlvAXkE4 uMJIzbCavPGVwRP4SHhNLjUzbD4PlqZo8uNQGrEBemJCqfR4nfdLfnvyXsVeq3ndsB cIM6/AR/R8iBQ== Date: Fri, 23 Feb 2024 17:32:20 -0800 Subject: [GIT PULL 13/18] xfs: online repair of rmap btrees From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873805592.1891722.8112765815277030914.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 0dc63c8a1ce39c1ac7da536ee9174cdc714afae2: xfs: launder in-memory btree buffers before transaction commit (2024-02-22 12:43:36 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-rmap-btree-6.9_2024-02-23 for you to fetch changes up to 7e1b84b24d257700e417bc9cd724c1efdff653d7: xfs: hook live rmap operations during a repair operation (2024-02-22 12:43:40 -0800) ---------------------------------------------------------------- xfs: online repair of rmap btrees [v29.3 13/18] We have now constructed the four tools that we need to scan the filesystem looking for reverse mappings: an inode scanner, hooks to receive live updates from other writer threads, the ability to construct btrees in memory, and a btree bulk loader. This series glues those three together, enabling us to scan the filesystem for mappings and keep it up to date while other writers run, and then commit the new btree to disk atomically. To reduce the size of each patch, the functionality is left disabled until the end of the series and broken up into three patches: one to create the mechanics of scanning the filesystem, a second to transition to in-memory btrees, and a third to set up the live hooks. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (5): xfs: create a helper to decide if a file mapping targets the rt volume xfs: create agblock bitmap helper to count the number of set regions xfs: repair the rmapbt xfs: create a shadow rmap btree during rmap repair xfs: hook live rmap operations during a repair operation fs/xfs/Makefile | 1 + fs/xfs/libxfs/xfs_ag.c | 1 + fs/xfs/libxfs/xfs_ag.h | 4 + fs/xfs/libxfs/xfs_bmap.c | 49 +- fs/xfs/libxfs/xfs_bmap.h | 8 + fs/xfs/libxfs/xfs_inode_fork.c | 9 + fs/xfs/libxfs/xfs_inode_fork.h | 1 + fs/xfs/libxfs/xfs_rmap.c | 199 +++-- fs/xfs/libxfs/xfs_rmap.h | 31 +- fs/xfs/libxfs/xfs_rmap_btree.c | 163 +++- fs/xfs/libxfs/xfs_rmap_btree.h | 6 + fs/xfs/libxfs/xfs_shared.h | 10 + fs/xfs/scrub/agb_bitmap.h | 5 + fs/xfs/scrub/bitmap.c | 14 + fs/xfs/scrub/bitmap.h | 2 + fs/xfs/scrub/bmap.c | 2 +- fs/xfs/scrub/common.c | 5 +- fs/xfs/scrub/common.h | 1 + fs/xfs/scrub/newbt.c | 12 +- fs/xfs/scrub/newbt.h | 7 + fs/xfs/scrub/reap.c | 2 +- fs/xfs/scrub/repair.c | 59 +- fs/xfs/scrub/repair.h | 12 +- fs/xfs/scrub/rmap.c | 11 +- fs/xfs/scrub/rmap_repair.c | 1697 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/scrub.c | 6 +- fs/xfs/scrub/scrub.h | 4 +- fs/xfs/scrub/trace.c | 1 + fs/xfs/scrub/trace.h | 80 +- fs/xfs/xfs_stats.c | 3 +- fs/xfs/xfs_stats.h | 1 + 31 files changed, 2336 insertions(+), 70 deletions(-) create mode 100644 fs/xfs/scrub/rmap_repair.c From patchwork Sat Feb 24 01:32:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570223 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51F784688 for ; Sat, 24 Feb 2024 01:32:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738357; cv=none; b=ez9tUYQwEH2TmgO4zyUOtQvJpZYi8F8RpDK45HMyv7fPvLgxll9U9DHKCjdTUuKOUZpGNo/e0EBodmcJDmI/ks0H0MsUxAFZ5CR3IIHjzV2ai03gLuUKlAaVGIaH78DV7kRrar7FNeP/jzU7vbb85i/ZLzmNoRDc0hypwBGA1D8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738357; c=relaxed/simple; bh=+ma+bDQO0E92vjWxNSNah6czrS3mDBVru8Fwu914U8M=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=X5aQv8UpJpJ6QZcsbOsqFFAmsQGHaZmuu2ChpTmZmn2a3eeY3OlKRB05Kj77WN9wrscr0qg5I2afS2m2+p/+LZAjTNLgysTenTBr6G6HieuSBHfk0758gJoma2zPaxKj6Z+hY9TQ/bBZ55Ija8Xd4wu6f3CLW/C2oZiU2+PdLRk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LRU7d5I0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LRU7d5I0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D23A4C433F1; Sat, 24 Feb 2024 01:32:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738356; bh=+ma+bDQO0E92vjWxNSNah6czrS3mDBVru8Fwu914U8M=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=LRU7d5I0fzLpSwhlbaY+wa22hyA7qP8NLOc3MQkGCsOdTwUOyiWLdmESA4BI4Sz7G BKPnns75C5u2gErbwYbnVcmgTLS5JfyLy/G2rjTh5gGzI9xDoLmlbjc7Qw/lFm9lvA 51aErb4CIP9VNcWQseJhwzKpTX1H0rnIZYX6o0o2NtSDaOJUANPW6xMVz2PM7kcGyd zkbdTKq7tFt3WoUNCMUSTofBRUsqu8Ps3z2UKR0jch/Wxp/BOw4zxCMmJBS9T/TPmD Zeeuf3JaVmb8OxEHqVx4c6bYf0UEYfTcmwtXxZlNyXTrZMclcW9S49TJ0wv7B13FE4 /sRdPf1rDToXQ== Date: Fri, 23 Feb 2024 17:32:36 -0800 Subject: [GIT PULL 14/18] xfs: reduce refcount repair memory usage From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873806024.1891722.15628825342498983385.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 7e1b84b24d257700e417bc9cd724c1efdff653d7: xfs: hook live rmap operations during a repair operation (2024-02-22 12:43:40 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/repair-refcount-scalability-6.9_2024-02-23 for you to fetch changes up to 7fbaab57a80f1639add1c7d02adeb9d17bd50206: xfs: port refcount repair to the new refcount bag structure (2024-02-22 12:43:42 -0800) ---------------------------------------------------------------- xfs: reduce refcount repair memory usage [v29.3 14/18] The refcountbt repair code has serious memory usage problems when the block sharing factor of the filesystem is very high. This can happen if a deduplication tool has been run against the filesystem, or if the fs stores reflinked VM images that have been aging for a long time. Recall that the original reference counting algorithm walks the reverse mapping records of the filesystem to generate reference counts. For any given block in the AG, the rmap bag structure contains the all rmap records that cover that block; the refcount is the size of that bag. For online repair, the bag doesn't need the owner, offset, or state flag information, so it discards those. This halves the record size, but the bag structure still stores one excerpted record for each reverse mapping. If the sharing count is high, this will use a LOT of memory storing redundant records. In the extreme case, 100k mappings to the same piece of space will consume 100k*16 bytes = 1.6M of memory. For offline repair, the bag stores the owner values so that we know which inodes need to be marked as being reflink inodes. If a deduplication tool has been run and there are many blocks within a file pointing to the same physical space, this will stll use a lot of memory to store redundant records. The solution to this problem is to deduplicate the bag records when possible by adding a reference count to the bag record, and changing the bag add function to detect an existing record to bump the refcount. In the above example, the 100k mappings will now use 24 bytes of memory. These lookups can be done efficiently with a btree, so we create a new refcount bag btree type (inside of online repair). This is why we refactored the btree code in the previous patchset. The btree conversion also dramatically reduces the runtime of the refcount generation algorithm, because the code to delete all bag records that end at a given agblock now only has to delete one record instead of (using the example above) 100k records. As an added benefit, record deletion now gives back the unused xfile space, which it did not do previously. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (3): xfs: define an in-memory btree for storing refcount bag info during repairs xfs: create refcount bag structure for btree repairs xfs: port refcount repair to the new refcount bag structure fs/xfs/Makefile | 2 + fs/xfs/scrub/rcbag.c | 307 ++++++++++++++++++++++++++++++++++ fs/xfs/scrub/rcbag.h | 28 ++++ fs/xfs/scrub/rcbag_btree.c | 370 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/rcbag_btree.h | 81 +++++++++ fs/xfs/scrub/refcount.c | 12 ++ fs/xfs/scrub/refcount_repair.c | 164 +++++++----------- fs/xfs/scrub/repair.h | 2 + fs/xfs/xfs_stats.c | 3 +- fs/xfs/xfs_stats.h | 1 + fs/xfs/xfs_super.c | 10 +- 11 files changed, 872 insertions(+), 108 deletions(-) create mode 100644 fs/xfs/scrub/rcbag.c create mode 100644 fs/xfs/scrub/rcbag.h create mode 100644 fs/xfs/scrub/rcbag_btree.c create mode 100644 fs/xfs/scrub/rcbag_btree.h From patchwork Sat Feb 24 01:32:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570224 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92DBF4688 for ; Sat, 24 Feb 2024 01:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738372; cv=none; b=PRlnQAMwuAOoL5Yxypo2KNnXqx+5HyAsZ8cruGWTWUJHeJXGpxbsNy7nlXPq4PkG2abbCiI3sMX2B6WBpKGTkvSXzPeejwhaC8OPUD8Q6c9vKKEovNwMGC/YHe0tbd/g6ptmjRmN4K2yuk1FCdqLi3sQsFII7+kD0wyaaRpPI6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738372; c=relaxed/simple; bh=1PpPRfSyxqrcQQstLqdVE4i0UxOB93MuhJQH82WEJeo=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=R0O9uE7JqAdSrq/ZiqrGZwQUD7GfMEKIEKMhI/FCAGmFj24ufCs078utAqvTKJXscrym3PNrMUHfKY4DO0tHArbHVWEQcVbKK83YlrbyeEnGsPTJ2X1fxL0ki9SJ7uPnpBIU91VCcGksaHlPbTcXsgBTEM2QUdUXGTPUe16C3kU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GHfVR4Ap; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GHfVR4Ap" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6E002C433F1; Sat, 24 Feb 2024 01:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738372; bh=1PpPRfSyxqrcQQstLqdVE4i0UxOB93MuhJQH82WEJeo=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=GHfVR4Apbwyi/MkPl/xYKOUgb12a7as2E72rh90y82ifKSefny2+UJm/54RZTJYkE EN1fkQd4we7xYkNOWeSiDOs5qv6u/k1/MU8FXrqjqwINYUaAcSAAtuJvPvK5EFeGTB tG0tuwMrVmbDe2pjF/fvqsHzH+gdMLkTELJBvQXg93WtCLF2VjbVSR02v4CqVuiLs/ hcvKo2dbAKlImRkZGJrg3ilyQ+LvLGr9zcoemh3rdPoK2VhrEH8A6V1gXpG1K5B3Bp 9uamxNUBLML6PXvEVdZneVu9yQyyz4DNkInnvXqs6ph6WpYimUcAGN3uu8HKiyET8E dZDrMAsFDzkKw== Date: Fri, 23 Feb 2024 17:32:52 -0800 Subject: [GIT PULL 15/18] xfs: bmap log intent cleanups From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873806470.1891722.15784103742656368395.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 7fbaab57a80f1639add1c7d02adeb9d17bd50206: xfs: port refcount repair to the new refcount bag structure (2024-02-22 12:43:42 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/bmap-intent-cleanups-6.9_2024-02-23 for you to fetch changes up to c75f1a2c154979287ee12c336e2b8c3122832bf7: xfs: add a xattr_entry helper (2024-02-22 12:44:22 -0800) ---------------------------------------------------------------- xfs: bmap log intent cleanups [v29.3 15/18] The next major target of online repair are metadata that are persisted in blocks mapped by a file fork. In other words, we want to repair directories, extended attributes, symbolic links, and the realtime free space information. For file-based metadata, we assume that the space metadata is correct, which enables repair to construct new versions of the metadata in a temporary file. We then need to swap the file fork mappings of the two files atomically. With this patchset, we begin constructing such a facility based on the existing bmap log items and a new extent swap log item. This series cleans up a few parts of the file block mapping log intent code before we start adding support for realtime bmap intents. Most of it involves cleaning up tracepoints so that more of the data extraction logic ends up in the tracepoint code and not the tracepoint call site, which should reduce overhead further when tracepoints are disabled. There is also a change to pass bmap intents all the way back to the bmap code instead of unboxing the intent values and re-boxing them after the _finish_one function completes. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (7): xfs: split tracepoint classes for deferred items xfs: clean up bmap log intent item tracepoint callsites xfs: remove xfs_trans_set_bmap_flags xfs: add a bi_entry helper xfs: reuse xfs_bmap_update_cancel_item xfs: move xfs_bmap_defer_add to xfs_bmap_item.c xfs: add a xattr_entry helper fs/xfs/libxfs/xfs_bmap.c | 21 +--- fs/xfs/libxfs/xfs_bmap.h | 7 +- fs/xfs/xfs_attr_item.c | 11 +- fs/xfs/xfs_bmap_item.c | 95 +++++++++-------- fs/xfs/xfs_bmap_item.h | 4 + fs/xfs/xfs_trace.c | 1 + fs/xfs/xfs_trace.h | 267 ++++++++++++++++++++++++++++++----------------- 7 files changed, 237 insertions(+), 169 deletions(-) From patchwork Sat Feb 24 01:33:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570225 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30805625 for ; Sat, 24 Feb 2024 01:33:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738388; cv=none; b=rNMWCa9RTuVId4GGIIuLz1EHHxbnGV8plSqUvEpqwyt3r35zU9LBnKEGbwBAk2QLOp6dHnjjEkjqxrhfmFnLjbEV1znELI81AN/4TzXLlc+fNedlETHZuRgGJd/U6jhFf6DXZG5xu9rGS8q9pK16+ZlUia/itK05wQ2YHuqrEvI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738388; c=relaxed/simple; bh=VoaECzDnjEtcXqZNRB8bVxY77eJGipW/ebinzO2ZluE=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=g3UB9n4WwVKByOaDuN1R19vykvOB8C2bAahTppTuRnBduPiA73DVbpLpp+tdwmRLbASf+B4vWrq1RkJc8AOTQcjLvjhnkjbY3cUEycCsC/yL/6JT84cZ+542OMZ1JGWKjkjnm4AHMJmxWEJ4BbCrxiX5rQliq83d9bsz2KPx/2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rTMM2T/D; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rTMM2T/D" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F2B8C433F1; Sat, 24 Feb 2024 01:33:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738388; bh=VoaECzDnjEtcXqZNRB8bVxY77eJGipW/ebinzO2ZluE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=rTMM2T/D/L0KTFZCnp+hR3A5Y5X1TPU1OuGtlPsWG40hG1Q2OSdIjnkUHZhjSm7Xm KEZfY0kdYg0LUlzDI0eylATlzUK6+tI5VblKpGPrEbD0sMs87xsKhK5rRYHBw072w7 VN+xQoQmLF5SW2GfHwGawBOGEnlV+qidDxpsLaGhDgvfKY/hwiR5x74ZhQi2nTyNTC k2bWRvrR2n57twuB4b2lqE5vwprMAeegM+dwoo7+kphWJW0PGpyUfskDRCeD254qEe 5Npk3F1IxMfIVBJT4xil9m+44vkkpy1103wlmnYmudfy/yI+v175lqKKEB9FfdZ3VH 2soJNycns9H3A== Date: Fri, 23 Feb 2024 17:33:07 -0800 Subject: [GIT PULL 16/18] xfs: widen BUI formats to support realtime From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873806923.1891722.6359185979646674968.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit c75f1a2c154979287ee12c336e2b8c3122832bf7: xfs: add a xattr_entry helper (2024-02-22 12:44:22 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/realtime-bmap-intents-6.9_2024-02-23 for you to fetch changes up to 1b5453baed3a43dd4726eda0e8a5618c56a4f3f7: xfs: support recovering bmap intent items targetting realtime extents (2024-02-22 12:44:24 -0800) ---------------------------------------------------------------- xfs: widen BUI formats to support realtime [v29.3 16/18] Atomic extent swapping (and later, reverse mapping and reflink) on the realtime device needs to be able to defer file mapping and extent freeing work in much the same manner as is required on the data volume. Make the BUI log items operate on rt extents in preparation for atomic swapping and realtime rmap. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (3): xfs: fix xfs_bunmapi to allow unmapping of partial rt extents xfs: add a realtime flag to the bmap update log redo items xfs: support recovering bmap intent items targetting realtime extents fs/xfs/libxfs/xfs_bmap.c | 4 ++-- fs/xfs/libxfs/xfs_log_format.h | 4 +++- fs/xfs/xfs_bmap_item.c | 17 +++++++++++++++++ fs/xfs/xfs_trace.h | 23 ++++++++++++++++++----- 4 files changed, 40 insertions(+), 8 deletions(-) From patchwork Sat Feb 24 01:33:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570226 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CED334C69 for ; Sat, 24 Feb 2024 01:33:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738404; cv=none; b=f1ySEi0cv/SacT9j/VtEXfbOHNoWttP1ebo8dZbFdAHmNkW+lJomSOrJ34YUm69PMtFWL7VN1GB9jqEhTQXeaB7hjca83dgpx2f9p4ZB/Jbot3r/bPiSRTpX3lhpeUGKbE1aqFlzZXACluEQFHKkrQbR5VKLCNkSwUPwIWsrRSM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738404; c=relaxed/simple; bh=tLBreaeO1uRCaeA/KX1928qw5vWstCx/fifueKSiqRs=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=aZuZhIQGZrjYLnGw+a63Kt8JDGVDi01gnFWHDcBuen7EWpNb6ytuTj8GOJ6EFiV7aKcBHXigC/POeeNFaRKYdB62lT7BWtTzW4jHF0t0eoM+0Zh/uLPnlDokp94lBDIh9EIDYIllilSFHoZ++fbAE3Xzazt22vUrm4TXzzKDWI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bN8aoags; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bN8aoags" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A9C01C433F1; Sat, 24 Feb 2024 01:33:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738403; bh=tLBreaeO1uRCaeA/KX1928qw5vWstCx/fifueKSiqRs=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=bN8aoagsAztVpzVYovA/NT+35klF9mbGHkLjURXvb4gH4PQRw8yoXxk4/dq9L+NUy vzpgneYhprmCJ0wqMUXBmlJIHWYricl73OGRoMvFUKrzyoSEo9SJ/8zzvGtE0hozCO ghF2irCGSi9jr5KvZAIidtZMMzcU37z4EYCb4dumbL2Drwb/VuEIZ/MRPfcoOQopol y2BMJoV1IhfjedkCHF4ItNHbeQmsrmZ3iOW6wE/cUkM0xO6s/HCsd77UQu48gg3uBd JS6/kGBYDw2S0YueLaQQDbaRj8F8BqRntRes+jkRYEujmHdI+oVJ43/cJ8sn4JHOtY ci0zPjTM9OO7Q== Date: Fri, 23 Feb 2024 17:33:23 -0800 Subject: [GIT PULL 17/18] xfs: support attrfork and unwritten BUIs From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873807375.1891722.5277726854433420352.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 1b5453baed3a43dd4726eda0e8a5618c56a4f3f7: xfs: support recovering bmap intent items targetting realtime extents (2024-02-22 12:44:24 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/expand-bmap-intent-usage_2024-02-23 for you to fetch changes up to 6c8127e93e3ac9c2cf6a13b885dd2d057b7e7d50: xfs: xfs_bmap_finish_one should map unwritten extents properly (2024-02-22 12:45:00 -0800) ---------------------------------------------------------------- xfs: support attrfork and unwritten BUIs [v29.3 17/18] In preparation for atomic extent swapping and the online repair functionality that wants atomic extent swaps, enhance the BUI code so that we can support deferred work on the extended attribute fork and on unwritten extents. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (2): xfs: support deferred bmap updates on the attr fork xfs: xfs_bmap_finish_one should map unwritten extents properly fs/xfs/libxfs/xfs_bmap.c | 49 +++++++++++++++++++++--------------------------- fs/xfs/libxfs/xfs_bmap.h | 4 ++-- fs/xfs/xfs_bmap_util.c | 8 ++++---- fs/xfs/xfs_reflink.c | 8 ++++---- 4 files changed, 31 insertions(+), 38 deletions(-) From patchwork Sat Feb 24 01:33:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13570227 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2A523C39 for ; Sat, 24 Feb 2024 01:33:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738419; cv=none; b=dp7V2i0hIuBMAeAhR0qdaB5iVF1/eHD6wTiOhpDJe0S4fHKGwy6LVvaHMvjvkCqbzwGrsjWozsorFllS80h4DIwHH4DL167eAUDDo+KfH8BcJ3YQ6g+Fw/d9/n77eBhbUuEm11Dw2EeySylvBR8PWYlLstSyrojbgHLWpllMhWI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708738419; c=relaxed/simple; bh=+pQkiLUxY0FVuvExNJsUHMxGtJCBVN+b0kYZ7r52YqA=; h=Date:Subject:From:To:Cc:Message-ID:MIME-Version:In-Reply-To: References:Content-Type; b=WaBGscCw5vl8WVN3E10EMwPrqH3elshJ4gIg0wifoBJYtkHNyLixmvRNb5bVAKMChyBL5CRWgcYMHdc4RTlfZZ7VD2Vv7Ckzsf7hpBaT4Guk7Mo+5mDMuxtVP7d3NRU2BYIIZWoT7JiZuKrndWmSmcu/u/BZrewCGXhPO5uek+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QTrbVhSD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QTrbVhSD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3FE5DC433C7; Sat, 24 Feb 2024 01:33:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708738419; bh=+pQkiLUxY0FVuvExNJsUHMxGtJCBVN+b0kYZ7r52YqA=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=QTrbVhSDnvv7U/A8g501HNvcx3k7v3LihXf8o6hW8Eq1BW93S6z1k9+PrSuIk9ljw 0IFB4uwCnGuhjR9aYJ1uNXO0pvJj0CDiRn6CKokxPVvVnMrulg2njombnyjTKf31FU p4wq/iP2lqjvNYjVp83z5MjWa3O/lOCjRlm2KA55z6rYxG7CFf/QXs9AXJdO8xCmM8 wOWDudgtvgiWJj+foWTtxuVeuRca4yg7veA7RvS5VxS6/+FHydd0/kRpSJtNNtC58Q 5QrtfHgiufr7lqzH/sVNoGGhu/ArayiB5wpiqZenk1fr6iA9Gjy373B69I/Ui2mCoT WyhfIO3vsRTpA== Date: Fri, 23 Feb 2024 17:33:38 -0800 Subject: [GIT PULL 18/18] xfs: clean up symbolic link code From: "Darrick J. Wong" To: chandanbabu@kernel.org, djwong@kernel.org Cc: hch@lst.de, linux-xfs@vger.kernel.org Message-ID: <170873807841.1891722.16097328572481729299.stg-ugh@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <20240224010220.GN6226@frogsfrogsfrogs> References: <20240224010220.GN6226@frogsfrogsfrogs> Hi Chandan, Please pull this branch with changes for xfs for 6.9-rc1. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 6c8127e93e3ac9c2cf6a13b885dd2d057b7e7d50: xfs: xfs_bmap_finish_one should map unwritten extents properly (2024-02-22 12:45:00 -0800) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/symlink-cleanups-6.9_2024-02-23 for you to fetch changes up to b8102b61f7b8929ad8043e4574a1e26276398041: xfs: move symlink target write function to libxfs (2024-02-22 12:52:37 -0800) ---------------------------------------------------------------- xfs: clean up symbolic link code [v29.3 18/18] This series cleans up a few bits of the symbolic link code as needed for future projects. Online repair requires the ability to commit fixed fork-based filesystem metadata such as directories, xattrs, and symbolic links atomically, so we need to rearrange the symlink code before we land the atomic extent swapping. Accomplish this by moving the remote symlink target block code and declarations to xfs_symlink_remote.[ch]. This has been running on the djcloud for months with no problems. Enjoy! Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (3): xfs: move xfs_symlink_remote.c declarations to xfs_symlink_remote.h xfs: move remote symlink target read function to libxfs xfs: move symlink target write function to libxfs fs/xfs/libxfs/xfs_bmap.c | 1 + fs/xfs/libxfs/xfs_inode_fork.c | 1 + fs/xfs/libxfs/xfs_shared.h | 13 ---- fs/xfs/libxfs/xfs_symlink_remote.c | 155 ++++++++++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_symlink_remote.h | 26 +++++++ fs/xfs/scrub/inode_repair.c | 1 + fs/xfs/scrub/symlink.c | 3 +- fs/xfs/xfs_symlink.c | 146 ++-------------------------------- fs/xfs/xfs_symlink.h | 1 - 9 files changed, 192 insertions(+), 155 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_symlink_remote.h