From patchwork Thu Apr 13 00:50:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13209688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4E38C7619A for ; Thu, 13 Apr 2023 00:50:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229630AbjDMAuj (ORCPT ); Wed, 12 Apr 2023 20:50:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229451AbjDMAuh (ORCPT ); Wed, 12 Apr 2023 20:50:37 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43B9B2717 for ; Wed, 12 Apr 2023 17:50:36 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CE04663929 for ; Thu, 13 Apr 2023 00:50:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 31C5FC433EF; Thu, 13 Apr 2023 00:50:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681347035; bh=Qov28d88L36b3wt4RWARginD2Yi9ZYEmV23tmYVVj90=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EjJP8MyTjKkZfa70lc8bqULV5HTfbhN/vEgHgBcpYvXmiN9qSpqMrfOHEEbzsdnTr KmleBKBQrIT62w857QTmyrD1rjsW0FYVo1czoAwwsuEMeZ5RNIXDgoX6jvc1PA1gDS xrdFpUzLmteQZUpuvpKcLoEP+LMEdDgnSnL8rpUD+KRQVHAQepR0ECcSxgmWEaAtiT 6rs+kKmaYMvxfQCT3feF4Ma/tT7H5tcfKdNU8jLawJQzizaqsm8uM31Gs17gLZnU5b +XGBKIRpEwFGANE6fFVKs6RgPN/kXmyw91hKR7ASyUQyVyY77EiNQGbxlbH71P3eXF EHnlYpIIlG/7Q== Date: Wed, 12 Apr 2023 17:50:34 -0700 From: "Darrick J. Wong" To: david@fromorbit.com Cc: dchinner@redhat.com, linux-xfs@vger.kernel.org Subject: [GIT PULL v2 14/22] xfs: fix bugs in parent pointer checking Message-ID: <20230413005034.GS360889@frogsfrogsfrogs> References: <168127095051.417736.2174858080826643116.stg-ugh@frogsfrogsfrogs> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <168127095051.417736.2174858080826643116.stg-ugh@frogsfrogsfrogs> Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Hi Dave, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 6bb9209ceebb07fd07cec25af04eed1809c654de: xfs: always check the existence of a dirent's child inode (2023-04-11 19:00:18 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-parent-fixes-6.4_2023-04-12 for you to fetch changes up to 0916056eba4fd816f8042a3960597c316ea10256: xfs: fix parent pointer scrub racing with subdirectory reparenting (2023-04-11 19:00:20 -0700) ---------------------------------------------------------------- xfs: fix bugs in parent pointer checking [v24.5] Jan Kara pointed out that the VFS doesn't take i_rwsem of a child subdirectory that is being moved from one parent to another. Upon deeper analysis, I realized that this was the source of a very hard to trigger false corruption report in the parent pointer checking code. Now that we've refactored how directory walks work in scrub, we can also get rid of all the unnecessary and broken locking to make parent pointer scrubbing work properly. Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (3): xfs: remove xchk_parent_count_parent_dentries xfs: simplify xchk_parent_validate xfs: fix parent pointer scrub racing with subdirectory reparenting fs/xfs/scrub/common.c | 22 ----- fs/xfs/scrub/common.h | 1 - fs/xfs/scrub/parent.c | 226 +++++++++++++++++--------------------------------- 3 files changed, 76 insertions(+), 173 deletions(-) From patchwork Thu Apr 13 00:53:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13209689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85E7AC7619A for ; Thu, 13 Apr 2023 00:53:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229578AbjDMAxW (ORCPT ); Wed, 12 Apr 2023 20:53:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229484AbjDMAxV (ORCPT ); Wed, 12 Apr 2023 20:53:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1B7B2D66 for ; Wed, 12 Apr 2023 17:53:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7D9B262C28 for ; Thu, 13 Apr 2023 00:53:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DCB6FC433EF; Thu, 13 Apr 2023 00:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681347199; bh=1qdmVDFjFDJO6uUGLpjHi5WuLF0vOFO1yNGmpIt8xlE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=qZszoqL1+J9aYEEB034goWu71Em5RTUyGs6qRvriEKQWuU+4ymHS8AZJ0zzlDurAH hthYZqFNPfyEtHXdWI3K2ztc1BT2tN8t17arDMCjBoqGDy4/Dah6QmVQEecfSmka8R 1iUjk2k3QmO4jAaTK4NOuFgJQHSjJmdbFUvK4XoVW/6IHVjggz9i8praL81JqDelZZ x81RjZtAKbLtiYpVwKLAgl2AKZRN1x2YWH2wtC0RlJODucqlNjgPcspq8MQCWlc0n6 JiHlKo2wM2HgGjEKsem2b8xZz35jCLd1TBsbfHFJR6H6x4Rywwbwm2f4b8SFksEFT5 KRFisgGk2qBtA== Date: Wed, 12 Apr 2023 17:53:19 -0700 From: "Darrick J. Wong" To: david@fromorbit.com Cc: dchinner@redhat.com, linux-xfs@vger.kernel.org Subject: [GIT PULL v2 15/22] xfs: fix iget/irele usage in online fsck Message-ID: <20230413005319.GU360889@frogsfrogsfrogs> References: <168127095149.417736.13949355066335699103.stg-ugh@frogsfrogsfrogs> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <168127095149.417736.13949355066335699103.stg-ugh@frogsfrogsfrogs> Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Hi Dave, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 0916056eba4fd816f8042a3960597c316ea10256: xfs: fix parent pointer scrub racing with subdirectory reparenting (2023-04-11 19:00:20 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-iget-fixes-6.4_2023-04-12 for you to fetch changes up to 1fc7a0597d237c17b6501f8c33b76d3eaaae9079: xfs: don't take the MMAPLOCK when scrubbing file metadata (2023-04-11 19:00:22 -0700) ---------------------------------------------------------------- xfs: fix iget/irele usage in online fsck [v24.5] This patchset fixes a handful of problems relating to how we get and release incore inodes in the online scrub code. The first patch fixes how we handle DONTCACHE -- our reasons for setting (or clearing it) depend entirely on the runtime environment at irele time. Hence we can refactor iget and irele to use our own wrappers that set that context appropriately. The second patch fixes a race between the iget call in the inode core scrubber and other writer threads that are allocating or freeing inodes in the same AG by changing the behavior of xchk_iget (and the inode core scrub setup function) to return either an incore inode or the AGI buffer so that we can be sure that the inode cannot disappear on us. The final patch elides MMAPLOCK from scrub paths when possible. It did not fit anywhere else. Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (5): xfs: manage inode DONTCACHE status at irele time xfs: fix an inode lookup race in xchk_get_inode xfs: rename xchk_get_inode -> xchk_iget_for_scrubbing xfs: retain the AGI when we can't iget an inode to scrub the core xfs: don't take the MMAPLOCK when scrubbing file metadata fs/xfs/scrub/bmap.c | 9 +- fs/xfs/scrub/common.c | 306 +++++++++++++++++++++++++++++++++++++++++--------- fs/xfs/scrub/common.h | 10 +- fs/xfs/scrub/dir.c | 14 +-- fs/xfs/scrub/inode.c | 191 ++++++++++++++++++++++++++----- fs/xfs/scrub/parent.c | 13 +-- fs/xfs/scrub/scrub.c | 2 +- fs/xfs/xfs_icache.c | 3 +- fs/xfs/xfs_icache.h | 11 +- 9 files changed, 448 insertions(+), 111 deletions(-) From patchwork Thu Apr 13 00:54:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13209694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22BB6C7619A for ; Thu, 13 Apr 2023 00:54:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229599AbjDMAyV (ORCPT ); Wed, 12 Apr 2023 20:54:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229484AbjDMAyU (ORCPT ); Wed, 12 Apr 2023 20:54:20 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA24E2717 for ; Wed, 12 Apr 2023 17:54:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6542B62C28 for ; Thu, 13 Apr 2023 00:54:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE5E9C433D2; Thu, 13 Apr 2023 00:54:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681347258; bh=5ii9kNQkz3TAYMU9RaC3P44U35QpT02ilRul3PInDfM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=lGImrND4HRbWug17Xp9Z9sUZHCpA7rcO1vVzkVBFP8YN6hd/wRpGSqPrOki/hps7Y TIBU+f9kSbvxTcvg/QWI+znvGwY9Qp7RoONQreZivpQOzLDKrQqjF+gSH35uCXMhF3 F/9RP2VHLin72aJNODyvbI3PsTrG4QpKYp/aUqoDVNKPNXgDSAil31yrzstyGJfA+k f6FZPmwiEp+E1w6/YQnbJJchz77GP61I9lSPMgzsI5z1WXAHuLydUqlx3eta3oUjZv evPD2JWT25kfEFS7aeq5BlailvkmLn6Dc+SPxn/EtvLZl2Z6cmcInW86JIaj16Giak WEivmdTnbc5qQ== Date: Wed, 12 Apr 2023 17:54:18 -0700 From: "Darrick J. Wong" To: david@fromorbit.com Cc: dchinner@redhat.com, linux-xfs@vger.kernel.org Subject: [GIT PULL v2 16/22] xfs: merge bmap records for faster scrubs Message-ID: <20230413005418.GV360889@frogsfrogsfrogs> References: <168127095245.417736.9350032118598729884.stg-ugh@frogsfrogsfrogs> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <168127095245.417736.9350032118598729884.stg-ugh@frogsfrogsfrogs> Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Hi Dave, Please pull this branch with changes for xfs. As usual, I did a test-merge with the main upstream branch as of a few minutes ago, and didn't see any conflicts. Please let me know if you encounter any problems. --D The following changes since commit 1fc7a0597d237c17b6501f8c33b76d3eaaae9079: xfs: don't take the MMAPLOCK when scrubbing file metadata (2023-04-11 19:00:22 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-merge-bmap-records-6.4_2023-04-12 for you to fetch changes up to 1e59fdb7d6157ff685a250e0873a015a2b16a4f2: xfs: don't call xchk_bmap_check_rmaps for btree-format file forks (2023-04-11 19:00:26 -0700) ---------------------------------------------------------------- xfs: merge bmap records for faster scrubs [v24.5] I started looking into performance problems with the data fork scrubber in generic/333, and noticed a few things that needed improving. First, due to design reasons, it's possible for file forks btrees to contain multiple contiguous mappings to the same physical space. Instead of checking each ondisk mapping individually, it's much faster to combine them when possible and check the combined mapping because that's fewer trips through the rmap btree, and we can drop this check-around behavior that it does when an rmapbt lookup produces a record that starts before or ends after a particular bmbt mapping. Second, I noticed that the bmbt scrubber decides to walk every reverse mapping in the filesystem if the file fork is in btree format. This is very costly, and only necessary if the inode repair code had to zap a fork to convince iget to work. Constraining the full-rmap scan to this one case means we can skip it for normal files, which drives the runtime of this test from 8 hours down to 45 minutes (observed with realtime reflink and rebuild-all mode.) Signed-off-by: Darrick J. Wong ---------------------------------------------------------------- Darrick J. Wong (6): xfs: change bmap scrubber to store the previous mapping xfs: accumulate iextent records when checking bmap xfs: split xchk_bmap_xref_rmap into two functions xfs: alert the user about data/attr fork mappings that could be merged xfs: split the xchk_bmap_check_rmaps into a predicate xfs: don't call xchk_bmap_check_rmaps for btree-format file forks fs/xfs/libxfs/xfs_bmap.h | 2 +- fs/xfs/scrub/bmap.c | 379 ++++++++++++++++++++++++++++++----------------- 2 files changed, 242 insertions(+), 139 deletions(-)