From patchwork Thu Feb 16 21:08:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0084BC61DA4 for ; Thu, 16 Feb 2023 21:08:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230139AbjBPVIr (ORCPT ); Thu, 16 Feb 2023 16:08:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230135AbjBPVIo (ORCPT ); Thu, 16 Feb 2023 16:08:44 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFD0A3B642 for ; Thu, 16 Feb 2023 13:08:39 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3EBCA60C6D for ; Thu, 16 Feb 2023 21:08:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2113C4339B; Thu, 16 Feb 2023 21:08:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581718; bh=ALM0awXkokXMqNm2UNUZBeDFYcg4oup1/0xW8NIdjRQ=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ckSZwJlmpLKJqzWt+e/nIkWe1jfDLNlc0yCVGrwUen+YvQJQDxfJKQuUcUFuQUHzF jVC5+FBxJWwfm1kDqyZNx8bWT0YJQJA+zAsLdjTeA0adP1ot/j602BZC+q4il9GpgE V94aRCONnOFOhh82eedjcTHU8guJqN4qxs+XgHyS1hRmbQXCOx8Z0ck01LWxPuzXTz DGJ7K/HYHnYCmPtCSPiKh5z3rIVH7nONLh0SNpzyz/8oYNuklBkLaxXwImtIzhKCbW iYeW7TfNChVy6Ok89ArOd2eqlQIO2QDFhC3TL3cRMvyS2tDTGJz1w9ANI1oHFGNQyM 3IMFXeCS+QwNQ== Date: Thu, 16 Feb 2023 13:08:38 -0800 Subject: [PATCH 1/8] xfs_repair: build a parent pointer index From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657881978.3477807.7257312710668307765.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong When we're walking directories during phase 6, build an index of parent pointers that we expect to find. Signed-off-by: Darrick J. Wong --- repair/Makefile | 2 repair/phase6.c | 55 +++++++++++-- repair/pptr.c | 242 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ repair/pptr.h | 15 +++ 4 files changed, 307 insertions(+), 7 deletions(-) create mode 100644 repair/pptr.c create mode 100644 repair/pptr.h diff --git a/repair/Makefile b/repair/Makefile index 2c40e59a..18731613 100644 --- a/repair/Makefile +++ b/repair/Makefile @@ -23,6 +23,7 @@ HFILES = \ err_protos.h \ globals.h \ incore.h \ + pptr.h \ prefetch.h \ progress.h \ protos.h \ @@ -59,6 +60,7 @@ CFILES = \ phase5.c \ phase6.c \ phase7.c \ + pptr.c \ prefetch.c \ progress.c \ quotacheck.c \ diff --git a/repair/phase6.c b/repair/phase6.c index 0d253701..48ec236d 100644 --- a/repair/phase6.c +++ b/repair/phase6.c @@ -18,6 +18,7 @@ #include "dinode.h" #include "progress.h" #include "versions.h" +#include "repair/pptr.h" static struct cred zerocr; static struct fsxattr zerofsx; @@ -67,6 +68,7 @@ struct dir_hash_ent { struct dir_hash_ent *nextbyorder; /* next in order added */ xfs_dahash_t hashval; /* hash value of name */ uint32_t address; /* offset of data entry */ + uint32_t new_address; /* new address, if we rebuild */ xfs_ino_t inum; /* inode num of entry */ short junkit; /* name starts with / */ short seen; /* have seen leaf entry */ @@ -224,6 +226,7 @@ dir_hash_add( p->address = addr; p->inum = inum; p->seen = 0; + p->new_address = addr; /* Set up the name in the region trailing the hash entry. */ memcpy(p->namebuf, name, namelen); @@ -885,6 +888,7 @@ mk_orphanage(xfs_mount_t *mp) int error; const int mode = 0755; int nres; + xfs_dir2_dataptr_t diroffset; struct xfs_name xname; /* @@ -969,11 +973,13 @@ mk_orphanage(xfs_mount_t *mp) /* * create the actual entry */ - error = -libxfs_dir_createname(tp, pip, &xname, ip->i_ino, nres, NULL); + error = -libxfs_dir_createname(tp, pip, &xname, ip->i_ino, nres, + &diroffset); if (error) do_error( _("can't make %s, createname error %d\n"), ORPHANAGE, error); + add_parent_ptr(ip->i_ino, ORPHANAGE, diroffset, pip); /* * bump up the link count in the root directory to account @@ -1018,6 +1024,7 @@ mv_orphanage( int nres; int incr; ino_tree_node_t *irec; + xfs_dir2_dataptr_t diroffset; int ino_offset = 0; struct xfs_name xname; @@ -1066,7 +1073,7 @@ mv_orphanage( libxfs_trans_ijoin(tp, ino_p, 0); err = -libxfs_dir_createname(tp, orphanage_ip, &xname, - ino, nres, NULL); + ino, nres, &diroffset); if (err) do_error( _("name create failed in %s (%d)\n"), ORPHANAGE, err); @@ -1100,7 +1107,7 @@ mv_orphanage( err = -libxfs_dir_createname(tp, orphanage_ip, &xname, - ino, nres, NULL); + ino, nres, &diroffset); if (err) do_error( _("name create failed in %s (%d)\n"), ORPHANAGE, err); @@ -1147,7 +1154,7 @@ mv_orphanage( libxfs_trans_ijoin(tp, ino_p, 0); err = -libxfs_dir_createname(tp, orphanage_ip, &xname, ino, - nres, NULL); + nres, &diroffset); if (err) do_error( _("name create failed in %s (%d)\n"), ORPHANAGE, err); @@ -1160,6 +1167,11 @@ mv_orphanage( do_error( _("orphanage name create failed (%d)\n"), err); } + + if (xfs_has_parent(mp)) + add_parent_ptr(ino_p->i_ino, xname.name, diroffset, + orphanage_ip); + libxfs_irele(ino_p); libxfs_irele(orphanage_ip); } @@ -1330,7 +1342,7 @@ longform_dir2_rebuild( libxfs_trans_ijoin(tp, ip, 0); error = -libxfs_dir_createname(tp, ip, &p->name, p->inum, - nres, NULL); + nres, &p->new_address); if (error) { do_warn( _("name create failed in ino %" PRIu64 " (%d)\n"), ino, error); @@ -2459,6 +2471,7 @@ shortform_dir2_entry_check( struct xfs_dir2_sf_entry *next_sfep; struct xfs_ifork *ifp; struct ino_tree_node *irec; + xfs_dir2_dataptr_t diroffset; int max_size; int ino_offset; int i; @@ -2637,8 +2650,9 @@ shortform_dir2_entry_check( /* * check for duplicate names in directory. */ - if (!dir_hash_add(mp, hashtab, (xfs_dir2_dataptr_t) - (sfep - xfs_dir2_sf_firstentry(sfp)), + diroffset = xfs_dir2_byte_to_dataptr( + xfs_dir2_sf_get_offset(sfep)); + if (!dir_hash_add(mp, hashtab, diroffset, lino, sfep->namelen, sfep->name, libxfs_dir2_sf_get_ftype(mp, sfep))) { do_warn( @@ -2672,6 +2686,7 @@ _("entry \"%s\" (ino %" PRIu64 ") in dir %" PRIu64 " is a duplicate name"), next_sfep = shortform_dir2_junk(mp, sfp, sfep, lino, &max_size, &i, &bytes_deleted, ino_dirty); + dir_hash_junkit(hashtab, diroffset); continue; } else if (parent == ino) { add_inode_reached(irec, ino_offset); @@ -2696,6 +2711,7 @@ _("entry \"%s\" (ino %" PRIu64 ") in dir %" PRIu64 " is a duplicate name"), next_sfep = shortform_dir2_junk(mp, sfp, sfep, lino, &max_size, &i, &bytes_deleted, ino_dirty); + dir_hash_junkit(hashtab, diroffset); continue; } } @@ -2787,6 +2803,26 @@ _("entry \"%s\" (ino %" PRIu64 ") in dir %" PRIu64 " is a duplicate name"), } } +static void +dir_hash_add_parent_ptrs( + struct xfs_inode *dp, + struct dir_hash_tab *hashtab) +{ + struct dir_hash_ent *p; + + if (!xfs_has_parent(dp->i_mount)) + return; + + for (p = hashtab->first; p; p = p->nextbyorder) { + if (p->name.name[0] == '/' || (p->name.name[0] == '.' && + (p->name.len == 1 || (p->name.len == 2 && + p->name.name[1] == '.')))) + continue; + + add_parent_ptr(p->inum, p->name.name, p->new_address, dp); + } +} + /* * processes all reachable inodes in directories */ @@ -2913,6 +2949,7 @@ _("error %d fixing shortform directory %llu\n"), default: break; } + dir_hash_add_parent_ptrs(ip, hashtab); dir_hash_done(hashtab); /* @@ -3204,6 +3241,8 @@ phase6(xfs_mount_t *mp) ino_tree_node_t *irec; int i; + parent_ptr_init(mp); + memset(&zerocr, 0, sizeof(struct cred)); memset(&zerofsx, 0, sizeof(struct fsxattr)); orphanage_ino = 0; @@ -3304,4 +3343,6 @@ _(" - resetting contents of realtime bitmap and summary inodes\n")); irec = next_ino_rec(irec); } } + + parent_ptr_free(mp); } diff --git a/repair/pptr.c b/repair/pptr.c new file mode 100644 index 00000000..b10c7f41 --- /dev/null +++ b/repair/pptr.c @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2023 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#include "libxfs.h" +#include "libxfs/xfile.h" +#include "libxfs/xfblob.h" +#include "repair/err_protos.h" +#include "repair/slab.h" +#include "repair/pptr.h" + +#undef PPTR_DEBUG + +#ifdef PPTR_DEBUG +# define dbg_printf(f, a...) do {printf(f, ## a); fflush(stdout); } while (0) +#else +# define dbg_printf(f, a...) +#endif + +/* + * Parent Pointer Validation + * ========================= + * + * Phase 6 validates the connectivity of the directory tree after validating + * that all the space metadata are correct, and confirming all the inodes that + * we intend to keep. The first part of phase 6 walks the directories of the + * filesystem to ensure that every file that isn't the root directory has a + * parent. Unconnected files are attached to the orphanage. Filesystems with + * the directory parent pointer feature enabled must also ensure that for every + * directory entry that points to a child file, that child has a matching + * parent pointer. + * + * There are many ways that we could check the parent pointers, but the means + * that we have chosen is to build a per-AG master index of all parent pointers + * of all inodes stored in that AG, and use that as the basis for comparison. + * This consumes a lot of memory, but performing both a forward scan to check + * dirent -> parent pointer and a backwards scan of parent pointer -> dirent + * takes longer than the simple method presented here. Userspace adds the + * additional twist that inodes are not cached (and there are no ILOCKs), which + * makes that approach even less attractive. + * + * During the directory walk at the start of phase 6, we transform each child + * directory entry found into its parent pointer equivalent. In other words, + * the forward information: + * + * (dir_ino, dir_offset, name, child_ino) + * + * becomes this backwards information: + * + * (*child_agino, *dir_ino, dir_gen, *dir_offset, name) + * + * Key fields are starred. + * + * This tuple is recorded in the per-AG master parent pointer index. Note + * that names are stored separately in an xfblob data structure so that the + * rest of the information can be sorted and processed as fixed-size records. + * + * Once we've finished with the forward scan, we get to work on the backwards + * scan. Each AG is processed independently. First, we sort the per-AG master + * records in order of child_agino, dir_ino, and dir_offset. Each inode in the + * AG is then processed in numerical order. + * + * The first thing that happens to the file is that we read all the extended + * attributes to look for parent pointers. Attributes that claim to be parent + * pointers but are obviously garbage are thrown away. The rest of the parent + * pointers for that file are recorded in memory like this: + * + * (*dir_ino, dir_gen, *dir_offset, name) + * + * When we've concluded the xattr scan, these records are sorted in order of + * dir_ino and dir_offset. The master index cursor should point at the first + * record for the file that we're scanning, if everything is consistent. + * + * If not, there are two possibilities: + * + * A. The master index cursor points to a higher inode number than the one we + * are scanning. The file has apparently lost all parents, so all parent + * pointers (if any) must be deleted. This should only happen to metadata + * inodes. + * + * B. The cursor instead points to a lower inode number than the one we are + * scanning. This means that there exists a directory entry pointing at an + * inode that is free. We supposedly already settled which inodes are free + * and which aren't, which means in-memory information is inconsistent. Abort. + * + * Otherwise, we are ready to check the file parent pointers against the + * master. If the ondisk directory metadata are all consistent, this recordset + * should correspond exactly to the subset of the master records with a + * child_agino matching the file that we're scanning. We should be able to + * walk both sets in lockstep, and find one of the following outcomes: + * + * 1) The master index cursor is ahead of the ondisk index cursor. This means + * that the inode has parent pointers that were not found during the dirent + * scan. These should be deleted. + * + * 2) The ondisk index gets ahead of the master index. This means that the + * dirent scan found parent pointers that are not attached to the inode. + * These should be added. + * + * 3) The parent_gen or (dirent) name are not consistent. Update the parent + * pointer to the values that we found during the dirent scan. + * + * 4) Everything matches. Move on to the next parent pointer. + * + * The current implementation does not try to rebuild directories from parent + * pointer information, as this requires a lengthy scan of the filesystem for + * each broken directory. + */ + +struct ag_pptr { + /* parent directory handle */ + xfs_ino_t parent_ino; + unsigned int parent_gen; + + /* dirent offset */ + xfs_dir2_dataptr_t diroffset; + + /* dirent name length */ + unsigned int namelen; + + /* cookie for the actual dirent name */ + xfblob_cookie name_cookie; + + /* agino of the child file */ + xfs_agino_t child_agino; +}; + +struct ag_pptrs { + /* Lock to protect pptr_recs during the dirent scan. */ + pthread_mutex_t lock; + + /* Parent pointer records for files in this AG. */ + struct xfs_slab *pptr_recs; +}; + +/* Global names storage file. */ +static struct xfblob *names; +static pthread_mutex_t names_mutex = PTHREAD_MUTEX_INITIALIZER; +static struct ag_pptrs *fs_pptrs; + +void +parent_ptr_free( + struct xfs_mount *mp) +{ + xfs_agnumber_t agno; + + if (!xfs_has_parent(mp)) + return; + + for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) { + free_slab(&fs_pptrs[agno].pptr_recs); + pthread_mutex_destroy(&fs_pptrs[agno].lock); + } + free(fs_pptrs); + fs_pptrs = NULL; + + xfblob_destroy(names); +} + +void +parent_ptr_init( + struct xfs_mount *mp) +{ + xfs_agnumber_t agno; + int error; + + if (!xfs_has_parent(mp)) + return; + + error = -xfblob_create(mp, "parent pointer names", &names); + if (error) + do_error(_("init parent pointer names failed: %s\n"), + strerror(error)); + + fs_pptrs = calloc(mp->m_sb.sb_agcount, sizeof(struct ag_pptrs)); + if (!fs_pptrs) + do_error( + _("init parent pointer per-AG record array failed: %s\n"), + strerror(errno)); + + for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) { + error = pthread_mutex_init(&fs_pptrs[agno].lock, NULL); + if (error) + do_error( + _("init agno %u parent pointer lock failed: %s\n"), + agno, strerror(error)); + + error = -init_slab(&fs_pptrs[agno].pptr_recs, + sizeof(struct ag_pptr)); + if (error) + do_error( + _("init agno %u parent pointer recs failed: %s\n"), + agno, strerror(error)); + } +} + +/* Remember that @dp has a dirent (@fname, @ino) at @diroffset. */ +void +add_parent_ptr( + xfs_ino_t ino, + const unsigned char *fname, + xfs_dir2_dataptr_t diroffset, + struct xfs_inode *dp) +{ + struct xfs_mount *mp = dp->i_mount; + struct ag_pptr ag_pptr = { + .child_agino = XFS_INO_TO_AGINO(mp, ino), + .parent_ino = dp->i_ino, + .parent_gen = VFS_I(dp)->i_generation, + .diroffset = diroffset, + .namelen = strlen(fname), + }; + struct ag_pptrs *ag_pptrs; + xfs_agnumber_t agno = XFS_INO_TO_AGNO(mp, ino); + int error; + + if (!xfs_has_parent(mp)) + return; + + pthread_mutex_lock(&names_mutex); + error = -xfblob_store(names, &ag_pptr.name_cookie, fname, + ag_pptr.namelen); + pthread_mutex_unlock(&names_mutex); + if (error) + do_error(_("storing name '%s' failed: %s\n"), + fname, strerror(error)); + + ag_pptrs = &fs_pptrs[agno]; + pthread_mutex_lock(&ag_pptrs->lock); + error = -slab_add(ag_pptrs->pptr_recs, &ag_pptr); + pthread_mutex_unlock(&ag_pptrs->lock); + if (error) + do_error(_("storing name '%s' key failed: %s\n"), + fname, strerror(error)); + + dbg_printf( + _("%s: dp %llu fname '%s' diroffset %u ino %llu cookie 0x%llx\n"), + __func__, (unsigned long long)dp->i_ino, fname, + diroffset, (unsigned long long)ino, + (unsigned long long)ag_pptr.name_cookie); +} diff --git a/repair/pptr.h b/repair/pptr.h new file mode 100644 index 00000000..2c632ec9 --- /dev/null +++ b/repair/pptr.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2023 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#ifndef __REPAIR_PPTR_H__ +#define __REPAIR_PPTR_H__ + +void parent_ptr_free(struct xfs_mount *mp); +void parent_ptr_init(struct xfs_mount *mp); + +void add_parent_ptr(xfs_ino_t ino, const unsigned char *fname, + xfs_dir2_dataptr_t diroffset, struct xfs_inode *dp); + +#endif /* __REPAIR_PPTR_H__ */ From patchwork Thu Feb 16 21:08:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E5B3C636CC for ; Thu, 16 Feb 2023 21:09:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230137AbjBPVJE (ORCPT ); Thu, 16 Feb 2023 16:09:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230144AbjBPVJB (ORCPT ); Thu, 16 Feb 2023 16:09:01 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4720553800 for ; Thu, 16 Feb 2023 13:08:55 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C9B4860A65 for ; Thu, 16 Feb 2023 21:08:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F69DC433EF; Thu, 16 Feb 2023 21:08:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581734; bh=GxDTcqMcEpKAuLVUx9V6e96dnIo9tcX12+3HArJlnV0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=btpzlOgWbYwj0pD43bFof9TSUdgqvy+B5RKSSOzdMOqXA9e/6gocnDe5SaUqO2VYZ lzjSfLSYGu2n8Yw3EDILse6ILgzm6OsKe4MfWhXCiCrx27sNcWzPkGhBG1T+mGhtX5 Hg1mtPHVlc3bPA4hWX69B+myfZGfFgAGXH2LKXx3EmuXXryQarUGp8ijsGCAgJATew 0oRWu+/McbPNCgByg+Vhz4IwGIziwmmKoOVH5Uobqe5oT8pWAcepxEFsY/qaqliCFS w9WgD7PKW8WlLQRk+NMPozUKZWLratIX7+vzSlaE3b4GO8KQ0AFgUNqnB5RzPaXFaL 33XoMLSQ5pnoQ== Date: Thu, 16 Feb 2023 13:08:53 -0800 Subject: [PATCH 2/8] xfs_repair: check parent pointers From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657881991.3477807.18417077389314514199.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Use the parent pointer index that we constructed in the previous patch to check that each file's parent pointer records exactly match the directory entries that we recorded while walking directory entries. Signed-off-by: Darrick J. Wong --- libxfs/libxfs_api_defs.h | 5 libxfs/xfblob.c | 9 + libxfs/xfblob.h | 2 repair/Makefile | 2 repair/listxattr.c | 283 +++++++++++++++++++++ repair/listxattr.h | 15 + repair/phase6.c | 2 repair/pptr.c | 618 ++++++++++++++++++++++++++++++++++++++++++++++ repair/pptr.h | 2 9 files changed, 938 insertions(+) create mode 100644 repair/listxattr.c create mode 100644 repair/listxattr.h diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h index b8ee0247..92cdb6cc 100644 --- a/libxfs/libxfs_api_defs.h +++ b/libxfs/libxfs_api_defs.h @@ -34,7 +34,9 @@ #define xfs_alloc_vextent libxfs_alloc_vextent #define xfs_attr3_leaf_hdr_from_disk libxfs_attr3_leaf_hdr_from_disk +#define xfs_attr3_leaf_read libxfs_attr3_leaf_read #define xfs_attr_get libxfs_attr_get +#define xfs_attr_is_leaf libxfs_attr_is_leaf #define xfs_attr_leaf_newentsize libxfs_attr_leaf_newentsize #define xfs_attr_namecheck libxfs_attr_namecheck #define xfs_attr_set libxfs_attr_set @@ -63,6 +65,7 @@ #define xfs_bwrite libxfs_bwrite #define xfs_calc_dquots_per_chunk libxfs_calc_dquots_per_chunk #define xfs_da3_node_hdr_from_disk libxfs_da3_node_hdr_from_disk +#define xfs_da3_node_read libxfs_da3_node_read #define xfs_da_get_buf libxfs_da_get_buf #define xfs_da_hashname libxfs_da_hashname #define xfs_da_read_buf libxfs_da_read_buf @@ -130,6 +133,7 @@ #define xfs_inobt_stage_cursor libxfs_inobt_stage_cursor #define xfs_inode_from_disk libxfs_inode_from_disk #define xfs_inode_from_disk_ts libxfs_inode_from_disk_ts +#define xfs_inode_hasattr libxfs_inode_hasattr #define xfs_inode_to_disk libxfs_inode_to_disk #define xfs_inode_validate_cowextsize libxfs_inode_validate_cowextsize #define xfs_inode_validate_extsize libxfs_inode_validate_extsize @@ -142,6 +146,7 @@ #define xfs_mode_to_ftype libxfs_mode_to_ftype #define xfs_parent_add libxfs_parent_add #define xfs_parent_finish libxfs_parent_finish +#define xfs_parent_irec_from_disk libxfs_parent_irec_from_disk #define xfs_parent_start libxfs_parent_start #define xfs_perag_get libxfs_perag_get #define xfs_perag_put libxfs_perag_put diff --git a/libxfs/xfblob.c b/libxfs/xfblob.c index 6c1c8e6f..2c6e69a2 100644 --- a/libxfs/xfblob.c +++ b/libxfs/xfblob.c @@ -146,3 +146,12 @@ xfblob_free( xfile_discard(blob->xfile, cookie, sizeof(key) + key.xb_size); return 0; } + +/* Drop all the blobs. */ +void +xfblob_truncate( + struct xfblob *blob) +{ + xfile_discard(blob->xfile, PAGE_SIZE, blob->last_offset - PAGE_SIZE); + blob->last_offset = PAGE_SIZE; +} diff --git a/libxfs/xfblob.h b/libxfs/xfblob.h index d1282810..0d6de8ce 100644 --- a/libxfs/xfblob.h +++ b/libxfs/xfblob.h @@ -22,4 +22,6 @@ int xfblob_store(struct xfblob *blob, xfblob_cookie *cookie, const void *ptr, uint32_t size); int xfblob_free(struct xfblob *blob, xfblob_cookie cookie); +void xfblob_truncate(struct xfblob *blob); + #endif /* __XFS_SCRUB_XFBLOB_H__ */ diff --git a/repair/Makefile b/repair/Makefile index 18731613..925864c2 100644 --- a/repair/Makefile +++ b/repair/Makefile @@ -23,6 +23,7 @@ HFILES = \ err_protos.h \ globals.h \ incore.h \ + listxattr.h \ pptr.h \ prefetch.h \ progress.h \ @@ -53,6 +54,7 @@ CFILES = \ incore_ext.c \ incore_ino.c \ init.c \ + listxattr.c \ phase1.c \ phase2.c \ phase3.c \ diff --git a/repair/listxattr.c b/repair/listxattr.c new file mode 100644 index 00000000..484f9a00 --- /dev/null +++ b/repair/listxattr.c @@ -0,0 +1,283 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#include "libxfs.h" +#include "libxlog.h" +#include "libfrog/bitmap.h" +#include "repair/listxattr.h" + +/* Call a function for every entry in a shortform xattr structure. */ +STATIC int +xattr_walk_sf( + struct xfs_inode *ip, + xattr_walk_fn attr_fn, + void *priv) +{ + struct xfs_attr_shortform *sf; + struct xfs_attr_sf_entry *sfe; + unsigned int i; + int error; + + sf = (struct xfs_attr_shortform *)ip->i_af.if_u1.if_data; + for (i = 0, sfe = &sf->list[0]; i < sf->hdr.count; i++) { + error = attr_fn(ip, sfe->flags, sfe->nameval, sfe->namelen, + &sfe->nameval[sfe->namelen], sfe->valuelen, + priv); + if (error) + return error; + + sfe = xfs_attr_sf_nextentry(sfe); + } + + return 0; +} + +/* Call a function for every entry in this xattr leaf block. */ +STATIC int +xattr_walk_leaf_entries( + struct xfs_inode *ip, + xattr_walk_fn attr_fn, + struct xfs_buf *bp, + void *priv) +{ + struct xfs_attr3_icleaf_hdr ichdr; + struct xfs_mount *mp = ip->i_mount; + struct xfs_attr_leafblock *leaf = bp->b_addr; + struct xfs_attr_leaf_entry *entry; + unsigned int i; + int error; + + libxfs_attr3_leaf_hdr_from_disk(mp->m_attr_geo, &ichdr, leaf); + entry = xfs_attr3_leaf_entryp(leaf); + + for (i = 0; i < ichdr.count; entry++, i++) { + void *value; + char *name; + unsigned int namelen, valuelen; + + if (entry->flags & XFS_ATTR_LOCAL) { + struct xfs_attr_leaf_name_local *name_loc; + + name_loc = xfs_attr3_leaf_name_local(leaf, i); + name = name_loc->nameval; + namelen = name_loc->namelen; + value = &name_loc->nameval[name_loc->namelen]; + valuelen = be16_to_cpu(name_loc->valuelen); + } else { + struct xfs_attr_leaf_name_remote *name_rmt; + + name_rmt = xfs_attr3_leaf_name_remote(leaf, i); + name = name_rmt->name; + namelen = name_rmt->namelen; + value = NULL; + valuelen = be32_to_cpu(name_rmt->valuelen); + } + + error = attr_fn(ip, entry->flags, name, namelen, value, + valuelen, priv); + if (error) + return error; + + } + + return 0; +} + +/* + * Call a function for every entry in a leaf-format xattr structure. Avoid + * memory allocations for the loop detector since there's only one block. + */ +STATIC int +xattr_walk_leaf( + struct xfs_inode *ip, + xattr_walk_fn attr_fn, + void *priv) +{ + struct xfs_buf *leaf_bp; + int error; + + error = -libxfs_attr3_leaf_read(NULL, ip, 0, &leaf_bp); + if (error) + return error; + + error = xattr_walk_leaf_entries(ip, attr_fn, leaf_bp, priv); + libxfs_trans_brelse(NULL, leaf_bp); + return error; +} + +/* Find the leftmost leaf in the xattr dabtree. */ +STATIC int +xattr_walk_find_leftmost_leaf( + struct xfs_inode *ip, + struct bitmap *seen_blocks, + struct xfs_buf **leaf_bpp) +{ + struct xfs_da3_icnode_hdr nodehdr; + struct xfs_mount *mp = ip->i_mount; + struct xfs_da_intnode *node; + struct xfs_da_node_entry *btree; + struct xfs_buf *bp; + //xfs_failaddr_t fa; + xfs_dablk_t blkno = 0; + unsigned int expected_level = 0; + int error; + + for (;;) { + uint16_t magic; + + error = -libxfs_da3_node_read(NULL, ip, blkno, &bp, + XFS_ATTR_FORK); + if (error) + return error; + + node = bp->b_addr; + magic = be16_to_cpu(node->hdr.info.magic); + if (magic == XFS_ATTR_LEAF_MAGIC || + magic == XFS_ATTR3_LEAF_MAGIC) + break; + + error = EFSCORRUPTED; + if (magic != XFS_DA_NODE_MAGIC && + magic != XFS_DA3_NODE_MAGIC) + goto out_buf; + +#if 0 + fa = xfs_da3_node_header_check(bp, ip->i_ino); + if (fa) + goto out_buf; +#endif + + libxfs_da3_node_hdr_from_disk(mp, &nodehdr, node); + + if (nodehdr.count == 0 || nodehdr.level >= XFS_DA_NODE_MAXDEPTH) + goto out_buf; + + /* Check the level from the root node. */ + if (blkno == 0) + expected_level = nodehdr.level - 1; + else if (expected_level != nodehdr.level) + goto out_buf; + else + expected_level--; + + /* Remember that we've seen this node. */ + error = -bitmap_set(seen_blocks, blkno, 1); + if (error) + goto out_buf; + + /* Find the next level towards the leaves of the dabtree. */ + btree = nodehdr.btree; + blkno = be32_to_cpu(btree->before); + libxfs_trans_brelse(NULL, bp); + + /* Make sure we haven't seen this new block already. */ + if (bitmap_test(seen_blocks, blkno, 1)) + return EFSCORRUPTED; + } + + error = EFSCORRUPTED; +#if 0 + fa = xfs_attr3_leaf_header_check(bp, ip->i_ino); + if (fa) + goto out_buf; +#endif + + if (expected_level != 0) + goto out_buf; + + /* Remember that we've seen this leaf. */ + error = -bitmap_set(seen_blocks, blkno, 1); + if (error) + goto out_buf; + + *leaf_bpp = bp; + return 0; + +out_buf: + libxfs_trans_brelse(NULL, bp); + return error; +} + +/* Call a function for every entry in a node-format xattr structure. */ +STATIC int +xattr_walk_node( + struct xfs_inode *ip, + xattr_walk_fn attr_fn, + void *priv) +{ + struct xfs_attr3_icleaf_hdr leafhdr; + struct bitmap *seen_blocks; + struct xfs_mount *mp = ip->i_mount; + struct xfs_attr_leafblock *leaf; + struct xfs_buf *leaf_bp; + int error; + + bitmap_alloc(&seen_blocks); + + error = xattr_walk_find_leftmost_leaf(ip, seen_blocks, &leaf_bp); + if (error) + goto out_bitmap; + + for (;;) { + error = xattr_walk_leaf_entries(ip, attr_fn, leaf_bp, + priv); + if (error) + goto out_leaf; + + /* Find the right sibling of this leaf block. */ + leaf = leaf_bp->b_addr; + libxfs_attr3_leaf_hdr_from_disk(mp->m_attr_geo, &leafhdr, leaf); + if (leafhdr.forw == 0) + goto out_leaf; + + libxfs_trans_brelse(NULL, leaf_bp); + + /* Make sure we haven't seen this new leaf already. */ + if (bitmap_test(seen_blocks, leafhdr.forw, 1)) + goto out_bitmap; + + error = -libxfs_attr3_leaf_read(NULL, ip, leafhdr.forw, + &leaf_bp); + if (error) + goto out_bitmap; + + /* Remember that we've seen this new leaf. */ + error = -bitmap_set(seen_blocks, leafhdr.forw, 1); + if (error) + goto out_leaf; + } + +out_leaf: + libxfs_trans_brelse(NULL, leaf_bp); +out_bitmap: + bitmap_free(&seen_blocks); + return error; +} + +/* Call a function for every extended attribute in a file. */ +int +xattr_walk( + struct xfs_inode *ip, + xattr_walk_fn attr_fn, + void *priv) +{ + int error; + + if (!libxfs_inode_hasattr(ip)) + return 0; + + if (ip->i_af.if_format == XFS_DINODE_FMT_LOCAL) + return xattr_walk_sf(ip, attr_fn, priv); + + /* attr functions require that the attr fork is loaded */ + error = -libxfs_iread_extents(NULL, ip, XFS_ATTR_FORK); + if (error) + return error; + + if (libxfs_attr_is_leaf(ip)) + return xattr_walk_leaf(ip, attr_fn, priv); + + return xattr_walk_node(ip, attr_fn, priv); +} diff --git a/repair/listxattr.h b/repair/listxattr.h new file mode 100644 index 00000000..cd18fdd2 --- /dev/null +++ b/repair/listxattr.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#ifndef __REPAIR_LISTXATTR_H__ +#define __REPAIR_LISTXATTR_H__ + +typedef int (*xattr_walk_fn)(struct xfs_inode *ip, unsigned int attr_flags, + const unsigned char *name, unsigned int namelen, + const void *value, unsigned int valuelen, void *priv); + +int xattr_walk(struct xfs_inode *ip, xattr_walk_fn attr_fn, void *priv); + +#endif /* __REPAIR_LISTXATTR_H__ */ diff --git a/repair/phase6.c b/repair/phase6.c index 48ec236d..1994162a 100644 --- a/repair/phase6.c +++ b/repair/phase6.c @@ -3344,5 +3344,7 @@ _(" - resetting contents of realtime bitmap and summary inodes\n")); } } + /* Check and repair directory parent pointers, if enabled. */ + check_parent_ptrs(mp); parent_ptr_free(mp); } diff --git a/repair/pptr.c b/repair/pptr.c index b10c7f41..d1e7f5ee 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -6,8 +6,13 @@ #include "libxfs.h" #include "libxfs/xfile.h" #include "libxfs/xfblob.h" +#include "libfrog/workqueue.h" +#include "repair/globals.h" #include "repair/err_protos.h" #include "repair/slab.h" +#include "repair/listxattr.h" +#include "repair/threads.h" +#include "repair/incore.h" #include "repair/pptr.h" #undef PPTR_DEBUG @@ -126,6 +131,21 @@ struct ag_pptr { xfs_agino_t child_agino; }; +struct file_pptr { + /* parent directory handle */ + xfs_ino_t parent_ino; + unsigned int parent_gen; + + /* dirent offset */ + xfs_dir2_dataptr_t diroffset; + + /* parent pointer name length */ + unsigned int namelen; + + /* cookie for the file dirent name */ + xfblob_cookie name_cookie; +}; + struct ag_pptrs { /* Lock to protect pptr_recs during the dirent scan. */ pthread_mutex_t lock; @@ -134,11 +154,80 @@ struct ag_pptrs { struct xfs_slab *pptr_recs; }; +struct file_scan { + struct ag_pptrs *ag_pptrs; + + /* cursor for comparing ag_pptrs.pptr_recs against file_pptrs_recs */ + struct xfs_slab_cursor *ag_pptr_recs_cur; + + /* xfs_parent_name_rec records for a file that we're checking */ + struct xfs_slab *file_pptr_recs; + + /* cursor for comparing file_pptr_recs against pptrs_recs */ + struct xfs_slab_cursor *file_pptr_recs_cur; + + /* names associated with file_pptr_recs */ + struct xfblob *file_pptr_names; + + /* Number of parent pointers recorded for this file. */ + unsigned int nr_file_pptrs; + + /* Does this file have garbage xattrs with ATTR_PARENT set? */ + bool have_garbage; +}; + /* Global names storage file. */ static struct xfblob *names; static pthread_mutex_t names_mutex = PTHREAD_MUTEX_INITIALIZER; static struct ag_pptrs *fs_pptrs; +static int +cmp_ag_pptr( + const void *a, + const void *b) +{ + const struct ag_pptr *pa = a; + const struct ag_pptr *pb = b; + + if (pa->child_agino < pb->child_agino) + return -1; + if (pa->child_agino > pb->child_agino) + return 1; + + if (pa->parent_ino < pb->parent_ino) + return -1; + if (pa->parent_ino > pb->parent_ino) + return 1; + + if (pa->diroffset < pb->diroffset) + return -1; + if (pa->diroffset > pb->diroffset) + return 1; + + return 0; +} + +static int +cmp_file_pptr( + const void *a, + const void *b) +{ + const struct file_pptr *pa = a; + const struct file_pptr *pb = b; + + if (pa->parent_ino < pb->parent_ino) + return -1; + if (pa->parent_ino > pb->parent_ino) + return 1; + + if (pa->diroffset < pb->diroffset) + return -1; + if (pa->diroffset > pb->diroffset) + return 1; + + return 0; +} + void parent_ptr_free( struct xfs_mount *mp) @@ -240,3 +329,532 @@ add_parent_ptr( diroffset, (unsigned long long)ino, (unsigned long long)ag_pptr.name_cookie); } + +/* Schedule this extended attribute for deletion. */ +static void +record_garbage_xattr( + struct xfs_inode *ip, + struct file_scan *fscan, + unsigned int attr_filter, + const void *name, + unsigned int namelen) +{ + if (no_modify) { + if (!fscan->have_garbage) + do_warn( + _("would delete garbage parent pointer extended attributes in ino %llu\n"), + (unsigned long long)ip->i_ino); + fscan->have_garbage = true; + return; + } + + if (fscan->have_garbage) + return; + fscan->have_garbage = true; + + do_warn( + _("deleting garbage parent pointer extended attributes in ino %llu\n"), + (unsigned long long)ip->i_ino); + /* XXX do the work */ +} + +/* Decide if this is a directory parent pointer and stash it if so. */ +static int +examine_xattr( + struct xfs_inode *ip, + unsigned int attr_flags, + const unsigned char *name, + unsigned int namelen, + const void *value, + unsigned int valuelen, + void *priv) +{ + struct file_pptr file_pptr = { }; + struct xfs_parent_name_irec irec; + struct xfs_mount *mp = ip->i_mount; + struct file_scan *fscan = priv; + const struct xfs_parent_name_rec *rec = (const void *)name; + int error; + + /* Ignore anything that isn't a parent pointer. */ + if (!(attr_flags & XFS_ATTR_PARENT)) + return 0; + + /* No incomplete parent pointers. */ + if (attr_flags & XFS_ATTR_INCOMPLETE) + goto corrupt; + + /* Does the ondisk parent pointer structure make sense? */ + if (!xfs_parent_namecheck(mp, rec, namelen, attr_flags) || + !xfs_parent_valuecheck(mp, value, valuelen)) + goto corrupt; + + libxfs_parent_irec_from_disk(&irec, rec, value, valuelen); + + file_pptr.parent_ino = irec.p_ino; + file_pptr.parent_gen = irec.p_gen; + file_pptr.diroffset = irec.p_diroffset; + file_pptr.namelen = irec.p_namelen; + + error = -xfblob_store(fscan->file_pptr_names, + &file_pptr.name_cookie, irec.p_name, irec.p_namelen); + if (error) + do_error( + _("storing ino %llu parent pointer '%.*s' failed: %s\n"), + (unsigned long long)ip->i_ino, irec.p_namelen, + (const char *)irec.p_name, strerror(error)); + + error = -slab_add(fscan->file_pptr_recs, &file_pptr); + if (error) + do_error(_("storing ino %llu parent pointer rec failed: %s\n"), + (unsigned long long)ip->i_ino, strerror(error)); + + dbg_printf( + _("%s: dp %llu fname '%.*s' namelen %u diroffset %u ino %llu cookie 0x%llx\n"), + __func__, (unsigned long long)irec.p_ino, + irec.p_namelen, (const char *)irec.p_name, + irec.p_namelen, irec.p_diroffset, + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr.name_cookie); + fscan->nr_file_pptrs++; + return 0; +corrupt: + record_garbage_xattr(ip, fscan, attr_flags, name, namelen); + return 0; +} + +/* Remove all pptrs from @ip. */ +static void +clear_all_pptrs( + struct xfs_inode *ip) +{ + if (no_modify) { + do_warn(_("would delete unlinked ino %llu parent pointers\n"), + (unsigned long long)ip->i_ino); + return; + } + + do_warn(_("deleting unlinked ino %llu parent pointers\n"), + (unsigned long long)ip->i_ino); + /* XXX actually do the work */ +} + +/* Add @ag_pptr to @ip. */ +static void +add_missing_parent_ptr( + struct xfs_inode *ip, + struct file_scan *fscan, + const struct ag_pptr *ag_pptr) +{ + unsigned char name[MAXNAMELEN]; + int error; + + error = -xfblob_load(names, ag_pptr->name_cookie, name, + ag_pptr->namelen); + if (error) + do_error( + _("loading missing name for ino %llu parent pointer (ino %llu gen 0x%x diroffset %u namecookie 0x%llx) failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + (unsigned long long)ag_pptr->name_cookie, + strerror(error)); + + if (no_modify) { + do_warn( + _("would add missing ino %llu parent pointer (ino %llu gen 0x%x diroffset %u name '%.*s')\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + ag_pptr->namelen, name); + return; + } + + do_warn( + _("adding missing ino %llu parent pointer (ino %llu gen 0x%x diroffset %u name '%.*s')\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + ag_pptr->namelen, name); + + /* XXX actually do the work */ +} + +/* Remove @file_pptr from @ip. */ +static void +remove_incorrect_parent_ptr( + struct xfs_inode *ip, + struct file_scan *fscan, + const struct file_pptr *file_pptr) +{ + unsigned char name[MAXNAMELEN] = { }; + int error; + + error = -xfblob_load(fscan->file_pptr_names, file_pptr->name_cookie, + name, file_pptr->namelen); + if (error) + do_error( + _("loading incorrect name for ino %llu parent pointer (ino %llu gen 0x%x diroffset %u namecookie 0x%llx) failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + (unsigned long long)file_pptr->name_cookie, + strerror(error)); + + if (no_modify) { + do_warn( + _("would remove bad ino %llu parent pointer (ino %llu gen 0x%x diroffset %u name '%.*s')\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + file_pptr->namelen, name); + return; + } + + do_warn( + _("removing bad ino %llu parent pointer (ino %llu gen 0x%x diroffset %u name '%.*s')\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + file_pptr->namelen, name); + + /* XXX actually do the work */ +} + +/* + * We found parent pointers that point to the same inode and directory offset. + * Make sure they have the same generation number and dirent name. + */ +static void +compare_parent_pointers( + struct xfs_inode *ip, + struct file_scan *fscan, + const struct ag_pptr *ag_pptr, + const struct file_pptr *file_pptr) +{ + unsigned char name1[MAXNAMELEN] = { }; + unsigned char name2[MAXNAMELEN] = { }; + int error; + + error = -xfblob_load(names, ag_pptr->name_cookie, name1, + ag_pptr->namelen); + if (error) + do_error( + _("loading master-list name for ino %llu parent pointer (ino %llu gen 0x%x diroffset %u namecookie 0x%llx namelen %u) failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + (unsigned long long)ag_pptr->name_cookie, + ag_pptr->namelen, strerror(error)); + + error = -xfblob_load(fscan->file_pptr_names, file_pptr->name_cookie, + name2, file_pptr->namelen); + if (error) + do_error( + _("loading file-list name for ino %llu parent pointer (ino %llu gen 0x%x diroffset %u namecookie 0x%llx namelen %u) failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + (unsigned long long)file_pptr->name_cookie, + ag_pptr->namelen, strerror(error)); + + if (ag_pptr->parent_gen != file_pptr->parent_gen) + goto reset; + if (ag_pptr->namelen != file_pptr->namelen) + goto reset; + if (memcmp(name1, name2, ag_pptr->namelen)) + goto reset; + + return; + +reset: + if (no_modify) { + do_warn( + _("would update ino %llu parent pointer (ino %llu gen 0x%x diroffset %u name '%.*s')\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + ag_pptr->namelen, name1); + return; + } + + do_warn( + _("updating ino %llu parent pointer (ino %llu gen 0x%x diroffset %u name '%.*s')\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + ag_pptr->namelen, name1); + + /* XXX do the work */ +} + +/* + * Make sure that the parent pointers we observed match the ones ondisk. + * + * Earlier, we generated a master list of parent pointers for files in this AG + * based on what we saw during the directory walk at the start of phase 6. + * Now that we've read in all of this file's parent pointers, make sure the + * lists match. + */ +static void +crosscheck_file_parent_ptrs( + struct xfs_inode *ip, + struct file_scan *fscan) +{ + struct ag_pptr *ag_pptr; + struct file_pptr *file_pptr; + struct xfs_mount *mp = ip->i_mount; + xfs_agino_t agino = XFS_INO_TO_AGINO(mp, ip->i_ino); + int error; + + ag_pptr = peek_slab_cursor(fscan->ag_pptr_recs_cur); + + if (!ag_pptr || ag_pptr->child_agino > agino) { + /* + * The cursor for the master pptr list has gone beyond this + * file that we're scanning. Evidently it has no parents at + * all, so we better not have found any pptrs attached to the + * file. + */ + if (fscan->nr_file_pptrs > 0) + clear_all_pptrs(ip); + + return; + } + + if (ag_pptr->child_agino < agino) { + /* + * The cursor for the master pptr list is behind the file that + * we're scanning. This suggests that the incore inode tree + * doesn't know about a file that is mentioned by a dirent. + * At this point the inode liveness is supposed to be settled, + * which means our incore information is inconsistent. + */ + do_error( + _("found dirent referring to ino %llu even though inobt scan moved on to ino %llu?!\n"), + (unsigned long long)XFS_AGINO_TO_INO(mp, + XFS_INO_TO_AGNO(mp, ip->i_ino), + ag_pptr->child_agino), + (unsigned long long)ip->i_ino); + /* does not return */ + } + + /* + * The master pptr list cursor is pointing to the inode that we want + * to check. Sort the pptr records that we recorded from the ondisk + * pptrs for this file, then set up for the comparison. + */ + qsort_slab(fscan->file_pptr_recs, cmp_file_pptr); + + error = -init_slab_cursor(fscan->file_pptr_recs, cmp_file_pptr, + &fscan->file_pptr_recs_cur); + if (error) + do_error(_("init ino %llu parent pointer cursor failed: %s\n"), + (unsigned long long)ip->i_ino, strerror(error)); + + do { + file_pptr = peek_slab_cursor(fscan->file_pptr_recs_cur); + + dbg_printf( + _("%s: dp %llu dp_gen 0x%x namelen %u diroffset %u ino %llu namecookie 0x%llx (master)\n"), + __func__, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, + ag_pptr->namelen, + ag_pptr->diroffset, + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->name_cookie); + + if (file_pptr) { + dbg_printf( + _("%s: dp %llu dp_gen 0x%x namelen %u diroffset %u ino %llu namecookie 0x%llx (file)\n"), + __func__, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, + file_pptr->namelen, + file_pptr->diroffset, + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->name_cookie); + } else { + dbg_printf( + _("%s: ran out of parent pointers for ino %llu (file)\n"), + __func__, + (unsigned long long)ip->i_ino); + } + + if (!file_pptr || + file_pptr->parent_ino > ag_pptr->parent_ino || + file_pptr->diroffset > ag_pptr->diroffset) { + /* + * The master pptr list knows about pptrs that are not + * in the ondisk metadata. Add the missing pptr and + * advance only the master pptr cursor. + */ + add_missing_parent_ptr(ip, fscan, ag_pptr); + advance_slab_cursor(fscan->ag_pptr_recs_cur); + } else if (file_pptr->parent_ino < ag_pptr->parent_ino || + file_pptr->diroffset < ag_pptr->diroffset) { + /* + * The ondisk pptrs mention a link that is not in the + * master list. Delete the extra pptr and advance only + * the file pptr cursor. + */ + remove_incorrect_parent_ptr(ip, fscan, file_pptr); + advance_slab_cursor(fscan->file_pptr_recs_cur); + } else { + /* + * Exact match, make sure the parent_gen and dirent + * name parts of the parent pointer match. Move both + * cursors forward. + */ + compare_parent_pointers(ip, fscan, ag_pptr, file_pptr); + advance_slab_cursor(fscan->ag_pptr_recs_cur); + advance_slab_cursor(fscan->file_pptr_recs_cur); + } + + ag_pptr = peek_slab_cursor(fscan->ag_pptr_recs_cur); + } while (ag_pptr && ag_pptr->child_agino == agino); + + while ((file_pptr = pop_slab_cursor(fscan->file_pptr_recs_cur))) { + dbg_printf( + _("%s: dp %llu dp_gen 0x%x namelen %u diroffset %u ino %llu namecookie 0x%llx (excess)\n"), + __func__, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, + file_pptr->namelen, + file_pptr->diroffset, + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->name_cookie); + + /* + * The master pptr list does not have any more pptrs for this + * file, but we still have unprocessed ondisk pptrs. Delete + * all these ondisk pptrs. + */ + remove_incorrect_parent_ptr(ip, fscan, file_pptr); + } +} + +/* Ensure this file's parent pointers match what we found in the dirent scan. */ +static void +check_file_parent_ptrs( + struct xfs_inode *ip, + struct file_scan *fscan) +{ + int error; + + error = -init_slab(&fscan->file_pptr_recs, sizeof(struct file_pptr)); + if (error) + do_error(_("init file parent pointer recs failed: %s\n"), + strerror(error)); + + fscan->have_garbage = false; + fscan->nr_file_pptrs = 0; + + error = xattr_walk(ip, examine_xattr, fscan); + if (error && !no_modify) + do_error(_("ino %llu parent pointer scan failed: %s\n"), + (unsigned long long)ip->i_ino, + strerror(error)); + if (error) { + do_warn(_("ino %llu parent pointer scan failed: %s\n"), + (unsigned long long)ip->i_ino, + strerror(error)); + goto out_free; + } + + crosscheck_file_parent_ptrs(ip, fscan); + +out_free: + free_slab(&fscan->file_pptr_recs); + xfblob_truncate(fscan->file_pptr_names); +} + +/* Check all the parent pointers of files in this AG. */ +static void +check_ag_parent_ptrs( + struct workqueue *wq, + uint32_t agno, + void *arg) +{ + struct xfs_mount *mp = wq->wq_ctx; + struct file_scan fscan = { + .ag_pptrs = &fs_pptrs[agno], + }; + struct ag_pptrs *ag_pptrs = &fs_pptrs[agno]; + struct ino_tree_node *irec; + int error; + + qsort_slab(ag_pptrs->pptr_recs, cmp_ag_pptr); + + error = -init_slab_cursor(ag_pptrs->pptr_recs, cmp_ag_pptr, + &fscan.ag_pptr_recs_cur); + if (error) + do_error( + _("init agno %u parent pointer slab cursor failed: %s\n"), + agno, strerror(error)); + + error = -xfblob_create(mp, "file parent pointer names", + &fscan.file_pptr_names); + if (error) + do_error( + _("init agno %u file parent pointer names failed: %s\n"), + agno, strerror(error)); + + for (irec = findfirst_inode_rec(agno); + irec != NULL; + irec = next_ino_rec(irec)) { + unsigned int ino_offset; + + for (ino_offset = 0; + ino_offset < XFS_INODES_PER_CHUNK; + ino_offset++) { + struct xfs_inode *ip; + xfs_ino_t ino; + + if (is_inode_free(irec, ino_offset)) + continue; + + ino = XFS_AGINO_TO_INO(mp, agno, + irec->ino_startnum + ino_offset); + error = -libxfs_iget(mp, NULL, ino, 0, &ip); + if (error && !no_modify) + do_error( + _("loading ino %llu for parent pointer check failed: %s\n"), + (unsigned long long)ino, + strerror(error)); + if (error) { + do_warn( + _("loading ino %llu for parent pointer check failed: %s\n"), + (unsigned long long)ino, + strerror(error)); + continue; + } + + check_file_parent_ptrs(ip, &fscan); + libxfs_irele(ip); + } + } + + xfblob_destroy(fscan.file_pptr_names); + free_slab_cursor(&fscan.ag_pptr_recs_cur); +} + +/* Check all the parent pointers of all files in this filesystem. */ +void +check_parent_ptrs( + struct xfs_mount *mp) +{ + struct workqueue wq; + xfs_agnumber_t agno; + + if (!xfs_has_parent(mp)) + return; + + create_work_queue(&wq, mp, ag_stride); + + for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) + queue_work(&wq, check_ag_parent_ptrs, agno, NULL); + + destroy_work_queue(&wq); +} diff --git a/repair/pptr.h b/repair/pptr.h index 2c632ec9..d72c1ac2 100644 --- a/repair/pptr.h +++ b/repair/pptr.h @@ -12,4 +12,6 @@ void parent_ptr_init(struct xfs_mount *mp); void add_parent_ptr(xfs_ino_t ino, const unsigned char *fname, xfs_dir2_dataptr_t diroffset, struct xfs_inode *dp); +void check_parent_ptrs(struct xfs_mount *mp); + #endif /* __REPAIR_PPTR_H__ */ From patchwork Thu Feb 16 21:09:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143926 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63733C61DA4 for ; Thu, 16 Feb 2023 21:09:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230146AbjBPVJQ (ORCPT ); Thu, 16 Feb 2023 16:09:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230150AbjBPVJO (ORCPT ); Thu, 16 Feb 2023 16:09:14 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B833A528AE for ; Thu, 16 Feb 2023 13:09:10 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 521B860A65 for ; Thu, 16 Feb 2023 21:09:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B26D2C433EF; Thu, 16 Feb 2023 21:09:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581749; bh=kgUZXWN6Fg1ay1mmf46HAskb5uQXTRxNALfhqD8J3Z4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=duMQ/Ac+NNgJWc7dAcBHysK2W0rGH3uY/Gja3Nv/CBqqZJFwW6PH0X4mX+f/Jhlj5 aImbhObwnbZ7b2vl9S7jE/Ps0HaDC9KjiRNZCApHbQED8Cbsy/ZA2uKXuRTkB7RGDY 3vZf/eDetWY0phXdK2qt7Wu3SaoTgrg1/tAnI+xNCVmB2efMEUDjvAcObPK6+NZEUo Y62L8dxV4Wqhj4LsdKVuf4ZlCm3fGRaCODsA6HI51pz4eoLOfcQc+RyiidSsE7s7Ob KllJfs+rksICikVyQd/MwrWnCgMK/R605ul1gYoml8rGvRog7QDd8QCVo7ZYVLFIdQ Dr6fRoDhgIASg== Date: Thu, 16 Feb 2023 13:09:09 -0800 Subject: [PATCH 3/8] xfs_repair: dump garbage parent pointer attributes From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657882005.3477807.6914142822313284703.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Delete xattrs that have ATTR_PARENT set but are so garbage that they clearly aren't parent pointers. Signed-off-by: Darrick J. Wong --- repair/pptr.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 111 insertions(+), 3 deletions(-) diff --git a/repair/pptr.c b/repair/pptr.c index d1e7f5ee..695177ce 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -174,6 +174,23 @@ struct file_scan { /* Does this file have garbage xattrs with ATTR_PARENT set? */ bool have_garbage; + + /* xattrs that we have to remove from this file */ + struct xfs_slab *garbage_xattr_recs; + + /* attr names associated with garbage_xattr_recs */ + struct xfblob *garbage_xattr_names; +}; + +struct garbage_xattr { + /* xfs_da_args.attr_filter for the attribute being removed */ + unsigned int attr_filter; + + /* attribute name length */ + unsigned int attrnamelen; + + /* cookie for the attribute name */ + xfblob_cookie attrname_cookie; }; /* Global names storage file. */ @@ -330,7 +347,63 @@ add_parent_ptr( (unsigned long long)ag_pptr.name_cookie); } -/* Schedule this extended attribute for deletion. */ +/* Remove garbage extended attributes that have ATTR_PARENT set. */ +static void +remove_garbage_xattrs( + struct xfs_inode *ip, + struct file_scan *fscan) +{ + struct xfs_slab_cursor *cur; + struct garbage_xattr *ga; + int error; + + error = -init_slab_cursor(fscan->garbage_xattr_recs, NULL, &cur); + if (error) + do_error(_("init garbage xattr cursor failed: %s\n"), + strerror(error)); + + while ((ga = pop_slab_cursor(cur)) != NULL) { + struct xfs_da_args args = { + .dp = ip, + .attr_filter = ga->attr_filter, + .namelen = ga->attrnamelen, + }; + void *buf; + + buf = malloc(ga->attrnamelen); + if (!buf) + do_error( + _("allocating %u bytes to remove ino %llu garbage xattr failed: %s\n"), + ga->attrnamelen, + (unsigned long long)ip->i_ino, + strerror(errno)); + + error = -xfblob_load(fscan->garbage_xattr_names, + ga->attrname_cookie, buf, ga->attrnamelen); + if (error) + do_error( + _("loading garbage xattr name failed: %s\n"), + strerror(error)); + + args.name = buf; + error = -libxfs_attr_set(&args); + if (error) + do_error( + _("removing ino %llu garbage xattr failed: %s\n"), + (unsigned long long)ip->i_ino, + strerror(error)); + + free(buf); + } + + free_slab_cursor(&cur); + + free_slab(&fscan->garbage_xattr_recs); + xfblob_destroy(fscan->garbage_xattr_names); + fscan->garbage_xattr_names = NULL; +} + +/* Schedule this ATTR_PARENT extended attribute for deletion. */ static void record_garbage_xattr( struct xfs_inode *ip, @@ -339,6 +412,13 @@ record_garbage_xattr( const void *name, unsigned int namelen) { + struct garbage_xattr garbage_xattr = { + .attr_filter = attr_filter, + .attrnamelen = namelen, + }; + struct xfs_mount *mp = ip->i_mount; + int error; + if (no_modify) { if (!fscan->have_garbage) do_warn( @@ -349,13 +429,38 @@ record_garbage_xattr( } if (fscan->have_garbage) - return; + goto stuffit; fscan->have_garbage = true; do_warn( _("deleting garbage parent pointer extended attributes in ino %llu\n"), (unsigned long long)ip->i_ino); - /* XXX do the work */ + + error = -init_slab(&fscan->garbage_xattr_recs, + sizeof(struct garbage_xattr)); + if (error) + do_error(_("init garbage xattr recs failed: %s\n"), + strerror(error)); + + error = -xfblob_create(mp, "garbage xattr names", + &fscan->garbage_xattr_names); + if (error) + do_error("init garbage xattr names failed: %s\n", + strerror(error)); + +stuffit: + error = -xfblob_store(fscan->garbage_xattr_names, + &garbage_xattr.attrname_cookie, name, namelen); + if (error) + do_error(_("storing ino %llu garbage xattr failed: %s\n"), + (unsigned long long)ip->i_ino, + strerror(error)); + + error = -slab_add(fscan->garbage_xattr_recs, &garbage_xattr); + if (error) + do_error(_("storing ino %llu garbage xattr rec failed: %s\n"), + (unsigned long long)ip->i_ino, + strerror(error)); } /* Decide if this is a directory parent pointer and stash it if so. */ @@ -763,6 +868,9 @@ check_file_parent_ptrs( goto out_free; } + if (!no_modify && fscan->have_garbage) + remove_garbage_xattrs(ip, fscan); + crosscheck_file_parent_ptrs(ip, fscan); out_free: From patchwork Thu Feb 16 21:09:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DF5BC636CC for ; Thu, 16 Feb 2023 21:09:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229971AbjBPVJ3 (ORCPT ); Thu, 16 Feb 2023 16:09:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbjBPVJ3 (ORCPT ); Thu, 16 Feb 2023 16:09:29 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1638F2BF17 for ; Thu, 16 Feb 2023 13:09:28 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 956BFB826BA for ; Thu, 16 Feb 2023 21:09:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 44572C433EF; Thu, 16 Feb 2023 21:09:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581765; bh=QamHp6bJxobrpyIYtEWWFba66Omo9437Op6Oz1CYJHA=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=FH3MgZHPiRBHTwAhXKAAj2Ill1LaVUgGDiN9Bkx/MdhVMEl8Qgh8jNLtVrTn1wc/4 XYxaD+OzA2sGMy1xib/Gc505ebEhUspK3DvSAhrNtgOnOJkmnkIrO4fntNOihK4o1m mVHa5MdarKmjgDqoa1jgxTIicyH1cz3U9rBqSw7jAE5POh3zTqJZ97aJ2UhVH/zsD6 xbFgyKGJPx0UT5nMv7flh792gN/UeqJzzUOzFVp8op7fk/e9A9DyEBnQDNtrONi/Px xIO29YgXdkrVs1drfVDR7fNH0xCH30Y94VG+Tdza6ntlUtRLltP0BdlykLufbh3s0l /bEjKegrtFQuQ== Date: Thu, 16 Feb 2023 13:09:24 -0800 Subject: [PATCH 4/8] xfs_repair: update ondisk parent pointer records From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657882018.3477807.10657512746908096447.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Update the ondisk parent pointer records as necessary. Signed-off-by: Darrick J. Wong --- libxfs/libxfs_api_defs.h | 2 + repair/pptr.c | 74 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 73 insertions(+), 3 deletions(-) diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h index 92cdb6cc..ab8bdc1c 100644 --- a/libxfs/libxfs_api_defs.h +++ b/libxfs/libxfs_api_defs.h @@ -147,7 +147,9 @@ #define xfs_parent_add libxfs_parent_add #define xfs_parent_finish libxfs_parent_finish #define xfs_parent_irec_from_disk libxfs_parent_irec_from_disk +#define xfs_parent_set libxfs_parent_set #define xfs_parent_start libxfs_parent_start +#define xfs_parent_unset libxfs_parent_unset #define xfs_perag_get libxfs_perag_get #define xfs_perag_put libxfs_perag_put #define xfs_prealloc_blocks libxfs_prealloc_blocks diff --git a/repair/pptr.c b/repair/pptr.c index 695177ce..53ac1013 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -528,6 +528,42 @@ examine_xattr( return 0; } +/* Add an on disk parent pointer to a file. */ +static int +add_file_pptr( + struct xfs_inode *ip, + const struct ag_pptr *ag_pptr, + const unsigned char *name) +{ + struct xfs_parent_name_irec pptr_rec = { + .p_ino = ag_pptr->parent_ino, + .p_gen = ag_pptr->parent_gen, + .p_diroffset = ag_pptr->diroffset, + .p_namelen = ag_pptr->namelen, + }; + struct xfs_parent_scratch scratch; + + memcpy(pptr_rec.p_name, name, ag_pptr->namelen); + + return -libxfs_parent_set(ip, &pptr_rec, &scratch); +} + +/* Remove an on disk parent pointer from a file. */ +static int +remove_file_pptr( + struct xfs_inode *ip, + const struct file_pptr *file_pptr) +{ + struct xfs_parent_name_irec pptr_rec = { + .p_ino = file_pptr->parent_ino, + .p_gen = file_pptr->parent_gen, + .p_diroffset = file_pptr->diroffset, + }; + struct xfs_parent_scratch scratch; + + return -libxfs_parent_unset(ip, &pptr_rec, &scratch); +} + /* Remove all pptrs from @ip. */ static void clear_all_pptrs( @@ -582,7 +618,14 @@ add_missing_parent_ptr( ag_pptr->parent_gen, ag_pptr->diroffset, ag_pptr->namelen, name); - /* XXX actually do the work */ + error = add_file_pptr(ip, ag_pptr, name); + if (error) + do_error( + _("adding ino %llu pptr (ino %llu gen 0x%x diroffset %u name '%.*s') failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + ag_pptr->namelen, name, strerror(error)); } /* Remove @file_pptr from @ip. */ @@ -623,7 +666,14 @@ remove_incorrect_parent_ptr( file_pptr->parent_gen, file_pptr->diroffset, file_pptr->namelen, name); - /* XXX actually do the work */ + error = remove_file_pptr(ip, file_pptr); + if (error) + do_error( + _("removing ino %llu pptr (ino %llu gen 0x%x diroffset %u name '%.*s') failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + file_pptr->namelen, name, strerror(error)); } /* @@ -690,7 +740,25 @@ compare_parent_pointers( ag_pptr->parent_gen, ag_pptr->diroffset, ag_pptr->namelen, name1); - /* XXX do the work */ + if (ag_pptr->parent_gen != file_pptr->parent_gen) { + error = remove_file_pptr(ip, file_pptr); + if (error) + do_error( + _("erasing ino %llu pptr (ino %llu gen 0x%x diroffset %u name '%.*s') failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + file_pptr->namelen, name2, strerror(error)); + } + + error = add_file_pptr(ip, ag_pptr, name1); + if (error) + do_error( + _("updating ino %llu pptr (ino %llu gen 0x%x diroffset %u name '%.*s') failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)ag_pptr->parent_ino, + ag_pptr->parent_gen, ag_pptr->diroffset, + ag_pptr->namelen, name1, strerror(error)); } /* From patchwork Thu Feb 16 21:09:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34B63C636CC for ; Thu, 16 Feb 2023 21:09:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230152AbjBPVJw (ORCPT ); Thu, 16 Feb 2023 16:09:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbjBPVJo (ORCPT ); Thu, 16 Feb 2023 16:09:44 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74EB92BEC4 for ; Thu, 16 Feb 2023 13:09:43 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2BA05B828E1 for ; Thu, 16 Feb 2023 21:09:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8758C433D2; Thu, 16 Feb 2023 21:09:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581780; bh=pow9LpkeR9xlR8pOXmuEdz9b2fdAglGJeH13cxpqZ3w=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=DA7f1ccHMIdJllq4NV4SNDQki7XXehEUZvP8L09pDylNXKyyroddEe3gdlZSiZCbh w5ElYmPwyyg8DB5epLzA92V00G9kl2EmkcCILRqlL796TnTSetf1Rb2DvFjddq52mF NiZIiVchnNodl2WbJAoq8krSXhDkTI/275VQhHfsPJpyOQU9pOK91PJnWJ4+kPKmZA 7IIiVsoiTldKAg+woJ9dHvlmyKJ0YinlTTfjug1Myf1DW8h27znDoTCX2KVNcAPimJ arLDecPI2x2KqJSxedELpvWKTC90EujvMZ6VT4PyvQD4GRbX+cfyivz/qr37gscYY8 RXbpx1s6qLRDQ== Date: Thu, 16 Feb 2023 13:09:40 -0800 Subject: [PATCH 5/8] xfs_repair: wipe ondisk parent pointers when there are none From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657882031.3477807.10832130110488009545.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Erase all the parent pointers when there aren't any found by the directory entry scan. Signed-off-by: Darrick J. Wong --- repair/pptr.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/repair/pptr.c b/repair/pptr.c index 53ac1013..b1f5fb4e 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -567,8 +567,13 @@ remove_file_pptr( /* Remove all pptrs from @ip. */ static void clear_all_pptrs( - struct xfs_inode *ip) + struct xfs_inode *ip, + struct file_scan *fscan) { + struct xfs_slab_cursor *cur; + struct file_pptr *file_pptr; + int error; + if (no_modify) { do_warn(_("would delete unlinked ino %llu parent pointers\n"), (unsigned long long)ip->i_ino); @@ -577,7 +582,25 @@ clear_all_pptrs( do_warn(_("deleting unlinked ino %llu parent pointers\n"), (unsigned long long)ip->i_ino); - /* XXX actually do the work */ + + error = -init_slab_cursor(fscan->file_pptr_recs, NULL, &cur); + if (error) + do_error(_("init ino %llu pptr cursor failed: %s\n"), + (unsigned long long)ip->i_ino, + strerror(error)); + + while ((file_pptr = pop_slab_cursor(cur)) != NULL) { + error = remove_file_pptr(ip, file_pptr); + if (error) + do_error( + _("wiping ino %llu pptr (ino %llu gen 0x%x diroffset %u) failed: %s\n"), + (unsigned long long)ip->i_ino, + (unsigned long long)file_pptr->parent_ino, + file_pptr->parent_gen, file_pptr->diroffset, + strerror(error)); + } + + free_slab_cursor(&cur); } /* Add @ag_pptr to @ip. */ @@ -790,7 +813,7 @@ crosscheck_file_parent_ptrs( * file. */ if (fscan->nr_file_pptrs > 0) - clear_all_pptrs(ip); + clear_all_pptrs(ip, fscan); return; } From patchwork Thu Feb 16 21:09:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D03FCC61DA4 for ; Thu, 16 Feb 2023 21:10:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230141AbjBPVKB (ORCPT ); Thu, 16 Feb 2023 16:10:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbjBPVKA (ORCPT ); Thu, 16 Feb 2023 16:10:00 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 097423B23C for ; Thu, 16 Feb 2023 13:09:59 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B8BA9B826BA for ; Thu, 16 Feb 2023 21:09:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70ECDC4339C; Thu, 16 Feb 2023 21:09:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581796; bh=zCFzbFJnogEByhUNcN/M+j232qS8x0E8wwoI7cMHtdw=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=qH93aydrpHnZZG8BjqtooPX/VjVBHLRKpQjcTTYqN8A0WRYJlKuwezWs+8ScZiAFP ihH8j1m+QgCacmi05+q6pG9GL4E0zWRMeDF/esJH1PHaEylnVMf8cxGJq02YZDrDo4 i/BWfpr1L+AFQ5bbmubY/pDpzx3JLayB+Bm4qvUHZk+/4nYlytujzTYooN1DwDl+Ri KvZPsk0cvlplxP7XhOF3WNbDnNV4YxGGHG2ZZW/P84TuNd64mrwLXT0fZ4vBEeDxY3 tuv/ZTvDOkEdBcsKR6TrAYXVdC6Dg1gbopGiR/MgptnvAgbou0ehGjeGsESOjkOczu g7SbTQLfsZWlg== Date: Thu, 16 Feb 2023 13:09:55 -0800 Subject: [PATCH 6/8] xfs_repair: move the global dirent name store to a separate object From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657882044.3477807.12305464936599915965.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Abstract the main parent pointer dirent names xfblob object into a separate data structure to hide implementation details. This is the first step towards deduplicating the names. Signed-off-by: Darrick J. Wong --- repair/Makefile | 2 + repair/pptr.c | 13 +++++---- repair/strblobs.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++++ repair/strblobs.h | 20 +++++++++++++ 4 files changed, 109 insertions(+), 6 deletions(-) create mode 100644 repair/strblobs.c create mode 100644 repair/strblobs.h diff --git a/repair/Makefile b/repair/Makefile index 925864c2..48ddcdd1 100644 --- a/repair/Makefile +++ b/repair/Makefile @@ -33,6 +33,7 @@ HFILES = \ rt.h \ scan.h \ slab.h \ + strblobs.h \ threads.h \ versions.h @@ -71,6 +72,7 @@ CFILES = \ sb.c \ scan.c \ slab.c \ + strblobs.c \ threads.c \ versions.c \ xfs_repair.c diff --git a/repair/pptr.c b/repair/pptr.c index b1f5fb4e..20d66884 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -14,6 +14,7 @@ #include "repair/threads.h" #include "repair/incore.h" #include "repair/pptr.h" +#include "repair/strblobs.h" #undef PPTR_DEBUG @@ -194,7 +195,7 @@ struct garbage_xattr { }; /* Global names storage file. */ -static struct xfblob *names; +static struct strblobs *nameblobs; static pthread_mutex_t names_mutex = PTHREAD_MUTEX_INITIALIZER; static struct ag_pptrs *fs_pptrs; @@ -261,7 +262,7 @@ parent_ptr_free( free(fs_pptrs); fs_pptrs = NULL; - xfblob_destroy(names); + strblobs_destroy(&nameblobs); } void @@ -274,7 +275,7 @@ parent_ptr_init( if (!xfs_has_parent(mp)) return; - error = -xfblob_create(mp, "parent pointer names", &names); + error = strblobs_init(mp, "parent pointer names", &nameblobs); if (error) do_error(_("init parent pointer names failed: %s\n"), strerror(error)); @@ -325,7 +326,7 @@ add_parent_ptr( return; pthread_mutex_lock(&names_mutex); - error = -xfblob_store(names, &ag_pptr.name_cookie, fname, + error = strblobs_store(nameblobs, &ag_pptr.name_cookie, fname, ag_pptr.namelen); pthread_mutex_unlock(&names_mutex); if (error) @@ -613,7 +614,7 @@ add_missing_parent_ptr( unsigned char name[MAXNAMELEN]; int error; - error = -xfblob_load(names, ag_pptr->name_cookie, name, + error = strblobs_load(nameblobs, ag_pptr->name_cookie, name, ag_pptr->namelen); if (error) do_error( @@ -714,7 +715,7 @@ compare_parent_pointers( unsigned char name2[MAXNAMELEN] = { }; int error; - error = -xfblob_load(names, ag_pptr->name_cookie, name1, + error = strblobs_load(nameblobs, ag_pptr->name_cookie, name1, ag_pptr->namelen); if (error) do_error( diff --git a/repair/strblobs.c b/repair/strblobs.c new file mode 100644 index 00000000..2b7a7a5e --- /dev/null +++ b/repair/strblobs.c @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2023 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#include "libxfs.h" +#include "libxfs/xfile.h" +#include "libxfs/xfblob.h" +#include "repair/strblobs.h" + +/* + * String Blob Structure + * ===================== + * + * This data structure wraps the storage of strings with explicit length in an + * xfblob structure. + */ +struct strblobs { + struct xfblob *strings; +}; + +/* Initialize a string blob structure. */ +int +strblobs_init( + struct xfs_mount *mp, + const char *descr, + struct strblobs **sblobs) +{ + struct strblobs *sb; + int error; + + sb = malloc(sizeof(struct strblobs)); + if (!sb) + return ENOMEM; + + error = -xfblob_create(mp, descr, &sb->strings); + if (error) + goto out_free; + + *sblobs = sb; + return 0; + +out_free: + free(sb); + return error; +} + +/* Deconstruct a string blob structure. */ +void +strblobs_destroy( + struct strblobs **sblobs) +{ + struct strblobs *sb = *sblobs; + + xfblob_destroy(sb->strings); + free(sb); + *sblobs = NULL; +} + +/* Store a string and return a cookie for its retrieval. */ +int +strblobs_store( + struct strblobs *sblobs, + xfblob_cookie *str_cookie, + const unsigned char *str, + unsigned int str_len) +{ + return -xfblob_store(sblobs->strings, str_cookie, str, str_len); +} + +/* Retrieve a previously stored string. */ +int +strblobs_load( + struct strblobs *sblobs, + xfblob_cookie str_cookie, + unsigned char *str, + unsigned int str_len) +{ + return -xfblob_load(sblobs->strings, str_cookie, str, str_len); +} diff --git a/repair/strblobs.h b/repair/strblobs.h new file mode 100644 index 00000000..f5680175 --- /dev/null +++ b/repair/strblobs.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2023 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + */ +#ifndef __REPAIR_STRBLOBS_H__ +#define __REPAIR_STRBLOBS_H__ + +struct strblobs; + +int strblobs_init(struct xfs_mount *mp, const char *descr, + struct strblobs **sblobs); +void strblobs_destroy(struct strblobs **sblobs); + +int strblobs_store(struct strblobs *sblobs, xfblob_cookie *str_cookie, + const unsigned char *str, unsigned int str_len); +int strblobs_load(struct strblobs *sblobs, xfblob_cookie str_cookie, + unsigned char *str, unsigned int str_len); + +#endif /* __REPAIR_STRBLOBS_H__ */ From patchwork Thu Feb 16 21:10:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 466CEC636CC for ; Thu, 16 Feb 2023 21:10:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230144AbjBPVKP (ORCPT ); Thu, 16 Feb 2023 16:10:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbjBPVKO (ORCPT ); Thu, 16 Feb 2023 16:10:14 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1ECE73B233 for ; Thu, 16 Feb 2023 13:10:13 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AC69D60BFE for ; Thu, 16 Feb 2023 21:10:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1074BC433D2; Thu, 16 Feb 2023 21:10:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581812; bh=K6uNE8ZVEc72klVqDQ8zki64Zg6/lXZCxNKm/eF4hgE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=pY2R4V7ME9uJPBTgZSuD8YJtdhy4Fqs4VpGa1iHStBf6FZZkiopKlOvt96PDH9ar+ CiE8tzpPDV3WVGRJD95/pjyBstQhs8WiDOgPDMWS5oa95Wxsrjqq6qvVcFeTIkeqcK n6Cp5mCE3myEKlUcTM+umHALZvxRO841HXElxFBy4VX5T8i3NXUBcAf8yfBmcSFaqC kXLRNvXC3vb2p8z6jWrqk7pzTG7bxF1W+DYle3GL/8zifB2V2e7oxGPixID4Pxa1gW PuOFCGMpzJUfoTZhiHkPNBKHETVZH69CLN2TujxLMzLODjjEZUP4E7Vx8Oymr0dKKB eCgfOB6mI9i9Q== Date: Thu, 16 Feb 2023 13:10:11 -0800 Subject: [PATCH 7/8] xfs_repair: deduplicate strings stored in string blob From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657882057.3477807.13885221277476227815.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Reduce the memory requirements of the string blob structure by deduplicating the strings stored within. Signed-off-by: Darrick J. Wong --- repair/pptr.c | 5 ++ repair/strblobs.c | 141 ++++++++++++++++++++++++++++++++++++++++++++++++++++- repair/strblobs.h | 4 +- 3 files changed, 145 insertions(+), 5 deletions(-) diff --git a/repair/pptr.c b/repair/pptr.c index 20d66884..c1cd9060 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -269,13 +269,16 @@ void parent_ptr_init( struct xfs_mount *mp) { + uint64_t iused; xfs_agnumber_t agno; int error; if (!xfs_has_parent(mp)) return; - error = strblobs_init(mp, "parent pointer names", &nameblobs); + /* One hash bucket per inode, up to about 8M of memory on 64-bit. */ + iused = min(mp->m_sb.sb_icount - mp->m_sb.sb_ifree, 1048573); + error = strblobs_init(mp, "parent pointer names", iused, &nameblobs); if (error) do_error(_("init parent pointer names failed: %s\n"), strerror(error)); diff --git a/repair/strblobs.c b/repair/strblobs.c index 2b7a7a5e..fb5929e9 100644 --- a/repair/strblobs.c +++ b/repair/strblobs.c @@ -13,23 +13,43 @@ * ===================== * * This data structure wraps the storage of strings with explicit length in an - * xfblob structure. + * xfblob structure. It stores a hashtable of string checksums to provide + * fast(ish) lookups of existing strings to enable deduplication of the strings + * contained within. */ +struct strblob_hashent { + struct strblob_hashent *next; + + xfblob_cookie str_cookie; + unsigned int str_len; + xfs_dahash_t str_hash; +}; + struct strblobs { struct xfblob *strings; + unsigned int nr_buckets; + + struct strblob_hashent *buckets[]; }; +static inline size_t strblobs_sizeof(unsigned int nr_buckets) +{ + return sizeof(struct strblobs) + + (nr_buckets * sizeof(struct strblobs_hashent *)); +} + /* Initialize a string blob structure. */ int strblobs_init( struct xfs_mount *mp, const char *descr, + unsigned int hash_buckets, struct strblobs **sblobs) { struct strblobs *sb; int error; - sb = malloc(sizeof(struct strblobs)); + sb = calloc(strblobs_sizeof(hash_buckets), 1); if (!sb) return ENOMEM; @@ -37,6 +57,7 @@ strblobs_init( if (error) goto out_free; + sb->nr_buckets = hash_buckets; *sblobs = sb; return 0; @@ -51,12 +72,114 @@ strblobs_destroy( struct strblobs **sblobs) { struct strblobs *sb = *sblobs; + struct strblob_hashent *ent, *ent_next; + unsigned int bucket; + + for (bucket = 0; bucket < sb->nr_buckets; bucket++) { + ent = sb->buckets[bucket]; + while (ent != NULL) { + ent_next = ent->next; + free(ent); + ent = ent_next; + } + } xfblob_destroy(sb->strings); free(sb); *sblobs = NULL; } +/* + * Search the string hashtable for a matching entry. Sets sets the cookie and + * returns 0 if one is found; ENOENT if there is no match; or a positive errno. + */ +static int +__strblobs_lookup( + struct strblobs *sblobs, + xfblob_cookie *str_cookie, + const unsigned char *str, + unsigned int str_len, + xfs_dahash_t str_hash) +{ + struct strblob_hashent *ent; + char *buf = NULL; + unsigned int bucket; + int error; + + bucket = str_hash % sblobs->nr_buckets; + ent = sblobs->buckets[bucket]; + + for (ent = sblobs->buckets[bucket]; ent != NULL; ent = ent->next) { + if (ent->str_len != str_len || ent->str_hash != str_hash) + continue; + + if (!buf) { + buf = malloc(str_len); + if (!buf) + return ENOMEM; + } + + error = strblobs_load(sblobs, ent->str_cookie, buf, str_len); + if (error) + goto out; + + if (memcmp(str, buf, str_len)) + continue; + + *str_cookie = ent->str_cookie; + goto out; + } + error = ENOENT; + +out: + free(buf); + return error; +} + +/* + * Search the string hashtable for a matching entry. Sets sets the cookie and + * returns 0 if one is found; ENOENT if there is no match; or a positive errno. + */ +int +strblobs_lookup( + struct strblobs *sblobs, + xfblob_cookie *str_cookie, + const unsigned char *str, + unsigned int str_len) +{ + xfs_dahash_t str_hash; + + str_hash = libxfs_da_hashname(str, str_len); + return __strblobs_lookup(sblobs, str_cookie, str, str_len, str_hash); +} + +/* Remember a string in the hashtable. */ +static int +strblobs_hash( + struct strblobs *sblobs, + xfblob_cookie str_cookie, + const unsigned char *str, + unsigned int str_len, + xfs_dahash_t str_hash) +{ + struct strblob_hashent *ent; + unsigned int bucket; + + bucket = str_hash % sblobs->nr_buckets; + + ent = malloc(sizeof(struct strblob_hashent)); + if (!ent) + return ENOMEM; + + ent->str_cookie = str_cookie; + ent->str_len = str_len; + ent->str_hash = str_hash; + ent->next = sblobs->buckets[bucket]; + + sblobs->buckets[bucket] = ent; + return 0; +} + /* Store a string and return a cookie for its retrieval. */ int strblobs_store( @@ -65,7 +188,19 @@ strblobs_store( const unsigned char *str, unsigned int str_len) { - return -xfblob_store(sblobs->strings, str_cookie, str, str_len); + int error; + xfs_dahash_t str_hash; + + str_hash = libxfs_da_hashname(str, str_len); + error = __strblobs_lookup(sblobs, str_cookie, str, str_len, str_hash); + if (error != ENOENT) + return error; + + error = -xfblob_store(sblobs->strings, str_cookie, str, str_len); + if (error) + return error; + + return strblobs_hash(sblobs, *str_cookie, str, str_len, str_hash); } /* Retrieve a previously stored string. */ diff --git a/repair/strblobs.h b/repair/strblobs.h index f5680175..b8b059e2 100644 --- a/repair/strblobs.h +++ b/repair/strblobs.h @@ -9,12 +9,14 @@ struct strblobs; int strblobs_init(struct xfs_mount *mp, const char *descr, - struct strblobs **sblobs); + unsigned int hash_buckets, struct strblobs **sblobs); void strblobs_destroy(struct strblobs **sblobs); int strblobs_store(struct strblobs *sblobs, xfblob_cookie *str_cookie, const unsigned char *str, unsigned int str_len); int strblobs_load(struct strblobs *sblobs, xfblob_cookie str_cookie, unsigned char *str, unsigned int str_len); +int strblobs_lookup(struct strblobs *sblobs, xfblob_cookie *str_cookie, + const unsigned char *str, unsigned int str_len); #endif /* __REPAIR_STRBLOBS_H__ */ From patchwork Thu Feb 16 21:10:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13143931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20F32C61DA4 for ; Thu, 16 Feb 2023 21:10:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230127AbjBPVKa (ORCPT ); Thu, 16 Feb 2023 16:10:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229884AbjBPVK3 (ORCPT ); Thu, 16 Feb 2023 16:10:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B54DF3B233 for ; Thu, 16 Feb 2023 13:10:28 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5141E60C0D for ; Thu, 16 Feb 2023 21:10:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF81AC433D2; Thu, 16 Feb 2023 21:10:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676581827; bh=ycq5naSFWC8fEbYYk82YgIhMkm7FhUqOQsEMuy6hqXc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=JL1vgYMDdw81+4nDr8TJORfT4bqVEFVXD3K6l6PBHjGEEQZmReLAT3kn1ZMBI1A7w 3y8ZaOSoaB3dRHaXRrq3bSEou3Fpkdyayvx3p2fsFI5zmMNYEW9nT1U2E+kXce5bVZ 2O8pla/pdDb6syQbTG1nrjnQMXP+isd7nc0Q3tu+hbELi2gaMk/xRjVb6JpPDzlgsw Gx/SNWwpLq3qH/9ku8SiMEYDgixFiKjUTvnW1i9jH0T3ujU9GfIfl2uHnA/ueUQkUx KFGY4rv84QUcwJPB1SvSiwXaBW0tSxq0L+irJDZG/VMKg6vGghWh5IUangKtI9CeQx NIhfYlTfKIlYw== Date: Thu, 16 Feb 2023 13:10:27 -0800 Subject: [PATCH 8/8] xfs_repair: try to reuse nameblob names for file pptr scan names From: "Darrick J. Wong" To: djwong@kernel.org Cc: allison.henderson@oracle.com, linux-xfs@vger.kernel.org Message-ID: <167657882071.3477807.2425230947352621618.stgit@magnolia> In-Reply-To: <167657881963.3477807.5005383731904631094.stgit@magnolia> References: <167657881963.3477807.5005383731904631094.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong When we're scanning a file's parent pointers, see if the name already exists in the nameblobs structure to save memory. If not, we'll continue to use the file scan xfblob, because we don't want to pollute the nameblob structure with names we didn't see in the directory walk. Each of the parent pointer scanner threads can access the nameblob structure locklessly since they don't modify the nameblob. Signed-off-by: Darrick J. Wong --- repair/pptr.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 55 insertions(+), 7 deletions(-) diff --git a/repair/pptr.c b/repair/pptr.c index c1cd9060..a5cf89b9 100644 --- a/repair/pptr.c +++ b/repair/pptr.c @@ -133,8 +133,11 @@ struct ag_pptr { }; struct file_pptr { + /* Is the name stored in the global nameblobs structure? */ + unsigned int name_in_nameblobs:1; + /* parent directory handle */ - xfs_ino_t parent_ino; + unsigned long long parent_ino:63; unsigned int parent_gen; /* dirent offset */ @@ -467,6 +470,32 @@ record_garbage_xattr( strerror(error)); } +/* + * Store this file parent pointer's name in the file scan namelist unless it's + * already in the global list. + */ +static int +store_file_pptr_name( + struct file_scan *fscan, + struct file_pptr *file_pptr, + const struct xfs_parent_name_irec *irec) +{ + int error; + + error = strblobs_lookup(nameblobs, &file_pptr->name_cookie, + irec->p_name, irec->p_namelen); + if (!error) { + file_pptr->name_in_nameblobs = true; + return 0; + } + if (error != ENOENT) + return error; + + file_pptr->name_in_nameblobs = false; + return -xfblob_store(fscan->file_pptr_names, &file_pptr->name_cookie, + irec->p_name, irec->p_namelen); +} + /* Decide if this is a directory parent pointer and stash it if so. */ static int examine_xattr( @@ -505,8 +534,7 @@ examine_xattr( file_pptr.diroffset = irec.p_diroffset; file_pptr.namelen = irec.p_namelen; - error = -xfblob_store(fscan->file_pptr_names, - &file_pptr.name_cookie, irec.p_name, irec.p_namelen); + error = store_file_pptr_name(fscan, &file_pptr, &irec); if (error) do_error( _("storing ino %llu parent pointer '%.*s' failed: %s\n"), @@ -568,6 +596,21 @@ remove_file_pptr( return -libxfs_parent_unset(ip, &pptr_rec, &scratch); } +/* Load a file parent pointer name from wherever we stored it. */ +static int +load_file_pptr_name( + struct file_scan *fscan, + const struct file_pptr *file_pptr, + unsigned char *name) +{ + if (file_pptr->name_in_nameblobs) + return strblobs_load(nameblobs, file_pptr->name_cookie, + name, file_pptr->namelen); + + return -xfblob_load(fscan->file_pptr_names, file_pptr->name_cookie, + name, file_pptr->namelen); +} + /* Remove all pptrs from @ip. */ static void clear_all_pptrs( @@ -665,8 +708,7 @@ remove_incorrect_parent_ptr( unsigned char name[MAXNAMELEN] = { }; int error; - error = -xfblob_load(fscan->file_pptr_names, file_pptr->name_cookie, - name, file_pptr->namelen); + error = load_file_pptr_name(fscan, file_pptr, name); if (error) do_error( _("loading incorrect name for ino %llu parent pointer (ino %llu gen 0x%x diroffset %u namecookie 0x%llx) failed: %s\n"), @@ -729,8 +771,7 @@ compare_parent_pointers( (unsigned long long)ag_pptr->name_cookie, ag_pptr->namelen, strerror(error)); - error = -xfblob_load(fscan->file_pptr_names, file_pptr->name_cookie, - name2, file_pptr->namelen); + error = load_file_pptr_name(fscan, file_pptr, name2); if (error) do_error( _("loading file-list name for ino %llu parent pointer (ino %llu gen 0x%x diroffset %u namecookie 0x%llx namelen %u) failed: %s\n"), @@ -1051,6 +1092,13 @@ check_parent_ptrs( struct workqueue wq; xfs_agnumber_t agno; + /* + * We only store the lower 63 bits of the inode number in struct + * file_pptr to save space, so we must guarantee that we'll never + * encounter an inumber with the top bit set. + */ + BUILD_BUG_ON((1ULL << 63) & XFS_MAXINUMBER); + if (!xfs_has_parent(mp)) return;