From patchwork Wed Apr 18 02:42:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 10347185 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8BF8C60216 for ; Wed, 18 Apr 2018 02:42:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D088284CE for ; Wed, 18 Apr 2018 02:42:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 71863284F5; Wed, 18 Apr 2018 02:42:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 94A66284CE for ; Wed, 18 Apr 2018 02:42:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753320AbeDRCmN (ORCPT ); Tue, 17 Apr 2018 22:42:13 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:45936 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753076AbeDRCmM (ORCPT ); Tue, 17 Apr 2018 22:42:12 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w3I2g5Yl100997 for ; Wed, 18 Apr 2018 02:42:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=2ZnZ9I/m8FkLaxEviIJ5tnxvq9m/j58pwnqeP5G6guM=; b=m0ZrcOr712jVWyu4apMj1TUpJoFHEZLYW5AOWQSL5C6OABxU6RuMIrs682Rqy1e8sbmE 3+rbtB0YMvLdnGaj7CaJjkaU9Sz0XdOEq6/K1gnGjhd0SQLvd6RzswbTtxJ9Z24sOAia 0pxjyL51kNNMS6dAmEBJRj0wlPA/+Csyjp1DnBYw6Q8O5mhSe4uSb7pDL5ZWIV3HV1e7 jt9F43En0uZAn+5UE3JZtEg+YHD4iLslmm1yq2HAjegRJ9Rg03gCsYBZo9tGiOEdQKO9 vl6ULWEoGoBCJ12P3+LP7XlOYUAzCvTa548Ke3WE2LJTZ8i5pkarIaRwQ46khAGag6fE mQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2120.oracle.com with ESMTP id 2hdrxn8juu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 18 Apr 2018 02:42:11 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w3I2gBX0009804 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 18 Apr 2018 02:42:11 GMT Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w3I2gBY3031313 for ; Wed, 18 Apr 2018 02:42:11 GMT Received: from localhost (/10.159.132.124) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 17 Apr 2018 19:42:10 -0700 Subject: [PATCH 03/24] xfs: add helpers to deal with transaction allocation and rolling From: "Darrick J. Wong" To: darrick.wong@oracle.com Cc: linux-xfs@vger.kernel.org Date: Tue, 17 Apr 2018 19:42:09 -0700 Message-ID: <152401932899.12079.11463794891965919567.stgit@magnolia> In-Reply-To: <152401930978.12079.2000682561964128780.stgit@magnolia> References: <152401930978.12079.2000682561964128780.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8866 signatures=668698 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1804180023 Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Darrick J. Wong For repairs, we need to reserve at least as many blocks as we think we're going to need to rebuild the data structure, and we're going to need some helpers to roll transactions while maintaining locks on the AG headers so that other threads cannot wander into the middle of a repair. Signed-off-by: Darrick J. Wong --- fs/xfs/scrub/bmap.c | 2 - fs/xfs/scrub/common.c | 21 ++++++- fs/xfs/scrub/common.h | 2 - fs/xfs/scrub/inode.c | 4 + fs/xfs/scrub/repair.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 12 ++++ 6 files changed, 186 insertions(+), 7 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/fs/xfs/scrub/bmap.c b/fs/xfs/scrub/bmap.c index 3f8fd10..df04215 100644 --- a/fs/xfs/scrub/bmap.c +++ b/fs/xfs/scrub/bmap.c @@ -74,7 +74,7 @@ xfs_scrub_setup_inode_bmap( } /* Got the inode, lock it and we're ready to go. */ - error = xfs_scrub_trans_alloc(sc); + error = xfs_scrub_trans_alloc(sc, 0); if (error) goto out; sc->ilock_flags |= XFS_ILOCK_EXCL; diff --git a/fs/xfs/scrub/common.c b/fs/xfs/scrub/common.c index 93f9e7d..0833fa5 100644 --- a/fs/xfs/scrub/common.c +++ b/fs/xfs/scrub/common.c @@ -49,6 +49,7 @@ #include "scrub/common.h" #include "scrub/trace.h" #include "scrub/btree.h" +#include "scrub/repair.h" /* Common code for the metadata scrubbers. */ @@ -571,11 +572,22 @@ xfs_scrub_ag_init( /* * Grab an empty transaction so that we can re-grab locked buffers if * one of our btrees turns out to be cyclic. + * + * If we're going to repair something, we need to ask for the largest possible + * log reservation so that we can handle the worst case scenario for metadata + * updates while rebuilding a metadata item. We also need to reserve as many + * blocks in the head transaction as we think we're going to need to rebuild + * the metadata object. */ int xfs_scrub_trans_alloc( - struct xfs_scrub_context *sc) + struct xfs_scrub_context *sc, + uint resblks) { + if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR) + return xfs_trans_alloc(sc->mp, &M_RES(sc->mp)->tr_itruncate, + resblks, 0, 0, &sc->tp); + return xfs_trans_alloc_empty(sc->mp, &sc->tp); } @@ -585,7 +597,10 @@ xfs_scrub_setup_fs( struct xfs_scrub_context *sc, struct xfs_inode *ip) { - return xfs_scrub_trans_alloc(sc); + uint resblks; + + resblks = xfs_repair_calc_ag_resblks(sc); + return xfs_scrub_trans_alloc(sc, resblks); } /* Set us up with AG headers and btree cursors. */ @@ -715,7 +730,7 @@ xfs_scrub_setup_inode_contents( /* Got the inode, lock it and we're ready to go. */ sc->ilock_flags = XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL; xfs_ilock(sc->ip, sc->ilock_flags); - error = xfs_scrub_trans_alloc(sc); + error = xfs_scrub_trans_alloc(sc, resblks); if (error) goto out; sc->ilock_flags |= XFS_ILOCK_EXCL; diff --git a/fs/xfs/scrub/common.h b/fs/xfs/scrub/common.h index 5d3e9c5..510aa8a 100644 --- a/fs/xfs/scrub/common.h +++ b/fs/xfs/scrub/common.h @@ -38,7 +38,7 @@ xfs_scrub_should_terminate( return false; } -int xfs_scrub_trans_alloc(struct xfs_scrub_context *sc); +int xfs_scrub_trans_alloc(struct xfs_scrub_context *sc, uint resblks); bool xfs_scrub_process_error(struct xfs_scrub_context *sc, xfs_agnumber_t agno, xfs_agblock_t bno, int *error); bool xfs_scrub_fblock_process_error(struct xfs_scrub_context *sc, int whichfork, diff --git a/fs/xfs/scrub/inode.c b/fs/xfs/scrub/inode.c index e15b1bc..a807ad4 100644 --- a/fs/xfs/scrub/inode.c +++ b/fs/xfs/scrub/inode.c @@ -67,7 +67,7 @@ xfs_scrub_setup_inode( break; case -EFSCORRUPTED: case -EFSBADCRC: - return xfs_scrub_trans_alloc(sc); + return xfs_scrub_trans_alloc(sc, 0); default: return error; } @@ -75,7 +75,7 @@ xfs_scrub_setup_inode( /* Got the inode, lock it and we're ready to go. */ sc->ilock_flags = XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL; xfs_ilock(sc->ip, sc->ilock_flags); - error = xfs_scrub_trans_alloc(sc); + error = xfs_scrub_trans_alloc(sc, 0); if (error) goto out; sc->ilock_flags |= XFS_ILOCK_EXCL; diff --git a/fs/xfs/scrub/repair.c b/fs/xfs/scrub/repair.c index be30825..ab053d0 100644 --- a/fs/xfs/scrub/repair.c +++ b/fs/xfs/scrub/repair.c @@ -128,3 +128,155 @@ xfs_repair_probe( return 0; } + +/* + * Roll a transaction, keeping the AG headers locked and reinitializing + * the btree cursors. + */ +int +xfs_repair_roll_ag_trans( + struct xfs_scrub_context *sc) +{ + int error; + + /* Keep the AG header buffers locked so we can keep going. */ + xfs_trans_bhold(sc->tp, sc->sa.agi_bp); + xfs_trans_bhold(sc->tp, sc->sa.agf_bp); + xfs_trans_bhold(sc->tp, sc->sa.agfl_bp); + + /* Roll the transaction. */ + error = xfs_trans_roll(&sc->tp); + if (error) + goto out_release; + + /* Join AG headers to the new transaction. */ + xfs_trans_bjoin(sc->tp, sc->sa.agi_bp); + xfs_trans_bjoin(sc->tp, sc->sa.agf_bp); + xfs_trans_bjoin(sc->tp, sc->sa.agfl_bp); + + return 0; + +out_release: + /* + * Rolling failed, so release the hold on the buffers. The + * buffers will be released during teardown on our way out + * of the kernel. + */ + xfs_trans_bhold_release(sc->tp, sc->sa.agi_bp); + xfs_trans_bhold_release(sc->tp, sc->sa.agf_bp); + xfs_trans_bhold_release(sc->tp, sc->sa.agfl_bp); + + return error; +} + +/* + * Does the given AG have enough space to rebuild a btree? Neither AG + * reservation can be critical, and we must have enough space (factoring + * in AG reservations) to construct a whole btree. + */ +bool +xfs_repair_ag_has_space( + struct xfs_perag *pag, + xfs_extlen_t nr_blocks, + enum xfs_ag_resv_type type) +{ + return !xfs_ag_resv_critical(pag, XFS_AG_RESV_AGFL) && + !xfs_ag_resv_critical(pag, XFS_AG_RESV_METADATA) && + pag->pagf_freeblks > xfs_ag_resv_needed(pag, type) + nr_blocks; +} + +/* + * Figure out how many blocks to reserve for an AG repair. We calculate the + * worst case estimate for the number of blocks we'd need to rebuild one of + * any type of per-AG btree. + */ +xfs_extlen_t +xfs_repair_calc_ag_resblks( + struct xfs_scrub_context *sc) +{ + struct xfs_mount *mp = sc->mp; + struct xfs_scrub_metadata *sm = sc->sm; + struct xfs_perag *pag; + struct xfs_buf *bp; + xfs_agino_t icount = 0; + xfs_extlen_t aglen = 0; + xfs_extlen_t usedlen; + xfs_extlen_t freelen; + xfs_extlen_t bnobt_sz; + xfs_extlen_t inobt_sz; + xfs_extlen_t rmapbt_sz; + xfs_extlen_t refcbt_sz; + int error; + + if (!(sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR)) + return 0; + + /* Use in-core counters if possible. */ + pag = xfs_perag_get(mp, sm->sm_agno); + if (pag->pagi_init) + icount = pag->pagi_count; + xfs_perag_put(pag); + + /* + * Otherwise try to get the actual counters from disk; if not, make + * some worst case assumptions. + */ + if (icount == 0) { + error = xfs_ialloc_read_agi(mp, NULL, sm->sm_agno, &bp); + if (error) { + icount = mp->m_sb.sb_agblocks / mp->m_sb.sb_inopblock; + } else { + pag = xfs_perag_get(mp, sm->sm_agno); + icount = pag->pagi_count; + xfs_perag_put(pag); + xfs_buf_relse(bp); + } + } + + /* Now grab the block counters from the AGF. */ + error = xfs_alloc_read_agf(mp, NULL, sm->sm_agno, 0, &bp); + if (error) { + aglen = mp->m_sb.sb_agblocks; + freelen = aglen; + usedlen = aglen; + } else { + pag = xfs_perag_get(mp, sm->sm_agno); + aglen = be32_to_cpu(XFS_BUF_TO_AGF(bp)->agf_length); + freelen = pag->pagf_freeblks; + usedlen = aglen - freelen; + xfs_perag_put(pag); + xfs_buf_relse(bp); + } + + trace_xfs_repair_calc_ag_resblks(mp, sm->sm_agno, icount, aglen, + freelen, usedlen); + + /* + * Figure out how many blocks we'd need worst case to rebuild + * each type of btree. Note that we can only rebuild the + * bnobt/cntbt or inobt/finobt as pairs. + */ + bnobt_sz = 2 * xfs_allocbt_calc_size(mp, freelen); + if (xfs_sb_version_hassparseinodes(&mp->m_sb)) + inobt_sz = xfs_iallocbt_calc_size(mp, icount / + XFS_INODES_PER_HOLEMASK_BIT); + else + inobt_sz = xfs_iallocbt_calc_size(mp, icount / + XFS_INODES_PER_CHUNK); + if (xfs_sb_version_hasfinobt(&mp->m_sb)) + inobt_sz *= 2; + if (xfs_sb_version_hasreflink(&mp->m_sb)) { + rmapbt_sz = xfs_rmapbt_calc_size(mp, aglen); + refcbt_sz = xfs_refcountbt_calc_size(mp, usedlen); + } else { + rmapbt_sz = xfs_rmapbt_calc_size(mp, usedlen); + refcbt_sz = 0; + } + if (!xfs_sb_version_hasrmapbt(&mp->m_sb)) + rmapbt_sz = 0; + + trace_xfs_repair_calc_ag_resblks_btsize(mp, sm->sm_agno, bnobt_sz, + inobt_sz, rmapbt_sz, refcbt_sz); + + return max(max(bnobt_sz, inobt_sz), max(rmapbt_sz, refcbt_sz)); +} diff --git a/fs/xfs/scrub/repair.h b/fs/xfs/scrub/repair.h index 83170dd..8d181dc 100644 --- a/fs/xfs/scrub/repair.h +++ b/fs/xfs/scrub/repair.h @@ -32,6 +32,10 @@ static inline int xfs_repair_notsupported(struct xfs_scrub_context *sc) int xfs_repair_attempt(struct xfs_inode *ip, struct xfs_scrub_context *sc, bool *fixed); void xfs_repair_failure(struct xfs_mount *mp); +int xfs_repair_roll_ag_trans(struct xfs_scrub_context *sc); +bool xfs_repair_ag_has_space(struct xfs_perag *pag, xfs_extlen_t nr_blocks, + enum xfs_ag_resv_type type); +xfs_extlen_t xfs_repair_calc_ag_resblks(struct xfs_scrub_context *sc); /* Metadata repairers */ @@ -49,6 +53,14 @@ static inline int xfs_repair_attempt( static inline void xfs_repair_failure(struct xfs_mount *mp) {} +static inline xfs_extlen_t +xfs_repair_calc_ag_resblks( + struct xfs_scrub_context *sc) +{ + ASSERT(!(sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR)); + return 0; +} + #define xfs_repair_probe xfs_repair_notsupported #endif /* CONFIG_XFS_ONLINE_REPAIR */