[PATCHSET,v6,00/12] xfs: try harder to reclaim space when we run out

Message ID	161188666613.1943978.971196931920996596.stgit@magnolia (mailing list archive)
Headers	show Return-Path: <linux-xfs-owner@kernel.org> Subject: [PATCHSET v6 00/12] xfs: try harder to reclaim space when we run out From: "Darrick J. Wong" <djwong@kernel.org> To: djwong@kernel.org Cc: Brian Foster <bfoster@redhat.com>, Christoph Hellwig <hch@lst.de>, linux-xfs@vger.kernel.org, hch@infradead.org, david@fromorbit.com, bfoster@redhat.com Date: Thu, 28 Jan 2021 18:17:46 -0800 Message-ID: <161188666613.1943978.971196931920996596.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Precedence: bulk
Series	xfs: try harder to reclaim space when we run out \| expand [PATCHSET,v6,00/12] xfs: try harder to reclaim space when we run out [01/12] xfs: trigger all block gc scans when low on quota space [02/12] xfs: don't stall cowblocks scan if we can't take locks [03/12] xfs: xfs_inode_free_quota_blocks should scan project quota [04/12] xfs: move and rename xfs_inode_free_quota_blocks to avoid conflicts [05/12] xfs: pass flags and return gc errors from xfs_blockgc_free_quota [06/12] xfs: try worst case space reservation upfront in xfs_reflink_remap_extent [07/12] xfs: flush eof/cowblocks if we can't reserve quota for file blocks [08/12] xfs: flush eof/cowblocks if we can't reserve quota for inode creation [09/12] xfs: flush eof/cowblocks if we can't reserve quota for chown [10/12] xfs: add a tracepoint for blockgc scans [11/12] xfs: refactor xfs_icache_free_{eof,cow}blocks call sites [12/12] xfs: flush speculative space allocations when we run out of space

Message ID

161188666613.1943978.971196931920996596.stgit@magnolia (mailing list archive)

Headers

Subject: [PATCHSET v6 00/12] xfs: try harder to reclaim space when we run out
From: "Darrick J. Wong" <djwong@kernel.org>
To: djwong@kernel.org
Cc: Brian Foster <bfoster@redhat.com>, Christoph Hellwig <hch@lst.de>,
        linux-xfs@vger.kernel.org, hch@infradead.org, david@fromorbit.com,
        bfoster@redhat.com
Date: Thu, 28 Jan 2021 18:17:46 -0800
Message-ID: <161188666613.1943978.971196931920996596.stgit@magnolia>
User-Agent: StGit/0.19
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk

Series

xfs: try harder to reclaim space when we run out | expand

Message

Darrick J. Wong Jan. 29, 2021, 2:17 a.m. UTC

Hi all,

Historically, when users ran out of space or quota when trying to write
to the filesystem, XFS didn't try very hard to reclaim space that it
might have speculatively allocated for the purpose of speeding up
front-end filesystem operations (appending writes, cow staging).  The
upcoming deferred inactivation series will greatly increase the amount
of allocated space that isn't actively being used to store user data.

Therefore, try to reduce the circumstances where we return EDQUOT or
ENOSPC to userspace by teaching the write paths to try to clear space
and retry the operation one time before giving up.

Previous iterations of this patchset made massive changes to the
codebase, but thanks to the transaction allocation helpers that Brian
pushed for (in the previous patchset) this is mostly no longer
necessary. :)

v2: clean up and rebase against 5.11.
v3: restructure the retry loops per dchinner suggestion
v4: simplify the calling convention of xfs_trans_reserve_quota_nblks
v5: constrain the open-coded 'goto retry' loops to the helpers created
    in the previous patchset
v6: move "xfs: try worst case space reservation upfront in
    xfs_reflink_remap_extent" to this series, open-code the qretry
    helpers in the (now very few) places they are used

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=reclaim-space-harder-5.12
---
 fs/xfs/xfs_file.c        |   24 +++---
 fs/xfs/xfs_icache.c      |  184 ++++++++++++++++++++++++++++++----------------
 fs/xfs/xfs_icache.h      |    7 +-
 fs/xfs/xfs_ioctl.c       |   12 +++
 fs/xfs/xfs_iops.c        |   11 +++
 fs/xfs/xfs_quota.h       |   16 ++--
 fs/xfs/xfs_reflink.c     |   30 +++++++-
 fs/xfs/xfs_trace.c       |    1 
 fs/xfs/xfs_trace.h       |   42 +++++++++++
 fs/xfs/xfs_trans.c       |   34 ++++++++-
 fs/xfs/xfs_trans_dquot.c |  116 ++++++++++++++++++++++++++---
 11 files changed, 370 insertions(+), 107 deletions(-)