[v3,0/2] ->total_bytes_pinned fixes for early ENOSPC issues

Message

Josef Bacik Jan. 15, 2021, 9:48 p.m. UTC

v2->v3:
- Updated the changelog in patch 2 to refer to the patchset that inspired the
  change.
- Added Nik's reviewed-by for patch 2.
v1->v2:
- Rebase onto latest misc-next.
- Added Nikolay's reviewed-by for the first patch.

--- Original email ---
Hello,

Nikolay discovered a regression with generic/371 with my delayed refs patches
applied.  This isn't actually a regression caused by those patches, but rather
those patches uncovered a problem that has existed forever, we just have papered
over it with our aggressive delayed ref flushing.  Enter these two patches.

The first patch is more of a refactoring and a simplification.  We've been
passing in a &old_refs and a &new_refs into the delayed ref code, and
duplicating the

if (old_refs >= 0 && new_refs < 0)
	->total_bytes_pinned += bytes;
else if (old_refs < 0 && new_refs >= 0)
	->total_bytes_pinned -= bytes;

logic for data and metadata.  This was made even more confusing by the fact that
we clean up this accounting when we drop the delayed ref, but also include it
when we actually pin the extents down properly.  It took me quite a while to
realize that we weren't mis-counting ->total_bytes_pinned because of how weirdly
complicated the accounting was.

I've refactored this code to make the handling distinct.  We modify it in the
delayed refs code itself, which allows us to clean up a bunch of function
arguments and duplicated code.  It also unifies how we handle the delayed ref
side of the ->total_bytes_pinned modification.  Now it's a little easier to see
who is responsible for the modification and where.

The second patch is the actual fix for the problem.  Previously we had simply
been assuming that ->total_ref_mod < 0 meant that we were freeing stuff.
However if we allocate and free in the same transaction, ->total_ref_mod == 0
also means we're freeing.  Adding that case is what fixes the problem Nikolay
was seeing.  Thanks,

Josef

Josef Bacik (2):
  btrfs: handle ->total_bytes_pinned inside the delayed ref itself
  btrfs: account for new extents being deleted in total_bytes_pinned

 fs/btrfs/block-group.c |  11 ++--
 fs/btrfs/delayed-ref.c |  60 ++++++++++++-------
 fs/btrfs/delayed-ref.h |  16 +++--
 fs/btrfs/extent-tree.c | 129 ++++++++++-------------------------------
 fs/btrfs/space-info.h  |  17 ++++++
 5 files changed, 103 insertions(+), 130 deletions(-)

Comments

David Sterba Jan. 24, 2021, 12:58 p.m. UTC | #1

On Fri, Jan 15, 2021 at 04:48:54PM -0500, Josef Bacik wrote:
> v2->v3:
> - Updated the changelog in patch 2 to refer to the patchset that inspired the
>   change.
> - Added Nik's reviewed-by for patch 2.

Thanks, the patches have been in for-next, adding v3 to misc-next.

I've tagged it for stable 5.10, though it looks like it would apply to
at least 5.4. There's a minor conflict with the way how the pinned
extents get tracked, changed in 5.7. The patchset switching to
per-transaction is not easily backportable to stable though it does not
seem to be a functionality conflict.