[12/35] btrfs: add ALLOC_CHUNK_FORCE to the flushing code
diff mbox series

Message ID 20180830174225.2200-13-josef@toxicpanda.com
State New
Headers show
Series
  • My current patch queue
Related show

Commit Message

Josef Bacik Aug. 30, 2018, 5:42 p.m. UTC
With my change to no longer take into account the global reserve for
metadata allocation chunks we have this side-effect for mixed block
group fs'es where we are no longer allocating enough chunks for the
data/metadata requirements.  To deal with this add a ALLOC_CHUNK_FORCE
step to the flushing state machine.  This will only get used if we've
already made a full loop through the flushing machinery and tried
committing the transaction.  If we have then we can try and force a
chunk allocation since we likely need it to make progress.  This
resolves the issues I was seeing with the mixed bg tests in xfstests
with my previous patch.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/btrfs/ctree.h             | 3 ++-
 fs/btrfs/extent-tree.c       | 7 ++++++-
 include/trace/events/btrfs.h | 1 +
 3 files changed, 9 insertions(+), 2 deletions(-)

Comments

Nikolay Borisov Sept. 3, 2018, 2:19 p.m. UTC | #1
On 30.08.2018 20:42, Josef Bacik wrote:
> +		if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
> +			flush_state++;

This is a bit obscure. So if we allocated a chunk and !commit_cycles
just break from the loop? What's the reasoning behind this ?
Josef Bacik Sept. 4, 2018, 5:57 p.m. UTC | #2
On Mon, Sep 03, 2018 at 05:19:19PM +0300, Nikolay Borisov wrote:
> 
> 
> On 30.08.2018 20:42, Josef Bacik wrote:
> > +		if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
> > +			flush_state++;
> 
> This is a bit obscure. So if we allocated a chunk and !commit_cycles
> just break from the loop? What's the reasoning behind this ?

I'll add a comment, but it doesn't break the loop, it just goes to COMMIT_TRANS.
The idea is we don't want to force a chunk allocation if we're experiencing a
little bit of pressure, because we could end up with a drive full of empty
metadata chunks.  We want to try committing the transaction first, and then if
we still have issues we can force a chunk allocation.  Thanks,

Josef
Nikolay Borisov Sept. 4, 2018, 6:22 p.m. UTC | #3
On  4.09.2018 20:57, Josef Bacik wrote:
> On Mon, Sep 03, 2018 at 05:19:19PM +0300, Nikolay Borisov wrote:
>>
>>
>> On 30.08.2018 20:42, Josef Bacik wrote:
>>> +		if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
>>> +			flush_state++;
>>
>> This is a bit obscure. So if we allocated a chunk and !commit_cycles
>> just break from the loop? What's the reasoning behind this ?
> 
> I'll add a comment, but it doesn't break the loop, it just goes to COMMIT_TRANS.
> The idea is we don't want to force a chunk allocation if we're experiencing a
> little bit of pressure, because we could end up with a drive full of empty
> metadata chunks.  We want to try committing the transaction first, and then if
> we still have issues we can force a chunk allocation.  Thanks,

I think it will be better if this check is moved up somewhere before the
the if (flush_state > commit trans).

> 
> Josef
>

Patch
diff mbox series

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 0a4e55703d48..791e287c2292 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2731,7 +2731,8 @@  enum btrfs_flush_state {
 	FLUSH_DELALLOC		=	5,
 	FLUSH_DELALLOC_WAIT	=	6,
 	ALLOC_CHUNK		=	7,
-	COMMIT_TRANS		=	8,
+	ALLOC_CHUNK_FORCE	=	8,
+	COMMIT_TRANS		=	9,
 };
 
 int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 783341e3653e..22e1f9f55f4f 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4907,6 +4907,7 @@  static void flush_space(struct btrfs_fs_info *fs_info,
 		btrfs_end_transaction(trans);
 		break;
 	case ALLOC_CHUNK:
+	case ALLOC_CHUNK_FORCE:
 		trans = btrfs_join_transaction(root);
 		if (IS_ERR(trans)) {
 			ret = PTR_ERR(trans);
@@ -4914,7 +4915,9 @@  static void flush_space(struct btrfs_fs_info *fs_info,
 		}
 		ret = do_chunk_alloc(trans,
 				     btrfs_metadata_alloc_profile(fs_info),
-				     CHUNK_ALLOC_NO_FORCE);
+				     (state == ALLOC_CHUNK) ?
+				     CHUNK_ALLOC_NO_FORCE :
+				     CHUNK_ALLOC_FORCE);
 		btrfs_end_transaction(trans);
 		if (ret > 0 || ret == -ENOSPC)
 			ret = 0;
@@ -5060,6 +5063,8 @@  static void btrfs_async_reclaim_metadata_space(struct work_struct *work)
 			}
 		}
 		spin_unlock(&space_info->lock);
+		if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
+			flush_state++;
 	} while (flush_state <= COMMIT_TRANS);
 }
 
diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h
index 7d205e50b09c..fdb23181b5b7 100644
--- a/include/trace/events/btrfs.h
+++ b/include/trace/events/btrfs.h
@@ -1051,6 +1051,7 @@  TRACE_EVENT(btrfs_trigger_flush,
 		{ FLUSH_DELAYED_REFS_NR,	"FLUSH_DELAYED_REFS_NR"},	\
 		{ FLUSH_DELAYED_REFS,		"FLUSH_ELAYED_REFS"},		\
 		{ ALLOC_CHUNK,			"ALLOC_CHUNK"},			\
+		{ ALLOC_CHUNK_FORCE,		"ALLOC_CHUNK_FORCE"},		\
 		{ COMMIT_TRANS,			"COMMIT_TRANS"})
 
 TRACE_EVENT(btrfs_flush_space,