From patchwork Thu Aug 15 19:37:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krister Johansen X-Patchwork-Id: 13765108 Received: from buffalo.tulip.relay.mailchannels.net (buffalo.tulip.relay.mailchannels.net [23.83.218.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5F1B79B87 for ; Thu, 15 Aug 2024 19:37:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=23.83.218.24 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723750668; cv=pass; b=Dbyv4ddU60yXtmkevyjvH5+MYRR/PFuD0HhZkA3h3uVgFJCsyIm3lMomH+XNYTlbt4T5RgM6BXGajDmTqkA15UtLnB3vhkP87YVLQl0UhKTMn6vET/qmMhhj3eNWojfudVlhGLe/+eKncKmw/jLxiSItpWWMHJ86uVaPKhgimv8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723750668; c=relaxed/simple; bh=UU3XHi7Hih7iaFgEwTyXlMxqZdauo5OmThWTVmv7d4A=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=HKn+w5t8YxyFLDfPO/jcrERcJt2CsSuPC7Ak0BAo1fV1NIOwCZxqTzmbJRnvceY2Fsxose0Pgx/1tFVmxoHHA56BoxG5VqIAZbNnVR1NnqD7OXPJhnwK6LNfsqS+h78OXlC6qtO1fbSmHkFx7WXlNm4lRySWFlAfxiyhbzBfC7g= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=templeofstupid.com; spf=pass smtp.mailfrom=templeofstupid.com; dkim=pass (2048-bit key) header.d=templeofstupid.com header.i=@templeofstupid.com header.b=UmyL015B; arc=pass smtp.client-ip=23.83.218.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=templeofstupid.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=templeofstupid.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=templeofstupid.com header.i=@templeofstupid.com header.b="UmyL015B" X-Sender-Id: dreamhost|x-authsender|kjlx@templeofstupid.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 42C32C630B for ; Thu, 15 Aug 2024 19:37:45 +0000 (UTC) Received: from pdx1-sub0-mail-a210.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id E53E0C50D3 for ; Thu, 15 Aug 2024 19:37:44 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1723750664; a=rsa-sha256; cv=none; b=Dzf+ADxtbJgIfr/B+nIp1cKbNjeo8PHtBLiSdJMP8TBej553FZRmAwOuCMkzUkdcGeagod XJRvPEqw1j3Wou0B3QDLhi2sKjz5HbDk6IX+U5OoqB69kNuH6pqi38E/2lMuDfABjJ+oHi 11a4nkf6jjM0a+sKp/Pt8Xow0YhDcAir8CfJzdMpLz4TclJl0NrS2tzSwHKpjBg/fqcsnh zqa0MMPgaUh+EVNxqSOWaP/G4uu/r24u73s6amnOHxyJ54zubxfAKWehNf/kzhOni48w6B tlLVNTlW2ihQUHJ+YOSKMlp39swg0zZxljycmhdKLEHnJ4MfRfvqGLlm2rEoBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1723750664; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ifarn2MUWQYbQq5OO7YyBq3UvZOk0iHxIPqUL1z3Xq0=; b=vo7qBtLHq7GNMTEoxHll6YBBuFIgLqiQAikDsocg+wrSJhDFn+t3eXo9F9MRJyUgqscIK3 HHfStLf+UFfui9fgfq2SDlt+hl/0RhO8MJtNGP+yYOfSnnJK/ogEOu5AQOaZ7By7/4RbOy pKOY7XaGt9k8HzwLfEoXXBLtt+jFHdYplI0w+da8Xpj4benaua3J4r70rUWDc454YuMC0C V/Hf7omxe1INPXhExntqEPTLQu2gJIJIT6NAQUkcolgJuLeA6X6ThsgmlgSs38Nmy6RawB J6txaJMEFo1R372/dEJSQ/kBoERiAt4Hvt8nR1rWV1BQDqVvdY9x7vuF8eFsUQ== ARC-Authentication-Results: i=1; rspamd-587694846-xgzwp; auth=pass smtp.auth=dreamhost smtp.mailfrom=kjlx@templeofstupid.com X-Sender-Id: dreamhost|x-authsender|kjlx@templeofstupid.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|kjlx@templeofstupid.com X-MailChannels-Auth-Id: dreamhost X-Abaft-Dime: 3de8a55f08a95a1a_1723750665190_729296425 X-MC-Loop-Signature: 1723750665190:2387654369 X-MC-Ingress-Time: 1723750665190 Received: from pdx1-sub0-mail-a210.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.117.75.39 (trex/7.0.2); Thu, 15 Aug 2024 19:37:45 +0000 Received: from kmjvbox.templeofstupid.com (c-73-70-109-47.hsd1.ca.comcast.net [73.70.109.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kjlx@templeofstupid.com) by pdx1-sub0-mail-a210.dreamhost.com (Postfix) with ESMTPSA id 4WlFlD4fLKzNs for ; Thu, 15 Aug 2024 12:37:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=templeofstupid.com; s=dreamhost; t=1723750664; bh=Ifarn2MUWQYbQq5OO7YyBq3UvZOk0iHxIPqUL1z3Xq0=; h=Date:From:To:Cc:Subject:Content-Type; b=UmyL015BIesdMdR3Dg7/GoY9ZM2JRvK78nt4ze1y+oPZcjWBjNNwEpPKR/sQ9EwLt 1b9jWrkx7eDzwmR05KfsIXSmb1t+q+MMlPmdnubElhY/g1kOob/mbCozj3XxB3Nqql Fgk2zXIJptZSpwOUJ4VZTBLnx3LbeT+x2OZxKb1DSrCnNtQuiQM87tWtzyuiFostNc Fi4xvnGYM3bJp6WGqmDULRuBxo82QxA0f34LanlWtzQfSX817tn8xZAqvQAD1rV7r1 o7xm/qvDmRQJuiTtP1b/bN3gj26+GI2OBQwddveLcRWD4uw4KvhDgUjwPmoasV0aNp 4513iBONQbRGg== Received: from johansen (uid 1000) (envelope-from kjlx@templeofstupid.com) id e0064 by kmjvbox.templeofstupid.com (DragonFly Mail Agent v0.12); Thu, 15 Aug 2024 12:37:43 -0700 Date: Thu, 15 Aug 2024 12:37:43 -0700 From: Krister Johansen To: Chandan Babu R , "Darrick J. Wong" , Dave Chinner Cc: Dave Chinner , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org Subject: [RFC PATCH 1/2] xfs/608: a test case for xfs_bmap_extents_to_btree allocation failure Message-ID: <93406e287e0def5605754f053d8680b11a4a44dd.1723690703.git.kjlx@templeofstupid.com> References: Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Add a test case that reproduces the xfs_bmap_extents_to_btree warnings that can be seen when a filesystem runs out of space while performing a dependent allocation. This test should be 100% reproducible on older kernels, prior to the AG-aware allocator re-write. The test runs a busy work job to keep another AG occupied in order to trigger the problem regardless of kernel version. However, this is only partially successful. On newer kernels, wiht the AG-aware allocator, this test now triggers the failure around 40-50% of the time on the author's test machine. Signed-off-by: Krister Johansen --- tests/xfs/608 | 372 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/608.out | 2 + 2 files changed, 374 insertions(+) create mode 100755 tests/xfs/608 create mode 100644 tests/xfs/608.out diff --git a/tests/xfs/608 b/tests/xfs/608 new file mode 100755 index 00000000..9db8d21f --- /dev/null +++ b/tests/xfs/608 @@ -0,0 +1,372 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024 YOUR NAME HERE. All Rights Reserved. +# +# FS QA Test 608 +# +# This test reproduces the xfs_bmap_extents_to_btree WARN that occurs when XFS +# fails to allocate a block for the new b-tree in the desired AG. This should +# reproduce the issue on kernels that predate the fix for the AG-aware extent +# allocator (< 6.3). +# +. ./common/preamble +_begin_fstest dangerous insert prealloc punch + +# Override the default cleanup function. + _cleanup() +{ + cd / + _destroy_loop_device $LOOP_DEV + rm -f $tmp.* $LOOP_FILE +} + +declare -a workers + +busy_work() +{ + while : + do + $XFS_IO_PROG -f -c "falloc 0 $BLOCK_SIZE" $BUSY_FILE + $XFS_IO_PROG -f -c "fpunch 0 $BLOCK_SIZE" $BUSY_FILE + done +} + +kill_busy_procs() +{ + for pid in ${workers[@]}; do + kill $pid + wait $pid + done + # Despite killing the workers and waiting for them to exit, we still + # sometimes get an EBUSY unmounting the loop device. Wait a second + # before returning to give lingering refcounts a chance to reach zero. + sleep 1 +} + +find_freesp() +{ + umount $LOOP_MNT + local freesp=$($XFS_DB_PROG $LOOP_DEV -c "agf 1" -c "print freeblks" \ + | awk '{print $3}') + mount -o nodiscard $LOOP_DEV $LOOP_MNT + echo $freesp +} + +find_biggest_freesp() +{ + umount $LOOP_MNT + local freesp=$($XFS_DB_PROG $LOOP_DEV -c 'agf 1' -c 'addr cntroot' \ + -c btdump | sed -rn 's/^[0-9]+:\[[0-9]+,([0-9]+)\].*/\1/p' \ + | tail -1) + mount -o nodiscard $LOOP_DEV $LOOP_MNT + echo $freesp +} + + +# Import common functions. +. ./common/filter + +# real QA test starts here + +# Modify as appropriate. +_supported_fs xfs +_require_test +_require_xfs_io_command "falloc" +_require_xfs_io_command "finsert" +_require_xfs_io_command "fpunch" + +# Require loop devices so that this test can create a small filesystem with +# a specific geometry that assists in making this easier to reproduce. +_require_loop + +LOOP_FILE=$TEST_DIR/$seq.img +LOOP_MNT=$TEST_DIR/$seq.mnt +mkdir -p $LOOP_MNT +$XFS_IO_PROG -ft -c "truncate 2g" $LOOP_FILE >> $seqres.full +LOOP_DEV=`_create_loop_device $LOOP_FILE` +loop_mkfs_addl_opts= + +$MKFS_XFS_PROG 2>&1 | grep -q rmapbt && \ + loop_mkfs_addl_opts="$loop_mkfs_addl_opts,rmapbt=0" +$MKFS_XFS_PROG 2>&1 | grep -q reflink && \ + loop_mkfs_addl_opts="$loop_mkfs_addl_opts,reflink=1" + +_mkfs_dev "-b size=4096 -m crc=1$loop_mkfs_addl_opts -d size=1708m,agcount=2" \ + " -l size=1986b" $LOOP_DEV >> $seqres.full +# nodiscard makes the unbusying of extents more predictable which makes the test +# repeatable. +_mount -o nodiscard $LOOP_DEV $LOOP_MNT + +BLOCK_SIZE=$(_get_file_block_size $LOOP_MNT) + +# Add a directory under the root of LOOP_MNT in order to ensure the files placed +# there end up in AG 1. + +TEST_SUBDIR=testdir/testsubdir +mkdir -p $LOOP_MNT/$TEST_SUBDIR + +# There are 3 files in this test. The first is allocated to consume most of the +# space in the AG. The test then punches holes in the file in order to allow +# the second file to allocate the fragmented blocks as individual extents. The +# second file is written such that it is only a couple of operations away from +# being converted from in-line extents to a b-tree in its bmbt. The third file +# simply exists for us to allocate spare blocks to in order to take the AG right +# up against ENOSPC. + +FILE1=$LOOP_MNT/$TEST_SUBDIR/file1 +FILE2=$LOOP_MNT/$TEST_SUBDIR/file2 +FILE3=$LOOP_MNT/$TEST_SUBDIR/file3 + +# BUSY_FILE is here to keep AG0 busy on versions of XFS where the allocator is +# allowed to check lower numbered AGs if it fails in a higher numbered one. +BUSY_FILE=$LOOP_MNT/testdir/busyfile + +# Calculate the number of extents we need in the cnobt and bnobt. This should +# be the maximum b-tree leaf size minus one, so that after adding two extents a +# split is triggered. + +# The test is currently set to always use CRC, otherwise this would be 56 if CRC +# and 16 if not. +alloc_block_len=56 +allocbt_leaf_maxrecs=$(((BLOCK_SIZE - alloc_block_len) / 8)) + +# Look at the attrfork offset in FILE2's inode in order to determine the number +# of extents before this splits to a b-tree. This test assumes a v5 filesystem +# so if forkoff is zero, it falls back to LITINO of 336 and uses a bmbt_rec_size +# of 16. +touch $FILE2 +file2_inode=$(stat -c '%i' "$FILE2") +# Temporarily unmount while parameters are gathered via xfs_db +umount $LOOP_MNT +forkoff=$($XFS_DB_PROG $LOOP_DEV -c "inode $file2_inode" \ + -c "print core.forkoff" | awk '{print $3}') +freeblks=$($XFS_DB_PROG $LOOP_DEV -c "agf 1" \ + -c "print freeblks" | awk '{print $3}') +mount -o nodiscard $LOOP_DEV $LOOP_MNT + +# We'll recreate FILE2 later. For now be as empty as we can. +rm $FILE2 + +# Some versions of xfs_db contain the agresv command, but not all do. +# Additionally, some parameters about how much space is actually allocatable +# aren't visible from xfs_db. Tracepoints have been helpful in figuring this +# out when developing the test by hand. Instead of trying to parse ftrace data +# and hope that the right tracepoints are available, brute force the actual +# allocatable maximum size by repeatedly trying to allocate larger offsets +# subtracted from $freeblks until one succeeds. +for (( i = 0, ag = 0; ; i++ )) +do + $XFS_IO_PROG -f -c "falloc 0 $(( (freeblks-i)*BLOCK_SIZE ))" $FILE1 + ag=$(xfs_bmap -v -n 1 $FILE1 | tail -1 | awk '{print $4}') + rm $FILE1 + (( ag == 1 )) && break +done + +# Let free'd extents unbusy +sleep 30 + +# At this point, $i is one larger than whatever the allocator thinks the maximum +# available space is. This is based upon the asssumption that the data +# allocation we made above set minleft = 1, so the allocation that finally fit +# into AG 1 has had any reservation withheld along with the space the allocator +# requested be withheld for any bmbt expansion. +freeblkseff=$((freeblks - i - 1)) +blocks_withheld=$((i+1)) + +iforksize=$((forkoff > 0 ? forkoff * 8 : 336)) +maxextents=$((iforksize / 16)) +# We'll need to allocate maxextents - 1 for this test, so that the last +# allocation to the file forces an extents -> b-tree conversion. +wanted_extents=$((maxextents - 1)) + +# The first allocation is just a big chunk into the first file. Grab the +# majority of the free blocks. +first_alloc_blocks=$((freeblkseff - 8192)) + +$XFS_IO_PROG -f -c "falloc 0 $((BLOCK_SIZE * first_alloc_blocks))" $FILE1 >> \ + $seqres.full + +# Insert space in the middle in order to help the allocator pick sequential +# blocks when we add space back later. If we don't do this, then it can break +# up larger extents instead of grabbing the ones we fpunch'd out. +# +# The insert offset was chosen arbitrarily and placed towards the beginning of +# the file for the conveinence of humans. The insert_blocks size needs to be +# larger than the space we'll later punch out of file 2 and insert back into +# file 1. This is on the order of 7 blocks, so 512 should always be large +# enough. +first_insert_offset=$((BLOCK_SIZE * 2083)) +first_insert_blocks=$((BLOCK_SIZE * 512)) + +$XFS_IO_PROG -f -c "finsert $first_insert_offset $first_insert_blocks" \ + $FILE1 >> $seqres.full + +# Punch 3-block holes into the file. This number was chosen so that we could +# re-allocate blocks from these chunks without causing the extent to get removed +# from the free-space btrees. +# +# Punch enough holes to ensure that the bnobt and cnobt end up two extents away +# from a b-tree split, and overshoot that value by the number we need to consume +# in the second file to end up with wanted_extents - 1. +num_holes=$(((allocbt_leaf_maxrecs-2) + (wanted_extents - 1))) +end_hole_offset=$((num_holes * 4 - 3)) +hole_blocks=$((BLOCK_SIZE * 3)) + +for i in $(seq 1 4 $end_hole_offset); do + $XFS_IO_PROG -f -c "fpunch $((BLOCK_SIZE * i)) $hole_blocks" \ + $FILE1 >> $seqres.full +done + +# Use the newly created holes to create extents in our victim file. The goal is +# to allocate up to the point of b-tree conversion minus 2. The remaining space +# is placed in the n-1 extent, and then the last is reserved for the split we +# trigger later. The holes are placed after a gap that's left towards the front +# of the file to allocate the rest of the space. This is done to get the +# allocator to give us the contiguous free chunk that would have previously been +# occupied by the per-AG reservation's free space. +alloc_hole_seq=$(((wanted_extents - 1) * 4 - 3)) + +# The offset for the placement of the holes needs to be after the remaining +# freespace chunk so calculate how big that needs to be first. We may need to +# recalculate this value to account for blocks freed from the AGFL later. +biggest_freesp=$(find_biggest_freesp) + +# 3x the biggest chunk of free blocks should be a big enough gap +hole_offset=$((biggest_freesp * 3)) + +for i in $(seq 1 4 $alloc_hole_seq); do + $XFS_IO_PROG -f \ + -c "falloc $((BLOCK_SIZE * (i+hole_offset))) $hole_blocks" \ + $FILE2 >> $seqres.full +done + +# Attempt to compensate for any late-breaking over/undershoot in the desired +# extent count by checking the number of extents in the bnobt and adding or +# removing space to try to arrive at the desired number. +umount $LOOP_MNT +current_extents=$($XFS_DB_PROG $LOOP_DEV -c 'agf 1' -c 'addr bnoroot' \ + -c 'btdump' | grep recs | sed -rn 's/^recs\[.*\-([0-9]+)\].*/\1/p' | \ + awk '{a +=int($1)} END{printf("%d\n", a);}') +mount -o nodiscard $LOOP_DEV $LOOP_MNT + +wanted_allocbt=$((allocbt_leaf_maxrecs-2)) +if [[ $current_extents -gt $wanted_allocbt ]]; then + ext_diff=$(( current_extents - wanted_allocbt )) + end_offset=$(( ext_diff * 4 - 3 )) + for i in $(seq 1 4 $end_offset); do + $XFS_IO_PROG -f -c "falloc $((BLOCK_SIZE * i)) $hole_blocks" \ + $FILE1 >> $seqres.full + done +elif [[ $current_extents -lt $wanted_allocbt ]]; then + ext_diff=$(( wanted_allocbt - current_extents )) + end_offset=$(( (ext_diff * 4 - 3) + end_hole_offset )) + for i in $(seq $end_hole_offset 4 $end_offset); do + $XFS_IO_PROG -f -c "fpunch $((BLOCK_SIZE * i )) $hole_blocks" \ + $FILE1 >> $seqres.full + done +fi + +# The previous falloc should have triggered a reverse-split of the freespace +# btrees. The next alloc should cause the freelist to be drained. Recompute +# the available freespace with the understanding that we'll need to do this +# again after the AGFL is trimmed by the next allocation. Leave a few blocks +# free so that we can use FILE3 to create the last needed set of free extents +# before triggering a split while simultaneously using the remaining space. +freesp_remaining=$(find_freesp) +f2_alloc_blocks=$((freesp_remaining - blocks_withheld - 10)) + +$XFS_IO_PROG -f -c "falloc 0 $((BLOCK_SIZE * f2_alloc_blocks))" \ + $FILE2 >> $seqres.full + +# Recompute the remaining blocks and let FILE3 consume the remainder of the +# space. This is intended to both leave one more free extent in the btrees and +# take us down to being right before ENOSPC. +freesp_remaining=$(find_freesp) +f3_alloc_blocks=$((freesp_remaining - blocks_withheld)) +biggest_freesp=$(find_biggest_freesp) + +# Due to variance outside of the control of the test, the remaining freespace +# may be broken into smaller chunks than it's possible to allocate in a single +# attempt. If the test tries to allocate one big chunk, that allocation will +# fail and consult the next AG. To prevent that from happening, check the size +# of the remaining freespace in AG1 and break this allocation into smaller +# chunks that a) consume space from AG1 and b) do not cause the extents we've +# carefully added to the freespace trees to get removed. +if [[ $f3_alloc_blocks -lt $biggest_freesp ]]; then + $XFS_IO_PROG -f -c "falloc 0 $((BLOCK_SIZE * f3_alloc_blocks))" \ + $FILE3 >> $seqres.full +else + alloc_left=$f3_alloc_blocks + alloc_blocks=$((biggest_freesp - 1)) + alloc_ofst=0 + while ((alloc_left > 0)); do + size=$((alloc_blocks * BLOCK_SIZE)) + $XFS_IO_PROG -f -c "falloc $alloc_ofst $size" \ + $FILE3 >> $seqres.full + alloc_left=$((alloc_left - alloc_blocks)) + alloc_ofst=$((alloc_ofst + (1000*BLOCK_SIZE))) + biggest_freesp=$(find_biggest_freesp) + if [[ $alloc_left -lt $biggest_freesp ]]; then + alloc_blocks=$alloc_left + else + alloc_blocks=$((biggest_freesp - 1)) + fi + done +fi + +# That's it for the setup. Now the test punches out a 12 block extent as one 6 +# block chunk in the middle, followed by two 3 block chunks on either side. It +# sleeps after the 6 block chunk so that portion of the extent will un-busy, but +# the 3 block chunks on either side stay (temporarily) unavailable. While the +# chunks on either side are busy, re-allocate some of the space that's been +# free'd back to FILE1 so that the final falloc to FILE2 brings us to ENOSPC. +f2_off=2560 +$XFS_IO_PROG -f -c "fpunch $((BLOCK_SIZE * f2_off)) $((BLOCK_SIZE * 6))" \ + $FILE2 >> $seqres.full +# Before we finish punching the final holes, start up some busy workers to keep +# the _other_ AG's locks contended. This isn't needed to reproduce the problem +# prior to the AG-aware allocator's arrival. Ncpu * 4 has been successful at +# reproducing the problem in places where a lower number of workers succeeds +# intermittently (or not at all). +ncpu=$(nproc) +for ((i=0 ; i < ncpu*4; i++)); do + busy_work & + workers[$i]=$! +done + +# Wait for first fpunch to unbusy, and then continue with remainder. +sleep 30 +$XFS_IO_PROG -f -c "fpunch $((BLOCK_SIZE * (f2_off + 6))) $((BLOCK_SIZE * 3))" \ + $FILE2 >> $seqres.full +$XFS_IO_PROG -f -c "fpunch $((BLOCK_SIZE * (f2_off - 3))) $((BLOCK_SIZE * 3))" \ + $FILE2 >> $seqres.full + +# Put 7 blocks back into FILE1 to consume some of the space free'd above. +# The offset here was picked so that the allocator takes blocks from the 3 block +# chunks we punched earlier, but leaves the extents intact in the freespace +# trees. +f1_off=1956 +f1_wanted_blocks=7 +f1_alloc_seq=$((f1_wanted_blocks * 4 - 3)) +for i in $(seq 1 4 $f1_alloc_seq); do + $XFS_IO_PROG -f -c "falloc $((BLOCK_SIZE * (i+f1_off))) $BLOCK_SIZE" \ + $FILE1 >> $seqres.full +done + +# This next falloc should result in FILE2's bmbt getting converted from extents +# to btree while simultaneously splitting the bnotbt and cnobt. The first +# allocation succeeds, splits the free space trees, consumes all the blocks in +# the agfl, and leaves us in a situation where the second allocation to convert +# from extents to a btree fails. +$XFS_IO_PROG -f -c "falloc $((BLOCK_SIZE * f2_off)) $((BLOCK_SIZE * 5))" \ + $FILE2 >> $seqres.full + +# Terminate the busy workers or else umount will return EBUSY. +kill_busy_procs + +umount $LOOP_MNT + +# success, all done +echo "Silence is golden" +status=0 +exit diff --git a/tests/xfs/608.out b/tests/xfs/608.out new file mode 100644 index 00000000..1e534458 --- /dev/null +++ b/tests/xfs/608.out @@ -0,0 +1,2 @@ +QA output created by 608 +Silence is golden From patchwork Thu Aug 15 19:40:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krister Johansen X-Patchwork-Id: 13765110 Received: from weasel.tulip.relay.mailchannels.net (weasel.tulip.relay.mailchannels.net [23.83.218.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFF801DDF5 for ; Thu, 15 Aug 2024 19:40:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=23.83.218.247 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723750811; cv=pass; b=kCMM6qntGzCW0GSMVOs9JqInnDLTQcrP6OM0C1CnuuF9DPZrvjSDD4bnejruA1lZHhLQa+2GV5pyGQIFe4VmZ652Ljt0gyrLQ3sAqptaCMbckX+3WjI8VQOqlGSRMWGNyxdxbKUq6tSUXByPPm4luIlvIQoM8plh3AuhbplZrlQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723750811; c=relaxed/simple; bh=DYv0XX5TH/96rVPN7kpkY+c8IJDR1Wxw81yxlCaYMZE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Y/n17kLOOE8iCLjia6c5Rp3HjV379I0EOibnUkHSD+5rRxpuHVeJnA3wMA58lfoLwcgAG+nOtka6Lv6iVMyntloI5Z4AZciP13484GC7uxWgArBmqw+AO2Sla/pdDIk0du6bNeyUiRUfcTgwTd71MeI58zdsttsYP6JrXUP3t7Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=templeofstupid.com; spf=pass smtp.mailfrom=templeofstupid.com; dkim=pass (2048-bit key) header.d=templeofstupid.com header.i=@templeofstupid.com header.b=qLH8BhP1; arc=pass smtp.client-ip=23.83.218.247 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=templeofstupid.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=templeofstupid.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=templeofstupid.com header.i=@templeofstupid.com header.b="qLH8BhP1" X-Sender-Id: dreamhost|x-authsender|kjlx@templeofstupid.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 6CBC54C581F for ; Thu, 15 Aug 2024 19:40:09 +0000 (UTC) Received: from pdx1-sub0-mail-a210.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 22AE14C58C8 for ; Thu, 15 Aug 2024 19:40:09 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1723750809; a=rsa-sha256; cv=none; b=aq63N73MnK3T7RIax7bB2tBarFKhYw9ojzu61QCNZzzSU+sVqPaWJHlRjTNcN6M8ChW40b JSFGgMyAT0vwls7y8xo6xp2gpbzxGDc0gW8+OH8cWpM9bF5j5QWs0par6H/3g8+i0osK+3 4GJtgQdvGSc22dRV3yTDCl9P56OpYCx7nQwuPccBNHT3fTRLi4Smp2mG84hY2FZTAnmmac 8kqxKhDZdcZVd0AC9wDONUwJBhYGv4VEYwPwRMi0jOJiomMUsIurau/eD658/NlBzXkCRE mmHZhQNeU1N++wIMubYT4TSX9oXDi5HvGzNybuX1NdNNXBrJcbwAuhIcEhixng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1723750809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pcTxIWGPeu1R01B3vemkbii6+5znxxejEkyXOzhEKmU=; b=Ejl/gwvr34ZjhsibdSpe6gnxSKkoEQtCiIJ320UfHEIET8Pc1rQAeonM7MyLeXn/g3NaRo x5CWa7f+AvYoUhJA3gIJuhtrj5uN2A3LtC7EhfZhSXDRXOkx7adgRsrsL8sCUm7BsHuX1Q AI/1y8TvVzZP3ZcEVgCRNfQ1+oe2B01/VDQqkkt+RP/y8d76Ygdqm0F/cr3yj7boq9Qjhh uHGwYSjrt8tNrLg+W3pCLx8bQooLJUuH5CcsOGS6kNbgCFswUIW/ZPI2BZNRmphzlCv7rf Zb7RchNbsjg+twIp/FjqhNmmry0TY9eOOmSpkep2SlyqynmQHsmA6e+/9VDm1A== ARC-Authentication-Results: i=1; rspamd-587694846-g59mf; auth=pass smtp.auth=dreamhost smtp.mailfrom=kjlx@templeofstupid.com X-Sender-Id: dreamhost|x-authsender|kjlx@templeofstupid.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|kjlx@templeofstupid.com X-MailChannels-Auth-Id: dreamhost X-Army-Cooperative: 304b302e7a94489c_1723750809358_910484132 X-MC-Loop-Signature: 1723750809358:1763193105 X-MC-Ingress-Time: 1723750809358 Received: from pdx1-sub0-mail-a210.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.117.75.39 (trex/7.0.2); Thu, 15 Aug 2024 19:40:09 +0000 Received: from kmjvbox.templeofstupid.com (c-73-70-109-47.hsd1.ca.comcast.net [73.70.109.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kjlx@templeofstupid.com) by pdx1-sub0-mail-a210.dreamhost.com (Postfix) with ESMTPSA id 4WlFp04MNdzT6 for ; Thu, 15 Aug 2024 12:40:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=templeofstupid.com; s=dreamhost; t=1723750808; bh=pcTxIWGPeu1R01B3vemkbii6+5znxxejEkyXOzhEKmU=; h=Date:From:To:Cc:Subject:Content-Type; b=qLH8BhP1IV9VF75GQ8se/mGUGeUpGZtXpA1kWGt3O+YPraUSVS6wbO1KRHM9Hg7Lf G0qYpbkc7h6Hf3uJhdk1aOPzERZ7VOA9LwpJWGAb1uUXMmv2ooy64I8WxBdvsdGKzi URN+gaHOXw/efANGUg6EDVIGad1mm4dF2KnlUg1gGshG6NqFEtOiqiFyXRYrDIOL4d OqSSighHaLuJzHT7JIaAqT6q3evzyIkXnkxdghc9YGCM+36AG/irQBFKJPzYaMKHKS dgYKsWWkQaBjl161dxko0Bvuvv56QTVQEktsNYTDIqVXIwxa423Db4CKUV4ns0K287 mmQLKwNB3BsCw== Received: from johansen (uid 1000) (envelope-from kjlx@templeofstupid.com) id e0064 by kmjvbox.templeofstupid.com (DragonFly Mail Agent v0.12); Thu, 15 Aug 2024 12:40:07 -0700 Date: Thu, 15 Aug 2024 12:40:07 -0700 From: Krister Johansen To: Chandan Babu R , "Darrick J. Wong" , Dave Chinner Cc: Dave Chinner , Zorro Lang , linux-xfs@vger.kernel.org, fstests@vger.kernel.org Subject: [RFC PATCH 2/2] xfs/306: update resblks to account for increased per-ag reservation size Message-ID: <7a1a37011ed851a2dfdb8ff62e9e60236253580a.1723690703.git.kjlx@templeofstupid.com> References: Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The AGFL reservation patches increase the amount of reserved space that this test needs in order to succeed. Modify the resblks xfs_io call to account for the increased per-AG reservation. Without this change, the dd in the test gets an ENOSPC in the path where a delalloc is converted to a real allocation. This results in the test spinning in the dd command and generating endless xfs_discard_folio warnings. Since resblks is supposed to prevent the filesystem from getting empty enough to hit a case like this, increase it slightly to account for the modified per-ag reservation size. Signed-off-by: Krister Johansen --- tests/xfs/306 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/xfs/306 b/tests/xfs/306 index e21a5622..9acace85 100755 --- a/tests/xfs/306 +++ b/tests/xfs/306 @@ -47,7 +47,7 @@ for i in $(seq 0 3); do done # consume and fragment free space -$XFS_IO_PROG -xc "resblks 16" $SCRATCH_MNT >> $seqres.full 2>&1 +$XFS_IO_PROG -xc "resblks 17" $SCRATCH_MNT >> $seqres.full 2>&1 dd if=/dev/zero of=$SCRATCH_MNT/file bs=4k >> $seqres.full 2>&1 $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/file >> $seqres.full 2>&1 $here/src/punch-alternating $SCRATCH_MNT/file