| Message ID | 20220321115848.30397-1-p.raghav@samsung.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v3] btrfs/237: Use zone cap instead of zone size in fill_size and rest calculation | expand |
friendly ping.... On 2022-03-21 12:58, Pankaj Raghav wrote: > This test will break when zone capacity != zone size because the > calculation of the size to be filled is done using zone_size instead of > the actual capacity available per zone. Fix it by using zone capacity. > > As a zoned device can have variable capacity, use the btrfs utility to > get the zone capacity from the first data block group that the test will > be performed on. > > The support to extract zone capacity was added to blkzone only from > version 2.37. So zcap will be used only when the blkzone version is > greater or equal to 2.37. > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > --- > Changes since v1: > - Add a filter that can append a 'cap' column for blkzone version <2.37 > (Naohiro) > - Extract the capacity from the zone on which the test is performed as a > zoned device can have variable zone capacity(Naohiro) > > Changes since v2: > - Use btrfs inspect-internal utility to find the first data zone address > instead of hardcoding. (Naohiro) > > common/filter | 13 +++++++++++++ > tests/btrfs/237 | 21 +++++++++++++++++---- > 2 files changed, 30 insertions(+), 4 deletions(-) > > diff --git a/common/filter b/common/filter > index 257227c2..5fe86756 100644 > --- a/common/filter > +++ b/common/filter > @@ -634,5 +634,18 @@ _filter_bash() > sed -e "s/^bash: line 1: /bash: /" > } > > +# > +# blkzone report added zone capacity to be printed from v2.37. > +# This filter will add an extra column 'cap' with the same value of > +# 'len'(zone size) for blkzone version < 2.37 > +# > +# Before: start: 0x000100000, len 0x040000, wptr 0x000000 .. > +# After: start: 0x000100000, len 0x040000, cap 0x040000, wptr 0x000000 .. > +_filter_blkzone_report() > +{ > + $AWK_PROG -F "," 'BEGIN{OFS=",";} $3 !~ /cap/ {$2=$2","$2;} {print;}' |\ > + sed -e 's/len/cap/2' > +} > + > # make sure this script returns success > /bin/true > diff --git a/tests/btrfs/237 b/tests/btrfs/237 > index 96940549..cb897416 100755 > --- a/tests/btrfs/237 > +++ b/tests/btrfs/237 > @@ -35,8 +35,13 @@ get_data_bg() > grep -Eo "CHUNK_ITEM [[:digit:]]+" | cut -d ' ' -f 2 > } > > -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) > -zonesize=$((zonesize << 9)) > +get_data_bg_physical() > +{ > + # Assumes SINGLE data profile > + $BTRFS_UTIL_PROG inspect-internal dump-tree -t CHUNK $SCRATCH_DEV |\ > + grep -A 4 CHUNK_ITEM | grep -A 3 'type DATA\|SINGLE' |\ > + grep -Eo 'offset [[:digit:]]+'| cut -d ' ' -f 2 > +} > > _scratch_mkfs >/dev/null 2>&1 > _scratch_mount -o commit=1 # 1s commit time to speed up test > @@ -49,12 +54,20 @@ if [[ "$uuid" == "" ]]; then > exit 1 > fi > > +start_data_bg_phy=$(get_data_bg_physical) > +start_data_bg_phy=$((data_bg_zone >> 9)) > + > +size=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\ > + _filter_blkzone_report |\ > + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) > +size=$((size << 9)) > + > reclaim_threshold=75 > echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold > fill_percent=$((reclaim_threshold + 2)) > rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG > -fill_size=$((zonesize * fill_percent / 100)) > -rest=$((zonesize * rest_percent / 100)) > +fill_size=$((size * fill_percent / 100)) > +rest=$((size * rest_percent / 100)) > > # step 1, fill FS over $fillsize > $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full
On Mon, Mar 21, 2022 at 12:58:48PM +0100, Pankaj Raghav wrote: > This test will break when zone capacity != zone size because the > calculation of the size to be filled is done using zone_size instead of > the actual capacity available per zone. Fix it by using zone capacity. > > As a zoned device can have variable capacity, use the btrfs utility to > get the zone capacity from the first data block group that the test will > be performed on. > > The support to extract zone capacity was added to blkzone only from > version 2.37. So zcap will be used only when the blkzone version is > greater or equal to 2.37. > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> I tried this test, but it didn't work. See below. > --- > Changes since v1: > - Add a filter that can append a 'cap' column for blkzone version <2.37 > (Naohiro) > - Extract the capacity from the zone on which the test is performed as a > zoned device can have variable zone capacity(Naohiro) > > Changes since v2: > - Use btrfs inspect-internal utility to find the first data zone address > instead of hardcoding. (Naohiro) > > common/filter | 13 +++++++++++++ > tests/btrfs/237 | 21 +++++++++++++++++---- > 2 files changed, 30 insertions(+), 4 deletions(-) > > diff --git a/common/filter b/common/filter > index 257227c2..5fe86756 100644 > --- a/common/filter > +++ b/common/filter > @@ -634,5 +634,18 @@ _filter_bash() > sed -e "s/^bash: line 1: /bash: /" > } > > +# > +# blkzone report added zone capacity to be printed from v2.37. > +# This filter will add an extra column 'cap' with the same value of > +# 'len'(zone size) for blkzone version < 2.37 > +# > +# Before: start: 0x000100000, len 0x040000, wptr 0x000000 .. > +# After: start: 0x000100000, len 0x040000, cap 0x040000, wptr 0x000000 .. > +_filter_blkzone_report() > +{ > + $AWK_PROG -F "," 'BEGIN{OFS=",";} $3 !~ /cap/ {$2=$2","$2;} {print;}' |\ > + sed -e 's/len/cap/2' > +} > + > # make sure this script returns success > /bin/true > diff --git a/tests/btrfs/237 b/tests/btrfs/237 > index 96940549..cb897416 100755 > --- a/tests/btrfs/237 > +++ b/tests/btrfs/237 > @@ -35,8 +35,13 @@ get_data_bg() > grep -Eo "CHUNK_ITEM [[:digit:]]+" | cut -d ' ' -f 2 > } > > -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) > -zonesize=$((zonesize << 9)) > +get_data_bg_physical() > +{ > + # Assumes SINGLE data profile > + $BTRFS_UTIL_PROG inspect-internal dump-tree -t CHUNK $SCRATCH_DEV |\ > + grep -A 4 CHUNK_ITEM | grep -A 3 'type DATA\|SINGLE' |\ > + grep -Eo 'offset [[:digit:]]+'| cut -d ' ' -f 2 > +} > > _scratch_mkfs >/dev/null 2>&1 > _scratch_mount -o commit=1 # 1s commit time to speed up test > @@ -49,12 +54,20 @@ if [[ "$uuid" == "" ]]; then > exit 1 > fi > > +start_data_bg_phy=$(get_data_bg_physical) > +start_data_bg_phy=$((data_bg_zone >> 9)) start_data_bg_phy=$((start_data_bg_phy >> 9)) ? > + > +size=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\ > + _filter_blkzone_report |\ > + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) The pattern should be "cap 0x[[:xdigit:]]+" as it is hexadecimal. I missed this point before. > +size=$((size << 9)) > + > reclaim_threshold=75 > echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold > fill_percent=$((reclaim_threshold + 2)) > rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG > -fill_size=$((zonesize * fill_percent / 100)) > -rest=$((zonesize * rest_percent / 100)) > +fill_size=$((size * fill_percent / 100)) > +rest=$((size * rest_percent / 100)) > > # step 1, fill FS over $fillsize > $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full > -- > 2.25.1 >
On 2022-03-25 06:00, Naohiro Aota wrote: >> >> +start_data_bg_phy=$(get_data_bg_physical) >> +start_data_bg_phy=$((data_bg_zone >> 9)) > > start_data_bg_phy=$((start_data_bg_phy >> 9)) ? > Oops. I fixed this in my test machine but forgot to put these while sending my patch. Thanks. >> + >> +size=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\ >> + _filter_blkzone_report |\ >> + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) > > The pattern should be "cap 0x[[:xdigit:]]+" as it is hexadecimal. I > missed this point before. > I will fix it up. >> +size=$((size << 9)) >> + >> reclaim_threshold=75 >> echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold >> fill_percent=$((reclaim_threshold + 2)) >> rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG >> -fill_size=$((zonesize * fill_percent / 100)) >> -rest=$((zonesize * rest_percent / 100)) >> +fill_size=$((size * fill_percent / 100)) >> +rest=$((size * rest_percent / 100)) >> >> # step 1, fill FS over $fillsize >> $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full >> -- >> 2.25.1
diff --git a/common/filter b/common/filter index 257227c2..5fe86756 100644 --- a/common/filter +++ b/common/filter @@ -634,5 +634,18 @@ _filter_bash() sed -e "s/^bash: line 1: /bash: /" } +# +# blkzone report added zone capacity to be printed from v2.37. +# This filter will add an extra column 'cap' with the same value of +# 'len'(zone size) for blkzone version < 2.37 +# +# Before: start: 0x000100000, len 0x040000, wptr 0x000000 .. +# After: start: 0x000100000, len 0x040000, cap 0x040000, wptr 0x000000 .. +_filter_blkzone_report() +{ + $AWK_PROG -F "," 'BEGIN{OFS=",";} $3 !~ /cap/ {$2=$2","$2;} {print;}' |\ + sed -e 's/len/cap/2' +} + # make sure this script returns success /bin/true diff --git a/tests/btrfs/237 b/tests/btrfs/237 index 96940549..cb897416 100755 --- a/tests/btrfs/237 +++ b/tests/btrfs/237 @@ -35,8 +35,13 @@ get_data_bg() grep -Eo "CHUNK_ITEM [[:digit:]]+" | cut -d ' ' -f 2 } -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) -zonesize=$((zonesize << 9)) +get_data_bg_physical() +{ + # Assumes SINGLE data profile + $BTRFS_UTIL_PROG inspect-internal dump-tree -t CHUNK $SCRATCH_DEV |\ + grep -A 4 CHUNK_ITEM | grep -A 3 'type DATA\|SINGLE' |\ + grep -Eo 'offset [[:digit:]]+'| cut -d ' ' -f 2 +} _scratch_mkfs >/dev/null 2>&1 _scratch_mount -o commit=1 # 1s commit time to speed up test @@ -49,12 +54,20 @@ if [[ "$uuid" == "" ]]; then exit 1 fi +start_data_bg_phy=$(get_data_bg_physical) +start_data_bg_phy=$((data_bg_zone >> 9)) + +size=$($BLKZONE_PROG report -o $start_data_bg_phy -l 1 $SCRATCH_DEV |\ + _filter_blkzone_report |\ + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) +size=$((size << 9)) + reclaim_threshold=75 echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold fill_percent=$((reclaim_threshold + 2)) rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG -fill_size=$((zonesize * fill_percent / 100)) -rest=$((zonesize * rest_percent / 100)) +fill_size=$((size * fill_percent / 100)) +rest=$((size * rest_percent / 100)) # step 1, fill FS over $fillsize $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full
This test will break when zone capacity != zone size because the calculation of the size to be filled is done using zone_size instead of the actual capacity available per zone. Fix it by using zone capacity. As a zoned device can have variable capacity, use the btrfs utility to get the zone capacity from the first data block group that the test will be performed on. The support to extract zone capacity was added to blkzone only from version 2.37. So zcap will be used only when the blkzone version is greater or equal to 2.37. Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> --- Changes since v1: - Add a filter that can append a 'cap' column for blkzone version <2.37 (Naohiro) - Extract the capacity from the zone on which the test is performed as a zoned device can have variable zone capacity(Naohiro) Changes since v2: - Use btrfs inspect-internal utility to find the first data zone address instead of hardcoding. (Naohiro) common/filter | 13 +++++++++++++ tests/btrfs/237 | 21 +++++++++++++++++---- 2 files changed, 30 insertions(+), 4 deletions(-)