[PATCHi,RFC] fstest: btrfs/158 fix miss-aligned stripe and device
diff mbox series

Message ID 1576044519-28313-1-git-send-email-anand.jain@oracle.com
State New
Headers show
Series
  • [PATCHi,RFC] fstest: btrfs/158 fix miss-aligned stripe and device
Related show

Commit Message

Anand Jain Dec. 11, 2019, 6:08 a.m. UTC
We changed the order of the allocation on the devices, and
so the test cases which are hard coded to find specific stripe
on the specific device gets failed. So fix it with the new layout.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
Qu, Right we need to fix the dev in the test case as well.
    I saw your patches bit late. Here is what I had.. you may
    use it. So I am marking this patch as RFC.
Thanks.

 tests/btrfs/158     | 10 +++++-----
 tests/btrfs/158.out |  4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)

Comments

Qu Wenruo Dec. 11, 2019, 7:08 a.m. UTC | #1
On 2019/12/11 下午2:08, Anand Jain wrote:
> We changed the order of the allocation on the devices, and
> so the test cases which are hard coded to find specific stripe
> on the specific device gets failed. So fix it with the new layout.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
> Qu, Right we need to fix the dev in the test case as well.
>     I saw your patches bit late. Here is what I had.. you may
>     use it. So I am marking this patch as RFC.

I am crafting a better solution, to handle both behavior (and even
future behavior), by getting both devid and physical offset.

And I tend to remove the fail_make_request requirement from some tests,
and direct read with multiple try should be enough to trigger repair for
test btrfs/142 and btrfs/143.


In fact, I don't believe your current fix is good enough to handle both
old and new mkfs.btrfs.

So we need to investigate more for raid repair test cases to make them
future proof.

Thanks,
Qu
> Thanks.
> 
>  tests/btrfs/158     | 10 +++++-----
>  tests/btrfs/158.out |  4 ++--
>  2 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/btrfs/158 b/tests/btrfs/158
> index 603e8bea9b7e..7f2066384f55 100755
> --- a/tests/btrfs/158
> +++ b/tests/btrfs/158
> @@ -76,14 +76,14 @@ _scratch_unmount
>  
>  stripe_0=`get_physical_stripe0`
>  stripe_1=`get_physical_stripe1`
> -dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
> -dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
> +dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
> +dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>  
>  # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
> -echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
> +echo "step 2......simulate bitrot at offset $stripe_0 of device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
>  
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 | _filter_xfs_io
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 | _filter_xfs_io
>  
>  # step 3: scrub filesystem to repair the bitrot
>  echo "step 3......repair the bitrot" >> $seqres.full
> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
> index 1f5ad3f76917..5cdaeb238c62 100644
> --- a/tests/btrfs/158.out
> +++ b/tests/btrfs/158.out
> @@ -1,9 +1,9 @@
>  QA output created by 158
>  wrote 131072/131072 bytes at offset 0
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> +wrote 65536/65536 bytes at offset 22020096
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> +wrote 65536/65536 bytes at offset 1048576
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>  *
>
Anand Jain Dec. 11, 2019, 7:23 a.m. UTC | #2
On 12/11/19 3:08 PM, Qu Wenruo wrote:
> 
> 
> On 2019/12/11 下午2:08, Anand Jain wrote:
>> We changed the order of the allocation on the devices, and
>> so the test cases which are hard coded to find specific stripe
>> on the specific device gets failed. So fix it with the new layout.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>> ---
>> Qu, Right we need to fix the dev in the test case as well.
>>      I saw your patches bit late. Here is what I had.. you may
>>      use it. So I am marking this patch as RFC.
> 
> I am crafting a better solution, to handle both behavior (and even
> future behavior), by getting both devid and physical offset.

  Yep helper function using either,
    devid and btrfs fi show
    or
    uuid  and blkid

  would dynamically find the right device.

  I am ok with either.

> And I tend to remove the fail_make_request requirement from some tests,
> and direct read with multiple try should be enough to trigger repair for
> test btrfs/142 and btrfs/143.
> 

> In fact, I don't believe your current fix is good enough to handle both
> old and new mkfs.btrfs.

  It was designed to handle only forward compatible.

Thanks, Anand

> So we need to investigate more for raid repair test cases to make them
> future proof.
> 
> Thanks,
> Qu
>> Thanks.
>>
>>   tests/btrfs/158     | 10 +++++-----
>>   tests/btrfs/158.out |  4 ++--
>>   2 files changed, 7 insertions(+), 7 deletions(-)
>>
>> diff --git a/tests/btrfs/158 b/tests/btrfs/158
>> index 603e8bea9b7e..7f2066384f55 100755
>> --- a/tests/btrfs/158
>> +++ b/tests/btrfs/158
>> @@ -76,14 +76,14 @@ _scratch_unmount
>>   
>>   stripe_0=`get_physical_stripe0`
>>   stripe_1=`get_physical_stripe1`
>> -dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
>> -dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
>> +dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>> +dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>>   
>>   # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
>> -echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
>> +echo "step 2......simulate bitrot at offset $stripe_0 of device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
>>   
>> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
>> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 | _filter_xfs_io
>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 | _filter_xfs_io
>>   
>>   # step 3: scrub filesystem to repair the bitrot
>>   echo "step 3......repair the bitrot" >> $seqres.full
>> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
>> index 1f5ad3f76917..5cdaeb238c62 100644
>> --- a/tests/btrfs/158.out
>> +++ b/tests/btrfs/158.out
>> @@ -1,9 +1,9 @@
>>   QA output created by 158
>>   wrote 131072/131072 bytes at offset 0
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>> -wrote 65536/65536 bytes at offset 9437184
>> +wrote 65536/65536 bytes at offset 22020096
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>> -wrote 65536/65536 bytes at offset 9437184
>> +wrote 65536/65536 bytes at offset 1048576
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>   0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>>   *
>>
>
Qu Wenruo Dec. 11, 2019, 7:35 a.m. UTC | #3
On 2019/12/11 下午3:23, Anand Jain wrote:
> 
> 
> On 12/11/19 3:08 PM, Qu Wenruo wrote:
>>
>>
>> On 2019/12/11 下午2:08, Anand Jain wrote:
>>> We changed the order of the allocation on the devices, and
>>> so the test cases which are hard coded to find specific stripe
>>> on the specific device gets failed. So fix it with the new layout.
>>>
>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>> ---
>>> Qu, Right we need to fix the dev in the test case as well.
>>>      I saw your patches bit late. Here is what I had.. you may
>>>      use it. So I am marking this patch as RFC.
>>
>> I am crafting a better solution, to handle both behavior (and even
>> future behavior), by getting both devid and physical offset.
> 
>  Yep helper function using either,
>    devid and btrfs fi show
>    or
>    uuid  and blkid
> 
>  would dynamically find the right device.

My current helper is going to rely on the fact that all mkfs.btrfs
assigned devid sequentially.
Which means "mkfs.btrfs -f $dev1 $dev2" will always assigned devid 1 to
$dev1, and devid 2 to $dev2.

As long as we don't touch that part, we should be OK.
(And I really hope we won't touch that part).

> 
>  I am ok with either.
> 
>> And I tend to remove the fail_make_request requirement from some tests,
>> and direct read with multiple try should be enough to trigger repair for
>> test btrfs/142 and btrfs/143.
>>
> 
>> In fact, I don't believe your current fix is good enough to handle both
>> old and new mkfs.btrfs.
> 
>  It was designed to handle only forward compatible.

Then both Oracle and SUSE QA people will hate such tests...

Thanks,
Qu

> 
> Thanks, Anand
> 
>> So we need to investigate more for raid repair test cases to make them
>> future proof.
>>
>> Thanks,
>> Qu
>>> Thanks.
>>>
>>>   tests/btrfs/158     | 10 +++++-----
>>>   tests/btrfs/158.out |  4 ++--
>>>   2 files changed, 7 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/tests/btrfs/158 b/tests/btrfs/158
>>> index 603e8bea9b7e..7f2066384f55 100755
>>> --- a/tests/btrfs/158
>>> +++ b/tests/btrfs/158
>>> @@ -76,14 +76,14 @@ _scratch_unmount
>>>     stripe_0=`get_physical_stripe0`
>>>   stripe_1=`get_physical_stripe1`
>>> -dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
>>> -dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
>>> +dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>>> +dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>>>     # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
>>> -echo "step 2......simulate bitrot at offset $stripe_0 of
>>> device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
>>> +echo "step 2......simulate bitrot at offset $stripe_0 of
>>> device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
>>>   -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 |
>>> _filter_xfs_io
>>> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 |
>>> _filter_xfs_io
>>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 |
>>> _filter_xfs_io
>>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 |
>>> _filter_xfs_io
>>>     # step 3: scrub filesystem to repair the bitrot
>>>   echo "step 3......repair the bitrot" >> $seqres.full
>>> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
>>> index 1f5ad3f76917..5cdaeb238c62 100644
>>> --- a/tests/btrfs/158.out
>>> +++ b/tests/btrfs/158.out
>>> @@ -1,9 +1,9 @@
>>>   QA output created by 158
>>>   wrote 131072/131072 bytes at offset 0
>>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>> -wrote 65536/65536 bytes at offset 9437184
>>> +wrote 65536/65536 bytes at offset 22020096
>>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>> -wrote 65536/65536 bytes at offset 9437184
>>> +wrote 65536/65536 bytes at offset 1048576
>>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>   0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>>>   *
>>>
>>
David Sterba Dec. 11, 2019, 1:33 p.m. UTC | #4
On Wed, Dec 11, 2019 at 03:23:06PM +0800, Anand Jain wrote:
> > And I tend to remove the fail_make_request requirement from some tests,
> > and direct read with multiple try should be enough to trigger repair for
> > test btrfs/142 and btrfs/143.
> > 
> 
> > In fact, I don't believe your current fix is good enough to handle both
> > old and new mkfs.btrfs.
> 
>   It was designed to handle only forward compatible.

If possible the tests should be robust and check assumptions and also
not hardcode eg. fixed block offset values that could change for various
reasons. Forward compatibility makes it hard to test backports, not only
on enterprise kernels but also on older stable kernels. We're in a state
where developers have list of tests to ignore because they known to be
bogus or unreliable.

Patch
diff mbox series

diff --git a/tests/btrfs/158 b/tests/btrfs/158
index 603e8bea9b7e..7f2066384f55 100755
--- a/tests/btrfs/158
+++ b/tests/btrfs/158
@@ -76,14 +76,14 @@  _scratch_unmount
 
 stripe_0=`get_physical_stripe0`
 stripe_1=`get_physical_stripe1`
-dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
-dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
+dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
 
 # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
-echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
+echo "step 2......simulate bitrot at offset $stripe_0 of device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
 
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 | _filter_xfs_io
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 | _filter_xfs_io
 
 # step 3: scrub filesystem to repair the bitrot
 echo "step 3......repair the bitrot" >> $seqres.full
diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
index 1f5ad3f76917..5cdaeb238c62 100644
--- a/tests/btrfs/158.out
+++ b/tests/btrfs/158.out
@@ -1,9 +1,9 @@ 
 QA output created by 158
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
+wrote 65536/65536 bytes at offset 22020096
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
+wrote 65536/65536 bytes at offset 1048576
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
 *