diff mbox

FS unmountable after RAID/LVM problems

Message ID ghlgevjavx.fsf@lena.gouders.net (mailing list archive)
State New, archived
Headers show

Commit Message

Dirk Gouders March 14, 2018, 8:53 a.m. UTC
Qu Wenruo <quwenruo.btrfs@gmx.com> writes:

> On 2018年03月13日 22:49, Dirk Gouders wrote:
> [snip]
>>>
>>> # btrfs inspect dump-tree -b 848986112 /dev/loop0p1
>>> # btrfs inspect dump-tree -b 72089600 /dev/loop0p1
>> 
>> OK.
>> 
>> (This mail gets a bit long but I don't want to snip probably important
>>  information above.)
>> 
>
> Feel free to snip.
> As the involved tree block is not shown anywhere.
>
> So it's not any root node corrupted.
> It may be some extent tree node corrupted in this case.
>
> While to inspect it, we need some new functionality in btrfs inspect tree.
>
> Before that, would you please try the following patch and to see if it
> helps btrfs-restore to salvage any data?

I tried it and got the following output:

# btrfs restore /dev/loop0p1 /mnt/
checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
checksum verify failed on 363069440 found DC09290B wanted C630FD61
checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
bytenr mismatch, want=363069440, have=17552567724568668829
Could not open root, trying backup super
checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
checksum verify failed on 363069440 found DC09290B wanted C630FD61
checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
bytenr mismatch, want=363069440, have=17552567724568668829
Could not open root, trying backup super
ERROR: superblock bytenr 274877906944 is larger than device size 10741612544
Could not open root, trying backup super


Because the patch did not apply, I did the modification manually as
follows:

# git diff

Thanks,

Dirk
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Qu Wenruo March 14, 2018, 9:16 a.m. UTC | #1
On 2018年03月14日 16:53, Dirk Gouders wrote:
> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
> 
>> On 2018年03月13日 22:49, Dirk Gouders wrote:
>> [snip]
>>>>
>>>> # btrfs inspect dump-tree -b 848986112 /dev/loop0p1
>>>> # btrfs inspect dump-tree -b 72089600 /dev/loop0p1
>>>
>>> OK.
>>>
>>> (This mail gets a bit long but I don't want to snip probably important
>>>  information above.)
>>>
>>
>> Feel free to snip.
>> As the involved tree block is not shown anywhere.
>>
>> So it's not any root node corrupted.
>> It may be some extent tree node corrupted in this case.
>>
>> While to inspect it, we need some new functionality in btrfs inspect tree.
>>
>> Before that, would you please try the following patch and to see if it
>> helps btrfs-restore to salvage any data?
> 
> I tried it and got the following output:
> 
> # btrfs restore /dev/loop0p1 /mnt/
> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
> checksum verify failed on 363069440 found DC09290B wanted C630FD61
> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
> bytenr mismatch, want=363069440, have=17552567724568668829
> Could not open root, trying backup super
> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
> checksum verify failed on 363069440 found DC09290B wanted C630FD61
> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
> bytenr mismatch, want=363069440, have=17552567724568668829
> Could not open root, trying backup super
> ERROR: superblock bytenr 274877906944 is larger than device size 10741612544
> Could not open root, trying backup super

So it's still something important in the tree.

Would you please apply this patch?
https://patchwork.kernel.org/patch/10281329/

And then dump the tree again using that newly added -f option?
(Both stdout and stderr is needed)

The dump command would be:
# btrfs inspect dump-tree -f -b <bytenr>

Needed bytenrs would be:
848773120	tree root
848789504	extent root (My primary guess)
30408704	dev root
850509824	fs root (this could contain *FILENAME*, please censor
                         them if needed, and it may be large)
212353024	uuid tree (not really imporatant)

And if it's extent root, we could enhance btrfs-progs open_ctree() to
handle it for RO mode (needed by btrfs-restore)

Thanks,
Qu

> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Dirk Gouders March 14, 2018, 9:36 a.m. UTC | #2
Qu Wenruo <quwenruo.btrfs@gmx.com> writes:

> On 2018年03月14日 16:53, Dirk Gouders wrote:
>> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
>> 
>>> On 2018年03月13日 22:49, Dirk Gouders wrote:
>>> [snip]
>>>>>
>>>>> # btrfs inspect dump-tree -b 848986112 /dev/loop0p1
>>>>> # btrfs inspect dump-tree -b 72089600 /dev/loop0p1
>>>>
>>>> OK.
>>>>
>>>> (This mail gets a bit long but I don't want to snip probably important
>>>>  information above.)
>>>>
>>>
>>> Feel free to snip.
>>> As the involved tree block is not shown anywhere.
>>>
>>> So it's not any root node corrupted.
>>> It may be some extent tree node corrupted in this case.
>>>
>>> While to inspect it, we need some new functionality in btrfs inspect tree.
>>>
>>> Before that, would you please try the following patch and to see if it
>>> helps btrfs-restore to salvage any data?
>> 
>> I tried it and got the following output:
>> 
>> # btrfs restore /dev/loop0p1 /mnt/
>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>> checksum verify failed on 363069440 found DC09290B wanted C630FD61
>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>> bytenr mismatch, want=363069440, have=17552567724568668829
>> Could not open root, trying backup super
>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>> checksum verify failed on 363069440 found DC09290B wanted C630FD61
>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>> bytenr mismatch, want=363069440, have=17552567724568668829
>> Could not open root, trying backup super
>> ERROR: superblock bytenr 274877906944 is larger than device size 10741612544
>> Could not open root, trying backup super
>
> So it's still something important in the tree.
>
> Would you please apply this patch?
> https://patchwork.kernel.org/patch/10281329/
>
> And then dump the tree again using that newly added -f option?
> (Both stdout and stderr is needed)
>
> The dump command would be:
> # btrfs inspect dump-tree -f -b <bytenr>
>
> Needed bytenrs would be:
> 848773120	tree root
> 848789504	extent root (My primary guess)

I am currently preparing the diagnosis data but after the above bytenr
the log grew to already 28MB.  Should I send all that data to the list?

Thanks,

Dirk

> 30408704	dev root
> 850509824	fs root (this could contain *FILENAME*, please censor
>                          them if needed, and it may be large)
> 212353024	uuid tree (not really imporatant)
>
> And if it's extent root, we could enhance btrfs-progs open_ctree() to
> handle it for RO mode (needed by btrfs-restore)
>
> Thanks,
> Qu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Qu Wenruo March 14, 2018, 9:41 a.m. UTC | #3
On 2018年03月14日 17:36, Dirk Gouders wrote:
> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
> 
>> On 2018年03月14日 16:53, Dirk Gouders wrote:
>>> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
>>>
>>>> On 2018年03月13日 22:49, Dirk Gouders wrote:
>>>> [snip]
>>>>>>
>>>>>> # btrfs inspect dump-tree -b 848986112 /dev/loop0p1
>>>>>> # btrfs inspect dump-tree -b 72089600 /dev/loop0p1
>>>>>
>>>>> OK.
>>>>>
>>>>> (This mail gets a bit long but I don't want to snip probably important
>>>>>  information above.)
>>>>>
>>>>
>>>> Feel free to snip.
>>>> As the involved tree block is not shown anywhere.
>>>>
>>>> So it's not any root node corrupted.
>>>> It may be some extent tree node corrupted in this case.
>>>>
>>>> While to inspect it, we need some new functionality in btrfs inspect tree.
>>>>
>>>> Before that, would you please try the following patch and to see if it
>>>> helps btrfs-restore to salvage any data?
>>>
>>> I tried it and got the following output:
>>>
>>> # btrfs restore /dev/loop0p1 /mnt/
>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>> checksum verify failed on 363069440 found DC09290B wanted C630FD61
>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>> bytenr mismatch, want=363069440, have=17552567724568668829
>>> Could not open root, trying backup super
>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>> checksum verify failed on 363069440 found DC09290B wanted C630FD61
>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>> bytenr mismatch, want=363069440, have=17552567724568668829
>>> Could not open root, trying backup super
>>> ERROR: superblock bytenr 274877906944 is larger than device size 10741612544
>>> Could not open root, trying backup super
>>
>> So it's still something important in the tree.
>>
>> Would you please apply this patch?
>> https://patchwork.kernel.org/patch/10281329/
>>
>> And then dump the tree again using that newly added -f option?
>> (Both stdout and stderr is needed)
>>
>> The dump command would be:
>> # btrfs inspect dump-tree -f -b <bytenr>
>>
>> Needed bytenrs would be:
>> 848773120	tree root
>> 848789504	extent root (My primary guess)
> 
> I am currently preparing the diagnosis data but after the above bytenr
> the log grew to already 28MB.  Should I send all that data to the list?

Nope, stderr is enough.

Thanks,
Qu

> 
> Thanks,
> 
> Dirk
> 
>> 30408704	dev root
>> 850509824	fs root (this could contain *FILENAME*, please censor
>>                          them if needed, and it may be large)
>> 212353024	uuid tree (not really imporatant)
>>
>> And if it's extent root, we could enhance btrfs-progs open_ctree() to
>> handle it for RO mode (needed by btrfs-restore)
>>
>> Thanks,
>> Qu
Qu Wenruo March 14, 2018, 10:14 a.m. UTC | #4
On 2018年03月14日 18:06, Dirk Gouders wrote:
> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
> 
>> On 2018年03月14日 17:36, Dirk Gouders wrote:
>>> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
>>>
>>>> On 2018年03月14日 16:53, Dirk Gouders wrote:
>>>>> Qu Wenruo <quwenruo.btrfs@gmx.com> writes:
>>>>>
>>>>>> On 2018年03月13日 22:49, Dirk Gouders wrote:
>>>>>> [snip]
>>>>>>>>
>>>>>>>> # btrfs inspect dump-tree -b 848986112 /dev/loop0p1
>>>>>>>> # btrfs inspect dump-tree -b 72089600 /dev/loop0p1
>>>>>>>
>>>>>>> OK.
>>>>>>>
>>>>>>> (This mail gets a bit long but I don't want to snip probably important
>>>>>>>  information above.)
>>>>>>>
>>>>>>
>>>>>> Feel free to snip.
>>>>>> As the involved tree block is not shown anywhere.
>>>>>>
>>>>>> So it's not any root node corrupted.
>>>>>> It may be some extent tree node corrupted in this case.
>>>>>>
>>>>>> While to inspect it, we need some new functionality in btrfs inspect tree.
>>>>>>
>>>>>> Before that, would you please try the following patch and to see if it
>>>>>> helps btrfs-restore to salvage any data?
>>>>>
>>>>> I tried it and got the following output:
>>>>>
>>>>> # btrfs restore /dev/loop0p1 /mnt/
>>>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>>>> checksum verify failed on 363069440 found DC09290B wanted C630FD61
>>>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>>>> bytenr mismatch, want=363069440, have=17552567724568668829
>>>>> Could not open root, trying backup super
>>>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>>>> checksum verify failed on 363069440 found DC09290B wanted C630FD61
>>>>> checksum verify failed on 363069440 found 296FB15A wanted F0AFE59D
>>>>> bytenr mismatch, want=363069440, have=17552567724568668829
>>>>> Could not open root, trying backup super
>>>>> ERROR: superblock bytenr 274877906944 is larger than device size 10741612544
>>>>> Could not open root, trying backup super
>>>>
>>>> So it's still something important in the tree.
>>>>
>>>> Would you please apply this patch?
>>>> https://patchwork.kernel.org/patch/10281329/
>>>>
>>>> And then dump the tree again using that newly added -f option?
>>>> (Both stdout and stderr is needed)
>>>>
>>>> The dump command would be:
>>>> # btrfs inspect dump-tree -f -b <bytenr>
>>>>
>>>> Needed bytenrs would be:
>>>> 848773120	tree root
>>>> 848789504	extent root (My primary guess)
>>>
>>> I am currently preparing the diagnosis data but after the above bytenr
>>> the log grew to already 28MB.  Should I send all that data to the list?
>>
>> Nope, stderr is enough.
> 
> OK, I will attach the output to the end.  The output is separated by the
> command lines, so searching for "inspect" helps for navigation.
> 
> For extend root and fs root I provide only stderr, because they grew
> stdout by 28 resp. 150 MB.

Thanks for all your effort!

It's clear the problem is the extent tree.

I'll try to enhance open_ctree() to allow btrfs-restore to continue even
if extent tree is corrupted asap.

Thanks,
Qu

> 
> Thanks,
> 
> Dirk
> 
>>
>>>
>>> Thanks,
>>>
>>> Dirk
>>>
>>>> 30408704	dev root
>>>> 850509824	fs root (this could contain *FILENAME*, please censor
>>>>                          them if needed, and it may be large)
>>>> 212353024	uuid tree (not really imporatant)
>>>>
>>>> And if it's extent root, we could enhance btrfs-progs open_ctree() to
>>>> handle it for RO mode (needed by btrfs-restore)
>>>>
>>>> Thanks,
>>>> Qu
>
diff mbox

Patch

diff --git a/cmds-restore.c b/cmds-restore.c
index ade35f0f..e7b96a67 100644
--- a/cmds-restore.c
+++ b/cmds-restore.c
@@ -1282,7 +1282,7 @@  static struct btrfs_root *open_fs(const char *dev, u64 root_location,
        for (i = super_mirror; i < BTRFS_SUPER_MIRROR_MAX; i++) {
                bytenr = btrfs_sb_offset(i);
                fs_info = open_ctree_fs_info(dev, bytenr, root_location, 0,
-                                            OPEN_CTREE_PARTIAL);
+                                            OPEN_CTREE_PARTIAL | __OPEN_CTREE_RETURN_CHUNK_ROOT);
                if (fs_info)
                        break;
                fprintf(stderr, "Could not open root, trying backup super\n");