[v2,00/12] btrfs-progs: make check handle invalid bg items

Message

Josef Bacik Aug. 18, 2021, 9:33 p.m. UTC

v1->v2:
- Discovered that we also don't check bytes_super in the superblock, add that
  checking and repair ability since it's coupled with the block group used
  repair.
- Discovered that we haven't actually been setting --mode=lowmem for the initial
  image check if we do make test-check-lowmem, we only do it after the repair.
  Fixed this.
- Now that we're properly testing error detection in all of the test cases, I
  found 3 problems with the --mode=lowmem mode, one infinite loop and two places
  we weren't properly propagating the error code up to the user.
- My super repair thing tripped a case where we wouldn't clean up properly for
  unaligned extent records, fixed this as well.
- Add another test image for the corrupted super bytes.
- Realize that you need a special .lowmem_repairable file in order for the
  lowmem repair code to run against images, so did that for both testcases.

--- Original email ---
Hello,

While writing code for extent tree v2 I noticed that I was generating a fs with
an invalid block group ->used value.  However fsck wasn't catching this, because
we don't actuall check the used value of the block group items in normal mode.
lowmem mode does this properly thankfully, so this only needs to be added to the
normal fsck mode.

I've added code to btrfs-corrupt-block to generate the corrupt image I need for
the test case.  Then of course the actual patch to detect and fix the problem.
Thanks,

Josef

Josef Bacik (12):
  btrfs-progs: fix running lowmem check tests
  btrfs-progs: do not infinite loop on corrupt keys with lowmem mode
  btrfs-progs: propagate fs root errors in lowmem mode
  btrfs-progs: propagate extent item errors in lowmem mode
  btrfs-progs: do not double add unaligned extent records
  btrfs-progs: add the ability to corrupt block group items
  btrfs-progs: add the ability to corrupt fields of the super block
  btrfs-progs: make check detect and fix invalid used for block groups
  btrfs-progs: make check detect and fix problems with super_bytes_used
  btrfs-progs: check btrfs_super_used in lowmem check
  btrfs-progs: add a test image with a corrupt block group item
  btrfs-progs: add a test image with an invalid super bytes_used

 btrfs-corrupt-block.c                         | 172 +++++++++++++++++-
 check/common.h                                |   5 +
 check/main.c                                  | 124 ++++++++++++-
 check/mode-lowmem.c                           |  25 ++-
 check/mode-lowmem.h                           |   1 +
 tests/common                                  |   5 +-
 .../.lowmem_repairable                        |   0
 .../default.img.xz                            | Bin 0 -> 1036 bytes
 .../.lowmem_repairable                        |   0
 .../default.img.xz                            | Bin 0 -> 1060 bytes
 10 files changed, 322 insertions(+), 10 deletions(-)
 create mode 100644 tests/fsck-tests/050-invalid-block-group-used/.lowmem_repairable
 create mode 100644 tests/fsck-tests/050-invalid-block-group-used/default.img.xz
 create mode 100644 tests/fsck-tests/051-invalid-super-bytes-used/.lowmem_repairable
 create mode 100644 tests/fsck-tests/051-invalid-super-bytes-used/default.img.xz

Comments

David Sterba Aug. 23, 2021, 6:31 p.m. UTC | #1

On Wed, Aug 18, 2021 at 05:33:12PM -0400, Josef Bacik wrote:
> v1->v2:
> - Discovered that we also don't check bytes_super in the superblock, add that
>   checking and repair ability since it's coupled with the block group used
>   repair.
> - Discovered that we haven't actually been setting --mode=lowmem for the initial
>   image check if we do make test-check-lowmem, we only do it after the repair.
>   Fixed this.
> - Now that we're properly testing error detection in all of the test cases, I
>   found 3 problems with the --mode=lowmem mode, one infinite loop and two places
>   we weren't properly propagating the error code up to the user.
> - My super repair thing tripped a case where we wouldn't clean up properly for
>   unaligned extent records, fixed this as well.
> - Add another test image for the corrupted super bytes.
> - Realize that you need a special .lowmem_repairable file in order for the
>   lowmem repair code to run against images, so did that for both testcases.
> 
> --- Original email ---
> Hello,
> 
> While writing code for extent tree v2 I noticed that I was generating a fs with
> an invalid block group ->used value.  However fsck wasn't catching this, because
> we don't actuall check the used value of the block group items in normal mode.
> lowmem mode does this properly thankfully, so this only needs to be added to the
> normal fsck mode.
> 
> I've added code to btrfs-corrupt-block to generate the corrupt image I need for
> the test case.  Then of course the actual patch to detect and fix the problem.
> Thanks,
> 
> Josef
> 
> Josef Bacik (12):
>   btrfs-progs: fix running lowmem check tests
>   btrfs-progs: do not infinite loop on corrupt keys with lowmem mode
>   btrfs-progs: propagate fs root errors in lowmem mode
>   btrfs-progs: propagate extent item errors in lowmem mode
>   btrfs-progs: do not double add unaligned extent records
>   btrfs-progs: add the ability to corrupt block group items
>   btrfs-progs: add the ability to corrupt fields of the super block
>   btrfs-progs: make check detect and fix invalid used for block groups
>   btrfs-progs: make check detect and fix problems with super_bytes_used
>   btrfs-progs: check btrfs_super_used in lowmem check
>   btrfs-progs: add a test image with a corrupt block group item
>   btrfs-progs: add a test image with an invalid super bytes_used

There are some comments or question so I haven't merged the patches yet
(not merged: 2, 5, 8, 9, 11, 12). Please have a look, rebase on top of
devel should cleanly drop the merged patches from the series.