[mdadm,v1,00/14] Bug fixes and testing improvments

Message

Logan Gunthorpe June 9, 2022, 9:11 p.m. UTC

Hi,

This series tries to clean up the testing infrastructure to be a bit
more reliable. It doesn't fix all the broken tests but annotates those
that I see as broken so testing can continue.

As such, I've fixed all the kernel panics (in md-next now) and segfaults
that caused testing to halt regardless of whether --keep-going was
passed. I've also included some patches posted to the list from Sudhakar
and Himanshu which fix some more broken tests.

I've also included a patch which adds the --loop option to ./test which
runs tests for a specified number of iterations (or indefinitely if zero
is specified). This was very useful for ferreting out tests that failed
randomly.

The last two patches adds some infrastructure and annotation for known
broken tests so that they don't stop the processing (even if
--keep-going is not passed). Tests that are known to be broken  can
optionally be skipped with the --skip-broken or --skip-always-broken
flags.

With these changes it's possible to run './test --loop=0' for several
days without stopping.

There are still a number of broken tests which need more work, and there
may be other issues on other people's systems (kernel configurations,
etc) but that will have to be left to other developers.

Thanks,

Logan

--

Logan Gunthorpe (10):
  Makefile: Don't build static build with everything
  DDF: Cleanup validate_geometry_ddf_container()
  DDF: Fix NULL pointer dereference in validate_geometry_ddf()
  mdadm/Grow: Fix use after close bug by closing after fork
  monitor: Avoid segfault when calling NULL get_bad_blocks
  mdadm: Fix mdadm -r remove option regresision
  mdadm: Fix optional --write-behind parameter
  mdadm/test: Add a mode to repeat specified tests
  mdadm/test: Mark and ignore broken test failures
  tests: Add broken files for all broken tests

Sudhakar Panneerselvam (4):
  tests/00raid0: add a test that validates raid0 with layout fails for
    0.9
  tests: fix raid0 tests for 0.90 metadata
  tests/04update-metadata: avoid passing chunk size to raid1
  tests/02lineargrow: clear the superblock at every iteration

 Grow.c                                     |  4 +-
 Makefile                                   |  2 +-
 ReadMe.c                                   |  7 +-
 mdadm.c                                    |  6 +-
 mdadm.h                                    |  1 +
 monitor.c                                  |  3 +
 super-ddf.c                                | 94 ++++++++++------------
 test                                       | 71 +++++++++++++---
 tests/00raid0                              | 10 +--
 tests/00readonly                           |  4 +
 tests/01r5integ.broken                     |  7 ++
 tests/01raid6integ.broken                  |  7 ++
 tests/02lineargrow                         |  2 +
 tests/03r0assem                            |  6 +-
 tests/04r0update                           |  4 +-
 tests/04r5swap.broken                      |  7 ++
 tests/04update-metadata                    |  8 +-
 tests/07autoassemble.broken                |  8 ++
 tests/07autodetect.broken                  |  5 ++
 tests/07changelevelintr.broken             |  9 +++
 tests/07changelevels.broken                |  9 +++
 tests/07reshape5intr.broken                | 45 +++++++++++
 tests/07revert-grow.broken                 | 31 +++++++
 tests/07revert-shrink.broken               |  9 +++
 tests/07testreshape5.broken                | 12 +++
 tests/09imsm-assemble.broken               |  6 ++
 tests/09imsm-create-fail-rebuild.broken    |  5 ++
 tests/09imsm-overlap.broken                |  7 ++
 tests/10ddf-assemble-missing.broken        |  6 ++
 tests/10ddf-fail-create-race.broken        |  7 ++
 tests/10ddf-fail-two-spares.broken         |  5 ++
 tests/10ddf-incremental-wrong-order.broken |  9 +++
 tests/14imsm-r1_2d-grow-r1_3d.broken       |  5 ++
 tests/14imsm-r1_2d-takeover-r0_2d.broken   |  6 ++
 tests/18imsm-r10_4d-takeover-r0_2d.broken  |  5 ++
 tests/18imsm-r1_2d-takeover-r0_1d.broken   |  6 ++
 tests/19raid6auto-repair.broken            |  5 ++
 tests/19raid6repair.broken                 |  5 ++
 38 files changed, 363 insertions(+), 85 deletions(-)
 create mode 100644 tests/01r5integ.broken
 create mode 100644 tests/01raid6integ.broken
 create mode 100644 tests/04r5swap.broken
 create mode 100644 tests/07autoassemble.broken
 create mode 100644 tests/07autodetect.broken
 create mode 100644 tests/07changelevelintr.broken
 create mode 100644 tests/07changelevels.broken
 create mode 100644 tests/07reshape5intr.broken
 create mode 100644 tests/07revert-grow.broken
 create mode 100644 tests/07revert-shrink.broken
 create mode 100644 tests/07testreshape5.broken
 create mode 100644 tests/09imsm-assemble.broken
 create mode 100644 tests/09imsm-create-fail-rebuild.broken
 create mode 100644 tests/09imsm-overlap.broken
 create mode 100644 tests/10ddf-assemble-missing.broken
 create mode 100644 tests/10ddf-fail-create-race.broken
 create mode 100644 tests/10ddf-fail-two-spares.broken
 create mode 100644 tests/10ddf-incremental-wrong-order.broken
 create mode 100644 tests/14imsm-r1_2d-grow-r1_3d.broken
 create mode 100644 tests/14imsm-r1_2d-takeover-r0_2d.broken
 create mode 100644 tests/18imsm-r10_4d-takeover-r0_2d.broken
 create mode 100644 tests/18imsm-r1_2d-takeover-r0_1d.broken
 create mode 100644 tests/19raid6auto-repair.broken
 create mode 100644 tests/19raid6repair.broken


base-commit: 52c67fcdd6dadc4138ecad73e65599551804d445
--
2.30.2

Comments

Paul Menzel June 10, 2022, 10:14 a.m. UTC | #1

Dear Logan,


Am 09.06.22 um 23:11 schrieb Logan Gunthorpe:

[…]

> There are still a number of broken tests which need more work, and there
> may be other issues on other people's systems (kernel configurations,
> etc) but that will have to be left to other developers.

Thank you for all your work. Can you list the broken tests, you 
encountered, in the cover letter?


Kind regards,

Paul

Logan Gunthorpe June 10, 2022, 3:27 p.m. UTC | #2

On 2022-06-10 04:14, Paul Menzel wrote:
> Dear Logan,
> 
> 
> Am 09.06.22 um 23:11 schrieb Logan Gunthorpe:
> 
> […]
> 
>> There are still a number of broken tests which need more work, and there
>> may be other issues on other people's systems (kernel configurations,
>> etc) but that will have to be left to other developers.
> 
> Thank you for all your work. Can you list the broken tests, you
> encountered, in the cover letter?

For full details of broken tests (after this series) see patch 14.

Here they are again:

01r5integ.broken
01raid6integ.broken
04r5swap.broken
07autoassemble.broken
07autodetect.broken
07changelevelintr.broken
07changelevels.broken
07reshape5intr.broken
07revert-grow.broken
07revert-shrink.broken
07testreshape5.broken
09imsm-assemble.broken
09imsm-create-fail-rebuild.broken
09imsm-overlap.broken
10ddf-assemble-missing.broken
10ddf-fail-create-race.broken
10ddf-fail-two-spares.broken
10ddf-incremental-wrong-order.broken
14imsm-r1_2d-grow-r1_3d.broken
14imsm-r1_2d-takeover-r0_2d.broken
18imsm-r10_4d-takeover-r0_2d.broken
18imsm-r1_2d-takeover-r0_1d.broken
19raid6auto-repair.broken
19raid6repair.broken

I don't have a comprehensive list of tests that were fixed by this
series (or my kernel series) as some of the fixes fixed multiple tests
that I didn't keep track of.

Logan