mbox series

[RFC,net-next,v2,0/4] selftests/tc-testing: parallel tdc

Message ID 20230824153736.629961-1-pctammela@mojatatu.com (mailing list archive)
Headers show
Series selftests/tc-testing: parallel tdc | expand

Message

Pedro Tammela Aug. 24, 2023, 3:37 p.m. UTC 
As the number of tdc tests is growing, so is our completion wall time.
One of the ideas to improve this is to run tests in parallel, as they
are self contained.

This series allows for tests to run in parallel, in batches of 32 tests.
Not all tests can run in parallel as they might conflict with each other.
The code will still honor this requirement even when trying to run the
tests over the worker pool.

In order to make this happen we had to localize the test resources
(patches 1 and 2), where instead of having all tests sharing one single
namespace and veths devices each test now gets it's own local namespace and devices.

Even though the tests serialize over rtnl_lock in the kernel, we
measured a speedup of about 3x in a test VM.

v1->v2: https://lore.kernel.org/all/20230728154059.1866057-1-pctammela@mojatatu.com/
   - Add worker pool

Pedro Tammela (4):
  selftests/tc-testing: localize test resources
  selftests/tc-testing: update test definitions for local resources
  selftests/tc-testing: implement tdc parallel test run
  selftests/tc-testing: update tdc documentation

 tools/testing/selftests/tc-testing/README     |  65 +---
 .../testing/selftests/tc-testing/TdcPlugin.py |   4 +-
 .../selftests/tc-testing/TdcResults.py        |   3 +-
 .../tc-testing/plugin-lib/nsPlugin.py         | 194 ++++++++----
 .../tc-testing/plugin-lib/rootPlugin.py       |   4 +-
 .../tc-testing/plugin-lib/valgrindPlugin.py   |   5 +-
 .../tc-testing/tc-tests/actions/connmark.json |  45 +++
 .../tc-testing/tc-tests/actions/csum.json     |  69 +++++
 .../tc-testing/tc-tests/actions/ct.json       |  54 ++++
 .../tc-testing/tc-tests/actions/ctinfo.json   |  36 +++
 .../tc-testing/tc-tests/actions/gact.json     |  75 +++++
 .../tc-testing/tc-tests/actions/gate.json     |  36 +++
 .../tc-testing/tc-tests/actions/ife.json      | 144 +++++++++
 .../tc-testing/tc-tests/actions/mirred.json   |  72 +++++
 .../tc-testing/tc-tests/actions/mpls.json     | 159 ++++++++++
 .../tc-testing/tc-tests/actions/nat.json      |  81 +++++
 .../tc-testing/tc-tests/actions/pedit.json    | 198 ++++++++++++
 .../tc-testing/tc-tests/actions/police.json   | 102 +++++++
 .../tc-testing/tc-tests/actions/sample.json   |  87 ++++++
 .../tc-testing/tc-tests/actions/simple.json   |  27 ++
 .../tc-testing/tc-tests/actions/skbedit.json  |  90 ++++++
 .../tc-testing/tc-tests/actions/skbmod.json   |  54 ++++
 .../tc-tests/actions/tunnel_key.json          | 117 ++++++++
 .../tc-testing/tc-tests/actions/vlan.json     | 108 +++++++
 .../tc-testing/tc-tests/actions/xt.json       |  24 ++
 .../tc-testing/tc-tests/filters/bpf.json      |  10 +-
 .../tc-testing/tc-tests/filters/fw.json       | 266 ++++++++--------
 .../tc-testing/tc-tests/filters/matchall.json | 141 +++++----
 .../tc-testing/tc-tests/infra/actions.json    | 144 ++++-----
 .../tc-testing/tc-tests/infra/filter.json     |   9 +-
 .../tc-testing/tc-tests/qdiscs/cake.json      |  82 ++---
 .../tc-testing/tc-tests/qdiscs/cbs.json       |  38 +--
 .../tc-testing/tc-tests/qdiscs/choke.json     |  30 +-
 .../tc-testing/tc-tests/qdiscs/codel.json     |  34 +--
 .../tc-testing/tc-tests/qdiscs/drr.json       |  10 +-
 .../tc-testing/tc-tests/qdiscs/etf.json       |  18 +-
 .../tc-testing/tc-tests/qdiscs/ets.json       | 284 ++++++++++--------
 .../tc-testing/tc-tests/qdiscs/fifo.json      |  98 +++---
 .../tc-testing/tc-tests/qdiscs/fq.json        |  68 +----
 .../tc-testing/tc-tests/qdiscs/fq_codel.json  |  54 +---
 .../tc-testing/tc-tests/qdiscs/fq_pie.json    |   5 +-
 .../tc-testing/tc-tests/qdiscs/gred.json      |  28 +-
 .../tc-testing/tc-tests/qdiscs/hfsc.json      |  26 +-
 .../tc-testing/tc-tests/qdiscs/hhf.json       |  36 +--
 .../tc-testing/tc-tests/qdiscs/htb.json       |  46 +--
 .../tc-testing/tc-tests/qdiscs/ingress.json   |  36 ++-
 .../tc-testing/tc-tests/qdiscs/netem.json     |  62 +---
 .../tc-tests/qdiscs/pfifo_fast.json           |  18 +-
 .../tc-testing/tc-tests/qdiscs/plug.json      |  30 +-
 .../tc-testing/tc-tests/qdiscs/prio.json      |  85 +++---
 .../tc-testing/tc-tests/qdiscs/qfq.json       |  39 +--
 .../tc-testing/tc-tests/qdiscs/red.json       |  34 +--
 .../tc-testing/tc-tests/qdiscs/sfb.json       |  48 +--
 .../tc-testing/tc-tests/qdiscs/sfq.json       |  40 +--
 .../tc-testing/tc-tests/qdiscs/skbprio.json   |  16 +-
 .../tc-testing/tc-tests/qdiscs/tbf.json       |  36 +--
 .../tc-testing/tc-tests/qdiscs/teql.json      |  34 +--
 tools/testing/selftests/tc-testing/tdc.py     | 250 +++++++++++----
 58 files changed, 2720 insertions(+), 1288 deletions(-)

Comments

Davide Caratti Sept. 13, 2023, 1:06 p.m. UTC | #1 
hello Pedro,

On Thu, Aug 24, 2023 at 5:38 PM Pedro Tammela <pctammela@mojatatu.com> wrote:
>
> As the number of tdc tests is growing, so is our completion wall time.
> One of the ideas to improve this is to run tests in parallel, as they
> are self contained.
>
> This series allows for tests to run in parallel, in batches of 32 tests.

<...>

looks good to me!

it would be nice to extend tdc.py in a way that all the ns mounts are
deleted before re-running a test (e.g. after a failure or after tdc.py
is interrupted). But that can be done in a separate patch.

Tested-by: Davide Caratti <dcaratti@redhat.com>