| Message ID | 20211203025323.6052-1-zhijianx.li@intel.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 96f3896780153214040a6747974bebc1355307c0 |
| Headers | show |
| Series | [v3,1/3] selftests/tc-testing: add exit code | expand |
Hello: This series was applied to netdev/net.git (master) by David S. Miller <davem@davemloft.net>: On Fri, 3 Dec 2021 10:53:21 +0800 you wrote: > Mark the summary result as FAIL to prevent from confusing the selftest > framework if some of them are failed. > > Previously, the selftest framework always treats it as *ok* even though > some of them are failed actually. That's because the script tdc.sh always > return 0. > > [...] Here is the summary with links: - [v3,1/3] selftests/tc-testing: add exit code https://git.kernel.org/netdev/net/c/96f389678015 - [v3,2/3] selftests/tc-testing: add missing config https://git.kernel.org/netdev/net/c/a8c9505c53c5 - [v3,3/3] selftests/tc-testing: Fix cannot create /sys/bus/netdevsim/new_device: Directory nonexistent https://git.kernel.org/netdev/net/c/db925bca33a9 You are awesome, thank you!
All LGTM - but shouldnt these patches be independent of each other? i.e submit 3 patches each with 1/1 cheers, jamal On 2021-12-02 21:53, Li Zhijian wrote: > Mark the summary result as FAIL to prevent from confusing the selftest > framework if some of them are failed. > > Previously, the selftest framework always treats it as *ok* even though > some of them are failed actually. That's because the script tdc.sh always > return 0. > > # All test results: > # > # 1..97 > # ok 1 83be - Create FQ-PIE with invalid number of flows > # ok 2 8b6e - Create RED with no flags > [...snip] > # ok 6 5f15 - Create RED with flags ECN, harddrop > # ok 7 53e8 - Create RED with flags ECN, nodrop > # ok 8 d091 - Fail to create RED with only nodrop flag > # ok 9 af8e - Create RED with flags ECN, nodrop, harddrop > # not ok 10 ce7d - Add mq Qdisc to multi-queue device (4 queues) > # Could not match regex pattern. Verify command output: > # qdisc mq 1: root > # qdisc fq_codel 0: parent 1:4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > # qdisc fq_codel 0: parent 1:3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > [...snip] > # ok 96 6979 - Change quantum of a strict ETS band > # ok 97 9a7d - Change ETS strict band without quantum > # > # > # > # > ok 1 selftests: tc-testing: tdc.sh <<< summary result > > CC: Philip Li <philip.li@intel.com> > Reported-by: kernel test robot <lkp@intel.com> > Signed-off-by: Li Zhijian <zhijianx.li@intel.com> > Acked-by: Davide Caratti <dcaratti@redhat.com> > Acked-by: Jamal Hadi Salim <jhs@mojatatu.com> > --- > V3: repost to netdev > V2: Fix missing ':' > --- > tools/testing/selftests/tc-testing/tdc.py | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py > index a3e43189d940..ee22e3447ec7 100755 > --- a/tools/testing/selftests/tc-testing/tdc.py > +++ b/tools/testing/selftests/tc-testing/tdc.py > @@ -716,6 +716,7 @@ def set_operation_mode(pm, parser, args, remaining): > list_test_cases(alltests) > exit(0) > > + exit_code = 0 # KSFT_PASS > if len(alltests): > req_plugins = pm.get_required_plugins(alltests) > try: > @@ -724,6 +725,8 @@ def set_operation_mode(pm, parser, args, remaining): > print('The following plugins were not found:') > print('{}'.format(pde.missing_pg)) > catresults = test_runner(pm, args, alltests) > + if catresults.count_failures() != 0: > + exit_code = 1 # KSFT_FAIL > if args.format == 'none': > print('Test results output suppression requested\n') > else: > @@ -748,6 +751,8 @@ def set_operation_mode(pm, parser, args, remaining): > gid=int(os.getenv('SUDO_GID'))) > else: > print('No tests found\n') > + exit_code = 4 # KSFT_SKIP > + exit(exit_code) > > def main(): > """ > @@ -767,8 +772,5 @@ def main(): > > set_operation_mode(pm, parser, args, remaining) > > - exit(0) > - > - > if __name__ == "__main__": > main() >
diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py index a3e43189d940..ee22e3447ec7 100755 --- a/tools/testing/selftests/tc-testing/tdc.py +++ b/tools/testing/selftests/tc-testing/tdc.py @@ -716,6 +716,7 @@ def set_operation_mode(pm, parser, args, remaining): list_test_cases(alltests) exit(0) + exit_code = 0 # KSFT_PASS if len(alltests): req_plugins = pm.get_required_plugins(alltests) try: @@ -724,6 +725,8 @@ def set_operation_mode(pm, parser, args, remaining): print('The following plugins were not found:') print('{}'.format(pde.missing_pg)) catresults = test_runner(pm, args, alltests) + if catresults.count_failures() != 0: + exit_code = 1 # KSFT_FAIL if args.format == 'none': print('Test results output suppression requested\n') else: @@ -748,6 +751,8 @@ def set_operation_mode(pm, parser, args, remaining): gid=int(os.getenv('SUDO_GID'))) else: print('No tests found\n') + exit_code = 4 # KSFT_SKIP + exit(exit_code) def main(): """ @@ -767,8 +772,5 @@ def main(): set_operation_mode(pm, parser, args, remaining) - exit(0) - - if __name__ == "__main__": main()