[2/2] iotests: Run pylint and mypy in a testcase
diff mbox series

Message ID 20200511163529.349329-3-kwolf@redhat.com
State New
Headers show
Series
  • iotests: Run pylint and mypy in a testcase
Related show

Commit Message

Kevin Wolf May 11, 2020, 4:35 p.m. UTC
We made sure that iotests.py passes pylint. It would be a shame if we
allowed new patches in that break this again, so let's just add a
meta-test case that runs pylint on it.

While we don't pass mypy --strict yet, we can already run it with a few
options that would be part of --strict to make sure that we won't
regress on these aspects at least until we can enable the full thing.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/297     | 44 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/297.out |  3 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 48 insertions(+)
 create mode 100755 tests/qemu-iotests/297
 create mode 100644 tests/qemu-iotests/297.out

Comments

Max Reitz May 13, 2020, 12:14 p.m. UTC | #1
On 11.05.20 18:35, Kevin Wolf wrote:
> We made sure that iotests.py passes pylint. It would be a shame if we
> allowed new patches in that break this again, so let's just add a
> meta-test case that runs pylint on it.
> 
> While we don't pass mypy --strict yet, we can already run it with a few
> options that would be part of --strict to make sure that we won't
> regress on these aspects at least until we can enable the full thing.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  tests/qemu-iotests/297     | 44 ++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/297.out |  3 +++
>  tests/qemu-iotests/group   |  1 +
>  3 files changed, 48 insertions(+)
>  create mode 100755 tests/qemu-iotests/297
>  create mode 100644 tests/qemu-iotests/297.out

Bit of a shame that this takes 8 s (on my machine at least) and will run
with every format/protocol combination unless I explictly exclude it
with -x meta...  But I suppose the actual problem here is the fact that
tests still can’t just define a “This is the format/protocol combination
I require” and then you can just let all tests run once with that
default combination.  (And maybe afterwards run all tests again with
some custom combinations, but only when that makes sense.)

Well.  Not a new problem.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Kevin Wolf May 13, 2020, 2:54 p.m. UTC | #2
Am 13.05.2020 um 14:14 hat Max Reitz geschrieben:
> On 11.05.20 18:35, Kevin Wolf wrote:
> > We made sure that iotests.py passes pylint. It would be a shame if we
> > allowed new patches in that break this again, so let's just add a
> > meta-test case that runs pylint on it.
> > 
> > While we don't pass mypy --strict yet, we can already run it with a few
> > options that would be part of --strict to make sure that we won't
> > regress on these aspects at least until we can enable the full thing.
> > 
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> >  tests/qemu-iotests/297     | 44 ++++++++++++++++++++++++++++++++++++++
> >  tests/qemu-iotests/297.out |  3 +++
> >  tests/qemu-iotests/group   |  1 +
> >  3 files changed, 48 insertions(+)
> >  create mode 100755 tests/qemu-iotests/297
> >  create mode 100644 tests/qemu-iotests/297.out
> 
> Bit of a shame that this takes 8 s (on my machine at least) and will run
> with every format/protocol combination unless I explictly exclude it
> with -x meta...

Yes, it's surprising how slow these tools are. At least mypy caches some
stuff, so the second run is considerably faster, but pylint doesn't do
that.

I wonder if there is some overlap between mypy and pylint that we could
configure away in pylint to speed it up.

> But I suppose the actual problem here is the fact that
> tests still can’t just define a “This is the format/protocol combination
> I require” and then you can just let all tests run once with that
> default combination.  (And maybe afterwards run all tests again with
> some custom combinations, but only when that makes sense.)

It's probably not hard to find more "actual problems" in the test
harness...

Kevin

Patch
diff mbox series

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
new file mode 100755
index 0000000000..5c5420712b
--- /dev/null
+++ b/tests/qemu-iotests/297
@@ -0,0 +1,44 @@ 
+#!/usr/bin/env bash
+#
+# Copyright (C) 2020 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+seq=$(basename $0)
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+# get standard environment
+. ./common.rc
+
+if ! type -p "pylint-3" > /dev/null; then
+    _notrun "pylint-3 not found"
+fi
+if ! type -p "mypy" > /dev/null; then
+    _notrun "mypy not found"
+fi
+
+pylint-3 --score=n iotests.py
+
+MYPYPATH=../../python/ mypy --warn-unused-configs --disallow-subclassing-any \
+    --disallow-any-generics --disallow-incomplete-defs \
+    --disallow-untyped-decorators --no-implicit-optional \
+    --warn-redundant-casts --warn-unused-ignores \
+    --no-implicit-reexport iotests.py
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out
new file mode 100644
index 0000000000..6acc843649
--- /dev/null
+++ b/tests/qemu-iotests/297.out
@@ -0,0 +1,3 @@ 
+QA output created by 297
+Success: no issues found in 1 source file
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index fe649c5b73..c514975880 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -299,3 +299,4 @@ 
 289 rw quick
 290 rw auto quick
 292 rw auto quick
+297 meta