diff mbox series

[1/1] Travis-CI: run flake8 on Python code

Message ID 20180815210853.15495-1-nicolas.iooss@m4x.org (mailing list archive)
State Not Applicable
Headers show
Series [1/1] Travis-CI: run flake8 on Python code | expand

Commit Message

Nicolas Iooss Aug. 15, 2018, 9:08 p.m. UTC 
flake8 is a Python linter which is able to detect issues in Python code
(syntax errors, undefined variables, etc.). It has been used to find
bugs in the project. In order to prevent the introduction of new bugs
which can be detected by it, add a script which runs it and use it in
Travis-CI.

flake8 can be used to detect code which is not written according to PEP8
style guide (which forbids whitespaces in some places, enforces the use
of space-indenting, specifies how many blank lines are used between
functions, etc.). As SELinux code does not follow this style guide,
scripts/run-flake8 disables many warnings related to this when running
the linter.

In order to silence flake8 warnings, the Python code can also be
modified. However fixing every "do not use bare 'except'" in the project
needs to be done carefully and takes much time.
This is why the warnings which are disabled have been ordered in three
lists:
* The warnings which can be activated in a not-so-distant future after
  the code has been modified.
* The warnings related to PEP8 which cannot be activated without a major
  cleaning work of the codebase (for example to modify white spaces)
* The warnings which are introduced by code generated by SWIG 3.0.12,
  which would require patches in SWIG in order to be activated (there
  is right now only one such warning).

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
---
 .travis.yml        |  6 ++++
 scripts/run-flake8 | 79 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)
 create mode 100755 scripts/run-flake8

Comments

Nicolas Iooss Aug. 18, 2018, 10:10 a.m. UTC | #1 
On Thu, Aug 16, 2018 at 10:26 PM William Roberts
<bill.c.roberts@gmail.com> wrote:
>
> I have no problems with this, seems like a reasonable plan going forward.

Thanks. I applied this patch, so now all of our Python code gets
checked for syntax errors every time the project is updated :)

Nicolas

> On Wed, Aug 15, 2018 at 2:08 PM, Nicolas Iooss <nicolas.iooss@m4x.org> wrote:
>>
>> flake8 is a Python linter which is able to detect issues in Python code
>> (syntax errors, undefined variables, etc.). It has been used to find
>> bugs in the project. In order to prevent the introduction of new bugs
>> which can be detected by it, add a script which runs it and use it in
>> Travis-CI.
>>
>> flake8 can be used to detect code which is not written according to PEP8
>> style guide (which forbids whitespaces in some places, enforces the use
>> of space-indenting, specifies how many blank lines are used between
>> functions, etc.). As SELinux code does not follow this style guide,
>> scripts/run-flake8 disables many warnings related to this when running
>> the linter.
>>
>> In order to silence flake8 warnings, the Python code can also be
>> modified. However fixing every "do not use bare 'except'" in the project
>> needs to be done carefully and takes much time.
>> This is why the warnings which are disabled have been ordered in three
>> lists:
>> * The warnings which can be activated in a not-so-distant future after
>>   the code has been modified.
>> * The warnings related to PEP8 which cannot be activated without a major
>>   cleaning work of the codebase (for example to modify white spaces)
>> * The warnings which are introduced by code generated by SWIG 3.0.12,
>>   which would require patches in SWIG in order to be activated (there
>>   is right now only one such warning).
>>
>> Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
>> ---
>>  .travis.yml        |  6 ++++
>>  scripts/run-flake8 | 79 ++++++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 85 insertions(+)
>>  create mode 100755 scripts/run-flake8
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 0612eb5480e5..09dd4749975e 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -83,6 +83,9 @@ install:
>>          rm python.tar.bz2 ;
>>      fi
>>
>> +  # Install flake8 for the given python version
>> +  - $VIRTUAL_ENV/bin/pip install flake8
>> +
>>  before_script:
>>    # clang on Travis-CI 14.04 environment is too old to support -Wdouble-promotion
>>    - if "$CC" --version |grep -q clang; then sed 's/ -Wdouble-promotion / /' -i libselinux/src/Makefile libselinux/utils/Makefile ; fi
>> @@ -141,6 +144,9 @@ script:
>>    - $PYTHON -c 'import selinux;import selinux.audit2why;import semanage;print(selinux.is_selinux_enabled())'
>>    - $RUBY -e 'require "selinux";require "semanage";puts Selinux::is_selinux_enabled()'
>>
>> +  # Run Python linter
>> +  - PATH="$VIRTUAL_ENV/bin:$PATH" ./scripts/run-flake8
>> +
>>    # Remove every installed files
>>    - rm -rf "$DESTDIR"
>>
>> diff --git a/scripts/run-flake8 b/scripts/run-flake8
>> new file mode 100755
>> index 000000000000..8a1f490b8a62
>> --- /dev/null
>> +++ b/scripts/run-flake8
>> @@ -0,0 +1,79 @@
>> +#!/bin/sh
>> +# Run flake8 (Python linter) on the source directory or on the given files/directories
>> +
>> +# Run on the base directory if no argument has been given
>> +if [ $# -eq 0 ] ; then
>> +    cd "$(dirname -- "$0")/.." || exit $?
>> +fi
>> +
>> +# Assign each ignore warning on a line, in order to ease testing enabling the warning again
>> +IGNORE_LIST=''
>> +
>> +# Important warnings that should be fixed
>> +# (Comment one line and run this script in order to see where the warning occurs)
>> +IGNORE_LIST="$IGNORE_LIST,W191" # indentation contains tabs
>> +
>> +IGNORE_LIST="$IGNORE_LIST,E101" # indentation contains mixed spaces and tabs
>> +IGNORE_LIST="$IGNORE_LIST,E703" # statement ends with a semicolon
>> +IGNORE_LIST="$IGNORE_LIST,E711" # comparison to None should be 'if cond is not None:'
>> +IGNORE_LIST="$IGNORE_LIST,E712" # comparison to False should be 'if cond is False:' or 'if not cond:'
>> +IGNORE_LIST="$IGNORE_LIST,E722" # do not use bare 'except'
>> +IGNORE_LIST="$IGNORE_LIST,E999" # TabError: inconsistent use of tabs and spaces in indentation
>> +
>> +IGNORE_LIST="$IGNORE_LIST,F401" # module imported but unused
>> +IGNORE_LIST="$IGNORE_LIST,F812" # list comprehension redefines 'f', in lex.py and yacc.py
>> +IGNORE_LIST="$IGNORE_LIST,F841" # local variable '...' is assigned to but never used
>> +
>> +
>> +# Less-important warnings
>> +IGNORE_LIST="$IGNORE_LIST,W291" # trailing whitespace
>> +IGNORE_LIST="$IGNORE_LIST,W293" # blank line contains whitespace
>> +IGNORE_LIST="$IGNORE_LIST,W391" # blank line at end of file
>> +IGNORE_LIST="$IGNORE_LIST,W503" # line break before binary operator
>> +IGNORE_LIST="$IGNORE_LIST,W504" # line break after binary operator
>> +
>> +IGNORE_LIST="$IGNORE_LIST,E111" # indentation is not a multiple of four
>> +IGNORE_LIST="$IGNORE_LIST,E114" # indentation is not a multiple of four (comment)
>> +IGNORE_LIST="$IGNORE_LIST,E115" # expected an indented block (comment)
>> +IGNORE_LIST="$IGNORE_LIST,E116" # unexpected indentation (comment)
>> +IGNORE_LIST="$IGNORE_LIST,E122" # continuation line missing indentation or outdented
>> +IGNORE_LIST="$IGNORE_LIST,E123" # closing bracket does not match indentation of opening bracket's line
>> +IGNORE_LIST="$IGNORE_LIST,E126" # continuation line over-indented for hanging indent
>> +IGNORE_LIST="$IGNORE_LIST,E127" # continuation line over-indented for visual indent
>> +IGNORE_LIST="$IGNORE_LIST,E128" # continuation line under-indented for visual indent
>> +IGNORE_LIST="$IGNORE_LIST,E201" # whitespace after '['
>> +IGNORE_LIST="$IGNORE_LIST,E202" # whitespace before '}'
>> +IGNORE_LIST="$IGNORE_LIST,E203" # whitespace before ':'
>> +IGNORE_LIST="$IGNORE_LIST,E211" # whitespace before '('
>> +IGNORE_LIST="$IGNORE_LIST,E221" # multiple spaces before operator
>> +IGNORE_LIST="$IGNORE_LIST,E222" # multiple spaces after operator
>> +IGNORE_LIST="$IGNORE_LIST,E225" # missing whitespace around operator
>> +IGNORE_LIST="$IGNORE_LIST,E226" # missing whitespace around arithmetic operator
>> +IGNORE_LIST="$IGNORE_LIST,E231" # missing whitespace after ','
>> +IGNORE_LIST="$IGNORE_LIST,E241" # multiple spaces after ':'
>> +IGNORE_LIST="$IGNORE_LIST,E251" # unexpected spaces around keyword / parameter equals
>> +IGNORE_LIST="$IGNORE_LIST,E261" # at least two spaces before inline comment
>> +IGNORE_LIST="$IGNORE_LIST,E265" # block comment should start with '# '
>> +IGNORE_LIST="$IGNORE_LIST,E266" # too many leading '#' for block comment
>> +IGNORE_LIST="$IGNORE_LIST,E272" # multiple spaces before keyword
>> +IGNORE_LIST="$IGNORE_LIST,E301" # expected 1 blank line, found 0
>> +IGNORE_LIST="$IGNORE_LIST,E302" # expected 2 blank lines, found 1
>> +IGNORE_LIST="$IGNORE_LIST,E303" # too many blank lines
>> +IGNORE_LIST="$IGNORE_LIST,E305" # expected 2 blank lines after class or function definition, found 0
>> +IGNORE_LIST="$IGNORE_LIST,E306" # expected 1 blank line before a nested definition, found 0
>> +IGNORE_LIST="$IGNORE_LIST,E401" # multiple imports on one line
>> +IGNORE_LIST="$IGNORE_LIST,E402" # module level import not at top of file
>> +IGNORE_LIST="$IGNORE_LIST,E501" # line too long
>> +IGNORE_LIST="$IGNORE_LIST,E701" # multiple statements on one line (colon)
>> +IGNORE_LIST="$IGNORE_LIST,E704" # multiple statements on one line (def)
>> +IGNORE_LIST="$IGNORE_LIST,E731" # do not assign a lambda expression, use a def
>> +IGNORE_LIST="$IGNORE_LIST,E741" # ambiguous variable name 'l' / 'I'
>> +
>> +IGNORE_LIST="$IGNORE_LIST,F403" # 'from ... import *' used; unable to detect undefined names
>> +IGNORE_LIST="$IGNORE_LIST,F405" # '...' may be undefined, or defined from star imports
>> +
>> +# Ignore errors from files generated from SWIG
>> +IGNORE_LIST="$IGNORE_LIST,F811" # redefinition of unused ...
>> +
>> +
>> +exec flake8 --max-line-length=120 --builtins='_,unicode,lextab,parsetab' --ignore=",$IGNORE_LIST" "$@"
>> --
>> 2.18.0
>>
>> _______________________________________________
>> Selinux mailing list
>> Selinux@tycho.nsa.gov
>> To unsubscribe, send email to Selinux-leave@tycho.nsa.gov.
>> To get help, send an email containing "help" to Selinux-request@tycho.nsa.gov.
>
>
diff mbox series

Patch

diff --git a/.travis.yml b/.travis.yml
index 0612eb5480e5..09dd4749975e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -83,6 +83,9 @@  install:
         rm python.tar.bz2 ;
     fi
 
+  # Install flake8 for the given python version
+  - $VIRTUAL_ENV/bin/pip install flake8
+
 before_script:
   # clang on Travis-CI 14.04 environment is too old to support -Wdouble-promotion
   - if "$CC" --version |grep -q clang; then sed 's/ -Wdouble-promotion / /' -i libselinux/src/Makefile libselinux/utils/Makefile ; fi
@@ -141,6 +144,9 @@  script:
   - $PYTHON -c 'import selinux;import selinux.audit2why;import semanage;print(selinux.is_selinux_enabled())'
   - $RUBY -e 'require "selinux";require "semanage";puts Selinux::is_selinux_enabled()'
 
+  # Run Python linter
+  - PATH="$VIRTUAL_ENV/bin:$PATH" ./scripts/run-flake8
+
   # Remove every installed files
   - rm -rf "$DESTDIR"
 
diff --git a/scripts/run-flake8 b/scripts/run-flake8
new file mode 100755
index 000000000000..8a1f490b8a62
--- /dev/null
+++ b/scripts/run-flake8
@@ -0,0 +1,79 @@ 
+#!/bin/sh
+# Run flake8 (Python linter) on the source directory or on the given files/directories
+
+# Run on the base directory if no argument has been given
+if [ $# -eq 0 ] ; then
+    cd "$(dirname -- "$0")/.." || exit $?
+fi
+
+# Assign each ignore warning on a line, in order to ease testing enabling the warning again
+IGNORE_LIST=''
+
+# Important warnings that should be fixed
+# (Comment one line and run this script in order to see where the warning occurs)
+IGNORE_LIST="$IGNORE_LIST,W191" # indentation contains tabs
+
+IGNORE_LIST="$IGNORE_LIST,E101" # indentation contains mixed spaces and tabs
+IGNORE_LIST="$IGNORE_LIST,E703" # statement ends with a semicolon
+IGNORE_LIST="$IGNORE_LIST,E711" # comparison to None should be 'if cond is not None:'
+IGNORE_LIST="$IGNORE_LIST,E712" # comparison to False should be 'if cond is False:' or 'if not cond:'
+IGNORE_LIST="$IGNORE_LIST,E722" # do not use bare 'except'
+IGNORE_LIST="$IGNORE_LIST,E999" # TabError: inconsistent use of tabs and spaces in indentation
+
+IGNORE_LIST="$IGNORE_LIST,F401" # module imported but unused
+IGNORE_LIST="$IGNORE_LIST,F812" # list comprehension redefines 'f', in lex.py and yacc.py
+IGNORE_LIST="$IGNORE_LIST,F841" # local variable '...' is assigned to but never used
+
+
+# Less-important warnings
+IGNORE_LIST="$IGNORE_LIST,W291" # trailing whitespace
+IGNORE_LIST="$IGNORE_LIST,W293" # blank line contains whitespace
+IGNORE_LIST="$IGNORE_LIST,W391" # blank line at end of file
+IGNORE_LIST="$IGNORE_LIST,W503" # line break before binary operator
+IGNORE_LIST="$IGNORE_LIST,W504" # line break after binary operator
+
+IGNORE_LIST="$IGNORE_LIST,E111" # indentation is not a multiple of four
+IGNORE_LIST="$IGNORE_LIST,E114" # indentation is not a multiple of four (comment)
+IGNORE_LIST="$IGNORE_LIST,E115" # expected an indented block (comment)
+IGNORE_LIST="$IGNORE_LIST,E116" # unexpected indentation (comment)
+IGNORE_LIST="$IGNORE_LIST,E122" # continuation line missing indentation or outdented
+IGNORE_LIST="$IGNORE_LIST,E123" # closing bracket does not match indentation of opening bracket's line
+IGNORE_LIST="$IGNORE_LIST,E126" # continuation line over-indented for hanging indent
+IGNORE_LIST="$IGNORE_LIST,E127" # continuation line over-indented for visual indent
+IGNORE_LIST="$IGNORE_LIST,E128" # continuation line under-indented for visual indent
+IGNORE_LIST="$IGNORE_LIST,E201" # whitespace after '['
+IGNORE_LIST="$IGNORE_LIST,E202" # whitespace before '}'
+IGNORE_LIST="$IGNORE_LIST,E203" # whitespace before ':'
+IGNORE_LIST="$IGNORE_LIST,E211" # whitespace before '('
+IGNORE_LIST="$IGNORE_LIST,E221" # multiple spaces before operator
+IGNORE_LIST="$IGNORE_LIST,E222" # multiple spaces after operator
+IGNORE_LIST="$IGNORE_LIST,E225" # missing whitespace around operator
+IGNORE_LIST="$IGNORE_LIST,E226" # missing whitespace around arithmetic operator
+IGNORE_LIST="$IGNORE_LIST,E231" # missing whitespace after ','
+IGNORE_LIST="$IGNORE_LIST,E241" # multiple spaces after ':'
+IGNORE_LIST="$IGNORE_LIST,E251" # unexpected spaces around keyword / parameter equals
+IGNORE_LIST="$IGNORE_LIST,E261" # at least two spaces before inline comment
+IGNORE_LIST="$IGNORE_LIST,E265" # block comment should start with '# '
+IGNORE_LIST="$IGNORE_LIST,E266" # too many leading '#' for block comment
+IGNORE_LIST="$IGNORE_LIST,E272" # multiple spaces before keyword
+IGNORE_LIST="$IGNORE_LIST,E301" # expected 1 blank line, found 0
+IGNORE_LIST="$IGNORE_LIST,E302" # expected 2 blank lines, found 1
+IGNORE_LIST="$IGNORE_LIST,E303" # too many blank lines
+IGNORE_LIST="$IGNORE_LIST,E305" # expected 2 blank lines after class or function definition, found 0
+IGNORE_LIST="$IGNORE_LIST,E306" # expected 1 blank line before a nested definition, found 0
+IGNORE_LIST="$IGNORE_LIST,E401" # multiple imports on one line
+IGNORE_LIST="$IGNORE_LIST,E402" # module level import not at top of file
+IGNORE_LIST="$IGNORE_LIST,E501" # line too long
+IGNORE_LIST="$IGNORE_LIST,E701" # multiple statements on one line (colon)
+IGNORE_LIST="$IGNORE_LIST,E704" # multiple statements on one line (def)
+IGNORE_LIST="$IGNORE_LIST,E731" # do not assign a lambda expression, use a def
+IGNORE_LIST="$IGNORE_LIST,E741" # ambiguous variable name 'l' / 'I'
+
+IGNORE_LIST="$IGNORE_LIST,F403" # 'from ... import *' used; unable to detect undefined names
+IGNORE_LIST="$IGNORE_LIST,F405" # '...' may be undefined, or defined from star imports
+
+# Ignore errors from files generated from SWIG
+IGNORE_LIST="$IGNORE_LIST,F811" # redefinition of unused ...
+
+
+exec flake8 --max-line-length=120 --builtins='_,unicode,lextab,parsetab' --ignore=",$IGNORE_LIST" "$@"