mbox series

[v5,0/6] : add more rules and enable on CI

Message ID 20240718081605.452366-1-karthik.188@gmail.com (mailing list archive)
Headers show
Series : add more rules and enable on CI | expand

Message

karthik nayak July 18, 2024, 8:15 a.m. UTC
This is v5 of my series to add clang-format to the CI.

The series was mostly inspired by a patch sent recently to 'include
kh_foreach* macros in ForEachMacros' [1]. I was wondering why we don't
run the formatting on CI and reduce some of the workload of reviewers.

We have a '.clang-format' file to provide rules for code formatting.
The commits 1-3 aims to add more rules to the file.

Commit 4 enables CI action on GitHub and GitLab to also check for the
code formatting. Currently, it is allowed to fail. This way we can build
confidence and fine tune the values as needed and finally enforce the
check in a future patch. I'm not well versed with GitHub workflows, and
I mostly tried to copy existing work there. Expecting some feedback in
that section!

Commit 5 fixes an existing issue with the 'check-whitespace' job, which
is failing as success in the GitLab CI.

Commit 6 adds the `RemoveBracesLLVM` only in the context of the CI. If we
decide against it, we could drop this commit from the series.

1: https://lore.kernel.org/git/4e7893f5-2dd9-46cf-8a64-cf780f4e1730@web.de/

Changes against v4:
- While testing out temp file as source for the CI, I noticed that version
of clang-format on GitHub is much older than that of GitLab, in accordance
with that, I removed some of the rules which are newer and not supported.
- Cleaned up extra dependency addition in 'install-dependencies'. 
- Instead of murking the in-tree '.clang-format', now we create a temporary
file and use that. 

Test jobs:
- With error:
  - GitLab: https://gitlab.com/gitlab-org/git/-/jobs/7357893486
  - GitHub: https://github.com/git/git/actions/runs/9964605815/job/27533243002
- Without error: 
  - GitLab: https://gitlab.com/gitlab-org/git/-/jobs/7357903589
  - GitHub: https://github.com/git/git/actions/runs/9964631696/job/27533328240

Karthik Nayak (6):
  clang-format: indent preprocessor directives after hash
  clang-format: avoid spacing around bitfield colon
  clang-format: formalize some of the spacing rules
  ci: run style check on GitHub and GitLab
  check-whitespace: detect if no base_commit is provided
  ci/style-check: add `RemoveBracesLLVM` in CI job

 .clang-format                     | 25 +++++++++++++++++++++
 .github/workflows/check-style.yml | 34 ++++++++++++++++++++++++++++
 .gitlab-ci.yml                    | 37 ++++++++++++++++++++++++++++++-
 ci/check-whitespace.sh            | 10 +++++++--
 ci/install-dependencies.sh        |  4 ++++
 ci/run-style-check.sh             | 25 +++++++++++++++++++++
 6 files changed, 132 insertions(+), 3 deletions(-)
 create mode 100644 .github/workflows/check-style.yml
 create mode 100755 ci/run-style-check.sh

Range-diff against v4:
1:  6cf91ffc86 = 1:  6cf91ffc86 clang-format: indent preprocessor directives after hash
2:  beb002885f = 2:  beb002885f clang-format: avoid spacing around bitfield colon
3:  3031be43e7 < -:  ---------- clang-format: ensure files end with newlines
4:  bc1550e300 < -:  ---------- clang-format: replace deprecated option with 'SpacesInParens'
5:  4586c0094b ! 3:  3922529001 clang-format: formalize some of the spacing rules
    @@ .clang-format: PointerAlignment: Right
      # Put a space before opening parentheses only after control statement keywords.
      # void f() {
      #   if (true) {
    -@@ .clang-format: SpaceBeforeAssignmentOperators: true
    - # }
    - SpaceBeforeParens: ControlStatements
    +@@ .clang-format: SpaceBeforeParens: ControlStatements
    + # Don't insert spaces inside empty '()'
    + SpaceInEmptyParentheses: false
      
     +# No space before first '[' in arrays
     +# int a[5][5];     not      int a [5][5];
    @@ .clang-format: SpaceBeforeAssignmentOperators: true
     +# No space will be inserted into {}
     +# while (true) {}    not    while (true) { }
     +SpaceInEmptyBlock: false
    - 
    ++
      # The number of spaces before trailing line comments (// - comments).
      # This does not affect trailing block comments (/* - comments).
    + SpacesBeforeTrailingComments: 1
6:  c18cb23369 ! 4:  ae23eb5af8 ci: run style check on GitHub and GitLab
    @@ Commit message
         this job to fail, so we can validate if this is useful and eventually
         enforce it.
     
    -    For GitLab, we use the 'CI_MERGE_REQUEST_TARGET_BRANCH_SHA' variable by
    -    default to obtain the base SHA of the merged pipeline (which is only
    -    available for merged pipelines [1]). Otherwise we use the
    +    For GitHub, we allow the job to pass by adding 'continue-on-error: true'
    +    to the workflow. This means the job would show as passed, even if the
    +    style check failed. To know the status of the job, users have to
    +    manually check the logs.
    +
    +    For GitLab, we allow the job to pass by adding 'allow_failure: true', to
    +    the job. Unlike GitHub, here the job will show as failed with a yellow
    +    warning symbol, but the pipeline would still show as passed.
    +
    +    Also for GitLab, we use the 'CI_MERGE_REQUEST_TARGET_BRANCH_SHA'
    +    variable by default to obtain the base SHA of the merged pipeline (which
    +    is only available for merged pipelines [1]). Otherwise we use the
         'CI_MERGE_REQUEST_DIFF_BASE_SHA' variable.
     
         [1]: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html#predefined-variables-for-merge-request-pipelines
     
    +    Helped-by: Junio C Hamano <gitster@pobox.com>
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
      ## .github/workflows/check-style.yml (new) ##
    @@ .github/workflows/check-style.yml (new)
     +
     +jobs:
     +  check-style:
    ++    env:
    ++      CC: clang
    ++      jobname: ClangFormat
     +    runs-on: ubuntu-latest
     +    steps:
     +    - uses: actions/checkout@v4
     +      with:
     +        fetch-depth: 0
     +
    ++    - run: ci/install-dependencies.sh
    ++
     +    - name: git clang-format
     +      continue-on-error: true
     +      id: check_out
    @@ .gitlab-ci.yml: check-whitespace:
     +  allow_failure: true
     +  variables:
     +    CC: clang
    ++    jobname: ClangFormat
     +  before_script:
     +    - ./ci/install-dependencies.sh
     +  # Since $CI_MERGE_REQUEST_TARGET_BRANCH_SHA is only defined for merged
    @@ .gitlab-ci.yml: check-whitespace:
     +  # be defined in all pipelines.
     +  script:
     +    - |
    -+      if test -n "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA"
    -+      then
    -+        ./ci/run-style-check.sh "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA"
    -+      elif test -n "$CI_MERGE_REQUEST_DIFF_BASE_SHA"
    ++      R=${CI_MERGE_REQUEST_TARGET_BRANCH_SHA-${CI_MERGE_REQUEST_DIFF_BASE_SHA?}}
    ++
    ++      if test -z "$R"
     +      then
    -+        ./ci/run-style-check.sh "$CI_MERGE_REQUEST_DIFF_BASE_SHA"
    -+      else
    -+        echo "CI_MERGE_REQUEST_DIFF_BASE_SHA should always exist!"; exit 1
    ++        echo "CI_MERGE_REQUEST_DIFF_BASE_SHA should always exist!"
    ++        exit 1
     +      fi
    ++      ./ci/run-style-check.sh "$R"
     +  rules:
     +    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
     +
    @@ .gitlab-ci.yml: check-whitespace:
        variables:
     
      ## ci/install-dependencies.sh ##
    -@@ ci/install-dependencies.sh: ubuntu-*)
    - 		make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \
    - 		tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \
    - 		libemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \
    --		${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE
    -+		${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE clang-format
    +@@ ci/install-dependencies.sh: macos-*)
    + esac
      
    - 	mkdir --parents "$CUSTOM_PATH"
    - 	wget --quiet --directory-prefix="$CUSTOM_PATH" \
    + case "$jobname" in
    ++ClangFormat)
    ++	sudo apt-get -q update
    ++	sudo apt-get -q -y install clang-format
    ++	;;
    + StaticAnalysis)
    + 	sudo apt-get -q update
    + 	sudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \
     
      ## ci/run-style-check.sh (new) ##
     @@
7:  4d08c570bb ! 5:  a38cde03a8 check-whitespace: detect if no base_commit is provided
    @@ Commit message
     
         [1]: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html#predefined-variables-for-merge-request-pipelines
     
    +    Helped-by: Junio C Hamano <gitster@pobox.com>
         Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
     
      ## .gitlab-ci.yml ##
    @@ .gitlab-ci.yml: check-whitespace:
        script:
     -    - ./ci/check-whitespace.sh "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA"
     +    - |
    -+      if test -n "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA"
    -+      then
    -+        ./ci/check-whitespace.sh "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA"
    -+      elif test -n "$CI_MERGE_REQUEST_DIFF_BASE_SHA"
    ++      R=${CI_MERGE_REQUEST_TARGET_BRANCH_SHA-${CI_MERGE_REQUEST_DIFF_BASE_SHA?}}
    ++
    ++      if test -z "$R"
     +      then
    -+        ./ci/check-whitespace.sh "$CI_MERGE_REQUEST_DIFF_BASE_SHA"
    -+      else
    -+        echo "CI_MERGE_REQUEST_DIFF_BASE_SHA should always exist!"; exit 1
    ++        echo "CI_MERGE_REQUEST_DIFF_BASE_SHA should always exist!"
    ++        exit 1
     +      fi
    ++      ./ci/check-whitespace.sh "$R"
        rules:
          - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      
8:  2b39431f93 ! 6:  008e77bd0a ci/style-check: add `RemoveBracesLLVM` to '.clang-format'
    @@ Metadata
     Author: Karthik Nayak <karthik.188@gmail.com>
     
      ## Commit message ##
    -    ci/style-check: add `RemoveBracesLLVM` to '.clang-format'
    +    ci/style-check: add `RemoveBracesLLVM` in CI job
     
    -    For 'clang-format' setting  'RemoveBracesLLVM' to 'true', adds a check
    +    For 'clang-format', setting 'RemoveBracesLLVM' to 'true', adds a check
         to ensure we avoid curly braces for single-statement bodies in
         conditional blocks.
     
    @@ Commit message
         rule, adding it to the in-tree '.clang-format' could affect end-users.
         Let's only add it to the CI jobs for now. With time, we can evaluate
         its efficacy and decide if we want to add it to '.clang-format' or
    -    retract it entirely.
    +    retract it entirely. We do so, by adding the existing rules in
    +    '.clang-format' and this rule to a temp file outside the working tree,
    +    which is then used by 'git clang-format'. This ensures we don't murk
    +    with files in-tree.
     
         [1]: https://clang.llvm.org/docs/ClangFormatStyleOptions.html#removebracesllvm
     
    @@ ci/run-style-check.sh
      
      baseCommit=$1
      
    +-git clang-format --style file --diff --extensions c,h "$baseCommit"
     +# Remove optional braces of control statements (if, else, for, and while)
     +# according to the LLVM coding style. This avoids braces on simple
     +# single-statement bodies of statements but keeps braces if one side of
    @@ ci/run-style-check.sh
     +# While also ensuring that end-users are not affected directly.
     +#
     +# [1]: https://clang.llvm.org/docs/ClangFormatStyleOptions.html#removebracesllvm
    -+echo "RemoveBracesLLVM: true" >> .clang-format
    ++{
    ++	cat .clang-format
    ++	echo "RemoveBracesLLVM: true"
    ++} >/tmp/clang-format-rules
     +
    - git clang-format --style file --diff --extensions c,h "$baseCommit"
    ++git clang-format --style=file:/tmp/clang-format-rules \
    ++	--diff --extensions c,h "$baseCommit"

Comments

Junio C Hamano July 18, 2024, 3:09 p.m. UTC | #1
Karthik Nayak <karthik.188@gmail.com> writes:

> Changes against v4:
> - While testing out temp file as source for the CI, I noticed that version
> of clang-format on GitHub is much older than that of GitLab, in accordance
> with that, I removed some of the rules which are newer and not supported.

Let's see what we lost relative to the previous round.
 
        -# Insert a newline at end of file if missing
        -InsertNewlineAtEOF: true

I guess we can do "git diff --check $BASE...HEAD", if we wanted to,
which will trigger 'blank-at-eol' and other whitespace checks.

        +# Don't insert spaces inside empty '()'
        +SpaceInEmptyParentheses: false
        +
        -SpacesInParens: Custom
        -SpacesInParensOptions:
        -  # Don't insert spaces inside empty '()'
        -  InEmptyParentheses: false
        +# Don't insert spaces after '(' or before ')'
        +# f(arg);    not    f( arg );
        +SpacesInParentheses: false
 
This was to express what we used an older construct to express by
using a newer construct?  So we are not losing anything here.

So, I guess loss of changes to a few rules is nothing to shed tears
on.

Unrelated to the rules part, this caught my eyes, but

        diff --git c/ci/install-dependencies.sh w/ci/install-dependencies.sh
        index 8b01bfb89f..fb34ced8f0 100755
        --- c/ci/install-dependencies.sh
        +++ w/ci/install-dependencies.sh
        -		${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE clang-format
        +		${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE
 
this is to stop installing clang-format everywhere because we are
already installing it in ClangFormat job anyway?  A good change.

Thanks.