diff mbox series

[v3,4/4] Jobs based on custom runners: add job definitions for QEMU's machines

Message ID 20201014052140.1146924-5-crosa@redhat.com (mailing list archive)
State New, archived
Headers show
Series GitLab Custom Runners and Jobs (was: QEMU Gating CI) | expand

Commit Message

Cleber Rosa Oct. 14, 2020, 5:21 a.m. UTC
The QEMU project has two machines (aarch64 and s390) that can be used
for jobs that do build and run tests.  This introduces those jobs,
which are a mapping of custom scripts used for the same purpose.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
 1 file changed, 192 insertions(+)

Comments

Daniel P. Berrangé Oct. 14, 2020, 5:46 p.m. UTC | #1
On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> The QEMU project has two machines (aarch64 and s390) that can be used
> for jobs that do build and run tests.  This introduces those jobs,
> which are a mapping of custom scripts used for the same purpose.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
>  1 file changed, 192 insertions(+)
> 
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> index 3004da2bda..5b51d1b336 100644
> --- a/.gitlab-ci.d/custom-runners.yml
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -12,3 +12,195 @@
>  # strategy.
>  variables:
>    GIT_SUBMODULE_STRATEGY: recursive
> +
> +# All ubuntu-18.04 jobs should run successfully in an environment
> +# setup by the scripts/ci/setup/build-environment.yml task
> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> +ubuntu-18.04-s390x-all-linux-static:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'

IIRC, in the previous v2 (or was it v1) we discussed changing this
so that users who provide their own runners, don't have to always
use the "staging" branch name.

IIUC, the key thing is that we don't want the job running on the
"master" or "stable-*" branches in the primary QEMU git. So could
check

   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'

which would let it work on users forks no matter what branch names
they use

What happens to the job if the user doesn't have runners ? Is it
simply skipped, or does the pipeline stall and get marked as failed ?

If the jobs aren't auto-skiped, we would need to add an env variable

   (
   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'
   )
   ||
   $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'

and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
in the web UI for their fork

That all said, I don't mind if you postpone this rules change to a
followup patch.

> + script:
> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763

This bug links to

  https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh
arg from all these jobs

> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> + - mkdir build
> + - cd build
> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> + - make --output-sync -j`nproc` check-tcg V=1



Regards,
Daniel
Cleber Rosa Oct. 14, 2020, 9:13 p.m. UTC | #2
On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> > The QEMU project has two machines (aarch64 and s390) that can be used
> > for jobs that do build and run tests.  This introduces those jobs,
> > which are a mapping of custom scripts used for the same purpose.
> > 
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> >  .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
> >  1 file changed, 192 insertions(+)
> > 
> > diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> > index 3004da2bda..5b51d1b336 100644
> > --- a/.gitlab-ci.d/custom-runners.yml
> > +++ b/.gitlab-ci.d/custom-runners.yml
> > @@ -12,3 +12,195 @@
> >  # strategy.
> >  variables:
> >    GIT_SUBMODULE_STRATEGY: recursive
> > +
> > +# All ubuntu-18.04 jobs should run successfully in an environment
> > +# setup by the scripts/ci/setup/build-environment.yml task
> > +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> > +ubuntu-18.04-s390x-all-linux-static:
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> 
> IIRC, in the previous v2 (or was it v1) we discussed changing this
> so that users who provide their own runners, don't have to always
> use the "staging" branch name.
>

Right, and what I got from that is that users can use a *prefix* as a
flag to indicate that they want the extra set of jobs.

> IIUC, the key thing is that we don't want the job running on the
> "master" or "stable-*" branches in the primary QEMU git. So could
> check
> 
>    $CI_PROJECT_NAMESPACE == 'qemu-project'
>    &&
>    $CI_COMMIT_BRANCH !~ '^master$'
>    &&
>    $CI_COMMIT_BRANCH !~ '^stable-$'
> 
> which would let it work on users forks no matter what branch names
> they use
> 
> What happens to the job if the user doesn't have runners ? Is it
> simply skipped, or does the pipeline stall and get marked as failed ?
>

The pipeline gets "stuck" (literaly, that's the status name it gets).
That's the main issue that made me believe that opting *in* (by using
a common branch name prefix) was the simpler solution.

> If the jobs aren't auto-skiped, we would need to add an env variable
> 
>    (
>    $CI_PROJECT_NAMESPACE == 'qemu-project'
>    &&
>    $CI_COMMIT_BRANCH !~ '^master$'
>    &&
>    $CI_COMMIT_BRANCH !~ '^stable-$'
>    )
>    ||
>    $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
> 
> and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
> in the web UI for their fork
>

We can do that, but I think it's more than we need.  The odds that a
user will have all of the same runners, and will be able to run all
the extra jobs, are very very low IMO.  Right from the start, very few
people have an s390 machine running Ubuntu 18.04.

So, I believe that whenever a user pushes to a branch such as
"staging-topic-foo", he will have to deal with some of the extra jobs
(such as canceling the ones that will never run) anyway.  Having to
deal with those on every single push, or alternatively having to turn
on/off $QEMU_ENABLE_CUSTOM_RUNNERS doesn't the best experience to me.

The "staging" prefix convention (with a better prefix name now or in
the future?) seems to result in the best experience to me.

> That all said, I don't mind if you postpone this rules change to a
> followup patch.
>

OK, let me know if you agree with my explanation above, or if you
really want to see a followup patch.

> > + script:
> > + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
> 
> This bug links to
> 
>   https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
> 
> which is marked as fixed. So I'm thinking we can drop the --disable-libssh
> arg from all these jobs
>

OK, I'll double check that and, if the fix is comfirmed beyond the bug
tracker, I'll update it.

Thanks!
- Cleber.

> > + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> > + - mkdir build
> > + - cd build
> > + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> > + - make --output-sync -j`nproc`
> > + - make --output-sync -j`nproc` check V=1
> > + - make --output-sync -j`nproc` check-tcg V=1
> 
> 
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Cleber Rosa Oct. 14, 2020, 11:24 p.m. UTC | #3
On Wed, Oct 14, 2020 at 05:14:01PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> > 
> > This bug links to
> > 
> >   https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
> > 
> > which is marked as fixed. So I'm thinking we can drop the --disable-libssh
> > arg from all these jobs
> >
> 
> OK, I'll double check that and, if the fix is comfirmed beyond the bug
> tracker, I'll update it.
> 

I was still able to reproduce the build failure with the latest packages.

The last update on the bug, is very misleading, but there's a hint of
its outcome:

---
Martin Wimpress (flexiondotorg) on 2019-12-17
tags:	added: rls-bb-notfixing
tags:	removed: bionic
no longer affects:	libssh (Ubuntu Bionic)
---

So, *notfixing* means it no longer affects that package on that
distro? Right...

Anyway, keeping this AS IS.

Regards,
- Cleber.
Thomas Huth Oct. 15, 2020, 6:09 a.m. UTC | #4
On 15/10/2020 01.24, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 05:14:01PM -0400, Cleber Rosa wrote:
>> On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
>>>
>>> This bug links to
>>>
>>>   https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
>>>
>>> which is marked as fixed. So I'm thinking we can drop the --disable-libssh
>>> arg from all these jobs
>>>
>>
>> OK, I'll double check that and, if the fix is comfirmed beyond the bug
>> tracker, I'll update it.
>>
> 
> I was still able to reproduce the build failure with the latest packages.
> 
> The last update on the bug, is very misleading, but there's a hint of
> its outcome:
> 
> ---
> Martin Wimpress (flexiondotorg) on 2019-12-17
> tags:	added: rls-bb-notfixing
> tags:	removed: bionic
> no longer affects:	libssh (Ubuntu Bionic)
> ---
> 
> So, *notfixing* means it no longer affects that package on that
> distro? Right...
> 
> Anyway, keeping this AS IS.

I can confirm that the libssh bug is still there in Ubuntu 18.04 ... when I
was updating our .travis.yml some weeks ago to stop using Xenial, I also hit
the bug again on Bionic.

Maybe rather a question for Peter, but what about updating the runners to
20.04 (Focal) instead? The libssh bug is gone there.

 Thomas
Daniel P. Berrangé Oct. 15, 2020, 8:32 a.m. UTC | #5
On Wed, Oct 14, 2020 at 07:24:30PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 05:14:01PM -0400, Cleber Rosa wrote:
> > On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> > > 
> > > This bug links to
> > > 
> > >   https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
> > > 
> > > which is marked as fixed. So I'm thinking we can drop the --disable-libssh
> > > arg from all these jobs
> > >
> > 
> > OK, I'll double check that and, if the fix is comfirmed beyond the bug
> > tracker, I'll update it.
> > 
> 
> I was still able to reproduce the build failure with the latest packages.
> 
> The last update on the bug, is very misleading, but there's a hint of
> its outcome:
> 
> ---
> Martin Wimpress (flexiondotorg) on 2019-12-17
> tags:	added: rls-bb-notfixing
> tags:	removed: bionic
> no longer affects:	libssh (Ubuntu Bionic)
> ---
> 
> So, *notfixing* means it no longer affects that package on that
> distro? Right...

Urgh, launchpad is such a confusing bug tracker :-(

> Anyway, keeping this AS IS.

Yep


Regards,
Daniel
Daniel P. Berrangé Oct. 15, 2020, 8:34 a.m. UTC | #6
On Wed, Oct 14, 2020 at 05:13:56PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> > On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> > > The QEMU project has two machines (aarch64 and s390) that can be used
> > > for jobs that do build and run tests.  This introduces those jobs,
> > > which are a mapping of custom scripts used for the same purpose.
> > > 
> > > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > > ---
> > >  .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
> > >  1 file changed, 192 insertions(+)
> > > 
> > > diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> > > index 3004da2bda..5b51d1b336 100644
> > > --- a/.gitlab-ci.d/custom-runners.yml
> > > +++ b/.gitlab-ci.d/custom-runners.yml
> > > @@ -12,3 +12,195 @@
> > >  # strategy.
> > >  variables:
> > >    GIT_SUBMODULE_STRATEGY: recursive
> > > +
> > > +# All ubuntu-18.04 jobs should run successfully in an environment
> > > +# setup by the scripts/ci/setup/build-environment.yml task
> > > +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> > > +ubuntu-18.04-s390x-all-linux-static:
> > > + needs: []
> > > + stage: build
> > > + tags:
> > > + - ubuntu_18.04
> > > + - s390x
> > > + rules:
> > > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> > 
> > IIRC, in the previous v2 (or was it v1) we discussed changing this
> > so that users who provide their own runners, don't have to always
> > use the "staging" branch name.
> >
> 
> Right, and what I got from that is that users can use a *prefix* as a
> flag to indicate that they want the extra set of jobs.
> 
> > IIUC, the key thing is that we don't want the job running on the
> > "master" or "stable-*" branches in the primary QEMU git. So could
> > check
> > 
> >    $CI_PROJECT_NAMESPACE == 'qemu-project'
> >    &&
> >    $CI_COMMIT_BRANCH !~ '^master$'
> >    &&
> >    $CI_COMMIT_BRANCH !~ '^stable-$'
> > 
> > which would let it work on users forks no matter what branch names
> > they use
> > 
> > What happens to the job if the user doesn't have runners ? Is it
> > simply skipped, or does the pipeline stall and get marked as failed ?
> >
> 
> The pipeline gets "stuck" (literaly, that's the status name it gets).
> That's the main issue that made me believe that opting *in* (by using
> a common branch name prefix) was the simpler solution.

Hmm, that's very annoying behaviour.

> > If the jobs aren't auto-skiped, we would need to add an env variable
> > 
> >    (
> >    $CI_PROJECT_NAMESPACE == 'qemu-project'
> >    &&
> >    $CI_COMMIT_BRANCH !~ '^master$'
> >    &&
> >    $CI_COMMIT_BRANCH !~ '^stable-$'
> >    )
> >    ||
> >    $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
> > 
> > and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
> > in the web UI for their fork
> >
> 
> We can do that, but I think it's more than we need.  The odds that a
> user will have all of the same runners, and will be able to run all
> the extra jobs, are very very low IMO.  Right from the start, very few
> people have an s390 machine running Ubuntu 18.04.
> 
> So, I believe that whenever a user pushes to a branch such as
> "staging-topic-foo", he will have to deal with some of the extra jobs
> (such as canceling the ones that will never run) anyway.  Having to
> deal with those on every single push, or alternatively having to turn
> on/off $QEMU_ENABLE_CUSTOM_RUNNERS doesn't the best experience to me.
> 
> The "staging" prefix convention (with a better prefix name now or in
> the future?) seems to result in the best experience to me.

Well "staging" prefix wasn';t appealing to me since none of the branches
I work on have such a name prefix.

> > That all said, I don't mind if you postpone this rules change to a
> > followup patch.
> >
> 
> OK, let me know if you agree with my explanation above, or if you
> really want to see a followup patch.

Just ignore it for now. I'll do more thinking to see if I can figure
out a more attractive solution.



Regards,
Daniel
diff mbox series

Patch

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 3004da2bda..5b51d1b336 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -12,3 +12,195 @@ 
 # strategy.
 variables:
   GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-18.04-s390x-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-18.04-s390x-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-20.04-aarch64-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-20.04-aarch64-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-20.04-aarch64-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1