| Message ID | 20250325200026.344006-15-thuth@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Convert remaining Avocado tests to functional | expand |
On Tue, Mar 25, 2025 at 09:00:22PM +0100, Thomas Huth wrote: > From: Thomas Huth <thuth@redhat.com> > > Since we don't run the Avocado jobs in the CI anymore, rename > these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL_TESTING. > > Also, there was a mismatch between the documentation and the > implementation of QEMU_CI_AVOCADO_TESTING: While the documentation > said that you had to "Set this variable to have the tests using the > Avocado framework run automatically", you indeed needed to set it > to make the pipelines appear in your dashboard - but they were never > run automatically in forks and had to be triggered manually. Let's > improve this now: No need to hide these pipelines from the users > by default anymore (the functional tests should be stable enough > nowadays), and rather allow the users to run the pipelines auto- > matically with this switch now instead, as was documented. > > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > docs/devel/testing/ci-jobs.rst.inc | 19 +++++++++---------- > .gitlab-ci.d/base.yml | 8 ++------ > .gitlab-ci.d/buildtest-template.yml | 2 +- > 3 files changed, 12 insertions(+), 17 deletions(-) > > diff --git a/docs/devel/testing/ci-jobs.rst.inc b/docs/devel/testing/ci-jobs.rst.inc > index 3756bbe3554..4f4c96dd883 100644 > --- a/docs/devel/testing/ci-jobs.rst.inc > +++ b/docs/devel/testing/ci-jobs.rst.inc > @@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH > The job is for publishing content after a branch has been > merged into the upstream default branch. > > -QEMU_JOB_AVOCADO > -~~~~~~~~~~~~~~~~ > +QEMU_JOB_FUNCTIONAL > +~~~~~~~~~~~~~~~~~~~ > > -The job runs the Avocado integration test suite > +The job runs the functional test suite > > Contributor controlled runtime variables > ---------------------------------------- > @@ -149,13 +149,12 @@ the jobs to be manually started from the UI > Set this variable to 2 to create the pipelines and run all > the jobs immediately, as was the historical behaviour > > -QEMU_CI_AVOCADO_TESTING > -~~~~~~~~~~~~~~~~~~~~~~~ > -By default, tests using the Avocado framework are not run automatically in > -the pipelines (because multiple artifacts have to be downloaded, and if > -these artifacts are not already cached, downloading them make the jobs > -reach the timeout limit). Set this variable to have the tests using the > -Avocado framework run automatically. > +QEMU_CI_FUNCTIONAL_TESTING Both old and new terms are quite verbose, how about dropping '_TESTING' suffix ? With, or without that rename though: Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel
diff --git a/docs/devel/testing/ci-jobs.rst.inc b/docs/devel/testing/ci-jobs.rst.inc index 3756bbe3554..4f4c96dd883 100644 --- a/docs/devel/testing/ci-jobs.rst.inc +++ b/docs/devel/testing/ci-jobs.rst.inc @@ -126,10 +126,10 @@ QEMU_JOB_PUBLISH The job is for publishing content after a branch has been merged into the upstream default branch. -QEMU_JOB_AVOCADO -~~~~~~~~~~~~~~~~ +QEMU_JOB_FUNCTIONAL +~~~~~~~~~~~~~~~~~~~ -The job runs the Avocado integration test suite +The job runs the functional test suite Contributor controlled runtime variables ---------------------------------------- @@ -149,13 +149,12 @@ the jobs to be manually started from the UI Set this variable to 2 to create the pipelines and run all the jobs immediately, as was the historical behaviour -QEMU_CI_AVOCADO_TESTING -~~~~~~~~~~~~~~~~~~~~~~~ -By default, tests using the Avocado framework are not run automatically in -the pipelines (because multiple artifacts have to be downloaded, and if -these artifacts are not already cached, downloading them make the jobs -reach the timeout limit). Set this variable to have the tests using the -Avocado framework run automatically. +QEMU_CI_FUNCTIONAL_TESTING +~~~~~~~~~~~~~~~~~~~~~~~~~~ +By default, tests using the functional framework are not run automatically +in the pipelines (because multiple artifacts have to be downloaded, which +might cause a lot of network traffic). Set this variable to have the tests +using the functional framework run automatically. Other misc variables -------------------- diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml index 25b88aaa06a..172028e1731 100644 --- a/.gitlab-ci.d/base.yml +++ b/.gitlab-ci.d/base.yml @@ -69,10 +69,6 @@ variables: - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' when: never - # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set - - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' - when: never - ############################################################# # Stage 2: fine tune execution of jobs in specific scenarios @@ -101,8 +97,8 @@ variables: when: manual allow_failure: true - # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset - - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' + # Functional jobs can be manually started in forks + - if: '$QEMU_JOB_FUNCTIONAL && $QEMU_CI_FUNCTIONAL_TESTING != "1" && $CI_PROJECT_NAMESPACE != $QEMU_CI_UPSTREAM' when: manual allow_failure: true diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml index 13fa4f4a4fc..d4f145fdb58 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -114,4 +114,4 @@ - cd build - du -chs ${CI_PROJECT_DIR}/*-cache variables: - QEMU_JOB_AVOCADO: 1 + QEMU_JOB_FUNCTIONAL: 1