Message ID | 20210319004300.3800583-1-f4bug@amsat.org (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | gitlab-ci: Restrict jobs using Docker to runners having 'docker' tag | expand |
On 19/03/2021 01.43, Philippe Mathieu-Daudé wrote: > When a job is based on a Docker image [1], or is using a Docker > service, it requires a runner with Docker installed. > > Gitlab shared runners provide the 'docker' tag when they have it > installed. > > Are Gitlab shared runners are limited resources, we'd like to s/Are/As/ > add more runners to QEMU repositories hosted on Gitlab. If a > runner doesn't provide Docker, our jobs requiring it will fail. > > Use the standard 'docker' tag to mark the jobs requiring Docker > on the runner. > > [1] https://docs.gitlab.com/ee/ci/yaml/#image > [2] https://docs.gitlab.com/ee/ci/yaml/#services > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> [...] > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index f65cb11c4d3..d4511cf7dea 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -14,6 +14,8 @@ include: > - local: '/.gitlab-ci.d/crossbuilds.yml' > > .native_build_job_template: &native_build_job_definition > + tags: > + - docker > stage: build > image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest > before_script: > @@ -38,6 +40,8 @@ include: > fi > > .native_test_job_template: &native_test_job_definition > + tags: > + - docker > stage: test > image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest > script: If you add it to the templates ... won't this disable most of the jobs on the dedicated runners that don't have docker? Wouldn't it be better to add the tag only to the jobs that run "make check-tcg" ? Thomas
On 03/19/21 06:40, Thomas Huth wrote: > On 19/03/2021 01.43, Philippe Mathieu-Daudé wrote: >> When a job is based on a Docker image [1], or is using a Docker >> service, it requires a runner with Docker installed. >> >> Gitlab shared runners provide the 'docker' tag when they have it >> installed. >> >> Are Gitlab shared runners are limited resources, we'd like to > > s/Are/As/ > >> add more runners to QEMU repositories hosted on Gitlab. If a >> runner doesn't provide Docker, our jobs requiring it will fail. >> >> Use the standard 'docker' tag to mark the jobs requiring Docker >> on the runner. >> >> [1] https://docs.gitlab.com/ee/ci/yaml/#image >> [2] https://docs.gitlab.com/ee/ci/yaml/#services >> >> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > [...] >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >> index f65cb11c4d3..d4511cf7dea 100644 >> --- a/.gitlab-ci.yml >> +++ b/.gitlab-ci.yml >> @@ -14,6 +14,8 @@ include: >> - local: '/.gitlab-ci.d/crossbuilds.yml' >> .native_build_job_template: &native_build_job_definition >> + tags: >> + - docker >> stage: build >> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest >> before_script: >> @@ -38,6 +40,8 @@ include: >> fi >> .native_test_job_template: &native_test_job_definition >> + tags: >> + - docker >> stage: test >> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest >> script: > > If you add it to the templates ... won't this disable most of the jobs > on the dedicated runners that don't have docker? Wouldn't it be better > to add the tag only to the jobs that run "make check-tcg" ? (I don't know if the docker dependency is presently expressed with the exact granularity that we need, but I'm willing to ACK the edk2 part, on principle. We should be explicit about dependencies.) Thanks Laszlo
On 3/19/21 6:40 AM, Thomas Huth wrote: > On 19/03/2021 01.43, Philippe Mathieu-Daudé wrote: >> When a job is based on a Docker image [1], or is using a Docker >> service, it requires a runner with Docker installed. >> >> Gitlab shared runners provide the 'docker' tag when they have it >> installed. >> >> Are Gitlab shared runners are limited resources, we'd like to > > s/Are/As/ > >> add more runners to QEMU repositories hosted on Gitlab. If a >> runner doesn't provide Docker, our jobs requiring it will fail. >> >> Use the standard 'docker' tag to mark the jobs requiring Docker >> on the runner. >> >> [1] https://docs.gitlab.com/ee/ci/yaml/#image >> [2] https://docs.gitlab.com/ee/ci/yaml/#services >> >> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > [...] >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >> index f65cb11c4d3..d4511cf7dea 100644 >> --- a/.gitlab-ci.yml >> +++ b/.gitlab-ci.yml >> @@ -14,6 +14,8 @@ include: >> - local: '/.gitlab-ci.d/crossbuilds.yml' >> .native_build_job_template: &native_build_job_definition >> + tags: >> + - docker >> stage: build >> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest >> before_script: >> @@ -38,6 +40,8 @@ include: >> fi >> .native_test_job_template: &native_test_job_definition >> + tags: >> + - docker >> stage: test >> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest >> script: > > If you add it to the templates ... won't this disable most of the jobs > on the dedicated runners that don't have docker? Wouldn't it be better > to add the tag only to the jobs that run "make check-tcg" ? But this is the point, if a runner doesn't have Docker, it can not run the job...
On 4/14/21 12:10 PM, Philippe Mathieu-Daudé wrote: > On 3/19/21 6:40 AM, Thomas Huth wrote: >> On 19/03/2021 01.43, Philippe Mathieu-Daudé wrote: >>> When a job is based on a Docker image [1], or is using a Docker >>> service, it requires a runner with Docker installed. >>> >>> Gitlab shared runners provide the 'docker' tag when they have it >>> installed. >>> >>> Are Gitlab shared runners are limited resources, we'd like to >> >> s/Are/As/ >> >>> add more runners to QEMU repositories hosted on Gitlab. If a >>> runner doesn't provide Docker, our jobs requiring it will fail. >>> >>> Use the standard 'docker' tag to mark the jobs requiring Docker >>> on the runner. >>> >>> [1] https://docs.gitlab.com/ee/ci/yaml/#image >>> [2] https://docs.gitlab.com/ee/ci/yaml/#services >>> >>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> >> [...] >>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >>> index f65cb11c4d3..d4511cf7dea 100644 >>> --- a/.gitlab-ci.yml >>> +++ b/.gitlab-ci.yml >>> @@ -14,6 +14,8 @@ include: >>> - local: '/.gitlab-ci.d/crossbuilds.yml' >>> .native_build_job_template: &native_build_job_definition >>> + tags: >>> + - docker >>> stage: build >>> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest >>> before_script: >>> @@ -38,6 +40,8 @@ include: >>> fi >>> .native_test_job_template: &native_test_job_definition >>> + tags: >>> + - docker >>> stage: test >>> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest >>> script: >> >> If you add it to the templates ... won't this disable most of the jobs >> on the dedicated runners that don't have docker? Wouldn't it be better >> to add the tag only to the jobs that run "make check-tcg" ? > > But this is the point, if a runner doesn't have Docker, it can not > run the job... Apparently gitlab isn't clever enough to figure the 'image:' tag implies we are expecting Docker... I suppose they wanted to keep it simple and filter with runner tags. Now the public runners are named 'gitlab-org-docker', see: https://gitlab.com/gitlab-org/gitlab-ui/-/merge_requests/1267/diffs https://gitlab.com/gitlab-org/gitlab-docs/-/merge_requests/800/diffs https://gitlab.com/gitlab-org/gitlab-docs/blob/master/.gitlab-ci.yml#L483 https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/9685 Long term it would be simpler if we use gitlab recommended templates, so we don't have to update ours when they change. Meanwhile I'll simpy respin using 'gitlab-org-docker'. Regards, Phil.
diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index 33e4046e233..8e2a6a99889 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -1,4 +1,6 @@ .container_job_template: &container_job_definition + tags: + - docker image: docker:stable stage: containers services: diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml index d5098c986b8..e59fbfdc73f 100644 --- a/.gitlab-ci.d/crossbuilds.yml +++ b/.gitlab-ci.d/crossbuilds.yml @@ -1,4 +1,6 @@ .cross_system_build_job: + tags: + - docker stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest timeout: 80m @@ -18,6 +20,8 @@ # KVM), and set extra options (such disabling other accelerators) via the # $ACCEL_CONFIGURE_OPTS variable. .cross_accel_build_job: + tags: + - docker stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest timeout: 30m diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml index ba7280605c4..afbd8e4d915 100644 --- a/.gitlab-ci.d/edk2.yml +++ b/.gitlab-ci.d/edk2.yml @@ -5,6 +5,8 @@ docker-edk2: - .gitlab-ci.d/edk2.yml - .gitlab-ci.d/edk2/Dockerfile when: always + tags: + - docker image: docker:19.03.1 services: - docker:19.03.1-dind @@ -24,6 +26,8 @@ docker-edk2: - docker push $IMAGE_TAG build-edk2: + tags: + - docker stage: build needs: ['docker-edk2'] rules: # Only run this job when ... diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index f66cd1d9089..a4a93222c2d 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -5,6 +5,8 @@ docker-opensbi: - .gitlab-ci.d/opensbi.yml - .gitlab-ci.d/opensbi/Dockerfile when: always + tags: + - docker image: docker:19.03.1 services: - docker:19.03.1-dind @@ -24,6 +26,8 @@ docker-opensbi: - docker push $IMAGE_TAG build-opensbi: + tags: + - docker stage: build needs: ['docker-opensbi'] rules: # Only run this job when ... diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f65cb11c4d3..d4511cf7dea 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -14,6 +14,8 @@ include: - local: '/.gitlab-ci.d/crossbuilds.yml' .native_build_job_template: &native_build_job_definition + tags: + - docker stage: build image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest before_script: @@ -38,6 +40,8 @@ include: fi .native_test_job_template: &native_test_job_definition + tags: + - docker stage: test image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest script:
When a job is based on a Docker image [1], or is using a Docker service, it requires a runner with Docker installed. Gitlab shared runners provide the 'docker' tag when they have it installed. Are Gitlab shared runners are limited resources, we'd like to add more runners to QEMU repositories hosted on Gitlab. If a runner doesn't provide Docker, our jobs requiring it will fail. Use the standard 'docker' tag to mark the jobs requiring Docker on the runner. [1] https://docs.gitlab.com/ee/ci/yaml/#image [2] https://docs.gitlab.com/ee/ci/yaml/#services Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> --- If someone is interested in testing or filling the documentation gap, what I ran is: $ sudo usermod -aG docker,kvm gitlab-runner $ sudo gitlab-runner --log-format text --log-level debug \ register \ --non-interactive \ --url https://gitlab.com --registration-token MYTOKEN --description myrunner \ --tag-list 'docker,linux,x86_64,kvm' --run-untagged --limit 2 \ --executor docker --docker-image docker:dind --docker-cpus 4 \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock \ --docker-dns 8.8.8.8 --docker-volumes is for docker:dind else it was not working This comes from this 3 year old thread: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1986 We can not use the 'docker:dind' tag for a runner having docker:dind and /var/run/docker.sock volume because this is not a tag used by the shared runners, so we can't use them anymore. --- .gitlab-ci.d/containers.yml | 2 ++ .gitlab-ci.d/crossbuilds.yml | 4 ++++ .gitlab-ci.d/edk2.yml | 4 ++++ .gitlab-ci.d/opensbi.yml | 4 ++++ .gitlab-ci.yml | 4 ++++ 5 files changed, 18 insertions(+)