From patchwork Wed Feb 5 08:16:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Raman X-Patchwork-Id: 13960660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D0A9BC02192 for ; Wed, 5 Feb 2025 08:19:48 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9B10510E755; Wed, 5 Feb 2025 08:19:46 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=collabora.com header.i=@collabora.com header.b="nFuVkk5f"; dkim-atps=neutral Received: from bali.collaboradmins.com (bali.collaboradmins.com [148.251.105.195]) by gabe.freedesktop.org (Postfix) with ESMTPS id E36D310E747 for ; Wed, 5 Feb 2025 08:17:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1738743435; bh=ITu4rNsk022MBCsPh3lIeUk/cYgk8IW58QWuy/gXT5Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nFuVkk5fX/ltcJLrYPI+A8TVA57xkxHeqUAueFJAhQ9x6hw4vO/K4As4m3XAJErG4 /wyz4P8gnpO9w0XYEvf5lGqzXt3zAU2xCMi64iehQmnLKP/HJway08CwkJ3flJzUo5 ynWvjjG84FlIYlNVdj4LXSPm2IQ8gJrIG0lqQI2mCEFvsGb6I/m9/s3Q9UFpxusn66 ofQgAr81x+HhXqjiTxmfh0cAJcwZFvKUomTkOFcXq3p2rrfLeBL4kWWDfnJf2thXv8 8DK/z1+lzejHtZSIVzNXVEl3+Z5u7eqLSNaML5JBHeitwh2tkRbizbWuBFh3It6RSY qo3PKVjUWm0/A== Received: from localhost.localdomain (unknown [171.76.83.215]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: vignesh) by bali.collaboradmins.com (Postfix) with ESMTPSA id D947817E1519; Wed, 5 Feb 2025 09:17:12 +0100 (CET) From: Vignesh Raman To: dri-devel@lists.freedesktop.org Cc: daniels@collabora.com, helen.koike@collabora.com, airlied@gmail.com, simona.vetter@ffwll.ch, robdclark@gmail.com, guilherme.gallo@collabora.com, sergi.blanch.torne@collabora.com, valentine.burley@collabora.com, jani.nikula@linux.intel.com, dmitry.baryshkov@linaro.org, mripard@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/3] drm/ci: uprev mesa Date: Wed, 5 Feb 2025 13:46:46 +0530 Message-ID: <20250205081652.1928927-2-vignesh.raman@collabora.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250205081652.1928927-1-vignesh.raman@collabora.com> References: <20250205081652.1928927-1-vignesh.raman@collabora.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Uprev mesa to adapt to the latest changes in mesa ci which includes new container jobs and stages. Also update lava-submit script to adapt to the recent changes in mesa to use LAVA rootfs overlays. Signed-off-by: Vignesh Raman --- v2: - Uprev mesa to the latest version and update the yaml files and scripts. --- drivers/gpu/drm/ci/build.sh | 2 +- drivers/gpu/drm/ci/build.yml | 103 +++++++++++++++++++++++++++++- drivers/gpu/drm/ci/container.yml | 22 +++---- drivers/gpu/drm/ci/gitlab-ci.yml | 81 +++++++++++++++-------- drivers/gpu/drm/ci/igt_runner.sh | 13 ++-- drivers/gpu/drm/ci/image-tags.yml | 11 +++- drivers/gpu/drm/ci/lava-submit.sh | 100 +++++++++++++++++++++-------- drivers/gpu/drm/ci/test.yml | 17 +++-- 8 files changed, 264 insertions(+), 85 deletions(-) diff --git a/drivers/gpu/drm/ci/build.sh b/drivers/gpu/drm/ci/build.sh index 139b81db6312..19fe01257ab9 100644 --- a/drivers/gpu/drm/ci/build.sh +++ b/drivers/gpu/drm/ci/build.sh @@ -132,7 +132,7 @@ fi # Pass needed files to the test stage mkdir -p install cp -rfv .gitlab-ci/* install/. -cp -rfv ci/* install/. +cp -rfv bin/ci/* install/. cp -rfv install/common install/ci-common cp -rfv drivers/gpu/drm/ci/* install/. diff --git a/drivers/gpu/drm/ci/build.yml b/drivers/gpu/drm/ci/build.yml index 9c198239033d..a2baa00a20db 100644 --- a/drivers/gpu/drm/ci/build.yml +++ b/drivers/gpu/drm/ci/build.yml @@ -2,7 +2,7 @@ extends: - .build-rules - .container+build-rules - stage: build + stage: build-only artifacts: paths: - artifacts @@ -110,3 +110,104 @@ build-nodebugfs:arm64: build:x86_64: extends: .build:x86_64 + +# Disable build jobs that we won't use +alpine-build-testing: + rules: + - when: never + +debian-android: + rules: + - when: never + +debian-arm32: + rules: + - when: never + +debian-arm32-asan: + rules: + - when: never + +debian-arm64: + rules: + - when: never + +debian-arm64-asan: + rules: + - when: never + +debian-arm64-build-test: + rules: + - when: never + +debian-arm64-release: + rules: + - when: never + +debian-build-testing: + rules: + - when: never + +debian-clang: + rules: + - when: never + +debian-clang-release: + rules: + - when: never + +debian-no-libdrm: + rules: + - when: never + +debian-ppc64el: + rules: + - when: never + +debian-release: + rules: + - when: never + +debian-s390x: + rules: + - when: never + +debian-testing: + rules: + - when: never + +debian-testing-asan: + rules: + - when: never + +debian-testing-msan: + rules: + - when: never + +debian-vulkan: + rules: + - when: never + +debian-x86_32: + rules: + - when: never + +fedora-release: + rules: + - when: never + +rustfmt: + rules: + - when: never + +shader-db: + rules: + - when: never + +windows-msvc: + rules: + - when: never + +yaml-toml-shell-py-test: + rules: + - when: never diff --git a/drivers/gpu/drm/ci/container.yml b/drivers/gpu/drm/ci/container.yml index 2a94f54ce4cf..07dc13ff865d 100644 --- a/drivers/gpu/drm/ci/container.yml +++ b/drivers/gpu/drm/ci/container.yml @@ -24,7 +24,7 @@ alpine/x86_64_build: rules: - when: never -debian/x86_64_test-vk: +debian/arm64_test-gl: rules: - when: never @@ -32,7 +32,15 @@ debian/arm64_test-vk: rules: - when: never -debian/arm64_test-gl: +debian/ppc64el_build: + rules: + - when: never + +debian/s390x_build: + rules: + - when: never + +debian/x86_64_test-vk: rules: - when: never @@ -56,14 +64,6 @@ windows_test_msvc: rules: - when: never -.debian/x86_64_build-mingw: - rules: - - when: never - -rustfmt: - rules: - - when: never - windows_msvc: rules: - - when: never \ No newline at end of file + - when: never diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml index b7214718723c..48ab2de9c90b 100644 --- a/drivers/gpu/drm/ci/gitlab-ci.yml +++ b/drivers/gpu/drm/ci/gitlab-ci.yml @@ -1,6 +1,6 @@ variables: DRM_CI_PROJECT_PATH: &drm-ci-project-path mesa/mesa - DRM_CI_COMMIT_SHA: &drm-ci-commit-sha c6a9a9c3bce90923f7700219354e0b6e5a3c9ba6 + DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 7d3062470f3ccc6cb40540e772e902c7e2248024 UPSTREAM_REPO: https://gitlab.freedesktop.org/drm/kernel.git TARGET_BRANCH: drm-next @@ -20,6 +20,9 @@ variables: rm download-git-cache.sh set +o xtrace S3_JWT_FILE: /s3_jwt + S3_JWT_FILE_SCRIPT: |- + echo -n '${S3_JWT}' > '${S3_JWT_FILE}' && + unset CI_JOB_JWT S3_JWT # Unsetting vulnerable env variables S3_HOST: s3.freedesktop.org # This bucket is used to fetch the kernel image S3_KERNEL_BUCKET: mesa-rootfs @@ -31,11 +34,7 @@ variables: PIPELINE_ARTIFACTS_BASE: ${S3_HOST}/${S3_ARTIFACTS_BUCKET}/${CI_PROJECT_PATH}/${CI_PIPELINE_ID} # per-job artifact storage on MinIO JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID} - # default kernel for rootfs before injecting the current kernel tree - KERNEL_REPO: "gfx-ci/linux" - KERNEL_TAG: "v6.6.21-mesa-f8ea" KERNEL_IMAGE_BASE: https://${S3_HOST}/${S3_KERNEL_BUCKET}/${KERNEL_REPO}/${KERNEL_TAG} - PKG_REPO_REV: "3cc12a2a" LAVA_TAGS: subset-1-gfx LAVA_JOB_PRIORITY: 30 ARTIFACTS_BASE_URL: https://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_NAME}/-/jobs/${CI_JOB_ID}/artifacts @@ -50,16 +49,13 @@ default: - export SCRIPTS_DIR=$(mktemp -d) - curl -L -s --retry 4 -f --retry-all-errors --retry-delay 60 -O --output-dir "${SCRIPTS_DIR}" "${DRM_CI_PROJECT_URL}/-/raw/${DRM_CI_COMMIT_SHA}/.gitlab-ci/setup-test-env.sh" - source ${SCRIPTS_DIR}/setup-test-env.sh - - echo -e "\e[0Ksection_start:$(date +%s):unset_env_vars_section[collapsed=true]\r\e[0KUnsetting vulnerable environment variables" - - echo -n "${S3_JWT}" > "${S3_JWT_FILE}" - - unset CI_JOB_JWT S3_JWT - - echo -e "\e[0Ksection_end:$(date +%s):unset_env_vars_section\r\e[0K" + - eval "$S3_JWT_FILE_SCRIPT" - echo -e "\e[0Ksection_start:$(date +%s):drm_ci_download_section[collapsed=true]\r\e[0KDownloading mesa from $DRM_CI_PROJECT_URL/-/archive/$DRM_CI_COMMIT_SHA/mesa-$DRM_CI_COMMIT_SHA.tar.gz" - cd $CI_PROJECT_DIR - curl --output - $DRM_CI_PROJECT_URL/-/archive/$DRM_CI_COMMIT_SHA/mesa-$DRM_CI_COMMIT_SHA.tar.gz | tar -xz - mv mesa-$DRM_CI_COMMIT_SHA/.gitlab-ci* . - - mv mesa-$DRM_CI_COMMIT_SHA/bin/ci . + - mv mesa-$DRM_CI_COMMIT_SHA/bin . - rm -rf mesa-$DRM_CI_COMMIT_SHA/ - echo -e "\e[0Ksection_end:$(date +%s):drm_ci_download_section\r\e[0K" @@ -85,6 +81,7 @@ include: - project: *drm-ci-project-path ref: *drm-ci-commit-sha file: + - '/.gitlab-ci/build/gitlab-ci.yml' - '/.gitlab-ci/container/gitlab-ci.yml' - '/.gitlab-ci/farm-rules.yml' - '/.gitlab-ci/lava/lava-gitlab-ci.yml' @@ -115,9 +112,10 @@ include: stages: - sanity - container - - code-validation - git-archive - - build + - build-for-tests + - build-only + - code-validation - amdgpu - i915 - mediatek @@ -197,7 +195,7 @@ stages: .ci-deqp-artifacts: artifacts: - name: "mesa_${CI_JOB_NAME}" + name: "${CI_PROJECT_NAME}_${CI_JOB_NAME}" when: always untracked: false paths: @@ -264,30 +262,63 @@ sanity: rules: - if: *is-pre-merge when: on_success - # Other cases default to never + - when: never variables: GIT_STRATEGY: none script: # ci-fairy check-commits --junit-xml=check-commits.xml - ci-fairy check-merge-request --require-allow-collaboration --junit-xml=check-merge-request.xml + - | + set -eu + image_tags=( + ALPINE_X86_64_LAVA_SSH_TAG + CONTAINER_TAG + DEBIAN_BASE_TAG + DEBIAN_BUILD_TAG + DEBIAN_PYUTILS_TAG + DEBIAN_TEST_GL_TAG + KERNEL_ROOTFS_TAG + KERNEL_TAG + PKG_REPO_REV + ) + for var in "${image_tags[@]}" + do + if [ "$(echo -n "${!var}" | wc -c)" -gt 20 ] + then + echo "$var is too long; please make sure it is at most 20 chars." + exit 1 + fi + done artifacts: when: on_failure reports: junit: check-*.xml + tags: + - placeholder-job -# Rules for tests that should not block merging, but should be available to -# optionally run with the "play" button in the UI in pre-merge non-marge -# pipelines. This should appear in "extends:" after any includes of -# test-source-dep.yml rules, so that these rules replace those. -.test-manual-mr: + +mr-label-maker-test: + extends: + - .fdo.ci-fairy + stage: sanity rules: - - !reference [.no_scheduled_pipelines-rules, rules] - - if: *is-forked-branch-or-pre-merge-not-for-marge - when: manual + - !reference [.mr-label-maker-rules, rules] variables: - JOB_TIMEOUT: 80 - + GIT_STRATEGY: fetch + timeout: 10m + script: + - set -eu + - python3 -m venv .venv + - source .venv/bin/activate + - pip install git+https://gitlab.freedesktop.org/freedesktop/mr-label-maker + - mr-label-maker --dry-run --mr $CI_MERGE_REQUEST_IID # Jobs that need to pass before spending hardware resources on further testing .required-for-hardware-jobs: - needs: [] + needs: + - job: clang-format + optional: true + - job: rustfmt + optional: true + - job: toml-lint + optional: true diff --git a/drivers/gpu/drm/ci/igt_runner.sh b/drivers/gpu/drm/ci/igt_runner.sh index f38836ec837c..68b042e43b7f 100755 --- a/drivers/gpu/drm/ci/igt_runner.sh +++ b/drivers/gpu/drm/ci/igt_runner.sh @@ -47,7 +47,7 @@ else ARCH="x86_64" fi -curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s ${FDO_HTTP_CACHE_URI:-}$PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd -v -x -C / +curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s $PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd -v -x -C / TESTLIST="/igt/libexec/igt-gpu-tools/ci-testlist.txt" @@ -69,7 +69,7 @@ igt-runner \ run \ --igt-folder /igt/libexec/igt-gpu-tools \ --caselist $TESTLIST \ - --output /results \ + --output $RESULTS_DIR \ -vvvv \ $IGT_SKIPS \ $IGT_FLAKES \ @@ -80,13 +80,10 @@ set -e deqp-runner junit \ --testsuite IGT \ - --results /results/failures.csv \ - --output /results/junit.xml \ + --results $RESULTS_DIR/failures.csv \ + --output $RESULTS_DIR/junit.xml \ --limit 50 \ - --template "See https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/results/{{testcase}}.xml" - -# Store the results also in the simpler format used by the runner in ChromeOS CI -#sed -r 's/(dmesg-warn|pass)/success/g' /results/results.txt > /results/results_simple.txt + --template "See $ARTIFACTS_BASE_URL/results/{{testcase}}.xml" cd $oldpath exit $ret diff --git a/drivers/gpu/drm/ci/image-tags.yml b/drivers/gpu/drm/ci/image-tags.yml index 8d8b9e71852e..20049f3626b2 100644 --- a/drivers/gpu/drm/ci/image-tags.yml +++ b/drivers/gpu/drm/ci/image-tags.yml @@ -1,5 +1,5 @@ variables: - CONTAINER_TAG: "2024-09-09-uprevs" + CONTAINER_TAG: "20250204-mesa-uprev" DEBIAN_X86_64_BUILD_BASE_IMAGE: "debian/x86_64_build-base" DEBIAN_BASE_TAG: "${CONTAINER_TAG}" @@ -7,9 +7,16 @@ variables: DEBIAN_BUILD_TAG: "${CONTAINER_TAG}" KERNEL_ROOTFS_TAG: "${CONTAINER_TAG}" + # default kernel for rootfs before injecting the current kernel tree + KERNEL_TAG: "v6.13-rc4-mesa-5e77" + KERNEL_REPO: "gfx-ci/linux" + PKG_REPO_REV: "bca9635d" DEBIAN_X86_64_TEST_BASE_IMAGE: "debian/x86_64_test-base" DEBIAN_X86_64_TEST_IMAGE_GL_PATH: "debian/x86_64_test-gl" DEBIAN_TEST_GL_TAG: "${CONTAINER_TAG}" - ALPINE_X86_64_LAVA_SSH_TAG: "${CONTAINER_TAG}" \ No newline at end of file + DEBIAN_PYUTILS_IMAGE: "debian/x86_64_pyutils" + DEBIAN_PYUTILS_TAG: "${CONTAINER_TAG}" + + ALPINE_X86_64_LAVA_SSH_TAG: "${CONTAINER_TAG}" diff --git a/drivers/gpu/drm/ci/lava-submit.sh b/drivers/gpu/drm/ci/lava-submit.sh index 6add15083c78..6e5ac51e8c0a 100755 --- a/drivers/gpu/drm/ci/lava-submit.sh +++ b/drivers/gpu/drm/ci/lava-submit.sh @@ -1,58 +1,102 @@ -#!/bin/bash +#!/usr/bin/env bash # SPDX-License-Identifier: MIT +# shellcheck disable=SC2086 # we want word splitting +# shellcheck disable=SC1091 # paths only become valid at runtime -set -e -set -x +# If we run in the fork (not from mesa or Marge-bot), reuse mainline kernel and rootfs, if exist. +_check_artifact_path() { + _url="https://${1}/${2}" + if curl -s -o /dev/null -I -L -f --retry 4 --retry-delay 15 "${_url}"; then + echo -n "${_url}" + fi +} -# Try to use the kernel and rootfs built in mainline first, so we're more -# likely to hit cache -if curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s "https://${BASE_SYSTEM_MAINLINE_HOST_PATH}/done"; then - BASE_SYSTEM_HOST_PATH="${BASE_SYSTEM_MAINLINE_HOST_PATH}" -else - BASE_SYSTEM_HOST_PATH="${BASE_SYSTEM_FORK_HOST_PATH}" -fi +get_path_to_artifact() { + _mainline_artifact="$(_check_artifact_path ${BASE_SYSTEM_MAINLINE_HOST_PATH} ${1})" + if [ -n "${_mainline_artifact}" ]; then + echo -n "${_mainline_artifact}" + return + fi + _fork_artifact="$(_check_artifact_path ${BASE_SYSTEM_FORK_HOST_PATH} ${1})" + if [ -n "${_fork_artifact}" ]; then + echo -n "${_fork_artifact}" + return + fi + set +x + error "Sorry, I couldn't find a viable built path for ${1} in either mainline or a fork." >&2 + echo "" >&2 + echo "If you're working on CI, this probably means that you're missing a dependency:" >&2 + echo "this job ran ahead of the job which was supposed to upload that artifact." >&2 + echo "" >&2 + echo "If you aren't working on CI, please ping @mesa/ci-helpers to see if we can help." >&2 + echo "" >&2 + echo "This job is going to fail, because I can't find the resources I need. Sorry." >&2 + set -x + exit 1 +} + +. "${SCRIPTS_DIR}/setup-test-env.sh" + +section_start prepare_rootfs "Preparing root filesystem" + +set -ex + +section_switch rootfs "Assembling root filesystem" +ROOTFS_URL="$(get_path_to_artifact lava-rootfs.tar.zst)" +[ $? != 1 ] || exit 1 rm -rf results mkdir -p results/job-rootfs-overlay/ -cp artifacts/ci-common/capture-devcoredump.sh results/job-rootfs-overlay/ +artifacts/ci-common/generate-env.sh > results/job-rootfs-overlay/set-job-env-vars.sh cp artifacts/ci-common/init-*.sh results/job-rootfs-overlay/ -cp artifacts/ci-common/intel-gpu-freq.sh results/job-rootfs-overlay/ cp "$SCRIPTS_DIR"/setup-test-env.sh results/job-rootfs-overlay/ -# Prepare env vars for upload. -section_start variables "Variables passed through:" -KERNEL_IMAGE_BASE="https://${BASE_SYSTEM_HOST_PATH}" \ - artifacts/ci-common/generate-env.sh | tee results/job-rootfs-overlay/set-job-env-vars.sh -section_end variables - tar zcf job-rootfs-overlay.tar.gz -C results/job-rootfs-overlay/ . ci-fairy s3cp --token-file "${S3_JWT_FILE}" job-rootfs-overlay.tar.gz "https://${JOB_ROOTFS_OVERLAY_PATH}" +# Prepare env vars for upload. +section_switch variables "Environment variables passed through to device:" +cat results/job-rootfs-overlay/set-job-env-vars.sh + +section_switch lava_submit "Submitting job for scheduling" + touch results/lava.log tail -f results/lava.log & - PYTHONPATH=artifacts/ artifacts/lava/lava_job_submitter.py \ - submit \ + --farm "${FARM}" \ + --device-type "${DEVICE_TYPE}" \ + --boot-method "${BOOT_METHOD}" \ + --job-timeout-min $((CI_JOB_TIMEOUT/60 - 5)) \ --dump-yaml \ --pipeline-info "$CI_JOB_NAME: $CI_PIPELINE_URL on $CI_COMMIT_REF_NAME ${CI_NODE_INDEX}/${CI_NODE_TOTAL}" \ - --rootfs-url-prefix "https://${BASE_SYSTEM_HOST_PATH}" \ + --rootfs-url "${ROOTFS_URL}" \ --kernel-url-prefix "https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}" \ - --build-url "${FDO_HTTP_CACHE_URI:-}https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/kernel-files.tar.zst" \ - --job-rootfs-overlay-url "${FDO_HTTP_CACHE_URI:-}https://${JOB_ROOTFS_OVERLAY_PATH}" \ - --job-timeout-min ${JOB_TIMEOUT:-80} \ + --kernel-external "${EXTERNAL_KERNEL_TAG}" \ --first-stage-init artifacts/ci-common/init-stage1.sh \ - --ci-project-dir "${CI_PROJECT_DIR}" \ - --device-type "${DEVICE_TYPE}" \ - --farm "${FARM}" \ --dtb-filename "${DTB}" \ --jwt-file "${S3_JWT_FILE}" \ --kernel-image-name "${KERNEL_IMAGE_NAME}" \ --kernel-image-type "${KERNEL_IMAGE_TYPE}" \ - --boot-method "${BOOT_METHOD}" \ --visibility-group "${VISIBILITY_GROUP}" \ --lava-tags "${LAVA_TAGS}" \ --mesa-job-name "$CI_JOB_NAME" \ --structured-log-file "results/lava_job_detail.json" \ --ssh-client-image "${LAVA_SSH_CLIENT_IMAGE}" \ + --project-name "${CI_PROJECT_NAME}" \ + --starting-section "${CURRENT_SECTION}" \ + --job-submitted-at "${CI_JOB_STARTED_AT}" \ + - append-overlay \ + --name=kernel-build \ + --url="${FDO_HTTP_CACHE_URI:-}https://${PIPELINE_ARTIFACTS_BASE}/${DEBIAN_ARCH}/kernel-files.tar.zst" \ + --compression=zstd \ + --path="${CI_PROJECT_DIR}" \ + --format=tar \ + - append-overlay \ + --name=job-overlay \ + --url="https://${JOB_ROOTFS_OVERLAY_PATH}" \ + --compression=gz \ + --path="/" \ + --format=tar \ + - submit \ >> results/lava.log diff --git a/drivers/gpu/drm/ci/test.yml b/drivers/gpu/drm/ci/test.yml index f0ef60c8f56d..c5c786bf419b 100644 --- a/drivers/gpu/drm/ci/test.yml +++ b/drivers/gpu/drm/ci/test.yml @@ -1,21 +1,21 @@ .test-rules: rules: - - if: '$FD_FARM == "offline" && $RUNNER_TAG =~ /^google-freedreno-/' - when: never - - if: '$COLLABORA_FARM == "offline" && $RUNNER_TAG =~ /^mesa-ci-x86-64-lava-/' - when: never - !reference [.no_scheduled_pipelines-rules, rules] + - !reference [.collabora-farm-rules, rules] + - !reference [.google-freedreno-farm-rules, rules] - when: on_success .lava-test: extends: - .test-rules + - .build-rules + - .container+build-rules timeout: "1h30m" script: # Note: Build dir (and thus install) may be dirty due to GIT_STRATEGY - rm -rf install - tar -xf artifacts/install.tar - - mv install/* artifacts/. + - mv -n install/* artifacts/. # Override it with our lava-submit.sh script - ./artifacts/lava-submit.sh @@ -32,6 +32,7 @@ - alpine/x86_64_lava_ssh_client - kernel+rootfs_arm32 - debian/x86_64_build + - python-artifacts - testing:arm32 - igt:arm32 @@ -48,6 +49,7 @@ - alpine/x86_64_lava_ssh_client - kernel+rootfs_arm64 - debian/x86_64_build + - python-artifacts - testing:arm64 - igt:arm64 @@ -64,6 +66,7 @@ - alpine/x86_64_lava_ssh_client - kernel+rootfs_x86_64 - debian/x86_64_build + - python-artifacts - testing:x86_64 - igt:x86_64 @@ -453,8 +456,6 @@ virtio_gpu:none: script: - ln -sf $CI_PROJECT_DIR/install /install - mv install/bzImage /lava-files/bzImage - - mkdir -p $CI_PROJECT_DIR/results - - ln -sf $CI_PROJECT_DIR/results /results - install/crosvm-runner.sh install/igt_runner.sh needs: - debian/x86_64_test-gl @@ -476,8 +477,6 @@ vkms:none: - ln -sf $CI_PROJECT_DIR/install /install - mv install/bzImage /lava-files/bzImage - mkdir -p /lib/modules - - mkdir -p $CI_PROJECT_DIR/results - - ln -sf $CI_PROJECT_DIR/results /results - ./install/crosvm-runner.sh ./install/igt_runner.sh needs: - debian/x86_64_test-gl From patchwork Wed Feb 5 08:16:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Raman X-Patchwork-Id: 13960658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7D8D9C02192 for ; Wed, 5 Feb 2025 08:19:45 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 086F310E74B; Wed, 5 Feb 2025 08:19:45 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=collabora.com header.i=@collabora.com header.b="QB0bpXcU"; dkim-atps=neutral Received: from bali.collaboradmins.com (bali.collaboradmins.com [148.251.105.195]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7D5B110E747 for ; Wed, 5 Feb 2025 08:17:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1738743438; bh=TwcKijoDdvB2KESuaUkAlxu5ofJu4K0qwDbtn1z0Cik=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QB0bpXcUGT4xIO9nWGQPdhAQRe0o6xvQ3MYVZXqP23z5JK4Mf/tzMD7R0CxRcwt7g ZcZ/Dvrk5RzGIpO7U9FZRbERReksXqm4UvYBjzF5EtciQmQRchnhot2B4Wwlavf0KZ 6mWoa6/fcHH5I8/zC5bHyO4p0HJHI0/xjATeH1GAPRmcLTMDjhjDAQnXYyAyqRcviP yn4jYtunmCVFxM2DQ/lWrrkIpIZr6cIg9TLgwT99iG10VSmtfh+5XupvfgInHbzGYO 0u3Cu8NrIBdkEKWmeosj/SDnQUmL5oDqQrjJN+DmOo6DqLBPZtuFiNozpc78VcBysa kRRnP1UGoXu4Q== Received: from localhost.localdomain (unknown [171.76.83.215]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: vignesh) by bali.collaboradmins.com (Postfix) with ESMTPSA id E6F2017E153C; Wed, 5 Feb 2025 09:17:15 +0100 (CET) From: Vignesh Raman To: dri-devel@lists.freedesktop.org Cc: daniels@collabora.com, helen.koike@collabora.com, airlied@gmail.com, simona.vetter@ffwll.ch, robdclark@gmail.com, guilherme.gallo@collabora.com, sergi.blanch.torne@collabora.com, valentine.burley@collabora.com, jani.nikula@linux.intel.com, dmitry.baryshkov@linaro.org, mripard@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/3] drm/ci: update gitlab rules Date: Wed, 5 Feb 2025 13:46:47 +0530 Message-ID: <20250205081652.1928927-3-vignesh.raman@collabora.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250205081652.1928927-1-vignesh.raman@collabora.com> References: <20250205081652.1928927-1-vignesh.raman@collabora.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Update gitlab rules to include scheduled pipelines. Signed-off-by: Vignesh Raman --- v2: - No changes. --- drivers/gpu/drm/ci/build.yml | 1 - drivers/gpu/drm/ci/gitlab-ci.yml | 116 +++++++++++++------------------ drivers/gpu/drm/ci/test.yml | 26 +++---- 3 files changed, 63 insertions(+), 80 deletions(-) diff --git a/drivers/gpu/drm/ci/build.yml b/drivers/gpu/drm/ci/build.yml index a2baa00a20db..274f118533a7 100644 --- a/drivers/gpu/drm/ci/build.yml +++ b/drivers/gpu/drm/ci/build.yml @@ -1,6 +1,5 @@ .build: extends: - - .build-rules - .container+build-rules stage: build-only artifacts: diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml index 48ab2de9c90b..f04aabe8327c 100644 --- a/drivers/gpu/drm/ci/gitlab-ci.yml +++ b/drivers/gpu/drm/ci/gitlab-ci.yml @@ -41,6 +41,7 @@ variables: # Python scripts for structured logger PYTHONPATH: "$PYTHONPATH:$CI_PROJECT_DIR/install" + default: id_tokens: S3_JWT: @@ -67,6 +68,7 @@ default: export S3_JWT="$(<${S3_JWT_FILE})" && rm "${S3_JWT_FILE}" + include: - project: 'freedesktop/ci-templates' ref: 16bc29078de5e0a067ff84a1a199a3760d3b3811 @@ -126,33 +128,27 @@ stages: - rockchip - software-driver + # YAML anchors for rule conditions # -------------------------------- .rules-anchors: rules: - # Pipeline for forked project branch - - if: &is-forked-branch '$CI_COMMIT_BRANCH && $CI_PROJECT_NAMESPACE != "mesa"' - when: manual - # Forked project branch / pre-merge pipeline not for Marge bot - - if: &is-forked-branch-or-pre-merge-not-for-marge '$CI_PROJECT_NAMESPACE != "mesa" || ($GITLAB_USER_LOGIN != "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event")' - when: manual - # Pipeline runs for the main branch of the upstream Mesa project - - if: &is-mesa-main '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH && $CI_COMMIT_BRANCH' - when: always - # Post-merge pipeline - - if: &is-post-merge '$CI_PROJECT_NAMESPACE == "mesa" && $CI_COMMIT_BRANCH' - when: on_success - # Post-merge pipeline, not for Marge Bot - - if: &is-post-merge-not-for-marge '$CI_PROJECT_NAMESPACE == "mesa" && $GITLAB_USER_LOGIN != "marge-bot" && $CI_COMMIT_BRANCH' - when: on_success + # do not duplicate pipelines on merge pipelines + - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push" + when: never + # merge pipeline + - if: &is-merge-attempt $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event" + # post-merge pipeline + - if: &is-post-merge $GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "push" # Pre-merge pipeline - - if: &is-pre-merge '$CI_PIPELINE_SOURCE == "merge_request_event"' - when: on_success - # Pre-merge pipeline for Marge Bot - - if: &is-pre-merge-for-marge '$GITLAB_USER_LOGIN == "marge-bot" && $CI_PIPELINE_SOURCE == "merge_request_event"' - when: on_success + - if: &is-pre-merge $CI_PIPELINE_SOURCE == "merge_request_event" # Push to a branch on a fork - - &is-fork-push '$CI_PROJECT_NAMESPACE != "mesa" && $CI_PIPELINE_SOURCE == "push"' + - if: &is-fork-push $CI_PROJECT_NAMESPACE != "mesa" && $CI_PIPELINE_SOURCE == "push" + # nightly pipeline + - if: &is-scheduled-pipeline $CI_PIPELINE_SOURCE == "schedule" + # pipeline for direct pushes that bypassed the CI + - if: &is-direct-push $CI_PROJECT_NAMESPACE == "mesa" && $CI_PIPELINE_SOURCE == "push" && $GITLAB_USER_LOGIN != "marge-bot" + # Rules applied to every job in the pipeline .common-rules: @@ -160,39 +156,48 @@ stages: - if: *is-fork-push when: manual + .never-post-merge-rules: rules: - if: *is-post-merge when: never -# Rule to filter for only scheduled pipelines. -.scheduled_pipeline-rules: - rules: - - if: &is-scheduled-pipeline '$CI_PIPELINE_SOURCE == "schedule"' - when: on_success - -# Generic rule to not run the job during scheduled pipelines. Jobs that aren't -# something like a nightly run should include this rule. -.no_scheduled_pipelines-rules: - rules: - - if: *is-scheduled-pipeline - when: never -# When to automatically run the CI for build jobs -.build-rules: - rules: - - !reference [.no_scheduled_pipelines-rules, rules] - - !reference [.never-post-merge-rules, rules] - # Run automatically once all dependency jobs have passed - - when: on_success - -# When to automatically run the CI for container jobs .container+build-rules: rules: - - !reference [.no_scheduled_pipelines-rules, rules] + - !reference [.common-rules, rules] + # Run when re-enabling a disabled farm, but not when disabling it + - !reference [.disable-farm-mr-rules, rules] + # Never run immediately after merging, as we just ran everything - !reference [.never-post-merge-rules, rules] + # Build everything in merge pipelines, if any files affecting the pipeline + # were changed + - if: *is-merge-attempt + changes: &all_paths + - drivers/gpu/drm/ci/**/* + when: on_success + # Same as above, but for pre-merge pipelines + - if: *is-pre-merge + changes: + *all_paths + when: manual + # Skip everything for pre-merge and merge pipelines which don't change + # anything in the build + - if: *is-merge-attempt + when: never + - if: *is-pre-merge + when: never + # Build everything after someone bypassed the CI + - if: *is-direct-push + when: on_success + # Build everything in scheduled pipelines + - if: *is-scheduled-pipeline + when: on_success + # Allow building everything in fork pipelines, but build nothing unless + # manually triggered - when: manual + .ci-deqp-artifacts: artifacts: name: "${CI_PROJECT_NAME}_${CI_JOB_NAME}" @@ -206,31 +211,7 @@ stages: - _build/meson-logs/strace -.container-rules: - rules: - - !reference [.no_scheduled_pipelines-rules, rules] - - !reference [.never-post-merge-rules, rules] - # Run pipeline by default in the main project if any CI pipeline - # configuration files were changed, to ensure docker images are up to date - - if: *is-post-merge - changes: - - drivers/gpu/drm/ci/**/* - when: on_success - # Run pipeline by default if it was triggered by Marge Bot, is for a - # merge request, and any files affecting the pipeline were changed - - if: *is-pre-merge-for-marge - when: on_success - # Run pipeline by default in the main project if it was not triggered by - # Marge Bot, and any files affecting the pipeline were changed - - if: *is-post-merge-not-for-marge - when: on_success - # Allow triggering jobs manually in other cases - - when: manual - - - # Git archive - make git archive: extends: - .fdo.ci-fairy @@ -313,6 +294,7 @@ mr-label-maker-test: - pip install git+https://gitlab.freedesktop.org/freedesktop/mr-label-maker - mr-label-maker --dry-run --mr $CI_MERGE_REQUEST_IID + # Jobs that need to pass before spending hardware resources on further testing .required-for-hardware-jobs: needs: diff --git a/drivers/gpu/drm/ci/test.yml b/drivers/gpu/drm/ci/test.yml index c5c786bf419b..6a1e059858e5 100644 --- a/drivers/gpu/drm/ci/test.yml +++ b/drivers/gpu/drm/ci/test.yml @@ -1,16 +1,11 @@ -.test-rules: - rules: - - !reference [.no_scheduled_pipelines-rules, rules] - - !reference [.collabora-farm-rules, rules] - - !reference [.google-freedreno-farm-rules, rules] - - when: on_success - .lava-test: extends: - - .test-rules - - .build-rules - .container+build-rules timeout: "1h30m" + rules: + - !reference [.scheduled_pipeline-rules, rules] + - !reference [.collabora-farm-rules, rules] + - when: on_success script: # Note: Build dir (and thus install) may be dirty due to GIT_STRATEGY - rm -rf install @@ -74,8 +69,11 @@ extends: - .baremetal-test-arm64 - .use-debian/baremetal_arm64_test - - .test-rules timeout: "1h30m" + rules: + - !reference [.scheduled_pipeline-rules, rules] + - !reference [.google-freedreno-farm-rules, rules] + - when: on_success variables: FDO_CI_CONCURRENT: 10 HWCI_TEST_SCRIPT: "/install/igt_runner.sh" @@ -444,13 +442,15 @@ panfrost:g12b: virtio_gpu:none: stage: software-driver timeout: "1h30m" + rules: + - !reference [.scheduled_pipeline-rules, rules] + - when: on_success variables: CROSVM_GALLIUM_DRIVER: llvmpipe DRIVER_NAME: virtio_gpu GPU_VERSION: none extends: - .test-gl - - .test-rules tags: - kvm script: @@ -465,12 +465,14 @@ virtio_gpu:none: vkms:none: stage: software-driver timeout: "1h30m" + rules: + - !reference [.scheduled_pipeline-rules, rules] + - when: on_success variables: DRIVER_NAME: vkms GPU_VERSION: none extends: - .test-gl - - .test-rules tags: - kvm script: From patchwork Wed Feb 5 08:16:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Raman X-Patchwork-Id: 13960659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 452F8C02197 for ; Wed, 5 Feb 2025 08:19:47 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3E67510E753; Wed, 5 Feb 2025 08:19:46 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=collabora.com header.i=@collabora.com header.b="b690HBL0"; dkim-atps=neutral Received: from bali.collaboradmins.com (bali.collaboradmins.com [148.251.105.195]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3837F10E747 for ; Wed, 5 Feb 2025 08:17:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1738743441; bh=IuOxI/ygHiJ4+esRB4KPb824pwCYUZ+ubtGSSeOB49g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b690HBL0Y58AYX8OZ+w0KvHB0iE9/an3YndS61YkZzLOv/X4sEo2LrpscRhGTNDfV 9he71z23AT3JRlSGt1M/aWOBjpD/zMq0sMn+Le1+MqrldNXxLT4iEFaTYGvNlsXZt5 iwQw/dF6BrrlPyfia6xF19nVROkMP+hQCpcuB3FxhEkB9teOw1GljVdURD+aEVKW6f KOkx97qug9ZYvZ/NcWD60zIDxu2FqNNEJRWFbEq4WRV9GYqyzP2JKn/xvyJwDjFlWk LteKtmZmrEjpRI6JHnnm9aY1gqdxkvplahnTjn5ETPRO441ug4VAKae/JPDuXVqoqL ZSL3N0F6ydZMg== Received: from localhost.localdomain (unknown [171.76.83.215]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: vignesh) by bali.collaboradmins.com (Postfix) with ESMTPSA id 0750C17E0DD8; Wed, 5 Feb 2025 09:17:18 +0100 (CET) From: Vignesh Raman To: dri-devel@lists.freedesktop.org Cc: daniels@collabora.com, helen.koike@collabora.com, airlied@gmail.com, simona.vetter@ffwll.ch, robdclark@gmail.com, guilherme.gallo@collabora.com, sergi.blanch.torne@collabora.com, valentine.burley@collabora.com, jani.nikula@linux.intel.com, dmitry.baryshkov@linaro.org, mripard@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/3] drm/ci: update expectation files Date: Wed, 5 Feb 2025 13:46:48 +0530 Message-ID: <20250205081652.1928927-4-vignesh.raman@collabora.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250205081652.1928927-1-vignesh.raman@collabora.com> References: <20250205081652.1928927-1-vignesh.raman@collabora.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Update expectation files for the mesa uprev. Signed-off-by: Vignesh Raman --- v2: - New commit in v2. --- drivers/gpu/drm/ci/xfails/vkms-none-fails.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt b/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt index 85647eca4e55..6ebcc7d89fbd 100644 --- a/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt +++ b/drivers/gpu/drm/ci/xfails/vkms-none-fails.txt @@ -13,6 +13,8 @@ kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail kms_cursor_legacy@flip-vs-cursor-legacy,Fail kms_flip@flip-vs-modeset-vs-hang,Fail kms_flip@flip-vs-panning-vs-hang,Fail +kms_flip@flip-vs-suspend,Fail +kms_flip@flip-vs-suspend-interruptible,Fail kms_lease@lease-uevent,Fail kms_writeback@writeback-check-output,Fail kms_writeback@writeback-check-output-XRGB2101010,Fail