From patchwork Thu Nov 17 09:39:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 13046463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 927A9C4332F for ; Thu, 17 Nov 2022 09:40:56 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.444827.699921 (Exim 4.92) (envelope-from ) id 1ovbO1-0002Nn-A3; Thu, 17 Nov 2022 09:40:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 444827.699921; Thu, 17 Nov 2022 09:40:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovbO1-0002Ng-7H; Thu, 17 Nov 2022 09:40:45 +0000 Received: by outflank-mailman (input) for mailman id 444827; Thu, 17 Nov 2022 09:40:43 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovbNy-0001D0-QA for xen-devel@lists.xenproject.org; Thu, 17 Nov 2022 09:40:43 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id e6c6e6c4-665b-11ed-8fd2-01056ac49cbb; Thu, 17 Nov 2022 10:40:42 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BDD4113D5; Thu, 17 Nov 2022 01:40:47 -0800 (PST) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A2AB53F73B; Thu, 17 Nov 2022 01:40:40 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e6c6e6c4-665b-11ed-8fd2-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v4 1/3] automation: Create Yocto docker images Date: Thu, 17 Nov 2022 09:39:49 +0000 Message-Id: <6d48a0f83324d2f42597d43c41ce72d6d80150e6.1668677493.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Add containers suitable to run yocto kirkstone build based on ubuntu 22.04. It contains all packages required by Yocto and a checkout of the layers required to build Xen with Yocto. Add a generic docker image template to be used to automatically generate docker files for different configurations: - specific yocto version - different targets (qemu arm, arm64 and x86) - different host platforms (x86 or arm64) During a call to 'make all', only the images for the current host platform will be generated. If needed, images for an other host platform can be generated manually by calling the right make target (see make help). Add a build script to build and run xen on qemu using Yocto. The script supports arm32, arm64 and x86_64 and checks that dom0 is properly booting. At this stage this does not run any guest on top of dom0. The script is to be executed in one of the docker images to build and run a system using a Xen source tree. Signed-off-by: Bertrand Marquis --- Changes in v4: - Rework the system to have one dockerfile template from which make will generate the required dockerfiles for the wanted configuration - add support for different host architectures - Merge the generation system into one single dockerfile - Merge patches 1 and 2 in a single patch - Introduce CONTAINERS_EXTRA to have extra containers not built by default (for those not used by CI but useful to users) Changes in v3: - limit number of jobs in yocto by default to 8 and add --num-jobs option to the script to set a custom number of jobs - do not copy anymore the build-yocto.sh script inside the main image so that the current one in the repository is used when running Changes in v2: - add a --dump-log command line option to build-yocto.sh script to dump the logs if an error occurs. Changes in v1: - add --image command line argument to build-yocto.sh to allow building something different than xen-image-minimal. - modify dockerfile to have one layer per line and make it easier to add other. I kept the for loop to keep the number of docker steps lower - update commit message to warn that no guest are tested. - fix build-yocto script to properly return with an error if expect script ends up in timeout or EOF. --- automation/build/Makefile | 14 +- automation/build/yocto/build-yocto.sh | 349 +++++++++++++++++++++ automation/build/yocto/yocto.dockerfile.in | 114 +++++++ automation/build/yocto/yocto.inc | 41 +++ 4 files changed, 516 insertions(+), 2 deletions(-) create mode 100755 automation/build/yocto/build-yocto.sh create mode 100644 automation/build/yocto/yocto.dockerfile.in create mode 100644 automation/build/yocto/yocto.inc diff --git a/automation/build/Makefile b/automation/build/Makefile index a4b2b85178cf..72a5335baec1 100644 --- a/automation/build/Makefile +++ b/automation/build/Makefile @@ -1,13 +1,18 @@ # the base of where these containers will appear REGISTRY := registry.gitlab.com/xen-project/xen -CONTAINERS = $(subst .dockerfile,,$(wildcard */*.dockerfile)) +CONTAINERS = $(filter-out yocto/%,$(subst .dockerfile,,$(wildcard */*.dockerfile))) +CONTAINERS_EXTRA = DOCKER_CMD ?= docker +include yocto/yocto.inc + help: @echo "Builds containers for building Xen based on different distros" @echo "To build one run 'make DISTRO/VERSION'. Available containers:" @$(foreach file,$(sort $(CONTAINERS)),echo ${file};) + @echo "Extra containers (not built using make all):" + @$(foreach file,$(sort $(CONTAINERS_EXTRA)),echo ${file};) @echo "To push container builds, set the env var PUSH" %: %.dockerfile ## Builds containers @@ -16,5 +21,10 @@ help: $(DOCKER_CMD) push $(REGISTRY)/$(@D):$(@F); \ fi -.PHONY: all +.PHONY: all clean all: $(CONTAINERS) + +# Remove generated dockerfiles for yocto +clean: + rm -f yocto/*.dockerfiles + diff --git a/automation/build/yocto/build-yocto.sh b/automation/build/yocto/build-yocto.sh new file mode 100755 index 000000000000..d0c93dfaffe0 --- /dev/null +++ b/automation/build/yocto/build-yocto.sh @@ -0,0 +1,349 @@ +#!/bin/bash +# +# Yocto meta virtualization build and run script +# +# This script is building Yocto xen-image-minimal for qemu targets and run +# them using runqemu inside yocto to check that dom0 is booting properly +# The build is using a local xen source tree so that specific patches can be +# tested. +# In order to optimize the build time, a build cache is used so that only xen +# packages and its dependencies are rebuilt (qemu and final image mainly). +# + +# Directories +YOCTODIR="$HOME/yocto-layers" +CACHEDIR="$HOME/yocto-cache" +LOGDIR="$HOME/logs" +XENDIR="$HOME/xen" +BUILDDIR="$HOME/build" + +# what yocto bsp we support +TARGET_SUPPORTED="qemuarm qemuarm64 qemux86-64" +VERBOSE="n" +TARGETLIST="" +BUILDJOBS="8" + +# actions to do +do_clean="n" +do_build="y" +do_run="y" +do_localsrc="n" +do_dump="n" +build_result=0 + +# layers to include in the project +build_layerlist="poky/meta poky/meta-poky poky/meta-yocto-bsp \ + meta-openembedded/meta-oe meta-openembedded/meta-python \ + meta-openembedded/meta-filesystems \ + meta-openembedded/meta-networking meta-virtualization" + +# yocto image to build +build_image="xen-image-minimal" + +function print_progress() { + echo -n "$(date +%T) $*" +} + +function run_task() { + local task_name="$1" + local task_target="$2" + + task_log="${task_name//project_}-${task_target}" + + mkdir -p "${LOGDIR}" + print_progress + echo -n "${task_name//project_} ${task_target}: " + if [ "${VERBOSE}" = "n" ]; then + $@ > "${LOGDIR}/${task_log}.log" 2>&1 + else + $@ 2>&1 | tee "${LOGDIR}/${task_log}.log" + fi + + if [ ${?} -ne 0 ]; then + echo "Error" + build_result=$((build_result+1)) + if [ "${do_dump}" = "y" ]; then + echo + echo "############ LOGS-START ############" + cat "${LOGDIR}/${task_log}.log" + echo "############ LOGS-END ############" + echo + fi + return 1 + else + echo "OK" + return 0 + fi +} + +function project_create() { + target="${1:?}" + destdir="${BUILDDIR}/${target}" + + ( + # init yocto project + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" + + # add needed layers + for layer in ${build_layerlist}; do + bitbake-layers add-layer ${YOCTODIR}/${layer} || exit 1 + done + ) || return 1 + + # customize project configuration + cat <> "${destdir}/conf/local.conf" +# Yocto BSP +MACHINE = "${target}" + +# Use local cache to reuse previous builds results +SSTATE_DIR = "${CACHEDIR}/sstate-cache" +DL_DIR = "${CACHEDIR}/downloads" + +# Enable xen and virtualization +DISTRO_FEATURES = " virtualization xen ipv4" + +# Speed up run by not generating ssh host keys +IMAGE_INSTALL:append:pn-xen-image-minimal = " ssh-pregen-hostkeys" + +# Save some disk space +INHERIT += "rm_work" + +# Reduce number of jobs +BB_NUMBER_THREADS="${BUILDJOBS}" + +EOF + + if [ "${do_localsrc}" = "y" ]; then + XENVERS=$(grep -e "^XEN_REL" \ + "${YOCTODIR}"/meta-virtualization/recipes-extended/xen/xen_*.bb \ + 2> /dev/null | tr -d ' ' | tr -d '?' | tr -d '"' \ + | sed -e "s/.*=//" | sort -V | tail -n 1) + + XENBASE=$(dirname "$(realpath -m "${XENDIR}")") + XENSUB=$(basename "$(realpath -m "${XENDIR}")") + + cat <> "${destdir}/conf/local.conf" +# Use local sources for xen and xen-tools +FILESEXTRAPATHS:prepend:pn-xen := "${XENBASE}:" +FILESEXTRAPATHS:prepend:pn-xen-tools := "${XENBASE}:" + +SRC_URI:pn-xen = "file://${XENSUB}/;subdir=local-xen/" +SRC_URI:pn-xen-tools = "file://${XENSUB}/;subdir=local-xen/" + +PREFERRED_VERSION:pn-xen = "${XENVERS}%" +PREFERRED_VERSION:pn-xen-tools = "${XENVERS}%" + +S:pn-xen = "\${WORKDIR}/local-xen/${XENSUB}" +S:pn-xen-tools = "\${WORKDIR}/local-xen/${XENSUB}" + +SRCREV:pn-xen = "\${AUTOREV}" +SRCREV:pn-xen-tools = "\${AUTOREV}" + +SRCPV:pn-xen = "1" +SRCPV:pn-xen-tools = "1" + +# Disable all QA errors as the recipe is not up to date with changes in Xen +# when we use local sources +ERROR_QA:pn-xen = "arch" +ERROR_QA:pn-xen-tools = "arch" + +EOF + fi +} + +function project_build() { + target="${1:?}" + destdir="${BUILDDIR}/${target}" + + ( + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" + + bitbake "${build_image}" || exit 1 + ) || return 1 +} + +function project_clean() { + target="${1:?}" + destdir="${BUILDDIR}/${target}" + + rm -rf "${destdir}" +} + +function project_run() { + target="${1:?}" + destdir="${BUILDDIR}/${target}" + ( + source ${YOCTODIR}/poky/oe-init-build-env "${destdir}" > /dev/null 2>&1 + + /usr/bin/expect < $$@ + cat $$< | \ + sed -e "s,##YOCTOVERSION##,$(1),g" | \ + sed -e "s,##YOCTOTARGET##,$(2),g" | \ + sed -e "s,##DOCKERPLAT##,$(3)/,g" > $$@ + +endef + +$(eval $(foreach vers,$(YOCTO_VERSION),\ + $(foreach tar,$(YOCTO_TARGETS),\ + $(foreach hst,$(YOCTO_HOSTS),\ + $(call GEN_DOCKER,$(vers),$(tar),$(hst),$(if $(filter amd64,$(hst)),,-$(hst))))))) From patchwork Thu Nov 17 09:39:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 13046461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D065CC433FE for ; Thu, 17 Nov 2022 09:40:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.444828.699926 (Exim 4.92) (envelope-from ) id 1ovbO1-0002Qo-Lr; Thu, 17 Nov 2022 09:40:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 444828.699926; Thu, 17 Nov 2022 09:40:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovbO1-0002PW-EN; Thu, 17 Nov 2022 09:40:45 +0000 Received: by outflank-mailman (input) for mailman id 444828; Thu, 17 Nov 2022 09:40:43 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovbNz-0001D0-QU for xen-devel@lists.xenproject.org; Thu, 17 Nov 2022 09:40:43 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id e74c36f9-665b-11ed-8fd2-01056ac49cbb; Thu, 17 Nov 2022 10:40:42 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A247B1477; Thu, 17 Nov 2022 01:40:48 -0800 (PST) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A08ED3F73B; Thu, 17 Nov 2022 01:40:41 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e74c36f9-665b-11ed-8fd2-01056ac49cbb From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v4 2/3] automation: Add a clean rule for containers Date: Thu, 17 Nov 2022 09:39:50 +0000 Message-Id: <3a127e836d744fbb2954690293ee47d2c0c899e2.1668677493.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Add make clean support to remove the containers from the local docker registry: - make clean: remove all images - clean-yocto/kirkstone-qemuarm: remove yocto kirkstone for qemuarm image Signed-off-by: Bertrand Marquis --- Changes in v4: - also generate clean rule for CONTAINERS_EXTRA Changes in v3: - none Changes in v2: - none Changes in v1: - patch added --- automation/build/Makefile | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/automation/build/Makefile b/automation/build/Makefile index 72a5335baec1..5b5f10c63ea4 100644 --- a/automation/build/Makefile +++ b/automation/build/Makefile @@ -28,3 +28,15 @@ all: $(CONTAINERS) clean: rm -f yocto/*.dockerfiles +define CLEAN_RULE +.PHONY: clean-$(1) +clean-$(1): +ifneq ($$(shell docker image ls -q $(REGISTRY)/$(subst /,:,$(1))),) + docker image rm $(REGISTRY)/$(subst /,:,$(1)) +endif + +clean: clean-$(1) + +endef + +$(eval $(foreach img,$(CONTAINERS) $(CONTAINERS_EXTRA),$(call CLEAN_RULE,$(img)))) From patchwork Thu Nov 17 09:39:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 13046462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0A17CC433FE for ; Thu, 17 Nov 2022 09:40:56 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.444830.699943 (Exim 4.92) (envelope-from ) id 1ovbO2-0002qi-U4; Thu, 17 Nov 2022 09:40:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 444830.699943; Thu, 17 Nov 2022 09:40:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovbO2-0002pQ-Mn; Thu, 17 Nov 2022 09:40:46 +0000 Received: by outflank-mailman (input) for mailman id 444830; Thu, 17 Nov 2022 09:40:46 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ovbO1-0002Nb-T0 for xen-devel@lists.xenproject.org; Thu, 17 Nov 2022 09:40:45 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id e7c4693f-665b-11ed-91b6-6bf2151ebd3b; Thu, 17 Nov 2022 10:40:44 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 857B1150C; Thu, 17 Nov 2022 01:40:49 -0800 (PST) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 85A663F73B; Thu, 17 Nov 2022 01:40:42 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e7c4693f-665b-11ed-91b6-6bf2151ebd3b From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: michal.orzel@amd.com, Doug Goldstein , Stefano Stabellini Subject: [PATCH v4 3/3] automation: Add CI test jobs for Yocto Date: Thu, 17 Nov 2022 09:39:51 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 From: Michal Orzel Populate test jobs for Yocto based tests using the provided containers. Due to the size restrictions, it is currently not possible to split the build and run tasks, therefore everything is done in a single step. Test jobs for the supported Yocto targets are generic to avoid the necessity to add new ones after each Yocto release. The only thing required to be changed after updating the containers is the variable YOCTO_VERSION stored in a .yocto-test template. The .yocto-test-arm64 template is to be used for jobs running on an arm64 hardware and .yocto-test-x86-64 for x86. Current jobs are using the arm64 version (x86 templates are here as a provision). Signed-off-by: Michal Orzel Signed-off-by: Bertrand Marquis --- Changes in v4: - add .yocto-test for arm64 and x86 - make yocto jobs use arm64 version Changes in v3: - patch added --- automation/gitlab-ci/test.yaml | 43 ++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index c7e0078e04f1..6ce2fd63eee6 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -54,6 +54,33 @@ tags: - x86_64 +.yocto-test: + extends: .test-jobs-common + script: + - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} + variables: + YOCTO_VERSION: kirkstone + CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}-${YOCTO_HOST} + artifacts: + paths: + - 'logs/*' + when: always + needs: [] + +.yocto-test-arm64: + extends: .yocto-test + variables: + YOCTO_HOST: arm64v8 + tags: + - arm64 + +.yocto-test-x86-64: + extends: .yocto-test + variables: + YOCTO_HOST: amd64 + tags: + - x86_64 + # Test jobs build-each-commit-gcc: extends: .test-jobs-common @@ -188,3 +215,19 @@ qemu-smoke-x86-64-clang-pvh: - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE} needs: - debian-unstable-clang-debug + +# Yocto test jobs +yocto-qemuarm64: + extends: .yocto-test-arm64 + variables: + YOCTO_BOARD: qemuarm64 + +yocto-qemuarm: + extends: .yocto-test-arm64 + variables: + YOCTO_BOARD: qemuarm + +yocto-qemux86-64: + extends: .yocto-test-arm64 + variables: + YOCTO_BOARD: qemux86-64