From patchwork Mon Nov 22 10:42:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: WANG Xuerui X-Patchwork-Id: 12631647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2970C433EF for ; Mon, 22 Nov 2021 11:04:24 +0000 (UTC) Received: from localhost ([::1]:57148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mp77W-0002Ze-PH for qemu-devel@archiver.kernel.org; Mon, 22 Nov 2021 06:04:22 -0500 Received: from eggs.gnu.org ([209.51.188.92]:53004) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mp6nM-000231-Ff for qemu-devel@nongnu.org; Mon, 22 Nov 2021 05:43:33 -0500 Received: from mail.xen0n.name ([115.28.160.31]:40916 helo=mailbox.box.xen0n.name) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mp6nH-0007jJ-PL for qemu-devel@nongnu.org; Mon, 22 Nov 2021 05:43:31 -0500 Received: from ld50.lan (unknown [101.88.31.179]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by mailbox.box.xen0n.name (Postfix) with ESMTPSA id B8B4160B08; Mon, 22 Nov 2021 18:42:57 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=xen0n.name; s=mail; t=1637577778; bh=tvMOZ8aNSmXtiCQg64zkCUTfT001SNnyD3hO+Up+MDU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZPXNT1xJHt8nxGZeHDpac/DyQq1FIt2lyfFKVXVWE/+KXxjSy/CYWd/EJYS24fxSn PUrZOFg23fW8hTlJfqQWi0DLyul6HxARgHvTxHGp7UqRicSXErKDCAQ/Oic4eEyBmw Qs1YDnb7m00qIIKF/GgT94H8BUDIGBtvDmISnZx4= From: WANG Xuerui To: qemu-devel@nongnu.org Subject: [PATCH v7 31/31] tests/docker: Add gentoo-loongarch64-cross image and run cross builds in GitLab Date: Mon, 22 Nov 2021 18:42:01 +0800 Message-Id: <20211122104201.112695-32-git@xen0n.name> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20211122104201.112695-1-git@xen0n.name> References: <20211122104201.112695-1-git@xen0n.name> MIME-Version: 1.0 Received-SPF: pass client-ip=115.28.160.31; envelope-from=git@xen0n.name; helo=mailbox.box.xen0n.name X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , XiaoJuan Yang , Richard Henderson , =?utf-8?q?Philippe_Mathie?= =?utf-8?q?u-Daud=C3=A9?= , Laurent Vivier , WANG Xuerui , =?utf-8?q?Alex_Benn=C3=A9e?= , Song Gao Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Normally this would be based on qemu/debian10 or qemu/ubuntu2004, but after a week-long struggle, I still cannot build stage2 gcc with the known-good LoongArch toolchain sources, so I chose the least-resistance path with Gentoo as base image. As this image is not expected to be re-built by CI, like hexagon, it should not take much maintenance effort; also it's expected to be replaced as soon as Debian is available. As the LoongArch target has not been merged yet, a check-tcg job is not added at the moment, but cross builds with the TCG port are already possible, and added to CI matrix. As a reference, the image takes about 27 minutes to rebuild on a Threadripper 3990X system with Docker operating on HDD; YMMV but it probably wouldn't become significantly shorter, as everything needs to be built from source in our case. Signed-off-by: WANG Xuerui --- .gitlab-ci.d/container-cross.yml | 27 ++++ .gitlab-ci.d/crossbuilds.yml | 16 +++ MAINTAINERS | 2 + tests/docker/Makefile.include | 21 +++ .../gentoo-loongarch64-cross.docker | 19 +++ .../build-toolchain.sh | 123 ++++++++++++++++++ 6 files changed, 208 insertions(+) create mode 100644 tests/docker/dockerfiles/gentoo-loongarch64-cross.docker create mode 100755 tests/docker/dockerfiles/gentoo-loongarch64-cross.docker.d/build-toolchain.sh diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml index a3b5b90552..7a8cc556cc 100644 --- a/.gitlab-ci.d/container-cross.yml +++ b/.gitlab-ci.d/container-cross.yml @@ -82,6 +82,33 @@ hppa-debian-cross-container: variables: NAME: debian-hppa-cross +# Similar to hexagon, we don't want to build loongarch64 in the CI either. +loongarch64-cross-container: + image: docker:stable + stage: containers + rules: + - if: '$CI_PROJECT_NAMESPACE == "qemu-project"' + when: never + - when: always + variables: + NAME: gentoo-loongarch64-cross + GIT_DEPTH: 1 + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest" + - docker info + - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - echo "TAG:$TAG" + - echo "COMMON_TAG:$COMMON_TAG" + - docker pull $COMMON_TAG + - docker tag $COMMON_TAG $TAG + - docker push "$TAG" + after_script: + - docker logout + m68k-debian-cross-container: extends: .container_job_template stage: containers-layer2 diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml index 17d6cb3e45..49f5b5525e 100644 --- a/.gitlab-ci.d/crossbuilds.yml +++ b/.gitlab-ci.d/crossbuilds.yml @@ -68,6 +68,22 @@ cross-i386-tci: EXTRA_CONFIGURE_OPTS: --target-list=i386-softmmu,i386-linux-user,aarch64-softmmu,aarch64-linux-user,ppc-softmmu,ppc-linux-user MAKE_CHECK_ARGS: check check-tcg +cross-loongarch64-system: + extends: .cross_system_build_job + allow_failure: true + needs: + job: loongarch64-cross-container + variables: + IMAGE: gentoo-loongarch64-cross + +cross-loongarch64-user: + extends: .cross_user_build_job + allow_failure: true + needs: + job: loongarch64-cross-container + variables: + IMAGE: gentoo-loongarch64-cross + cross-mips-system: extends: .cross_system_build_job needs: diff --git a/MAINTAINERS b/MAINTAINERS index a638fb6b31..165ef630bf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3141,6 +3141,8 @@ LoongArch64 TCG target M: WANG Xuerui S: Maintained F: tcg/loongarch64/ +F: tests/docker/dockerfiles/gentoo-loongarch64-cross.docker +F: tests/docker/dockerfiles/gentoo-loongarch64-cross.docker.d/ MIPS TCG target M: Philippe Mathieu-Daudé diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index f1a0c5db7a..a2cdf193bb 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -176,6 +176,27 @@ docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.dock qemu/debian-hexagon-cross --add-current-user, \ "PREPARE", "debian-hexagon-cross")) +# +# Same for loongarch64-cross. +# +docker-image-gentoo-loongarch64-cross: $(DOCKER_FILES_DIR)/gentoo-loongarch64-cross.docker + $(if $(NOCACHE), \ + $(call quiet-command, \ + $(DOCKER_SCRIPT) build -t qemu/gentoo-loongarch64-cross -f $< \ + $(if $V,,--quiet) --no-cache \ + --registry $(DOCKER_REGISTRY) --extra-files \ + $(DOCKER_FILES_DIR)/gentoo-loongarch64-cross.docker.d/build-toolchain.sh, \ + "BUILD", "gentoo-loongarch64-cross"), \ + $(call quiet-command, \ + $(DOCKER_SCRIPT) fetch $(if $V,,--quiet) \ + qemu/gentoo-loongarch64-cross $(DOCKER_REGISTRY), \ + "FETCH", "gentoo-loongarch64-cross") \ + $(call quiet-command, \ + $(DOCKER_SCRIPT) update $(if $V,,--quiet) \ + qemu/gentoo-loongarch64-cross --add-current-user, \ + "PREPARE", "gentoo-loongarch64-cross")) + + debian-toolchain-run = \ $(if $(NOCACHE), \ $(call quiet-command, \ diff --git a/tests/docker/dockerfiles/gentoo-loongarch64-cross.docker b/tests/docker/dockerfiles/gentoo-loongarch64-cross.docker new file mode 100644 index 0000000000..6f4582d8d2 --- /dev/null +++ b/tests/docker/dockerfiles/gentoo-loongarch64-cross.docker @@ -0,0 +1,19 @@ +# +# Docker LoongArch64 cross-compiler target +# +# This docker target is used for building loongarch64 tests. As it also +# needs to be able to build QEMU itself in CI we include it's +# build-deps. It is also a "stand-alone" image so as not to be +# triggered by re-builds on other base images given it takes a long +# time to build. +# + +FROM gentoo/stage3:20211116 + +ADD build-toolchain.sh /root/build-toolchain.sh +RUN cd /root && ./build-toolchain.sh + +# Specify the cross prefix for this image (see tests/docker/common.rc) +ENV QEMU_CONFIGURE_OPTS --cross-prefix=loongarch64-unknown-linux-gnu- +# We don't have target support at the moment, so just verify that things build +ENV DEF_TARGET_LIST x86_64-softmmu,x86_64-linux-user diff --git a/tests/docker/dockerfiles/gentoo-loongarch64-cross.docker.d/build-toolchain.sh b/tests/docker/dockerfiles/gentoo-loongarch64-cross.docker.d/build-toolchain.sh new file mode 100755 index 0000000000..90aa51c255 --- /dev/null +++ b/tests/docker/dockerfiles/gentoo-loongarch64-cross.docker.d/build-toolchain.sh @@ -0,0 +1,123 @@ +#!/bin/bash + +set -e + +TRIPLE=loongarch64-unknown-linux-gnu +CROSSDEV_OV=/opt/crossdev-overlay +LOONGSON_OV=/opt/loongson-overlay +CROSS_EMERGE="${TRIPLE}-emerge" + +J=$(expr $(nproc) / 2) +echo "MAKEOPTS=\"-j${J}\"" >> /etc/portage/make.conf +echo "EGIT_CLONE_TYPE=shallow" >> /etc/portage/make.conf + +# these features are not supported in Docker +export FEATURES="-ipc-sandbox -network-sandbox" + +# populate Portage tree +GENTOO_MIRROR='https://bouncer.gentoo.org/fetch/root/all' +PORTAGE_SNAPSHOT_FILE=gentoo-20211116.tar.xz +pushd /tmp + wget "${GENTOO_MIRROR}/snapshots/${PORTAGE_SNAPSHOT_FILE}" + + mkdir -p /var/db/repos/gentoo + pushd /var/db/repos/gentoo + tar -xf "/tmp/${PORTAGE_SNAPSHOT_FILE}" --strip-components=1 + popd + + rm "$PORTAGE_SNAPSHOT_FILE" +popd + +emerge -j crossdev dev-vcs/git + +# prepare for crossdev +mkdir /etc/portage/repos.conf +crossdev -t "$TRIPLE" --ov-output "$CROSSDEV_OV" --init-target + +git clone https://github.com/xen0n/loongson-overlay.git "$LOONGSON_OV" +pushd "$LOONGSON_OV" + git checkout 6aae40366a9c11e82ae0e9450b40b056ef58409c +popd + +pushd "${CROSSDEV_OV}/cross-${TRIPLE}" + rm binutils gcc glibc linux-headers + ln -s "${LOONGSON_OV}/sys-devel/binutils" . + ln -s "${LOONGSON_OV}/sys-devel/gcc" . + ln -s "${LOONGSON_OV}/sys-libs/glibc" . + ln -s "${LOONGSON_OV}/sys-kernel/linux-headers" . +popd + +cat > "${CROSSDEV_OV}/metadata/layout.conf" < repos.conf/loongson.conf <