From patchwork Thu Dec 7 13:53:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13483322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A573FC4167B for ; Thu, 7 Dec 2023 13:53:46 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.649873.1014935 (Exim 4.92) (envelope-from ) id 1rBEor-0002cn-Js; Thu, 07 Dec 2023 13:53:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 649873.1014935; Thu, 07 Dec 2023 13:53:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEor-0002cc-Gj; Thu, 07 Dec 2023 13:53:37 +0000 Received: by outflank-mailman (input) for mailman id 649873; Thu, 07 Dec 2023 13:53:36 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEoq-0002bv-6w for xen-devel@lists.xenproject.org; Thu, 07 Dec 2023 13:53:36 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 030ad929-9508-11ee-9b0f-b553b5be7939; Thu, 07 Dec 2023 14:53:34 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6AD19139F; Thu, 7 Dec 2023 05:54:19 -0800 (PST) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C348F3F6C4; Thu, 7 Dec 2023 05:53:30 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 030ad929-9508-11ee-9b0f-b553b5be7939 From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Henry Wang , Doug Goldstein , Stefano Stabellini , Michal Orzel , Julien Grall , Bertrand Marquis , Wei Chen Subject: [PATCH 1/5] automation: Add a Dockerfile for running FVP_Base jobs Date: Thu, 7 Dec 2023 21:53:14 +0800 Message-Id: <20231207135318.1912846-2-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231207135318.1912846-1-Henry.Wang@arm.com> References: <20231207135318.1912846-1-Henry.Wang@arm.com> MIME-Version: 1.0 Fixed Virtual Platforms (FVPs) are complete simulations of an Arm system, including processor, memory and peripherals. These are set out in a "programmer's view", which gives programmers a comprehensive model on which to build and test software. FVP can be configured to different setups by its cmdline parameters, and hence having the FVP in CI will provide us with the flexibility to test Arm features and setups that we find difficult to use real hardware or emulators. This commit adds a Dockerfile for the new arm64v8 container with FVP installed, based on the debian bookworm-arm64v8 image. This container will be used to run the FVP test jobs. Compared to the debian bookworm-arm64v8 image, the packages in the newly added FVP container does not contain the `u-boot-qemu`, and adds the `expect` to run expect scripts introduced by following commits, `telnet` to connect to FVP, and `tftpd-hpa` to provide the TFTP service for the FVP. Signed-off-by: Henry Wang Reviewed-by: Stefano Stabellini --- .../debian/bookworm-arm64v8-fvp.dockerfile | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 automation/build/debian/bookworm-arm64v8-fvp.dockerfile diff --git a/automation/build/debian/bookworm-arm64v8-fvp.dockerfile b/automation/build/debian/bookworm-arm64v8-fvp.dockerfile new file mode 100644 index 0000000000..3b87dc5a5b --- /dev/null +++ b/automation/build/debian/bookworm-arm64v8-fvp.dockerfile @@ -0,0 +1,64 @@ +FROM --platform=linux/arm64/v8 debian:bookworm +LABEL maintainer.name="The Xen Project" \ + maintainer.email="xen-devel@lists.xenproject.org" + +ARG FVP_BASE_VERSION="11.23_9_Linux64_armv8l" + +ENV DEBIAN_FRONTEND=noninteractive +ENV USER root + +RUN mkdir /build +WORKDIR /build + +# build depends +RUN apt-get update && \ + apt-get --quiet --yes install \ + build-essential \ + zlib1g-dev \ + libncurses5-dev \ + libssl-dev \ + python3-dev \ + python3-setuptools \ + xorg-dev \ + uuid-dev \ + libyajl-dev \ + libaio-dev \ + libglib2.0-dev \ + clang \ + libpixman-1-dev \ + pkg-config \ + flex \ + bison \ + acpica-tools \ + libfdt-dev \ + bin86 \ + bcc \ + liblzma-dev \ + libnl-3-dev \ + ocaml-nox \ + libfindlib-ocaml-dev \ + markdown \ + transfig \ + pandoc \ + checkpolicy \ + wget \ + git \ + nasm \ + # for test phase, fvp-smoke-* jobs + u-boot-tools \ + expect \ + device-tree-compiler \ + curl \ + cpio \ + busybox-static \ + telnet \ + tftpd-hpa \ + && \ + apt-get autoremove -y && \ + apt-get clean && \ + rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/* + +RUN wget https://developer.arm.com/-/media/Files/downloads/ecosystem-models/FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz && \ + mkdir -p /FVP/FVP_Base_RevC-2xAEMvA && \ + tar -xvzf FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz -C /FVP/FVP_Base_RevC-2xAEMvA && \ + rm FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz From patchwork Thu Dec 7 13:53:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13483325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D0B5C4167B for ; Thu, 7 Dec 2023 13:53:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.649874.1014945 (Exim 4.92) (envelope-from ) id 1rBEot-0002u1-Vv; Thu, 07 Dec 2023 13:53:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 649874.1014945; Thu, 07 Dec 2023 13:53:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEot-0002tu-T5; Thu, 07 Dec 2023 13:53:39 +0000 Received: by outflank-mailman (input) for mailman id 649874; Thu, 07 Dec 2023 13:53:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEos-0002Al-Lb for xen-devel@lists.xenproject.org; Thu, 07 Dec 2023 13:53:38 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 04f5b949-9508-11ee-98e7-6d05b1d4d9a1; Thu, 07 Dec 2023 14:53:37 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1699912FC; Thu, 7 Dec 2023 05:54:23 -0800 (PST) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6F7F23F6C4; Thu, 7 Dec 2023 05:53:34 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 04f5b949-9508-11ee-98e7-6d05b1d4d9a1 From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Henry Wang , Doug Goldstein , Stefano Stabellini , Michal Orzel , Julien Grall , Bertrand Marquis , Wei Chen Subject: [PATCH 2/5] automation: Add the Dockerfile to build TF-A and U-Boot for FVP Date: Thu, 7 Dec 2023 21:53:15 +0800 Message-Id: <20231207135318.1912846-3-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231207135318.1912846-1-Henry.Wang@arm.com> References: <20231207135318.1912846-1-Henry.Wang@arm.com> MIME-Version: 1.0 Unlike the emulators that currently being used in the CI pipelines, the FVP must start at EL3. Therefore we need the firmware, i.e. the TrustedFirmware-A (TF-A), for corresponding functionality. There is a dedicated board (vexpress_fvp) in U-Boot (serve as the BL33 of the TF-A) for the FVP platform, so the U-Boot should also be compiled for the FVP platform instead of reusing the U-Boot for the existing emulators used in the CI pipelines. To avoid compiling TF-A and U-Boot everytime in the job, adding a Dockerfile to the test artifacts to build TF-A v2.9.0 and U-Boot v2023.10 for FVP. The binaries for the TF-A and U-Boot, as well as the device tree for the FVP platform, will be saved (and exported by the CI job introduced by following commits). Note that, a patch for the TF-A will be applied before building to enable the virtio-net and the virtio-rng device on the FVP. The virtio-net device will provide the networking service for FVP, and the virtio-rng device will improve the speed of the FVP. Signed-off-by: Henry Wang Reviewed-by: Stefano Stabellini --- .../2023.10-2.9.0-arm64v8.dockerfile | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 automation/tests-artifacts/armfvp-uboot-tfa/2023.10-2.9.0-arm64v8.dockerfile diff --git a/automation/tests-artifacts/armfvp-uboot-tfa/2023.10-2.9.0-arm64v8.dockerfile b/automation/tests-artifacts/armfvp-uboot-tfa/2023.10-2.9.0-arm64v8.dockerfile new file mode 100644 index 0000000000..6566b60545 --- /dev/null +++ b/automation/tests-artifacts/armfvp-uboot-tfa/2023.10-2.9.0-arm64v8.dockerfile @@ -0,0 +1,48 @@ +FROM --platform=linux/arm64/v8 debian:bookworm +LABEL maintainer.name="The Xen Project" \ + maintainer.email="xen-devel@lists.xenproject.org" + +ENV DEBIAN_FRONTEND=noninteractive +ENV UBOOT_VERSION="2023.10" +ENV TFA_VERSION="v2.9.0" +ENV USER root + +RUN mkdir /build +WORKDIR /build + +# build depends +RUN apt-get update && \ + apt-get --quiet --yes install \ + build-essential \ + libssl-dev \ + bc \ + curl \ + flex \ + bison \ + git \ + device-tree-compiler && \ + apt-get autoremove -y && \ + apt-get clean && \ + rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/* + +# Build U-Boot and TF-A +RUN curl -fsSLO https://ftp.denx.de/pub/u-boot/u-boot-"$UBOOT_VERSION".tar.bz2 && \ + tar xvf u-boot-"$UBOOT_VERSION".tar.bz2 && \ + cd u-boot-"$UBOOT_VERSION" && \ + make -j$(nproc) V=1 vexpress_fvp_defconfig && \ + make -j$(nproc) V=1 all && \ + cd .. && \ + git clone --branch "$TFA_VERSION" --depth 1 https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git && \ + cd trusted-firmware-a && \ + curl -fsSLO https://git.yoctoproject.org/meta-arm/plain/meta-arm-bsp/recipes-bsp/trusted-firmware-a/files/fvp-base/0001-fdts-fvp-base-Add-stdout-path-and-virtio-net-and-rng.patch \ + --output 0001-fdts-fvp-base-Add-stdout-path-and-virtio-net-and-rng.patch && \ + git config --global user.email "you@example.com" && \ + git config --global user.name "Your Name" && \ + git am 0001-fdts-fvp-base-Add-stdout-path-and-virtio-net-and-rng.patch && \ + make -j$(nproc) DEBUG=1 PLAT=fvp ARCH=aarch64 FVP_DT_PREFIX=fvp-base-gicv3-psci-1t all && \ + make -j$(nproc) DEBUG=1 PLAT=fvp ARCH=aarch64 FVP_DT_PREFIX=fvp-base-gicv3-psci-1t fip BL33=../u-boot-"$UBOOT_VERSION"/u-boot.bin && \ + cp build/fvp/debug/bl1.bin / && \ + cp build/fvp/debug/fip.bin / && \ + cp build/fvp/debug/fdts/fvp-base-gicv3-psci-1t.dtb / && \ + cd /build && \ + rm -rf u-boot-"$UBOOT_VERSION" trusted-firmware-a From patchwork Thu Dec 7 13:53:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13483324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29846C46CA0 for ; Thu, 7 Dec 2023 13:53:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.649875.1014955 (Exim 4.92) (envelope-from ) id 1rBEox-0003E0-81; Thu, 07 Dec 2023 13:53:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 649875.1014955; Thu, 07 Dec 2023 13:53:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEox-0003Dl-4t; Thu, 07 Dec 2023 13:53:43 +0000 Received: by outflank-mailman (input) for mailman id 649875; Thu, 07 Dec 2023 13:53:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEow-0002Al-6g for xen-devel@lists.xenproject.org; Thu, 07 Dec 2023 13:53:42 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 071a17ec-9508-11ee-98e7-6d05b1d4d9a1; Thu, 07 Dec 2023 14:53:40 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C4DF312FC; Thu, 7 Dec 2023 05:54:26 -0800 (PST) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 29CE03F6C4; Thu, 7 Dec 2023 05:53:37 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 071a17ec-9508-11ee-98e7-6d05b1d4d9a1 From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Henry Wang , Doug Goldstein , Stefano Stabellini , Michal Orzel , Julien Grall , Bertrand Marquis , Wei Chen Subject: [PATCH 3/5] automation: Add the expect script with test case for FVP Date: Thu, 7 Dec 2023 21:53:16 +0800 Message-Id: <20231207135318.1912846-4-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231207135318.1912846-1-Henry.Wang@arm.com> References: <20231207135318.1912846-1-Henry.Wang@arm.com> MIME-Version: 1.0 To interact with the FVP (for example entering the U-Boot shell and transferring the files by TFTP), we need to connect the corresponding port by the telnet first. Use an expect script to simplify the automation of the whole "interacting with FVP" stuff. The expect script will firstly detect the IP of the host, then connect to the telnet port of the FVP, set the `serverip` and `ipaddr` for the TFTP service in the U-Boot shell, and finally boot Xen from U-Boot and wait for the expected results by Xen, Dom0 and DomU. Signed-off-by: Henry Wang --- .../expect/fvp-base-smoke-dom0-arm64.exp | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100755 automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp diff --git a/automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp b/automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp new file mode 100755 index 0000000000..25d9a5f81c --- /dev/null +++ b/automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp @@ -0,0 +1,73 @@ +#!/usr/bin/expect + +set timeout 2000 + +# Command to use to run must be given as first argument +# if options are required, quotes must be used: +# xxx.exp "cmd opt1 opt2" +set runcmd [lindex $argv 0] + +# Maximum number of line to be printed, this can be used to prevent runs to +# go forever on errors when Xen is rebooting +set maxline 1000 + +# Configure slow parameters used with send -s +# This allows us to slow down console writes to prevent issues with slow +# emulators or targets. +# Format here is: {NUM TIME} which reads as wait TIME seconds each NUM of +# characters, here we send 1 char each 100ms +set send_slow {1 .1} + +proc test_boot {{maxline} {host_ip}} { + expect_after { + -re "(.*)\r" { + if {$maxline != 0} { + set maxline [expr {$maxline - 1}] + if {$maxline <= 0} { + send_user "ERROR-Toomuch!\n" + exit 2 + } + } + exp_continue + } + timeout {send_user "ERROR-Timeout!\n"; exit 3} + eof {send_user "ERROR-EOF!\n"; exit 4} + } + + # Extract the telnet port numbers from FVP output, because the telnet ports + # are not guaranteed to be fixed numbers. + expect -re {terminal_0: Listening for serial connection on port [0-9]+} + set terminal_0 $expect_out(0,string) + if {[regexp {port (\d+)} $terminal_0 match port_0]} { + puts "terminal_0 port is $port_0" + } else { + puts "terminal_0 port not found" + exit 5 + } + + spawn bash -c "telnet localhost $port_0" + expect -re "Hit any key to stop autoboot.*" + send -s " \r" + send -s "setenv serverip $host_ip; setenv ipaddr $host_ip; tftpb 0x80200000 boot.scr; source 0x80200000\r" + + # Initial Xen boot + expect -re "\(XEN\).*Freed .* init memory." + + # Dom0 and DomU + expect -re "Domain-0.*" + expect -re "BusyBox.*" + expect -re "/ #.*" +} + +# Get host IP +spawn bash -c "hostname -I | awk '{print \$1}'" +expect -re {(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})} +set host_ip $expect_out(0,string) + +# Start the FVP and run the test +spawn bash -c "$runcmd" + +test_boot 2000 "$host_ip" + +send_user "\nExecution with SUCCESS\n" +exit 0 From patchwork Thu Dec 7 13:53:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13483326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91F5CC4167B for ; Thu, 7 Dec 2023 13:53:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.649876.1014965 (Exim 4.92) (envelope-from ) id 1rBEp2-0003eD-KL; Thu, 07 Dec 2023 13:53:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 649876.1014965; Thu, 07 Dec 2023 13:53:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEp2-0003dt-E4; Thu, 07 Dec 2023 13:53:48 +0000 Received: by outflank-mailman (input) for mailman id 649876; Thu, 07 Dec 2023 13:53:46 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEp0-0002bv-TQ for xen-devel@lists.xenproject.org; Thu, 07 Dec 2023 13:53:46 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 098ec8c9-9508-11ee-9b0f-b553b5be7939; Thu, 07 Dec 2023 14:53:45 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6114A12FC; Thu, 7 Dec 2023 05:54:30 -0800 (PST) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BA1723F6C4; Thu, 7 Dec 2023 05:53:41 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 098ec8c9-9508-11ee-9b0f-b553b5be7939 From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Henry Wang , Doug Goldstein , Stefano Stabellini , Michal Orzel , Julien Grall , Bertrand Marquis , Wei Chen Subject: [PATCH 4/5] automation: Add the script for the FVP smoke test Date: Thu, 7 Dec 2023 21:53:17 +0800 Message-Id: <20231207135318.1912846-5-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231207135318.1912846-1-Henry.Wang@arm.com> References: <20231207135318.1912846-1-Henry.Wang@arm.com> MIME-Version: 1.0 This commit adds the shell script for the FVP smoke test. Similarly as the QEMU jobs, the shell script will firstly prepare the DomU BusyBox image, use the ImageBuilder to arrange the binaries in memory and generate the U-Boot script, then start the test. To provide the TFTP service for the FVP, the shell script will also start the TFTP service, and copy the binaries needed by test to the TFTP directory used by the TFTP server. Signed-off-by: Henry Wang Reviewed-by: Stefano Stabellini --- .../scripts/fvp-base-smoke-dom0-arm64.sh | 117 ++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100755 automation/scripts/fvp-base-smoke-dom0-arm64.sh diff --git a/automation/scripts/fvp-base-smoke-dom0-arm64.sh b/automation/scripts/fvp-base-smoke-dom0-arm64.sh new file mode 100755 index 0000000000..716a63b0a8 --- /dev/null +++ b/automation/scripts/fvp-base-smoke-dom0-arm64.sh @@ -0,0 +1,117 @@ +#!/bin/bash + +set -ex + +# DomU Busybox +cd binaries +mkdir -p initrd +mkdir -p initrd/bin +mkdir -p initrd/sbin +mkdir -p initrd/etc +mkdir -p initrd/dev +mkdir -p initrd/proc +mkdir -p initrd/sys +mkdir -p initrd/lib +mkdir -p initrd/var +mkdir -p initrd/mnt +cp /bin/busybox initrd/bin/busybox +initrd/bin/busybox --install initrd/bin +echo "#!/bin/sh + +mount -t proc proc /proc +mount -t sysfs sysfs /sys +mount -t devtmpfs devtmpfs /dev +/bin/sh" > initrd/init +chmod +x initrd/init +cd initrd +find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz +cd .. + +mkdir -p rootfs +cd rootfs +tar xvzf ../initrd.tar.gz +mkdir proc +mkdir run +mkdir srv +mkdir sys +rm var/run +cp -ar ../dist/install/* . +mv ../initrd.cpio.gz ./root +cp ../Image ./root +echo "name=\"test\" +memory=512 +vcpus=1 +kernel=\"/root/Image\" +ramdisk=\"/root/initrd.cpio.gz\" +extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\" +" > root/test.cfg +echo "#!/bin/bash + +export LD_LIBRARY_PATH=/usr/local/lib +bash /etc/init.d/xencommons start + +xl list + +xl create -c /root/test.cfg + +" > etc/local.d/xen.start +chmod +x etc/local.d/xen.start +echo "rc_verbose=yes" >> etc/rc.conf +find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz +cd ../.. + +# Start a TFTP server to provide TFTP service to FVP +service tftpd-hpa start + +# ImageBuilder +echo 'MEMORY_START="0x80000000" +MEMORY_END="0xFF000000" + +DEVICE_TREE="fvp-base-gicv3-psci-1t.dtb" +XEN="xen" +DOM0_KERNEL="Image" +DOM0_RAMDISK="xen-rootfs.cpio.gz" +XEN_CMD="console=dtuart dom0_mem=1024M console_timestamps=boot" + +NUM_DOMUS=0 + +LOAD_CMD="tftpb" +UBOOT_SOURCE="boot.source" +UBOOT_SCRIPT="boot.scr"' > binaries/config +rm -rf imagebuilder +git clone https://gitlab.com/ViryaOS/imagebuilder +bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config + +# Copy files to the TFTP directory to use +cp ./binaries/boot.scr /srv/tftp/ +cp ./binaries/Image /srv/tftp/ +cp ./binaries/xen-rootfs.cpio.gz /srv/tftp/ +cp ./binaries/xen /srv/tftp/ +cp ./binaries/fvp-base-gicv3-psci-1t.dtb /srv/tftp/ + +# Start FVP +TERM0_CFG="-C bp.terminal_0.mode=telnet -C bp.terminal_0.start_telnet=0" +TERM1_CFG="-C bp.terminal_1.mode=telnet -C bp.terminal_1.start_telnet=0" +TERM2_CFG="-C bp.terminal_2.mode=telnet -C bp.terminal_2.start_telnet=0" +TERM3_CFG="-C bp.terminal_3.mode=telnet -C bp.terminal_3.start_telnet=0" + +VIRTIO_USER_NETWORK_CFG="-C bp.virtio_net.enabled=1 \ +-C bp.virtio_net.hostbridge.userNetworking=1 \ +-C bp.virtio_net.hostbridge.userNetPorts=8022=22 \ +-C bp.virtio_net.transport=legacy" + +./automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp \ + "/FVP/FVP_Base_RevC-2xAEMvA/Base_RevC_AEMvA_pkg/models/Linux64_armv8l_GCC-9.3/FVP_Base_RevC-2xAEMvA \ + -C bp.vis.disable_visualisation=1 \ + -C bp.ve_sysregs.exit_on_shutdown=1 \ + -C bp.secure_memory=0 \ + -C cache_state_modelled=0 \ + -C cluster0.has_arm_v8-4=1 \ + -C cluster1.has_arm_v8-4=1 \ + ${TERM0_CFG} ${TERM1_CFG} ${TERM2_CFG} ${TERM3_CFG} \ + ${VIRTIO_USER_NETWORK_CFG} \ + -C bp.secureflashloader.fname=$(pwd)/binaries/bl1.bin \ + -C bp.flashloader0.fname=$(pwd)/binaries/fip.bin" |& \ + tee smoke.serial + +exit 0 From patchwork Thu Dec 7 13:53:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13483327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C4046C10DC3 for ; Thu, 7 Dec 2023 13:54:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.649878.1014975 (Exim 4.92) (envelope-from ) id 1rBEp5-00044i-QT; Thu, 07 Dec 2023 13:53:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 649878.1014975; Thu, 07 Dec 2023 13:53:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEp5-00043y-N8; Thu, 07 Dec 2023 13:53:51 +0000 Received: by outflank-mailman (input) for mailman id 649878; Thu, 07 Dec 2023 13:53:50 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rBEp4-0002bv-AD for xen-devel@lists.xenproject.org; Thu, 07 Dec 2023 13:53:50 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 0bbdcb4f-9508-11ee-9b0f-b553b5be7939; Thu, 07 Dec 2023 14:53:48 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0187412FC; Thu, 7 Dec 2023 05:54:34 -0800 (PST) Received: from a015966.shanghai.arm.com (a015966.shanghai.arm.com [10.169.190.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5A40A3F6C4; Thu, 7 Dec 2023 05:53:45 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0bbdcb4f-9508-11ee-9b0f-b553b5be7939 From: Henry Wang To: xen-devel@lists.xenproject.org Cc: Henry Wang , Doug Goldstein , Stefano Stabellini , Michal Orzel , Julien Grall , Bertrand Marquis , Wei Chen Subject: [PATCH 5/5] automation: Add the arm64 FVP build and Dom0 smoke test jobs Date: Thu, 7 Dec 2023 21:53:18 +0800 Message-Id: <20231207135318.1912846-6-Henry.Wang@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231207135318.1912846-1-Henry.Wang@arm.com> References: <20231207135318.1912846-1-Henry.Wang@arm.com> MIME-Version: 1.0 Add a job in the build stage to export the TF-A, U-Boot and the device tree for the FVP platform from the test artifact container. Add a FVP smoke test job in the test stage to do the same test as the `qemu-smoke-dom0-arm64-gcc` job. Signed-off-by: Henry Wang Reviewed-by: Stefano Stabellini --- Although it does not affect the functionality, I am still quite confused about why the logs displayed by GitLab UI, for example [1], is much less than the actual output (saved in the artifacts, see [2]). Had a discussion with Michal on Matrix and we agree that the log in gitlab UI is usually capped. [1] https://gitlab.com/xen-project/people/henryw/xen/-/jobs/5700569676 [2] https://gitlab.com/xen-project/people/henryw/xen/-/jobs/5700569676/artifacts/file/smoke.serial --- automation/gitlab-ci/build.yaml | 17 +++++++++++++++++ automation/gitlab-ci/test.yaml | 22 ++++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml index 32af30cced..89d2f01302 100644 --- a/automation/gitlab-ci/build.yaml +++ b/automation/gitlab-ci/build.yaml @@ -344,6 +344,23 @@ kernel-6.1.19-export: tags: - x86_64 +armfvp-uboot-tfa-2023.10-2.9.0-arm64-export: + extends: .test-jobs-artifact-common + image: registry.gitlab.com/xen-project/xen/tests-artifacts/armfvp-uboot-tfa:2023.10-2.9.0-arm64v8 + script: + - | + mkdir binaries && \ + cp /bl1.bin binaries/bl1.bin && \ + cp /fip.bin binaries/fip.bin && \ + cp /fvp-base-gicv3-psci-1t.dtb binaries/fvp-base-gicv3-psci-1t.dtb + artifacts: + paths: + - binaries/bl1.bin + - binaries/fip.bin + - binaries/fvp-base-gicv3-psci-1t.dtb + tags: + - arm64 + # Jobs below this line # Build jobs needed for tests diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index 6aabdb9d15..47e00d0a0b 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -96,6 +96,19 @@ tags: - xilinx +.fvp-arm64: + extends: .test-jobs-common + variables: + CONTAINER: debian:bookworm-arm64v8-fvp + LOGFILE: fvp-smoke-arm64.log + artifacts: + paths: + - smoke.serial + - '*.log' + when: always + tags: + - arm64 + .adl-x86-64: extends: .test-jobs-common variables: @@ -459,3 +472,12 @@ qemu-smoke-ppc64le-powernv9-gcc: needs: - qemu-system-ppc64-8.1.0-ppc64-export - debian-bullseye-gcc-ppc64le-debug + +fvp-smoke-dom0-arm64-gcc-debug: + extends: .fvp-arm64 + script: + - ./automation/scripts/fvp-base-smoke-dom0-arm64.sh 2>&1 | tee ${LOGFILE} + needs: + - *arm64-test-needs + - armfvp-uboot-tfa-2023.10-2.9.0-arm64-export + - alpine-3.18-gcc-debug-arm64