diff mbox series

[isar-cip-core,2/2] .gitlab-ci.yml: Add ci for reproducible build checks

Message ID 20230221154138.15146-3-venkata.pyla@toshiba-tsip.com (mailing list archive)
State Superseded
Headers show
Series CI for reproducible build check | expand

Commit Message

Venkata Pyla Feb. 21, 2023, 3:41 p.m. UTC 
From: venkata pyla <venkata.pyla@toshiba-tsip.com>

Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
 .gitlab-ci.yml | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

Comments

Jan Kiszka Feb. 21, 2023, 8:42 p.m. UTC | #1 
On 21.02.23 16:41, venkata.pyla@toshiba-tsip.com wrote:
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 

Quite a bit of your explanation from the cover letter should actually go
here instead. This here will be archived in git, the cover only in the
email archive.

> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
>  .gitlab-ci.yml | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 93 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index f7c8c88..3458a2f 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -13,6 +13,7 @@ variables:
>  
>  stages:
>    - build
> +  - test
>  
>  default:
>    before_script:
> @@ -26,6 +27,8 @@ default:
>  
>  .build_base:
>    stage: build
> +  except:
> +    - schedules
>    tags:
>      - large
>    variables:
> @@ -42,6 +45,57 @@ default:
>      - if [ "${deploy}" = "enable" ]; then scripts/deploy-cip-core.sh ${release} ${target} ${extension} ${dtb} ${CI_COMMIT_REF_SLUG}; fi
>      - if [ "${deploy_kernelci}" = "enable" ]; then scripts/deploy-kernelci.py ${release} ${target} ${extension} ${dtb}; fi
>  
> +.repro-build:
> +  variables:
> +    use_rt: disable
> +    wic_targz: disable
> +    targz: enable
> +    deploy: disable
> +    deploy_kernelci: disable
> +    artifact_suffix: ".tar.gz"
> +    base_yaml: "kas-cip.yml:kas/board/${target}.yml"
> +  stage: build
> +  tags:
> +    - large
> +  only:
> +    - schedules
> +  script:
> +    - if [ -f kas/opt/reproducible.yml ]; then base_yaml="${base_yaml}:kas/opt/reproducible.yml"; fi
> +    # Build 1st time
> +    - !reference [.build_base, script]
> +    - mv build/tmp/deploy/images/${target}/cip-core-image-cip-core-${release}-${target}${artifact_suffix} image1
> +    # clean
> +    - sudo rm -rf build/tmp
> +    - sudo rm -rf build/sstate-cache
> +    # Build 2nd time
> +    - !reference [.build_base, script]
> +    - mv build/tmp/deploy/images/${target}/cip-core-image-cip-core-${release}-${target}${artifact_suffix} image2
> +  artifacts:
> +    expire_in: 1 day
> +    paths:
> +      - image1
> +      - image2
> +
> +.repro-test:
> +  image:
> +    name: debian:bullseye
> +  tags:
> +    - large
> +  stage: test
> +  only:
> +    - schedules
> +  variables:
> +    GIT_STRATEGY: none
> +  before_script:
> +    - apt update && DEBIAN_FRONTEND=noninteractive apt install -y diffoscope
> +  script:
> +    - diffoscope --text diffoscope_output.txt image1 image2
> +  artifacts:
> +    when: always
> +    expire_in: 1 day
> +    paths:
> +      - diffoscope_output.txt
> +
>  # base image
>  build:simatic-ipc227e-base:
>    extends:
> @@ -281,3 +335,42 @@ build:qemu-amd64-secure-boot-buster:
>      use_rt: disable
>      wic_targz: disable
>      deploy: disable
> +
> +# repro build
> +build:qemu-amd64-base-repro-build:
> +  extends:
> +    - .repro-build
> +  variables:
> +    target: qemu-amd64
> +
> +build:qemu-arm64-base-repro-build:
> +  extends:
> +    - .repro-build
> +  variables:
> +    target: qemu-arm64
> +
> +build:qemu-arm-base-repro-build:
> +  extends:
> +    - .repro-build
> +  variables:
> +    target: qemu-arm
> +
> +# repro build test
> +test:qemu-amd64-base-repro-test:
> +  extends:
> +    - .repro-test
> +  dependencies:
> +    - build:qemu-amd64-base-repro-build
> +
> +test:qemu-arm64-base-repro-test:
> +  extends:
> +    - .repro-test
> +  dependencies:
> +    - build:qemu-arm64-base-repro-build
> +
> +test:qemu-arm-base-repro-test:
> +  extends:
> +    - .repro-test
> +  dependencies:
> +    - build:qemu-arm-base-repro-build
> +

Our gitlab-ci file becomes larger and larger. How about splitting it
into different topics and include those split-up files?

Jan
Venkata Pyla Feb. 22, 2023, 4 a.m. UTC | #2 
>-----Original Message-----
>From: Jan Kiszka <jan.kiszka@siemens.com>
>Sent: 22 February 2023 02:12
>To: pyla venkata(ＴＳＩＰ TMIEC ODG Porting) <Venkata.Pyla@toshiba-
>tsip.com>; cip-dev@lists.cip-project.org
>Cc: dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-
>tsip.com>; hayashi kazuhiro(林 和宏 □ＳＷＣ◯ＡＣＴ)
><kazuhiro3.hayashi@toshiba.co.jp>
>Subject: Re: [isar-cip-core 2/2] .gitlab-ci.yml: Add ci for reproducible build
>checks
>
>On 21.02.23 16:41, venkata.pyla@toshiba-tsip.com wrote:
>> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>>
>
>Quite a bit of your explanation from the cover letter should actually go here
>instead. This here will be archived in git, the cover only in the email archive.
>
>> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
>> ---
>>  .gitlab-ci.yml | 93
>> ++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 93 insertions(+)
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f7c8c88..3458a2f
>> 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -13,6 +13,7 @@ variables:
>>
>>  stages:
>>    - build
>> +  - test
>>
>>  default:
>>    before_script:
>> @@ -26,6 +27,8 @@ default:
>>
>>  .build_base:
>>    stage: build
>> +  except:
>> +    - schedules
>>    tags:
>>      - large
>>    variables:
>> @@ -42,6 +45,57 @@ default:
>>      - if [ "${deploy}" = "enable" ]; then scripts/deploy-cip-core.sh ${release}
>${target} ${extension} ${dtb} ${CI_COMMIT_REF_SLUG}; fi
>>      - if [ "${deploy_kernelci}" = "enable" ]; then
>> scripts/deploy-kernelci.py ${release} ${target} ${extension} ${dtb};
>> fi
>>
>> +.repro-build:
>> +  variables:
>> +    use_rt: disable
>> +    wic_targz: disable
>> +    targz: enable
>> +    deploy: disable
>> +    deploy_kernelci: disable
>> +    artifact_suffix: ".tar.gz"
>> +    base_yaml: "kas-cip.yml:kas/board/${target}.yml"
>> +  stage: build
>> +  tags:
>> +    - large
>> +  only:
>> +    - schedules
>> +  script:
>> +    - if [ -f kas/opt/reproducible.yml ]; then
>base_yaml="${base_yaml}:kas/opt/reproducible.yml"; fi
>> +    # Build 1st time
>> +    - !reference [.build_base, script]
>> +    - mv build/tmp/deploy/images/${target}/cip-core-image-cip-core-
>${release}-${target}${artifact_suffix} image1
>> +    # clean
>> +    - sudo rm -rf build/tmp
>> +    - sudo rm -rf build/sstate-cache
>> +    # Build 2nd time
>> +    - !reference [.build_base, script]
>> +    - mv
>> +build/tmp/deploy/images/${target}/cip-core-image-cip-core-${release}-
>> +${target}${artifact_suffix} image2
>> +  artifacts:
>> +    expire_in: 1 day
>> +    paths:
>> +      - image1
>> +      - image2
>> +
>> +.repro-test:
>> +  image:
>> +    name: debian:bullseye
>> +  tags:
>> +    - large
>> +  stage: test
>> +  only:
>> +    - schedules
>> +  variables:
>> +    GIT_STRATEGY: none
>> +  before_script:
>> +    - apt update && DEBIAN_FRONTEND=noninteractive apt install -y
>> +diffoscope
>> +  script:
>> +    - diffoscope --text diffoscope_output.txt image1 image2
>> +  artifacts:
>> +    when: always
>> +    expire_in: 1 day
>> +    paths:
>> +      - diffoscope_output.txt
>> +
>>  # base image
>>  build:simatic-ipc227e-base:
>>    extends:
>> @@ -281,3 +335,42 @@ build:qemu-amd64-secure-boot-buster:
>>      use_rt: disable
>>      wic_targz: disable
>>      deploy: disable
>> +
>> +# repro build
>> +build:qemu-amd64-base-repro-build:
>> +  extends:
>> +    - .repro-build
>> +  variables:
>> +    target: qemu-amd64
>> +
>> +build:qemu-arm64-base-repro-build:
>> +  extends:
>> +    - .repro-build
>> +  variables:
>> +    target: qemu-arm64
>> +
>> +build:qemu-arm-base-repro-build:
>> +  extends:
>> +    - .repro-build
>> +  variables:
>> +    target: qemu-arm
>> +
>> +# repro build test
>> +test:qemu-amd64-base-repro-test:
>> +  extends:
>> +    - .repro-test
>> +  dependencies:
>> +    - build:qemu-amd64-base-repro-build
>> +
>> +test:qemu-arm64-base-repro-test:
>> +  extends:
>> +    - .repro-test
>> +  dependencies:
>> +    - build:qemu-arm64-base-repro-build
>> +
>> +test:qemu-arm-base-repro-test:
>> +  extends:
>> +    - .repro-test
>> +  dependencies:
>> +    - build:qemu-arm-base-repro-build
>> +
>
>Our gitlab-ci file becomes larger and larger. How about splitting it into
>different topics and include those split-up files?

Thanks for pointing out, I will do split-up and send the patch again.

>
>Jan
>
>--
>Siemens AG, Technology
>Competence Center Embedded Linux
diff mbox series

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f7c8c88..3458a2f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -13,6 +13,7 @@  variables:
 
 stages:
   - build
+  - test
 
 default:
   before_script:
@@ -26,6 +27,8 @@  default:
 
 .build_base:
   stage: build
+  except:
+    - schedules
   tags:
     - large
   variables:
@@ -42,6 +45,57 @@  default:
     - if [ "${deploy}" = "enable" ]; then scripts/deploy-cip-core.sh ${release} ${target} ${extension} ${dtb} ${CI_COMMIT_REF_SLUG}; fi
     - if [ "${deploy_kernelci}" = "enable" ]; then scripts/deploy-kernelci.py ${release} ${target} ${extension} ${dtb}; fi
 
+.repro-build:
+  variables:
+    use_rt: disable
+    wic_targz: disable
+    targz: enable
+    deploy: disable
+    deploy_kernelci: disable
+    artifact_suffix: ".tar.gz"
+    base_yaml: "kas-cip.yml:kas/board/${target}.yml"
+  stage: build
+  tags:
+    - large
+  only:
+    - schedules
+  script:
+    - if [ -f kas/opt/reproducible.yml ]; then base_yaml="${base_yaml}:kas/opt/reproducible.yml"; fi
+    # Build 1st time
+    - !reference [.build_base, script]
+    - mv build/tmp/deploy/images/${target}/cip-core-image-cip-core-${release}-${target}${artifact_suffix} image1
+    # clean
+    - sudo rm -rf build/tmp
+    - sudo rm -rf build/sstate-cache
+    # Build 2nd time
+    - !reference [.build_base, script]
+    - mv build/tmp/deploy/images/${target}/cip-core-image-cip-core-${release}-${target}${artifact_suffix} image2
+  artifacts:
+    expire_in: 1 day
+    paths:
+      - image1
+      - image2
+
+.repro-test:
+  image:
+    name: debian:bullseye
+  tags:
+    - large
+  stage: test
+  only:
+    - schedules
+  variables:
+    GIT_STRATEGY: none
+  before_script:
+    - apt update && DEBIAN_FRONTEND=noninteractive apt install -y diffoscope
+  script:
+    - diffoscope --text diffoscope_output.txt image1 image2
+  artifacts:
+    when: always
+    expire_in: 1 day
+    paths:
+      - diffoscope_output.txt
+
 # base image
 build:simatic-ipc227e-base:
   extends:
@@ -281,3 +335,42 @@  build:qemu-amd64-secure-boot-buster:
     use_rt: disable
     wic_targz: disable
     deploy: disable
+
+# repro build
+build:qemu-amd64-base-repro-build:
+  extends:
+    - .repro-build
+  variables:
+    target: qemu-amd64
+
+build:qemu-arm64-base-repro-build:
+  extends:
+    - .repro-build
+  variables:
+    target: qemu-arm64
+
+build:qemu-arm-base-repro-build:
+  extends:
+    - .repro-build
+  variables:
+    target: qemu-arm
+
+# repro build test
+test:qemu-amd64-base-repro-test:
+  extends:
+    - .repro-test
+  dependencies:
+    - build:qemu-amd64-base-repro-build
+
+test:qemu-arm64-base-repro-test:
+  extends:
+    - .repro-test
+  dependencies:
+    - build:qemu-arm64-base-repro-build
+
+test:qemu-arm-base-repro-test:
+  extends:
+    - .repro-test
+  dependencies:
+    - build:qemu-arm-base-repro-build
+