diff mbox series

[isar-cip-core,3/3] .gitlab-ci.yml: Add test targets specific for CIP core functionalities

Message ID 20240603070920.2942859-4-Sai.Sathujoda@toshiba-tsip.com (mailing list archive)
State Superseded
Headers show
Series Trigger CIP Core testing LAVA jobs | expand

Commit Message

Sai.Sathujoda@toshiba-tsip.com June 3, 2024, 7:09 a.m. UTC
From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>

Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
---
 .gitlab-ci.yml | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

Comments

Jan Kiszka June 3, 2024, 9:12 a.m. UTC | #1
On 03.06.24 09:09, Sai.Sathujoda@toshiba-tsip.com wrote:
> From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> 
> Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> ---
>  .gitlab-ci.yml | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index aab55be..0a22a3f 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -358,6 +358,39 @@ build:qemu-riscv64:
>      deploy: disable
>    allow_failure: true
>  
> +.test-cip-core:
> +  stage: test
> +  needs: ["build:qemu-amd64-base","build:qemu-arm64-base","build:qemu-arm-base"]
> +  tags:
> +    - large

Conceptually, we should not need large runners for these jobs. Do we
need that for other reasons.

> +  allow_failure: true

Why?

> +  artifacts:
> +    name: "$CI_JOB_NAME"
> +    when: always
> +    expire_in: 1 day
> +    paths:
> +      - results
> +    reports:
> +      junit: results/results*.xml
> +
> +test-qemu-secure-boot:
> +  extends:
> +    - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh secure-boot ${CI_COMMIT_SHA} ${release}
> +
> +test-qemu-swupdate:
> +  extends:
> +    - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh swupdate ${CI_COMMIT_SHA} ${release}
> +
> +test-qemu-IEC:
> +  extends:
> +   - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh IEC ${CI_COMMIT_SHA} ${release}
> +
>  cve-checks:
>    stage: cve-check
>    needs: []

Jan
Sai.Sathujoda@toshiba-tsip.com June 3, 2024, 9:29 a.m. UTC | #2
Hi Jan,

I added "allow_failure: true" because I felt every change which triggers a pipeline might now be related to the functionalities we are testing with these LAVA jobs. So, I thought it is not good to fail the entire pipeline just because one of these test jobs fail. So, I think it is better to give it as a warning instead of letting the pipeline fail.

Thanks and regards,
Sai Ashrith Sathujoda

-----Original Message-----
From: Jan Kiszka <jan.kiszka@siemens.com> 
Sent: Monday, June 3, 2024 2:42 PM
To: ashrith sai(TSIP) <Sai.Sathujoda@toshiba-tsip.com>; cip-dev@lists.cip-project.org
Cc: dinesh kumar(TSIP TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 DME ○DIG□MPS○MP4) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [isar-cip-core 3/3] .gitlab-ci.yml: Add test targets specific for CIP core functionalities

On 03.06.24 09:09, Sai.Sathujoda@toshiba-tsip.com wrote:
> From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> 
> Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> ---
>  .gitlab-ci.yml | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index aab55be..0a22a3f 
> 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -358,6 +358,39 @@ build:qemu-riscv64:
>      deploy: disable
>    allow_failure: true
>  
> +.test-cip-core:
> +  stage: test
> +  needs: 
> +["build:qemu-amd64-base","build:qemu-arm64-base","build:qemu-arm-base
> +"]
> +  tags:
> +    - large

Conceptually, we should not need large runners for these jobs. Do we need that for other reasons.

> +  allow_failure: true

Why?

> +  artifacts:
> +    name: "$CI_JOB_NAME"
> +    when: always
> +    expire_in: 1 day
> +    paths:
> +      - results
> +    reports:
> +      junit: results/results*.xml
> +
> +test-qemu-secure-boot:
> +  extends:
> +    - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh secure-boot ${CI_COMMIT_SHA} ${release}
> +
> +test-qemu-swupdate:
> +  extends:
> +    - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh swupdate ${CI_COMMIT_SHA} ${release}
> +
> +test-qemu-IEC:
> +  extends:
> +   - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh IEC ${CI_COMMIT_SHA} ${release}
> +
>  cve-checks:
>    stage: cve-check
>    needs: []

Jan

--
Siemens AG, Technology
Linux Expert Center
Sai.Sathujoda@toshiba-tsip.com June 3, 2024, 9:44 a.m. UTC | #3
I apologize for the typo "might now" -> "might not".

Regards,
Sai Ashrith Sathujoda

-----Original Message-----
From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On Behalf Of sai.sathujoda@toshiba-tsip.com
Sent: Monday, June 3, 2024 2:59 PM
To: jan.kiszka@siemens.com; cip-dev@lists.cip-project.org
Cc: dinesh kumar(TSIP TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 DME ○DIG□MPS○MP4) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [cip-dev] [isar-cip-core 3/3] .gitlab-ci.yml: Add test targets specific for CIP core functionalities

Hi Jan,

I added "allow_failure: true" because I felt every change which triggers a pipeline might now be related to the functionalities we are testing with these LAVA jobs. So, I thought it is not good to fail the entire pipeline just because one of these test jobs fail. So, I think it is better to give it as a warning instead of letting the pipeline fail.

Thanks and regards,
Sai Ashrith Sathujoda

-----Original Message-----
From: Jan Kiszka <jan.kiszka@siemens.com> 
Sent: Monday, June 3, 2024 2:42 PM
To: ashrith sai(TSIP) <Sai.Sathujoda@toshiba-tsip.com>; cip-dev@lists.cip-project.org
Cc: dinesh kumar(TSIP TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 DME ○DIG□MPS○MP4) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [isar-cip-core 3/3] .gitlab-ci.yml: Add test targets specific for CIP core functionalities

On 03.06.24 09:09, Sai.Sathujoda@toshiba-tsip.com wrote:
> From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> 
> Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> ---
>  .gitlab-ci.yml | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index aab55be..0a22a3f 
> 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -358,6 +358,39 @@ build:qemu-riscv64:
>      deploy: disable
>    allow_failure: true
>  
> +.test-cip-core:
> +  stage: test
> +  needs: 
> +["build:qemu-amd64-base","build:qemu-arm64-base","build:qemu-arm-base
> +"]
> +  tags:
> +    - large

Conceptually, we should not need large runners for these jobs. Do we need that for other reasons.

> +  allow_failure: true

Why?

> +  artifacts:
> +    name: "$CI_JOB_NAME"
> +    when: always
> +    expire_in: 1 day
> +    paths:
> +      - results
> +    reports:
> +      junit: results/results*.xml
> +
> +test-qemu-secure-boot:
> +  extends:
> +    - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh secure-boot ${CI_COMMIT_SHA} ${release}
> +
> +test-qemu-swupdate:
> +  extends:
> +    - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh swupdate ${CI_COMMIT_SHA} ${release}
> +
> +test-qemu-IEC:
> +  extends:
> +   - .test-cip-core
> +  script:
> +    - scripts/submit_lava.sh IEC ${CI_COMMIT_SHA} ${release}
> +
>  cve-checks:
>    stage: cve-check
>    needs: []

Jan

--
Siemens AG, Technology
Linux Expert Center
Jan Kiszka June 3, 2024, 11:59 a.m. UTC | #4
Ok, thanks for the clarification. Then we can also merge this after the
release.

Still, if you see any issues in isar-cip-core's swupdate while evolving
this pipeline, the upcoming release candidate phase would already
benefit from hints!

Thanks,
Jan

On 03.06.24 11:44, Sai.Sathujoda@toshiba-tsip.com wrote:
> I apologize for the typo "might now" -> "might not".
> 
> Regards,
> Sai Ashrith Sathujoda
> 
> -----Original Message-----
> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On Behalf Of sai.sathujoda@toshiba-tsip.com
> Sent: Monday, June 3, 2024 2:59 PM
> To: jan.kiszka@siemens.com; cip-dev@lists.cip-project.org
> Cc: dinesh kumar(TSIP TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 DME ○DIG□MPS○MP4) <kazuhiro3.hayashi@toshiba.co.jp>
> Subject: Re: [cip-dev] [isar-cip-core 3/3] .gitlab-ci.yml: Add test targets specific for CIP core functionalities
> 
> Hi Jan,
> 
> I added "allow_failure: true" because I felt every change which triggers a pipeline might now be related to the functionalities we are testing with these LAVA jobs. So, I thought it is not good to fail the entire pipeline just because one of these test jobs fail. So, I think it is better to give it as a warning instead of letting the pipeline fail.
> 
> Thanks and regards,
> Sai Ashrith Sathujoda
> 
> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com> 
> Sent: Monday, June 3, 2024 2:42 PM
> To: ashrith sai(TSIP) <Sai.Sathujoda@toshiba-tsip.com>; cip-dev@lists.cip-project.org
> Cc: dinesh kumar(TSIP TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 DME ○DIG□MPS○MP4) <kazuhiro3.hayashi@toshiba.co.jp>
> Subject: Re: [isar-cip-core 3/3] .gitlab-ci.yml: Add test targets specific for CIP core functionalities
> 
> On 03.06.24 09:09, Sai.Sathujoda@toshiba-tsip.com wrote:
>> From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
>>
>> Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
>> ---
>>  .gitlab-ci.yml | 33 +++++++++++++++++++++++++++++++++
>>  1 file changed, 33 insertions(+)
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index aab55be..0a22a3f 
>> 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -358,6 +358,39 @@ build:qemu-riscv64:
>>      deploy: disable
>>    allow_failure: true
>>  
>> +.test-cip-core:
>> +  stage: test
>> +  needs: 
>> +["build:qemu-amd64-base","build:qemu-arm64-base","build:qemu-arm-base
>> +"]
>> +  tags:
>> +    - large
> 
> Conceptually, we should not need large runners for these jobs. Do we need that for other reasons.
> 
>> +  allow_failure: true
> 
> Why?
> 
>> +  artifacts:
>> +    name: "$CI_JOB_NAME"
>> +    when: always
>> +    expire_in: 1 day
>> +    paths:
>> +      - results
>> +    reports:
>> +      junit: results/results*.xml
>> +
>> +test-qemu-secure-boot:
>> +  extends:
>> +    - .test-cip-core
>> +  script:
>> +    - scripts/submit_lava.sh secure-boot ${CI_COMMIT_SHA} ${release}
>> +
>> +test-qemu-swupdate:
>> +  extends:
>> +    - .test-cip-core
>> +  script:
>> +    - scripts/submit_lava.sh swupdate ${CI_COMMIT_SHA} ${release}
>> +
>> +test-qemu-IEC:
>> +  extends:
>> +   - .test-cip-core
>> +  script:
>> +    - scripts/submit_lava.sh IEC ${CI_COMMIT_SHA} ${release}
>> +
>>  cve-checks:
>>    stage: cve-check
>>    needs: []
> 
> Jan
> 
> --
> Siemens AG, Technology
> Linux Expert Center
Chris Paterson June 4, 2024, 8:25 a.m. UTC | #5
Hello,

> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On
> Behalf Of Jan Kiszka via lists.cip-project.org
> Sent: Monday, June 3, 2024 10:12 AM
> 
> On 03.06.24 09:09, Sai.Sathujoda@toshiba-tsip.com wrote:
> > From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> >
> > Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> > ---
> >  .gitlab-ci.yml | 33 +++++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> >
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index aab55be..0a22a3f 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -358,6 +358,39 @@ build:qemu-riscv64:
> >      deploy: disable
> >    allow_failure: true
> >
> > +.test-cip-core:
> > +  stage: test
> > +  needs: ["build:qemu-amd64-base","build:qemu-arm64-base","build:qemu-
> arm-base"]
> > +  tags:
> > +    - large
> 
> Conceptually, we should not need large runners for these jobs. Do we
> need that for other reasons.

Agreed.
Please use the "small" tag for jobs like this where a fast CPU isn't required.

Kind regards, Chris

> 
> > +  allow_failure: true
> 
> Why?
> 
> > +  artifacts:
> > +    name: "$CI_JOB_NAME"
> > +    when: always
> > +    expire_in: 1 day
> > +    paths:
> > +      - results
> > +    reports:
> > +      junit: results/results*.xml
> > +
> > +test-qemu-secure-boot:
> > +  extends:
> > +    - .test-cip-core
> > +  script:
> > +    - scripts/submit_lava.sh secure-boot ${CI_COMMIT_SHA} ${release}
> > +
> > +test-qemu-swupdate:
> > +  extends:
> > +    - .test-cip-core
> > +  script:
> > +    - scripts/submit_lava.sh swupdate ${CI_COMMIT_SHA} ${release}
> > +
> > +test-qemu-IEC:
> > +  extends:
> > +   - .test-cip-core
> > +  script:
> > +    - scripts/submit_lava.sh IEC ${CI_COMMIT_SHA} ${release}
> > +
> >  cve-checks:
> >    stage: cve-check
> >    needs: []
> 
> Jan
> 
> --
> Siemens AG, Technology
> Linux Expert Center
diff mbox series

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index aab55be..0a22a3f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -358,6 +358,39 @@  build:qemu-riscv64:
     deploy: disable
   allow_failure: true
 
+.test-cip-core:
+  stage: test
+  needs: ["build:qemu-amd64-base","build:qemu-arm64-base","build:qemu-arm-base"]
+  tags:
+    - large
+  allow_failure: true
+  artifacts:
+    name: "$CI_JOB_NAME"
+    when: always
+    expire_in: 1 day
+    paths:
+      - results
+    reports:
+      junit: results/results*.xml
+
+test-qemu-secure-boot:
+  extends:
+    - .test-cip-core
+  script:
+    - scripts/submit_lava.sh secure-boot ${CI_COMMIT_SHA} ${release}
+
+test-qemu-swupdate:
+  extends:
+    - .test-cip-core
+  script:
+    - scripts/submit_lava.sh swupdate ${CI_COMMIT_SHA} ${release}
+
+test-qemu-IEC:
+  extends:
+   - .test-cip-core
+  script:
+    - scripts/submit_lava.sh IEC ${CI_COMMIT_SHA} ${release}
+
 cve-checks:
   stage: cve-check
   needs: []