| Message ID | 20250123135342.1468787-1-vignesh.raman@collabora.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing | expand |
On Thu, 23 Jan 2025 at 05:56, Vignesh Raman <vignesh.raman@collabora.com> wrote: > > Documentation/ci/gitlab-ci/images/drm-vkms.png | Bin 0 -> 73810 bytes > .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 20000 bytes > .../ci/gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes > .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes > .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 62464 bytes > .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes This seems excessive. We have a few images in the tree, but they tend to be either the penguin logo or various source SVGs. Not a set of random screenshots or whatever. Linus
Le jeudi 23 janvier 2025 à 07:46 -0800, Linus Torvalds a écrit : > On Thu, 23 Jan 2025 at 05:56, Vignesh Raman <vignesh.raman@collabora.com> wrote: > > > > Documentation/ci/gitlab-ci/images/drm-vkms.png | Bin 0 -> 73810 bytes > > .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 20000 bytes > > .../ci/gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes > > .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes > > .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 62464 bytes > > .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes > > > This seems excessive. > > We have a few images in the tree, but they tend to be either the > penguin logo or various source SVGs. > > Not a set of random screenshots or whatever. I can only agree. Nicolas > > Linus
On Thu Jan 23, 2025 at 3:53 PM EET, Vignesh Raman wrote: > We are working towards creating a generic, upstream GitLab-CI pipeline > (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > is designed with a distributed infrastructure model, making it possible > to run on any gitLab instance. We plan to leverage KernelCI [2] as the > backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > integration. Why can't you keep the next version of your great pipeline outside the kernel tree? If there is a legit motivation for doing that, why it needs to be bound to Gitlab? Why can't you make script callable from any CI? BR, Jarkko
On Thu Jan 23, 2025 at 11:30 PM EET, Jarkko Sakkinen wrote: > On Thu Jan 23, 2025 at 3:53 PM EET, Vignesh Raman wrote: > > We are working towards creating a generic, upstream GitLab-CI pipeline > > (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > > is designed with a distributed infrastructure model, making it possible > > to run on any gitLab instance. We plan to leverage KernelCI [2] as the > > backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > > integration. > > Why can't you keep the next version of your great pipeline outside the > kernel tree? > > If there is a legit motivation for doing that, why it needs to be bound > to Gitlab? Why can't you make script callable from any CI? To add, most of the distributions have their CI pipelines but they don't proactively push them over here. BR, Jarkko
On Thu, 2025-01-23 at 19:23 +0530, Vignesh Raman wrote: > This patch series introduces "kci-gitlab," a GitLab CI pipeline > specifically designed for kernel testing. It provides kernel > developers with an integrated, efficient, and flexible testing > framework using GitLab's CI/CD capabilities. This patch includes > a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with > additional YAML and script files, to define a basic test pipeline > triggered by code pushes to a GitLab-CI instance. > The initial version implements: > > - Static checks: Includes checkpatch and smatch for code validation. > - Build tests: Covers various architectures and configurations. > - Boot tests: Utilizes virtme for basic boot testing. > > Additionally, it introduces a flexible "scenarios" mechanism to > support subsystem-specific extensions. > > This series also introduces a drm scenario that adds a job to run IGT > tests for vkms. This scenario includes helper scripts to build deqp-runner > and IGT, leveraging approaches from the drm-ci/mesa-ci project. > > We are working towards creating a generic, upstream GitLab-CI pipeline > (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > is designed with a distributed infrastructure model, making it possible > to run on any gitLab instance. We plan to leverage KernelCI [2] as the > backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > integration. > > For an example of a fully executed pipeline with drm scenario set, > including documentation generation, > see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262 > > Please refer to the documentation included in the patch, or check the > rendered version, here: > https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html > > Differences from v1 to v2: > - moved to tools/ci as suggested by Linus on the previous version > - add arm64 containers for native compilation > - added boot tests using virtme: this is the base structure for boot tests, > next steps would be adding other tests such as kselftests/kunit tests > - added DRM scenario testing on vkms: this should replace current vkms test > in drm-ci. This work shows how a test scenario can be used by different > subsystems to add their tests. > - update documentation > > For more details on the motivation behind this work, please refer to the > cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/20240228225527.1052240-1-helen.koike@collabora.com/ > > [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/ > [2] https://kernelci.org/ > [3] https://docs.kernelci.org/kcidb/ > > Helen Koike (3): > kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing > kci-gitlab: Add documentation > kci-gitlab: docs: Add images > > Vignesh Raman (2): > MAINTAINERS: Add an entry for ci automated testing > kci-gitlab: Add drm scenario Hi Vignesh Raman, I am very happy to see this project going forward :) It's been a few years since I first thought on finding a good way of helping kernel developers testing their patches, while making use of the free runner minutes Gitlab offers. It can greatly simplify the testing for people who are new to kernel development, or students trying to understand it better. And this patchset allows that to happen :) Actually, I spoke to Helen last year, and to enable it to run on the free Gitlab-CI runners, there is a small extra patch which is needed: https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ Could you please apply it on top of your tree? Some stuff changed places, but I can send a v2 with that fix if you want. While I have yet to review this v2 patchset, I applied it on my repo for testing, and cherry-picked the patch on above link, triggering a Pipeline: https://gitlab.com/linux-kernel/linux/-/pipelines/1638955600 It seems to be working fine, please check it, as you may be more used to the results. Thanks! Leo
On Fri, Jan 24, 2025 at 02:11:26AM -0300, Leonardo Brás wrote: > On Thu, 2025-01-23 at 19:23 +0530, Vignesh Raman wrote: > > This patch series introduces "kci-gitlab," a GitLab CI pipeline > > specifically designed for kernel testing. It provides kernel > > developers with an integrated, efficient, and flexible testing > > framework using GitLab's CI/CD capabilities. This patch includes > > a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with > > additional YAML and script files, to define a basic test pipeline > > triggered by code pushes to a GitLab-CI instance. > > The initial version implements: > > > > - Static checks: Includes checkpatch and smatch for code validation. > > - Build tests: Covers various architectures and configurations. > > - Boot tests: Utilizes virtme for basic boot testing. > > > > Additionally, it introduces a flexible "scenarios" mechanism to > > support subsystem-specific extensions. > > > > This series also introduces a drm scenario that adds a job to run IGT > > tests for vkms. This scenario includes helper scripts to build deqp-runner > > and IGT, leveraging approaches from the drm-ci/mesa-ci project. > > > > We are working towards creating a generic, upstream GitLab-CI pipeline > > (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > > is designed with a distributed infrastructure model, making it possible > > to run on any gitLab instance. We plan to leverage KernelCI [2] as the > > backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > > integration. > > > > For an example of a fully executed pipeline with drm scenario set, > > including documentation generation, > > see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262 > > > > Please refer to the documentation included in the patch, or check the > > rendered version, here: > > https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html > > > > Differences from v1 to v2: > > - moved to tools/ci as suggested by Linus on the previous version > > - add arm64 containers for native compilation > > - added boot tests using virtme: this is the base structure for boot tests, > > next steps would be adding other tests such as kselftests/kunit tests > > - added DRM scenario testing on vkms: this should replace current vkms test > > in drm-ci. This work shows how a test scenario can be used by different > > subsystems to add their tests. > > - update documentation > > > > For more details on the motivation behind this work, please refer to the > > cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/20240228225527.1052240-1-helen.koike@collabora.com/ > > > > [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/ > > [2] https://kernelci.org/ > > [3] https://docs.kernelci.org/kcidb/ > > > > Helen Koike (3): > > kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing > > kci-gitlab: Add documentation > > kci-gitlab: docs: Add images > > > > Vignesh Raman (2): > > MAINTAINERS: Add an entry for ci automated testing > > kci-gitlab: Add drm scenario > > Hi Vignesh Raman, > I am very happy to see this project going forward :) > > It's been a few years since I first thought on finding a good way of helping > kernel developers testing their patches, while making use of the free runner > minutes Gitlab offers. It can greatly simplify the testing for people who are > new to kernel development, or students trying to understand it better. > > And this patchset allows that to happen :) > > Actually, I spoke to Helen last year, and to enable it to run on the free > Gitlab-CI runners, there is a small extra patch which is needed: > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ Gitlab as an open-source software project (the community edition) is one thing, but can we please avoid advertising specific proprietary services in the kernel documentation ? > Could you please apply it on top of your tree? > Some stuff changed places, but I can send a v2 with that fix if you want. > > > While I have yet to review this v2 patchset, I applied it on my repo for > testing, and cherry-picked the patch on above link, triggering a Pipeline: > > https://gitlab.com/linux-kernel/linux/-/pipelines/1638955600 > > It seems to be working fine, please check it, as you may be more used to the > results. > > Thanks! > Leo
On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > Gitlab as an open-source software project (the community edition) is one > thing, but can we please avoid advertising specific proprietary services > in the kernel documentation ? I don't think we should have any of this in the mainline kernel. One angle is that "no regressions rule" applies also to the shenanigans. Do we really spend energy on this proprietary crap to the eternity? BR, Jarkko
Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu: > > It's been a few years since I first thought on finding a good way of helping > > kernel developers testing their patches, while making use of the free runner > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > new to kernel development, or students trying to understand it better. > > > > And this patchset allows that to happen :) > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ Sounds interesting! > Gitlab as an open-source software project (the community edition) is one > thing, but can we please avoid advertising specific proprietary services > in the kernel documentation ? Every time Gitlab is mentioned, the brand of the company that developed it and has been providing proprietary services is also advertised. If you're not happy with that, you should move to use a git forge developed by some open source community. The way I see, the best would be if the CI integration could work with more than one type of forge and being able to use any free Git??b-CI runners that would be available for developers to use, as this would allow testing more subsystems with CI, thus increasing code quality. Thanks, Mauro
Hi Jarkko, On 1/23/25 11:30 PM, Jarkko Sakkinen wrote: > On Thu Jan 23, 2025 at 3:53 PM EET, Vignesh Raman wrote: >> We are working towards creating a generic, upstream GitLab-CI pipeline >> (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline >> is designed with a distributed infrastructure model, making it possible >> to run on any gitLab instance. We plan to leverage KernelCI [2] as the >> backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] >> integration. > > Why can't you keep the next version of your great pipeline outside the > kernel tree? > > If there is a legit motivation for doing that, why it needs to be bound > to Gitlab? Why can't you make script callable from any CI? Greetings from the (today's) sunny Espoo! Of course we could keep it outside the kernel tree. However, the point of this contribution is to provide kernel maintainers and developers with an easy way to setup their CI pipeline on a GitLab instance (the main one, FreeDesktop one, or any other). Basically this is like a template or a library, if you wish, which helps you do that. Approved by Linus too. Why GitLab? Because it's one of the best, if not *the* best CI system these days, with lots of flexibility, and it's Open-Source too (well, at least open-core, which is still very capable). And also because a number of maintainers and companies are already using it. Sure, a script could be contributed too, but the value of this contribution is a ready-made integration. And we want to make it easily discoverable, and easily contributed to. BTW, here's the talk we gave at last year's LPC regarding current use of GitLab in the kernel and surrounding community: https://lpc.events/event/18/contributions/1728/ Nick
On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > It's been a few years since I first thought on finding a good way of helping > > > kernel developers testing their patches, while making use of the free runner > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > new to kernel development, or students trying to understand it better. > > > > > > And this patchset allows that to happen :) > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > Sounds interesting! > > > Gitlab as an open-source software project (the community edition) is one > > thing, but can we please avoid advertising specific proprietary services > > in the kernel documentation ? > > Every time Gitlab is mentioned, the brand of the company that > developed it and has been providing proprietary services is also > advertised. If you're not happy with that, you should move to use > a git forge developed by some open source community. I'm fine mentioning the gitlab community edition, I'm not fine advertising gitlab.com SaaS in the kernel source tree. > The way I see, the best would be if the CI integration could work > with more than one type of forge and being able to use any > free Git??b-CI runners that would be available for developers to > use, as this would allow testing more subsystems with CI, thus > increasing code quality.
On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: >> Gitlab as an open-source software project (the community edition) is one >> thing, but can we please avoid advertising specific proprietary services >> in the kernel documentation ? > > I don't think we should have any of this in the mainline kernel. > > One angle is that "no regressions rule" applies also to the shenanigans. > > Do we really spend energy on this proprietary crap to the eternity? This is not getting included into the kernel itself, the contributed code is, of course, open-source. And yes it would execute just fine on the fully open-source community-edition GitLab. I don't think "no regressions rule" should apply here. This is for developers only, and is a template for making your own pipeline mostly, with pieces which can be reused. Nick
Hi, Le vendredi 24 janvier 2025 à 15:00 +0200, Nikolai Kondrashov a écrit : > On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > > > Gitlab as an open-source software project (the community edition) is one > > > thing, but can we please avoid advertising specific proprietary services > > > in the kernel documentation ? > > > > I don't think we should have any of this in the mainline kernel. > > > > One angle is that "no regressions rule" applies also to the shenanigans. > > > > Do we really spend energy on this proprietary crap to the eternity? > > This is not getting included into the kernel itself, the contributed code is, > of course, open-source. And yes it would execute just fine on the fully > open-source community-edition GitLab. I don't think "no regressions rule" > should apply here. This is for developers only, and is a template for making > your own pipeline mostly, with pieces which can be reused. Perhpas worth clarifying that Media and DRM subsystem have opted for the Freedesktop instance. This instance is running the Open Source version of Gitlab, with hundreds of CI runners contributed and shared among many projects (which includes Mesa, GStreamer, Wayland, Pipewire, libcamera, just to name few). Nicolas
Em Fri, 24 Jan 2025 09:26:33 -0500 Nicolas Dufresne <nicolas.dufresne@collabora.com> escreveu: > Hi, > > Le vendredi 24 janvier 2025 à 15:00 +0200, Nikolai Kondrashov a écrit : > > On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > > > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > > > > Gitlab as an open-source software project (the community edition) is one > > > > thing, but can we please avoid advertising specific proprietary services > > > > in the kernel documentation ? > > > > > > I don't think we should have any of this in the mainline kernel. > > > > > > One angle is that "no regressions rule" applies also to the shenanigans. > > > > > > Do we really spend energy on this proprietary crap to the eternity? > > > > This is not getting included into the kernel itself, the contributed code is, > > of course, open-source. And yes it would execute just fine on the fully > > open-source community-edition GitLab. > > I don't think "no regressions rule" should apply here. It doesn't, as this is not a Kernel userspace API. It is just some facility to integrate Kernel builds using a CI infrastructure. This can change with time as needed. Still, once people start using it, developers need to take some care to avoid regressions that would cause CI builds to fail for the ones using such facilities. Also, ideally, this should be completely independent of the Kernel version, e.g. if one sets up an infra using the version that was there when, let's say, Kernel 6.14 is released, the same CI scripts should work just fine with stable Kernels and even future Kernels. Due to that, I'm not convinced that such kernel CI files should be hosted at the Kernel tree. IMO, this should be stored on a separate repository hosted at kernel.org, using Semantic Versoning (https://semver.org/) - e. g. when there are incompatible changes, major version number will be increased. > > This is for developers only, and is a template for making > > your own pipeline mostly, with pieces which can be reused. > > Perhpas worth clarifying that Media and DRM subsystem have opted for the > Freedesktop instance. This instance is running the Open Source version of > Gitlab, with hundreds of CI runners contributed and shared among many projects > (which includes Mesa, GStreamer, Wayland, Pipewire, libcamera, just to name > few). It doesn't matter much what git forge some projects are currently using, as things like that could change with time. Starting with supporting just one type of git forge sounds OK to me, but long term goal should be to make it generic enough to be used with as much CI engines as possible - not only forges developed by companies that provide paid services like Gitlab/Github, but also completely open source forges and even Jenkins. Thanks, Mauro
Le vendredi 24 janvier 2025 à 15:00 +0200, Laurent Pinchart a écrit : > On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > > > It's been a few years since I first thought on finding a good way of helping > > > > kernel developers testing their patches, while making use of the free runner > > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > > new to kernel development, or students trying to understand it better. > > > > > > > > And this patchset allows that to happen :) > > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > > > Sounds interesting! > > > > > Gitlab as an open-source software project (the community edition) is one > > > thing, but can we please avoid advertising specific proprietary services > > > in the kernel documentation ? > > > > Every time Gitlab is mentioned, the brand of the company that > > developed it and has been providing proprietary services is also > > advertised. If you're not happy with that, you should move to use > > a git forge developed by some open source community. > > I'm fine mentioning the gitlab community edition, I'm not fine > advertising gitlab.com SaaS in the kernel source tree. I've just looked attentively, the intention is just to explain you may need to set gitlab variable in your project fork in order to select correctly sized sized runners in your own instance. Its is not strictly about commercial gitlab.com instance. The default only works with the original project used instance (which is not gitlab.com as far as I know), but the comment refer to companies that will choose gitlab.com internally to reduce their IT cost. Its quite a stretch to call this "advertisement", that makes it looks very dramatic. I personally believe its quite ahead of most other gitlab CI to take into consideration running these pipelines on foreign (to the project) instances. Nicolas > > > The way I see, the best would be if the CI integration could work > > with more than one type of forge and being able to use any > > free Git??b-CI runners that would be available for developers to > > use, as this would allow testing more subsystems with CI, thus > > increasing code quality. >
On Fri Jan 24, 2025 at 2:58 PM EET, Nikolai Kondrashov wrote: > Hi Jarkko, > > On 1/23/25 11:30 PM, Jarkko Sakkinen wrote: > > On Thu Jan 23, 2025 at 3:53 PM EET, Vignesh Raman wrote: > >> We are working towards creating a generic, upstream GitLab-CI pipeline > >> (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > >> is designed with a distributed infrastructure model, making it possible > >> to run on any gitLab instance. We plan to leverage KernelCI [2] as the > >> backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > >> integration. > > > > Why can't you keep the next version of your great pipeline outside the > > kernel tree? > > > > If there is a legit motivation for doing that, why it needs to be bound > > to Gitlab? Why can't you make script callable from any CI? > > Greetings from the (today's) sunny Espoo! > > Of course we could keep it outside the kernel tree. However, the point of this > contribution is to provide kernel maintainers and developers with an easy way > to setup their CI pipeline on a GitLab instance (the main one, FreeDesktop > one, or any other). Basically this is like a template or a library, if you > wish, which helps you do that. Approved by Linus too. With all due respect, "approved by Linus" is not an argument :-) I get that Linus can make the "ultimate decision" but in argumentation I'm not a strong believer of authority based decision making process. Not downplaying this more than that I knowingly ignore this comment. > Why GitLab? Because it's one of the best, if not *the* best CI system these > days, with lots of flexibility, and it's Open-Source too (well, at least > open-core, which is still very capable). And also because a number of > maintainers and companies are already using it. There's also Github Runners and Woodpecker CI used by Codeberg. To add, some companies and other organizations prefer Jenkins. This one fit for all strategy is somewhat short-sighted. Also running Gitlab tasks locally is possible but is heavy-lifting. Can we then agree that any CI changes can be sent untested to LKML if a patch set needs to reflect on those? It's not reasonable to except local runs Gitlab from individuals for patch sets. It makes our lives more difficult, not easier. All maintainers I know test their patches for the most part with BuildRoot, distro VM's and stuff like that. Not claiming that they don't exist, but never heard of kernel maintainer who uses Gitlab as their main kernel testing tool. > Sure, a script could be contributed too, but the value of this contribution is > a ready-made integration. And we want to make it easily discoverable, and > easily contributed to. This is definitely NOT "lots of flexibility". Are you dead seriously claiming that DevOps engineers could not handle well defined CI scripts and bind to whatever CI makes sense to them? o_O > > BTW, here's the talk we gave at last year's LPC regarding current use of > GitLab in the kernel and surrounding community: > > https://lpc.events/event/18/contributions/1728/ This patch set should make the case, not an old presentation. > Nick BR, Jarkko
On Fri Jan 24, 2025 at 3:00 PM EET, Nikolai Kondrashov wrote: > On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > >> Gitlab as an open-source software project (the community edition) is one > >> thing, but can we please avoid advertising specific proprietary services > >> in the kernel documentation ? > > > > I don't think we should have any of this in the mainline kernel. > > > > One angle is that "no regressions rule" applies also to the shenanigans. > > > > Do we really spend energy on this proprietary crap to the eternity? > > This is not getting included into the kernel itself, the contributed code is, > of course, open-source. And yes it would execute just fine on the fully > open-source community-edition GitLab. I don't think "no regressions rule" > should apply here. This is for developers only, and is a template for making > your own pipeline mostly, with pieces which can be reused. Ah, OK then it is fine. It was a kernel list so I assumed it was for kernel. Then: I don't care about this ;-) I use a different testing tool (BuildRoot) and a different CI (Woodpecker in my local server). So go ahead and do what you like... > > Nick BR, Jarkko
Hi Jarkko, On 1/24/25 6:32 PM, Jarkko Sakkinen wrote: > On Fri Jan 24, 2025 at 2:58 PM EET, Nikolai Kondrashov wrote: >> Of course we could keep it outside the kernel tree. However, the point of this >> contribution is to provide kernel maintainers and developers with an easy way >> to setup their CI pipeline on a GitLab instance (the main one, FreeDesktop >> one, or any other). Basically this is like a template or a library, if you >> wish, which helps you do that. Approved by Linus too. > > With all due respect, "approved by Linus" is not an argument :-) I get > that Linus can make the "ultimate decision" but in argumentation I'm not > a strong believer of authority based decision making process. Not > downplaying this more than that I knowingly ignore this comment. Sure, fair enough! >> Why GitLab? Because it's one of the best, if not *the* best CI system these >> days, with lots of flexibility, and it's Open-Source too (well, at least >> open-core, which is still very capable). And also because a number of >> maintainers and companies are already using it. > > There's also Github Runners and Woodpecker CI used by Codeberg. To add, > some companies and other organizations prefer Jenkins. This one fit for > all strategy is somewhat short-sighted. We're not suggesting everyone should use this. We've just been using it, know it well, know it works, and so are trying to help people use it. Please feel free to use something else, and of course contribute the corresponding CI setup to the kernel repo! Not that I can stop anyone, of course, but that's the idea behind this. Note the "/ci/gitlab-ci/" path. Anything which could help maintainers/developers integrate testing with their development workflow cango under "/ci/", I think. > Also running Gitlab tasks locally is possible but is heavy-lifting. Yeah, it's not perfect. > Can we then agree that any CI changes can be sent untested to LKML if > a patch set needs to reflect on those? It's not reasonable to except > local runs Gitlab from individuals for patch sets. It makes our lives > more difficult, not easier. This is not really for individual contributors, but for maintainers/subsystems to help automate their team's (and contributor's) testing. And this is not saying you're required to run GitLab CI for your contribution. It's up to a particular subsystem to decide their policies: whether to just have the maintainers send contributions off to a GitLab CI instance during review, or larger merges, give co-maintainers access to it, or require contributors to submit an (automatically-tested) MR. If you send patches changing CI, then of course you're expected to test it, as with any contribution to any project. But if you're not running it, why would you do that? And if you or the recipient maintainer run it, then you get it tested automatically. > All maintainers I know test their patches for the most part with > BuildRoot, distro VM's and stuff like that. Not claiming that they > don't exist, but never heard of kernel maintainer who uses Gitlab > as their main kernel testing tool. Well, some of the people in this discussion do that, and more are mentioned in that talk I linked. But sure, I think contributing their setups to the kernel tree would help others who want to automate their testing. >> Sure, a script could be contributed too, but the value of this contribution is >> a ready-made integration. And we want to make it easily discoverable, and >> easily contributed to. > > This is definitely NOT "lots of flexibility". > > Are you dead seriously claiming that DevOps engineers could not handle > well defined CI scripts and bind to whatever CI makes sense to them? > o_O No, of course not. This is simply to make life easier for maintainers, who are not necessarily DevOps engineers themselves, and want to try it out, or have it setup. Of course, having well-defined CI scripts would be great, and we enjoy e.g. well-defined make interface in CI, for example. I assume engineers who implemented this (Helen et al.) would be happy to use such a script, if it existed and was suitable. The problem is that it would still need to interface with GitLab CI, which is while flexible, suffers the flip-side problem of being complex. And that's where the devil is, and this contribution is trying to help defeat it. >> BTW, here's the talk we gave at last year's LPC regarding current use of >> GitLab in the kernel and surrounding community: >> >> https://lpc.events/event/18/contributions/1728/ > > This patch set should make the case, not an old presentation. Well, I'm sure there are angles not covered by the cover letter and commit messages of this contribution, and I'm partially to blame, as I was one of the pre-reviewers. However, I'm sure we have left more stuff uncovered than we discuss here, even considering the discussion of the first version. And so I provided the link to the talk as a rectification. But if it would help, I can provide a summary of it here too. Nick
On Fri, Jan 24, 2025 at 06:32:07PM +0200, Jarkko Sakkinen wrote: > Can we then agree that any CI changes can be sent untested to LKML if > a patch set needs to reflect on those? It's not reasonable to except > local runs Gitlab from individuals for patch sets. It makes our lives > more difficult, not easier. I think the theory here is that this more shared building blocks for people who want to set something up using gitlab rather than the one true CI system that everyone has to use. Even with people who do end up using gitlab there's going to be issues with things like hardware access and just realistic resources which mean that not everyone can test everything. We generally have an expectation that people will make a reasonable effort to test things, not that they're going to cover everything, and I don't see why this should be different. > All maintainers I know test their patches for the most part with > BuildRoot, distro VM's and stuff like that. Not claiming that they > don't exist, but never heard of kernel maintainer who uses Gitlab > as their main kernel testing tool. There's a huge diversity in what people are using for their CI infrastructure, gitlab instances are definitely in there already. The last time I looked at the implementation the clang testing was driven through gitlab, AIUI the DRM and media systems also have something. One of the ways gitlab is handy is that it's something that companies are likely to have set up anyway which helps with getting things provisioned. > > Sure, a script could be contributed too, but the value of this contribution is > > a ready-made integration. And we want to make it easily discoverable, and > > easily contributed to. > This is definitely NOT "lots of flexibility". > Are you dead seriously claiming that DevOps engineers could not handle > well defined CI scripts and bind to whatever CI makes sense to them? > o_O There's going to be an audience out there for people who aren't specifically devops people and would find it helpful to have something to look at when getting started, and even where people are capable of doing the work it seems helpful to pool effort on the more common stuff.
On Fri, Jan 24, 2025 at 06:32:07PM +0200, Jarkko Sakkinen wrote: > On Fri Jan 24, 2025 at 2:58 PM EET, Nikolai Kondrashov wrote: > > On 1/23/25 11:30 PM, Jarkko Sakkinen wrote: > > > On Thu Jan 23, 2025 at 3:53 PM EET, Vignesh Raman wrote: > > >> We are working towards creating a generic, upstream GitLab-CI pipeline > > >> (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > > >> is designed with a distributed infrastructure model, making it possible > > >> to run on any gitLab instance. We plan to leverage KernelCI [2] as the > > >> backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > > >> integration. > > > > > > Why can't you keep the next version of your great pipeline outside the > > > kernel tree? > > > > > > If there is a legit motivation for doing that, why it needs to be bound > > > to Gitlab? Why can't you make script callable from any CI? > > > > Greetings from the (today's) sunny Espoo! > > > > Of course we could keep it outside the kernel tree. However, the point of this > > contribution is to provide kernel maintainers and developers with an easy way > > to setup their CI pipeline on a GitLab instance (the main one, FreeDesktop > > one, or any other). Basically this is like a template or a library, if you > > wish, which helps you do that. Approved by Linus too. > > With all due respect, "approved by Linus" is not an argument :-) I get > that Linus can make the "ultimate decision" but in argumentation I'm not > a strong believer of authority based decision making process. Not > downplaying this more than that I knowingly ignore this comment. > > > Why GitLab? Because it's one of the best, if not *the* best CI system these > > days, with lots of flexibility, and it's Open-Source too (well, at least > > open-core, which is still very capable). And also because a number of > > maintainers and companies are already using it. > > There's also Github Runners and Woodpecker CI used by Codeberg. To add, > some companies and other organizations prefer Jenkins. This one fit for > all strategy is somewhat short-sighted. > > Also running Gitlab tasks locally is possible but is heavy-lifting. That's my personal opinion too. In all the CI pipelines I've designed (which totals to an amount of exactly one) I've implemented as much as possible in shell scripts that are called from a minimal service-specific CI configuration. I find this architecture to ease running tests locally. > Can we then agree that any CI changes can be sent untested to LKML if > a patch set needs to reflect on those? It's not reasonable to except > local runs Gitlab from individuals for patch sets. It makes our lives > more difficult, not easier. > > All maintainers I know test their patches for the most part with > BuildRoot, distro VM's and stuff like that. Not claiming that they > don't exist, but never heard of kernel maintainer who uses Gitlab > as their main kernel testing tool. That's highly subsystem-specific, and depends on what kind of testing is performed. The Linux media subsystem recently adopted gitlab.freedesktop.org for compilation and other static tests. As a developer, I compile-test my kernel patches locally, and run them on hardware without going through CI pipelines. > > Sure, a script could be contributed too, but the value of this contribution is > > a ready-made integration. And we want to make it easily discoverable, and > > easily contributed to. > > This is definitely NOT "lots of flexibility". > > Are you dead seriously claiming that DevOps engineers could not handle > well defined CI scripts and bind to whatever CI makes sense to them? > o_O > > > BTW, here's the talk we gave at last year's LPC regarding current use of > > GitLab in the kernel and surrounding community: > > > > https://lpc.events/event/18/contributions/1728/ > > This patch set should make the case, not an old presentation.
Hi Mauro, ---- On Fri, 24 Jan 2025 12:29:16 -0300 Mauro Carvalho Chehab wrote --- > Em Fri, 24 Jan 2025 09:26:33 -0500 > Nicolas Dufresne nicolas.dufresne@collabora.com> escreveu: > > > Hi, > > > > Le vendredi 24 janvier 2025 à 15:00 +0200, Nikolai Kondrashov a écrit : > > > On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > > > > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > in the kernel documentation ? > > > > > > > > I don't think we should have any of this in the mainline kernel. > > > > > > > > One angle is that "no regressions rule" applies also to the shenanigans. > > > > > > > > Do we really spend energy on this proprietary crap to the eternity? > > > > > > This is not getting included into the kernel itself, the contributed code is, > > > of course, open-source. And yes it would execute just fine on the fully > > > open-source community-edition GitLab. > > > > I don't think "no regressions rule" should apply here. > > It doesn't, as this is not a Kernel userspace API. It is just some > facility to integrate Kernel builds using a CI infrastructure. This can > change with time as needed. > > Still, once people start using it, developers need to take some care to > avoid regressions that would cause CI builds to fail for the ones using > such facilities. > > Also, ideally, this should be completely independent of the Kernel version, > e.g. if one sets up an infra using the version that was there when, let's > say, Kernel 6.14 is released, the same CI scripts should work just fine > with stable Kernels and even future Kernels. Or you can just configure your GitLab CI to work and an older version of the script by just pointing the right yaml URL at that versions in the configs, or am I missing something? > > Due to that, I'm not convinced that such kernel CI files should be > hosted at the Kernel tree. > > IMO, this should be stored on a separate repository hosted at > kernel.org, using Semantic Versoning (https://semver.org/) - e. g. > when there are incompatible changes, major version number will be > increased. A key benefit of having it in-tree is the test expectation list, as seen with DRM-CI. This allows developers to track the state of drivers and progress over time by showing which tests are expected to pass or fail at a given point in history. (From what I see in DRM-CI, this adds a considerable amount of value.) Please check the VKMS patch in this patchset. Also, if we keep this tool out of tree, I’m concerned that subsystems like DRM and Media will continue not collaborating—each currently has its own solution when the base could be shared and reused. All static checks, build processes, and boot mechanisms are fundamentally the same, with only minor differences that are customizable. Other subsystems could use just the base or build their specific configurations/tests on top of it. Having it in-tree sends a message: "You can create your own GitLab CI pipeline, but why not use this existing one, contribute to it, and collaborate for everyone's benefit?". Since it's under the tools/ folder, it’s a helper tool. Make sense? Thanks, Helen > > > > This is for developers only, and is a template for making > > > your own pipeline mostly, with pieces which can be reused. > > > > Perhpas worth clarifying that Media and DRM subsystem have opted for the > > Freedesktop instance. This instance is running the Open Source version of > > Gitlab, with hundreds of CI runners contributed and shared among many projects > > (which includes Mesa, GStreamer, Wayland, Pipewire, libcamera, just to name > > few). > > It doesn't matter much what git forge some projects are currently using, as > things like that could change with time. > > Starting with supporting just one type of git forge sounds OK to me, > but long term goal should be to make it generic enough to be used with as > much CI engines as possible - not only forges developed by companies that > provide paid services like Gitlab/Github, but also completely open > source forges and even Jenkins. > > Thanks, > Mauro >
> -----Original Message----- > From: Helen Mae Koike Fornazier <helen.koike@collabora.com> > Hi Mauro, > > ---- On Fri, 24 Jan 2025 12:29:16 -0300 Mauro Carvalho Chehab wrote --- > > > Em Fri, 24 Jan 2025 09:26:33 -0500 > > Nicolas Dufresne nicolas.dufresne@collabora.com> escreveu: > > > > > Hi, > > > > > > Le vendredi 24 janvier 2025 à 15:00 +0200, Nikolai Kondrashov a écrit : > > > > On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > > > > > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > > in the kernel documentation ? > > > > > > > > > > I don't think we should have any of this in the mainline kernel. > > > > > > > > > > One angle is that "no regressions rule" applies also to the shenanigans. > > > > > > > > > > Do we really spend energy on this proprietary crap to the eternity? > > > > > > > > This is not getting included into the kernel itself, the contributed code is, > > > > of course, open-source. And yes it would execute just fine on the fully > > > > open-source community-edition GitLab. > > > > > > I don't think "no regressions rule" should apply here. > > > > It doesn't, as this is not a Kernel userspace API. It is just some > > facility to integrate Kernel builds using a CI infrastructure. This can > > change with time as needed. > > > > Still, once people start using it, developers need to take some care to > > avoid regressions that would cause CI builds to fail for the ones using > > such facilities. > > > > Also, ideally, this should be completely independent of the Kernel version, > > e.g. if one sets up an infra using the version that was there when, let's > > say, Kernel 6.14 is released, the same CI scripts should work just fine > > with stable Kernels and even future Kernels. > > Or you can just configure your GitLab CI to work and an older version of > the script by just pointing the right yaml URL at that versions in the configs, > or am I missing something? > > > > > Due to that, I'm not convinced that such kernel CI files should be > > hosted at the Kernel tree. > > > > IMO, this should be stored on a separate repository hosted at > > kernel.org, using Semantic Versoning (https://semver.org/) - e. g. > > when there are incompatible changes, major version number will be > > increased. > > A key benefit of having it in-tree is the test expectation list, as seen with > DRM-CI. This allows developers to track the state of drivers and progress over > time by showing which tests are expected to pass or fail at a given point in > history. (From what I see in DRM-CI, this adds a considerable amount of value.) > Please check the VKMS patch in this patchset. > > Also, if we keep this tool out of tree, I’m concerned that subsystems like DRM > and Media will continue not collaborating—each currently has its own solution > when the base could be shared and reused. All static checks, build processes, > and boot mechanisms are fundamentally the same, with only minor differences > that are customizable. Other subsystems could use just the base or build their > specific configurations/tests on top of it. > Having it in-tree sends a message: "You can create your own GitLab CI pipeline, > but why not use this existing one, contribute to it, and collaborate for > everyone's benefit?". > Since it's under the tools/ folder, it’s a helper tool. Although I don't use gitlab CI for my kernel testing (I use Fuego), I've been waiting for this so that I can see if it's possible to leverage the information contained in it for my own CI system. I may not end up using the info myself, but at a minimum it will show me the info needed by another CI system. Having this upstream is useful, IMHO, even if it just reflects a single CI system currently being used. -- Tim > > > > This is for developers only, and is a template for making > > > > your own pipeline mostly, with pieces which can be reused. > > > > > > Perhpas worth clarifying that Media and DRM subsystem have opted for the > > > Freedesktop instance. This instance is running the Open Source version of > > > Gitlab, with hundreds of CI runners contributed and shared among many projects > > > (which includes Mesa, GStreamer, Wayland, Pipewire, libcamera, just to name > > > few). > > > > It doesn't matter much what git forge some projects are currently using, as > > things like that could change with time. > > > > Starting with supporting just one type of git forge sounds OK to me, > > but long term goal should be to make it generic enough to be used with as > > much CI engines as possible - not only forges developed by companies that > > provide paid services like Gitlab/Github, but also completely open > > source forges and even Jenkins. > > > > Thanks, > > Mauro > > >
On Fri, 2025-01-24 at 13:52 +0100, Mauro Carvalho Chehab wrote: > Em Fri, 24 Jan 2025 10:12:50 +0200 > Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu: > > > > It's been a few years since I first thought on finding a good way of helping > > > kernel developers testing their patches, while making use of the free runner > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > new to kernel development, or students trying to understand it better. > > > > > > And this patchset allows that to happen :) > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > Sounds interesting! Thanks! :) > > > Gitlab as an open-source software project (the community edition) is one > > thing, but can we please avoid advertising specific proprietary services > > in the kernel documentation ? > > Every time Gitlab is mentioned, the brand of the company that > developed it and has been providing proprietary services is also > advertised. If you're not happy with that, you should move to use > a git forge developed by some open source community. > > The way I see, the best would be if the CI integration could work > with more than one type of forge and being able to use any > free Git??b-CI runners that would be available for developers to > use, as this would allow testing more subsystems with CI, thus > increasing code quality. Completely agree! I think that this Gitlab contribution will set a base for implementing other forges / CI systems as well, and allow us to make use of those resources for better kernel testing. Thanks! Leo > > Thanks, > Mauro
On Fri, 2025-01-24 at 10:45 -0500, Nicolas Dufresne wrote: > Le vendredi 24 janvier 2025 à 15:00 +0200, Laurent Pinchart a écrit : > > On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > > > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > > > > > It's been a few years since I first thought on finding a good way of helping > > > > > kernel developers testing their patches, while making use of the free runner > > > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > > > new to kernel development, or students trying to understand it better. > > > > > > > > > > And this patchset allows that to happen :) > > > > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > > > > > Sounds interesting! > > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > thing, but can we please avoid advertising specific proprietary services > > > > in the kernel documentation ? > > > > > > Every time Gitlab is mentioned, the brand of the company that > > > developed it and has been providing proprietary services is also > > > advertised. If you're not happy with that, you should move to use > > > a git forge developed by some open source community. > > > > I'm fine mentioning the gitlab community edition, I'm not fine > > advertising gitlab.com SaaS in the kernel source tree. Hello Laurent, I see your point, and I see no issue on removing the two last lines of CI_TAGS documentation. I just added this information on documentation because the default runner used for the Free Tier of Gitlab does not work for this CI, as it needs more resources to run. This information can be added on some other place, but at the time I thought it would be ok to let it be there. This other runner I mentioned in the patch is also available on the Free Tier (free as in beer). I would like to make it clear that I have no connection/affiliation to Gitlab, other than a free account in their system, which I use for some CI. I have no interest on advertising anything from them. My only objective is to make it easier to hobbyists/beginners to use those available free minutes to test some change before sending the patch, if they think that's valuable. > > I've just looked attentively, the intention is just to explain you may need to > set gitlab variable in your project fork in order to select correctly sized > sized runners in your own instance. That's correct > Its is not strictly about commercial gitlab.com instance. Exactly, the change is about being able to choose the runner you want. > The default only works with the original project used > instance (which is not gitlab.com as far as I know), but the comment refer to > companies that will choose gitlab.com internally to reduce their IT cost. Correct. Companies can benefit on that, but my focus was on hobbyist (or begginers) who may want to test their patches on free CI before sending them to the ML. > > Its quite a stretch to call this "advertisement", that makes it looks very > dramatic. I personally believe its quite ahead of most other gitlab CI to take > into consideration running these pipelines on foreign (to the project) > instances. > > Nicolas Thank you! Leo > > > > > > The way I see, the best would be if the CI integration could work > > > with more than one type of forge and being able to use any > > > free Git??b-CI runners that would be available for developers to > > > use, as this would allow testing more subsystems with CI, thus > > > increasing code quality. > > >
Hi Linus, On 23/01/25 21:16, Linus Torvalds wrote: > On Thu, 23 Jan 2025 at 05:56, Vignesh Raman <vignesh.raman@collabora.com> wrote: >> >> Documentation/ci/gitlab-ci/images/drm-vkms.png | Bin 0 -> 73810 bytes >> .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 20000 bytes >> .../ci/gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes >> .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes >> .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 62464 bytes >> .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes > > > This seems excessive. > > We have a few images in the tree, but they tend to be either the > penguin logo or various source SVGs. > > Not a set of random screenshots or whatever. We will remove this commit and update the documentation. Thanks. Regards, Vignesh > > Linus
On Fri, Jan 24, 2025 at 06:12:24PM -0300, Leonardo Brás wrote: > On Fri, 2025-01-24 at 10:45 -0500, Nicolas Dufresne wrote: > > Le vendredi 24 janvier 2025 à 15:00 +0200, Laurent Pinchart a écrit : > > > On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > > > > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > > > > > > > It's been a few years since I first thought on finding a good way of helping > > > > > > kernel developers testing their patches, while making use of the free runner > > > > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > > > > new to kernel development, or students trying to understand it better. > > > > > > > > > > > > And this patchset allows that to happen :) > > > > > > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > > > > > > > Sounds interesting! > > > > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > in the kernel documentation ? > > > > > > > > Every time Gitlab is mentioned, the brand of the company that > > > > developed it and has been providing proprietary services is also > > > > advertised. If you're not happy with that, you should move to use > > > > a git forge developed by some open source community. > > > > > > I'm fine mentioning the gitlab community edition, I'm not fine > > > advertising gitlab.com SaaS in the kernel source tree. > > Hello Laurent, > > I see your point, and I see no issue on removing the two last lines of CI_TAGS > documentation. > > I just added this information on documentation because the default runner used > for the Free Tier of Gitlab does not work for this CI, as it needs more > resources to run. This information can be added on some other place, but at the > time I thought it would be ok to let it be there. > This other runner I mentioned in the patch is also available on the Free Tier > (free as in beer). > > I would like to make it clear that I have no connection/affiliation to Gitlab, > other than a free account in their system, which I use for some CI. I have no > interest on advertising anything from them. > > My only objective is to make it easier to hobbyists/beginners to use those > available free minutes to test some change before sending the patch, if they > think that's valuable. Given the 400 free computes minute per month, and the fact that the saas-linux-medium-amd64 runner consumers two minutes per minute, how many of the proposed CI runs would be available per month ? CI pipeline runs always compile the kernel from scratch as far as can see. This may not be an issue for final testing before submission of a patch series, but it just won't work for incremental testing of changes. Think of how inefficient it would be to run a full pipeline just to get the checkpatch.pl output for instance. This is why I believe tests should focus first and foremost on ease of use in developers' local environments. A standardized, from-scratch, comprehensive test run as a gate keeper for integration has value as well, but that won't help beginners much. > > I've just looked attentively, the intention is just to explain you may need to > > set gitlab variable in your project fork in order to select correctly sized > > sized runners in your own instance. > > That's correct > > > Its is not strictly about commercial gitlab.com instance. > > Exactly, the change is about being able to choose the runner you want. > > > The default only works with the original project used > > instance (which is not gitlab.com as far as I know), but the comment refer to > > companies that will choose gitlab.com internally to reduce their IT cost. > > Correct. > Companies can benefit on that, but my focus was on hobbyist (or begginers) who > may want to test their patches on free CI before sending them to the ML. > > > Its quite a stretch to call this "advertisement", that makes it looks very > > dramatic. I personally believe its quite ahead of most other gitlab CI to take > > into consideration running these pipelines on foreign (to the project) > > instances. > > > > > > The way I see, the best would be if the CI integration could work > > > > with more than one type of forge and being able to use any > > > > free Git??b-CI runners that would be available for developers to > > > > use, as this would allow testing more subsystems with CI, thus > > > > increasing code quality.
Em Fri, 24 Jan 2025 16:49:30 -0300 Helen Mae Koike Fornazier <helen.koike@collabora.com> escreveu: > Hi Mauro, > > > > ---- On Fri, 24 Jan 2025 12:29:16 -0300 Mauro Carvalho Chehab wrote --- > > > Em Fri, 24 Jan 2025 09:26:33 -0500 > > Nicolas Dufresne nicolas.dufresne@collabora.com> escreveu: > > > > > Hi, > > > > > > Le vendredi 24 janvier 2025 à 15:00 +0200, Nikolai Kondrashov a écrit : > > > > On 1/24/25 2:16 PM, Jarkko Sakkinen wrote: > > > > > On Fri Jan 24, 2025 at 10:12 AM EET, Laurent Pinchart wrote: > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > > in the kernel documentation ? > > > > > > > > > > I don't think we should have any of this in the mainline kernel. > > > > > > > > > > One angle is that "no regressions rule" applies also to the shenanigans. > > > > > > > > > > Do we really spend energy on this proprietary crap to the eternity? > > > > > > > > This is not getting included into the kernel itself, the contributed code is, > > > > of course, open-source. And yes it would execute just fine on the fully > > > > open-source community-edition GitLab. > > > > > > I don't think "no regressions rule" should apply here. > > > > It doesn't, as this is not a Kernel userspace API. It is just some > > facility to integrate Kernel builds using a CI infrastructure. This can > > change with time as needed. > > > > Still, once people start using it, developers need to take some care to > > avoid regressions that would cause CI builds to fail for the ones using > > such facilities. > > > > Also, ideally, this should be completely independent of the Kernel version, > > e.g. if one sets up an infra using the version that was there when, let's > > say, Kernel 6.14 is released, the same CI scripts should work just fine > > with stable Kernels and even future Kernels. > > Or you can just configure your GitLab CI to work and an older version of > the script by just pointing the right yaml URL at that versions in the configs, > or am I missing something? The problem of placing them altogether is that it would be hard to track incompatible changes, as there's not separate versioning for the yaml files. > > > > Due to that, I'm not convinced that such kernel CI files should be > > hosted at the Kernel tree. > > > > IMO, this should be stored on a separate repository hosted at > > kernel.org, using Semantic Versoning (https://semver.org/) - e. g. > > when there are incompatible changes, major version number will be > > increased. > > A key benefit of having it in-tree is the test expectation list, as seen with > DRM-CI. This allows developers to track the state of drivers and progress over > time by showing which tests are expected to pass or fail at a given point in > history. (From what I see in DRM-CI, this adds a considerable amount of value.) > Please check the VKMS patch in this patchset. I'm not saying that everything should be on a separate tree. Things like expected tests and test results are dependent at Kernel versions. What I'm saying is that hosting CI-specific files a separate repository is better, as you can keep there what is generic. > Also, if we keep this tool out of tree, I’m concerned that subsystems like DRM > and Media will continue not collaborating—each currently has its own solution > when the base could be shared and reused. All static checks, build processes, > and boot mechanisms are fundamentally the same, with only minor differences > that are customizable. Other subsystems could use just the base or build their > specific configurations/tests on top of it. I'm in favor of having a common set of CI tooling for all subsystems interested on setting up CI. Yet moving part of it to be on a separate repository won't prevent collaboration. Also, either in or out of tree won't avoid different subsystems to use different solutions. What would make people use it is the capability of quickly setting it up on a new repository. For this to happen, documentation is really important. Equally important is to have support for multiple git forges and other CI engines. For instance, besides media and DRM, I'm also personally interested on having it setup for other subsystems that I also collaborate. > Having it in-tree sends a message: "You can create your own GitLab CI pipeline, > but why not use this existing one, contribute to it, and collaborate for > everyone's benefit?". > > Since it's under the tools/ folder, it’s a helper tool. It makes sense to have it documented at the Kernel Documentation folder and to have something at tools/ that would setup the git forge repositories used for CI to do Kernel builds. > Make sense? > > Thanks, > Helen > > > > > > > This is for developers only, and is a template for making > > > > your own pipeline mostly, with pieces which can be reused. > > > > > > Perhpas worth clarifying that Media and DRM subsystem have opted for the > > > Freedesktop instance. This instance is running the Open Source version of > > > Gitlab, with hundreds of CI runners contributed and shared among many projects > > > (which includes Mesa, GStreamer, Wayland, Pipewire, libcamera, just to name > > > few). > > > > It doesn't matter much what git forge some projects are currently using, as > > things like that could change with time. > > > > Starting with supporting just one type of git forge sounds OK to me, > > but long term goal should be to make it generic enough to be used with as > > much CI engines as possible - not only forges developed by companies that > > provide paid services like Gitlab/Github, but also completely open > > source forges and even Jenkins. > > > > Thanks, > > Mauro > > > Thanks, Mauro
Em Mon, 27 Jan 2025 08:07:38 +0200 Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu: > On Fri, Jan 24, 2025 at 06:12:24PM -0300, Leonardo Brás wrote: > > On Fri, 2025-01-24 at 10:45 -0500, Nicolas Dufresne wrote: > > > Le vendredi 24 janvier 2025 à 15:00 +0200, Laurent Pinchart a écrit : > > > > On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > > > > > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > > > > > > > > > It's been a few years since I first thought on finding a good way of helping > > > > > > > kernel developers testing their patches, while making use of the free runner > > > > > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > > > > > new to kernel development, or students trying to understand it better. > > > > > > > > > > > > > > And this patchset allows that to happen :) > > > > > > > > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > > > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > > > > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > > > > > > > > > Sounds interesting! > > > > > > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > > in the kernel documentation ? > > > > > > > > > > Every time Gitlab is mentioned, the brand of the company that > > > > > developed it and has been providing proprietary services is also > > > > > advertised. If you're not happy with that, you should move to use > > > > > a git forge developed by some open source community. > > > > > > > > I'm fine mentioning the gitlab community edition, I'm not fine > > > > advertising gitlab.com SaaS in the kernel source tree. > > > > Hello Laurent, > > > > I see your point, and I see no issue on removing the two last lines of CI_TAGS > > documentation. > > > > I just added this information on documentation because the default runner used > > for the Free Tier of Gitlab does not work for this CI, as it needs more > > resources to run. This information can be added on some other place, but at the > > time I thought it would be ok to let it be there. > > This other runner I mentioned in the patch is also available on the Free Tier > > (free as in beer). > > > > I would like to make it clear that I have no connection/affiliation to Gitlab, > > other than a free account in their system, which I use for some CI. I have no > > interest on advertising anything from them. > > > > My only objective is to make it easier to hobbyists/beginners to use those > > available free minutes to test some change before sending the patch, if they > > think that's valuable. > > Given the 400 free computes minute per month, and the fact that the > saas-linux-medium-amd64 runner consumers two minutes per minute, how > many of the proposed CI runs would be available per month ? > > CI pipeline runs always compile the kernel from scratch as far as can > see. This may not be an issue for final testing before submission of a > patch series, but it just won't work for incremental testing of changes. > Think of how inefficient it would be to run a full pipeline just to get > the checkpatch.pl output for instance. This is why I believe tests > should focus first and foremost on ease of use in developers' local > environments. A standardized, from-scratch, comprehensive test run as a > gate keeper for integration has value as well, but that won't help > beginners much. To better cope with free minutes, CI pipelines should allow setting up build just the subsystem instead of the entire Kernel. Not perfect, but still if CI would test multiple archs/Kconfig options, it would be useful to get early lots of potential issues. That's said, using developers' local environments is interesting. Yet, there's s potential risk of using it while having an external entity running a git forge, as it would mean that a remote entity would trigger something to run locally. Personally, I would So, if I were to setup a local build environment, I would either use a separate machine outside my internal network or I would opt to install a local git forge instance. No idea how easy/hard would be to maintain such local gitlab instance. Thanks, Mauro
Hi Leo, On 24/01/25 13:42, Laurent Pinchart wrote: > On Fri, Jan 24, 2025 at 02:11:26AM -0300, Leonardo Brás wrote: >> On Thu, 2025-01-23 at 19:23 +0530, Vignesh Raman wrote: >>> This patch series introduces "kci-gitlab," a GitLab CI pipeline >>> specifically designed for kernel testing. It provides kernel >>> developers with an integrated, efficient, and flexible testing >>> framework using GitLab's CI/CD capabilities. This patch includes >>> a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with >>> additional YAML and script files, to define a basic test pipeline >>> triggered by code pushes to a GitLab-CI instance. >>> The initial version implements: >>> >>> - Static checks: Includes checkpatch and smatch for code validation. >>> - Build tests: Covers various architectures and configurations. >>> - Boot tests: Utilizes virtme for basic boot testing. >>> >>> Additionally, it introduces a flexible "scenarios" mechanism to >>> support subsystem-specific extensions. >>> >>> This series also introduces a drm scenario that adds a job to run IGT >>> tests for vkms. This scenario includes helper scripts to build deqp-runner >>> and IGT, leveraging approaches from the drm-ci/mesa-ci project. >>> >>> We are working towards creating a generic, upstream GitLab-CI pipeline >>> (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline >>> is designed with a distributed infrastructure model, making it possible >>> to run on any gitLab instance. We plan to leverage KernelCI [2] as the >>> backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] >>> integration. >>> >>> For an example of a fully executed pipeline with drm scenario set, >>> including documentation generation, >>> see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262 >>> >>> Please refer to the documentation included in the patch, or check the >>> rendered version, here: >>> https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html >>> >>> Differences from v1 to v2: >>> - moved to tools/ci as suggested by Linus on the previous version >>> - add arm64 containers for native compilation >>> - added boot tests using virtme: this is the base structure for boot tests, >>> next steps would be adding other tests such as kselftests/kunit tests >>> - added DRM scenario testing on vkms: this should replace current vkms test >>> in drm-ci. This work shows how a test scenario can be used by different >>> subsystems to add their tests. >>> - update documentation >>> >>> For more details on the motivation behind this work, please refer to the >>> cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/20240228225527.1052240-1-helen.koike@collabora.com/ >>> >>> [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/ >>> [2] https://kernelci.org/ >>> [3] https://docs.kernelci.org/kcidb/ >>> >>> Helen Koike (3): >>> kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing >>> kci-gitlab: Add documentation >>> kci-gitlab: docs: Add images >>> >>> Vignesh Raman (2): >>> MAINTAINERS: Add an entry for ci automated testing >>> kci-gitlab: Add drm scenario >> >> Hi Vignesh Raman, >> I am very happy to see this project going forward :) >> >> It's been a few years since I first thought on finding a good way of helping >> kernel developers testing their patches, while making use of the free runner >> minutes Gitlab offers. It can greatly simplify the testing for people who are >> new to kernel development, or students trying to understand it better. >> >> And this patchset allows that to happen :) Thanks for testing the patch and providing feedback. >> >> Actually, I spoke to Helen last year, and to enable it to run on the free >> Gitlab-CI runners, there is a small extra patch which is needed: >> >> https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ Thanks for the patch. We can keep the variable name as KCI_CI_TAGS (like other variables) and add it in variables with default empty string. > > Gitlab as an open-source software project (the community edition) is one > thing, but can we please avoid advertising specific proprietary services > in the kernel documentation ? Agree. We could remove the lines which mentions gitlab.com SaaS in commit message and documentation. > >> Could you please apply it on top of your tree? >> Some stuff changed places, but I can send a v2 with that fix if you want. I will test your patch and we can add this for v3. Regards, Vignesh >> >> >> While I have yet to review this v2 patchset, I applied it on my repo for >> testing, and cherry-picked the patch on above link, triggering a Pipeline: >> >> https://gitlab.com/linux-kernel/linux/-/pipelines/1638955600 >> >> It seems to be working fine, please check it, as you may be more used to the >> results. >> >> Thanks! >> Leo >
Le lundi 27 janvier 2025 à 08:07 +0200, Laurent Pinchart a écrit : > On Fri, Jan 24, 2025 at 06:12:24PM -0300, Leonardo Brás wrote: > > On Fri, 2025-01-24 at 10:45 -0500, Nicolas Dufresne wrote: > > > Le vendredi 24 janvier 2025 à 15:00 +0200, Laurent Pinchart a écrit : > > > > On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > > > > > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > > > > > > > > > It's been a few years since I first thought on finding a good way of helping > > > > > > > kernel developers testing their patches, while making use of the free runner > > > > > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > > > > > new to kernel development, or students trying to understand it better. > > > > > > > > > > > > > > And this patchset allows that to happen :) > > > > > > > > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > > > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > > > > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > > > > > > > > > Sounds interesting! > > > > > > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > > in the kernel documentation ? > > > > > > > > > > Every time Gitlab is mentioned, the brand of the company that > > > > > developed it and has been providing proprietary services is also > > > > > advertised. If you're not happy with that, you should move to use > > > > > a git forge developed by some open source community. > > > > > > > > I'm fine mentioning the gitlab community edition, I'm not fine > > > > advertising gitlab.com SaaS in the kernel source tree. > > > > Hello Laurent, > > > > I see your point, and I see no issue on removing the two last lines of CI_TAGS > > documentation. > > > > I just added this information on documentation because the default runner used > > for the Free Tier of Gitlab does not work for this CI, as it needs more > > resources to run. This information can be added on some other place, but at the > > time I thought it would be ok to let it be there. > > This other runner I mentioned in the patch is also available on the Free Tier > > (free as in beer). > > > > I would like to make it clear that I have no connection/affiliation to Gitlab, > > other than a free account in their system, which I use for some CI. I have no > > interest on advertising anything from them. > > > > My only objective is to make it easier to hobbyists/beginners to use those > > available free minutes to test some change before sending the patch, if they > > think that's valuable. > > Given the 400 free computes minute per month, and the fact that the > saas-linux-medium-amd64 runner consumers two minutes per minute, how > many of the proposed CI runs would be available per month ? > > CI pipeline runs always compile the kernel from scratch as far as can > see. This may not be an issue for final testing before submission of a > patch series, but it just won't work for incremental testing of changes. > Think of how inefficient it would be to run a full pipeline just to get > the checkpatch.pl output for instance. This is why I believe tests > should focus first and foremost on ease of use in developers' local > environments. A standardized, from-scratch, comprehensive test run as a > gate keeper for integration has value as well, but that won't help > beginners much. Perhaps we can provide pointers how to run a private gitlab runner on your local computer. My experience is that this take few minutes, unlike figuring all the foreign bash scripts dependency, and debugging issues related to some tools not having the right version, which inevitably lead to folks criticizing distro choices and other counter productive discussions. Of course, it can only work if that process is used and tested. > > > > I've just looked attentively, the intention is just to explain you may need to > > > set gitlab variable in your project fork in order to select correctly sized > > > sized runners in your own instance. > > > > That's correct > > > > > Its is not strictly about commercial gitlab.com instance. > > > > Exactly, the change is about being able to choose the runner you want. > > > > > The default only works with the original project used > > > instance (which is not gitlab.com as far as I know), but the comment refer to > > > companies that will choose gitlab.com internally to reduce their IT cost. > > > > Correct. > > Companies can benefit on that, but my focus was on hobbyist (or begginers) who > > may want to test their patches on free CI before sending them to the ML. > > > > > Its quite a stretch to call this "advertisement", that makes it looks very > > > dramatic. I personally believe its quite ahead of most other gitlab CI to take > > > into consideration running these pipelines on foreign (to the project) > > > instances. > > > > > > > > The way I see, the best would be if the CI integration could work > > > > > with more than one type of forge and being able to use any > > > > > free Git??b-CI runners that would be available for developers to > > > > > use, as this would allow testing more subsystems with CI, thus > > > > > increasing code quality. >
Hi Nicolas, On Mon, Jan 27, 2025 at 09:43:23AM -0500, Nicolas Dufresne wrote: > Le lundi 27 janvier 2025 à 08:07 +0200, Laurent Pinchart a écrit : > > On Fri, Jan 24, 2025 at 06:12:24PM -0300, Leonardo Brás wrote: > > > On Fri, 2025-01-24 at 10:45 -0500, Nicolas Dufresne wrote: > > > > Le vendredi 24 janvier 2025 à 15:00 +0200, Laurent Pinchart a écrit : > > > > > On Fri, Jan 24, 2025 at 01:52:03PM +0100, Mauro Carvalho Chehab wrote: > > > > > > Em Fri, 24 Jan 2025 10:12:50 +0200 Laurent Pinchart escreveu: > > > > > > > > > > > > > > It's been a few years since I first thought on finding a good way of helping > > > > > > > > kernel developers testing their patches, while making use of the free runner > > > > > > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > > > > > > new to kernel development, or students trying to understand it better. > > > > > > > > > > > > > > > > And this patchset allows that to happen :) > > > > > > > > > > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > > > > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > > > > > > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > > > > > > > > > > > Sounds interesting! > > > > > > > > > > > > > Gitlab as an open-source software project (the community edition) is one > > > > > > > thing, but can we please avoid advertising specific proprietary services > > > > > > > in the kernel documentation ? > > > > > > > > > > > > Every time Gitlab is mentioned, the brand of the company that > > > > > > developed it and has been providing proprietary services is also > > > > > > advertised. If you're not happy with that, you should move to use > > > > > > a git forge developed by some open source community. > > > > > > > > > > I'm fine mentioning the gitlab community edition, I'm not fine > > > > > advertising gitlab.com SaaS in the kernel source tree. > > > > > > Hello Laurent, > > > > > > I see your point, and I see no issue on removing the two last lines of CI_TAGS > > > documentation. > > > > > > I just added this information on documentation because the default runner used > > > for the Free Tier of Gitlab does not work for this CI, as it needs more > > > resources to run. This information can be added on some other place, but at the > > > time I thought it would be ok to let it be there. > > > This other runner I mentioned in the patch is also available on the Free Tier > > > (free as in beer). > > > > > > I would like to make it clear that I have no connection/affiliation to Gitlab, > > > other than a free account in their system, which I use for some CI. I have no > > > interest on advertising anything from them. > > > > > > My only objective is to make it easier to hobbyists/beginners to use those > > > available free minutes to test some change before sending the patch, if they > > > think that's valuable. > > > > Given the 400 free computes minute per month, and the fact that the > > saas-linux-medium-amd64 runner consumers two minutes per minute, how > > many of the proposed CI runs would be available per month ? > > > > CI pipeline runs always compile the kernel from scratch as far as can > > see. This may not be an issue for final testing before submission of a > > patch series, but it just won't work for incremental testing of changes. > > Think of how inefficient it would be to run a full pipeline just to get > > the checkpatch.pl output for instance. This is why I believe tests > > should focus first and foremost on ease of use in developers' local > > environments. A standardized, from-scratch, comprehensive test run as a > > gate keeper for integration has value as well, but that won't help > > beginners much. > > Perhaps we can provide pointers how to run a private gitlab runner on your local > computer. My experience is that this take few minutes, unlike figuring all the > foreign bash scripts dependency, and debugging issues related to some tools not > having the right version, which inevitably lead to folks criticizing distro > choices and other counter productive discussions. Of course, it can only work if > that process is used and tested. I've run that before. It's not necessarily extremely complicated (we of course all know that in IT "a few minutes" means at least half a day - the rule of thumb is to multiple any time estimate by two and bump to the next unit :-)), but it will still take a massive amount of CPU time to run CI loads designed for a large number of powerful runners on a development laptop. As I said before, it has its uses, but mostly as a last check before merging (or sending patches), and not as an iterative process during development. And yes, if we run a gitlab runner locally we could also set up a cache that makes builds incremental, but that will only work if incremental builds are supported as a first class citizen by the CI pipeline developers. I don't see that happening very often. The point I'm trying to make is that I believe it would be more valuable for beginners (and also more experienced developers) if we could provide them with the ability to run individual tests incrementally with the lowest overheard, even if the setup time is a bit longer. > > > > I've just looked attentively, the intention is just to explain you may need to > > > > set gitlab variable in your project fork in order to select correctly sized > > > > sized runners in your own instance. > > > > > > That's correct > > > > > > > Its is not strictly about commercial gitlab.com instance. > > > > > > Exactly, the change is about being able to choose the runner you want. > > > > > > > The default only works with the original project used > > > > instance (which is not gitlab.com as far as I know), but the comment refer to > > > > companies that will choose gitlab.com internally to reduce their IT cost. > > > > > > Correct. > > > Companies can benefit on that, but my focus was on hobbyist (or begginers) who > > > may want to test their patches on free CI before sending them to the ML. > > > > > > > Its quite a stretch to call this "advertisement", that makes it looks very > > > > dramatic. I personally believe its quite ahead of most other gitlab CI to take > > > > into consideration running these pipelines on foreign (to the project) > > > > instances. > > > > > > > > > > The way I see, the best would be if the CI integration could work > > > > > > with more than one type of forge and being able to use any > > > > > > free Git??b-CI runners that would be available for developers to > > > > > > use, as this would allow testing more subsystems with CI, thus > > > > > > increasing code quality.
On Mon, 2025-01-27 at 13:02 +0530, Vignesh Raman wrote: > Hi Leo, > > On 24/01/25 13:42, Laurent Pinchart wrote: > > On Fri, Jan 24, 2025 at 02:11:26AM -0300, Leonardo Brás wrote: > > > On Thu, 2025-01-23 at 19:23 +0530, Vignesh Raman wrote: > > > > This patch series introduces "kci-gitlab," a GitLab CI pipeline > > > > specifically designed for kernel testing. It provides kernel > > > > developers with an integrated, efficient, and flexible testing > > > > framework using GitLab's CI/CD capabilities. This patch includes > > > > a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with > > > > additional YAML and script files, to define a basic test pipeline > > > > triggered by code pushes to a GitLab-CI instance. > > > > The initial version implements: > > > > > > > > - Static checks: Includes checkpatch and smatch for code validation. > > > > - Build tests: Covers various architectures and configurations. > > > > - Boot tests: Utilizes virtme for basic boot testing. > > > > > > > > Additionally, it introduces a flexible "scenarios" mechanism to > > > > support subsystem-specific extensions. > > > > > > > > This series also introduces a drm scenario that adds a job to run IGT > > > > tests for vkms. This scenario includes helper scripts to build deqp-runner > > > > and IGT, leveraging approaches from the drm-ci/mesa-ci project. > > > > > > > > We are working towards creating a generic, upstream GitLab-CI pipeline > > > > (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline > > > > is designed with a distributed infrastructure model, making it possible > > > > to run on any gitLab instance. We plan to leverage KernelCI [2] as the > > > > backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] > > > > integration. > > > > > > > > For an example of a fully executed pipeline with drm scenario set, > > > > including documentation generation, > > > > see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262 > > > > > > > > Please refer to the documentation included in the patch, or check the > > > > rendered version, here: > > > > https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html > > > > > > > > Differences from v1 to v2: > > > > - moved to tools/ci as suggested by Linus on the previous version > > > > - add arm64 containers for native compilation > > > > - added boot tests using virtme: this is the base structure for boot tests, > > > > next steps would be adding other tests such as kselftests/kunit tests > > > > - added DRM scenario testing on vkms: this should replace current vkms test > > > > in drm-ci. This work shows how a test scenario can be used by different > > > > subsystems to add their tests. > > > > - update documentation > > > > > > > > For more details on the motivation behind this work, please refer to the > > > > cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/20240228225527.1052240-1-helen.koike@collabora.com/ > > > > > > > > [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/ > > > > [2] https://kernelci.org/ > > > > [3] https://docs.kernelci.org/kcidb/ > > > > > > > > Helen Koike (3): > > > > kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing > > > > kci-gitlab: Add documentation > > > > kci-gitlab: docs: Add images > > > > > > > > Vignesh Raman (2): > > > > MAINTAINERS: Add an entry for ci automated testing > > > > kci-gitlab: Add drm scenario > > > > > > Hi Vignesh Raman, > > > I am very happy to see this project going forward :) > > > > > > It's been a few years since I first thought on finding a good way of helping > > > kernel developers testing their patches, while making use of the free runner > > > minutes Gitlab offers. It can greatly simplify the testing for people who are > > > new to kernel development, or students trying to understand it better. > > > > > > And this patchset allows that to happen :) > > Thanks for testing the patch and providing feedback. :) > > > > > > > Actually, I spoke to Helen last year, and to enable it to run on the free > > > Gitlab-CI runners, there is a small extra patch which is needed: > > > > > > https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ > > Thanks for the patch. Thank you for most of the work, it sure will be useful :) > > We can keep the variable name as KCI_CI_TAGS (like other variables) and > add it in variables with default empty string. Just tested, if it's an empty string, it will just be ignored, as it won't match anything (that is not an empty string) > > > > > Gitlab as an open-source software project (the community edition) is one > > thing, but can we please avoid advertising specific proprietary services > > in the kernel documentation ? > > Agree. We could remove the lines which mentions gitlab.com SaaS in > commit message and documentation. Done :) Added a generic reason for needing better instance > > > > > > Could you please apply it on top of your tree? > > > Some stuff changed places, but I can send a v2 with that fix if you want. > > I will test your patch and we can add this for v3. Thanks! That's the v2 with the required changes: https://lore.kernel.org/all/20250127185849.34131-1-leobras@redhat.com/T/#u Please let me know if any further change is needed. Thanks! Leo > > Regards, > Vignesh > > > > > > > > > > While I have yet to review this v2 patchset, I applied it on my repo for > > > testing, and cherry-picked the patch on above link, triggering a Pipeline: > > > > > > https://gitlab.com/linux-kernel/linux/-/pipelines/1638955600 > > > > > > It seems to be working fine, please check it, as you may be more used to the > > > results. > > > > > > Thanks! > > > Leo > >
Hi Leo, On 28/01/25 00:35, Leonardo Brás wrote: > On Mon, 2025-01-27 at 13:02 +0530, Vignesh Raman wrote: >> Hi Leo, >> >> On 24/01/25 13:42, Laurent Pinchart wrote: >>> On Fri, Jan 24, 2025 at 02:11:26AM -0300, Leonardo Brás wrote: >>>> On Thu, 2025-01-23 at 19:23 +0530, Vignesh Raman wrote: >>>>> This patch series introduces "kci-gitlab," a GitLab CI pipeline >>>>> specifically designed for kernel testing. It provides kernel >>>>> developers with an integrated, efficient, and flexible testing >>>>> framework using GitLab's CI/CD capabilities. This patch includes >>>>> a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with >>>>> additional YAML and script files, to define a basic test pipeline >>>>> triggered by code pushes to a GitLab-CI instance. >>>>> The initial version implements: >>>>> >>>>> - Static checks: Includes checkpatch and smatch for code validation. >>>>> - Build tests: Covers various architectures and configurations. >>>>> - Boot tests: Utilizes virtme for basic boot testing. >>>>> >>>>> Additionally, it introduces a flexible "scenarios" mechanism to >>>>> support subsystem-specific extensions. >>>>> >>>>> This series also introduces a drm scenario that adds a job to run IGT >>>>> tests for vkms. This scenario includes helper scripts to build deqp-runner >>>>> and IGT, leveraging approaches from the drm-ci/mesa-ci project. >>>>> >>>>> We are working towards creating a generic, upstream GitLab-CI pipeline >>>>> (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline >>>>> is designed with a distributed infrastructure model, making it possible >>>>> to run on any gitLab instance. We plan to leverage KernelCI [2] as the >>>>> backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] >>>>> integration. >>>>> >>>>> For an example of a fully executed pipeline with drm scenario set, >>>>> including documentation generation, >>>>> see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262 >>>>> >>>>> Please refer to the documentation included in the patch, or check the >>>>> rendered version, here: >>>>> https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html >>>>> >>>>> Differences from v1 to v2: >>>>> - moved to tools/ci as suggested by Linus on the previous version >>>>> - add arm64 containers for native compilation >>>>> - added boot tests using virtme: this is the base structure for boot tests, >>>>> next steps would be adding other tests such as kselftests/kunit tests >>>>> - added DRM scenario testing on vkms: this should replace current vkms test >>>>> in drm-ci. This work shows how a test scenario can be used by different >>>>> subsystems to add their tests. >>>>> - update documentation >>>>> >>>>> For more details on the motivation behind this work, please refer to the >>>>> cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/20240228225527.1052240-1-helen.koike@collabora.com/ >>>>> >>>>> [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/ >>>>> [2] https://kernelci.org/ >>>>> [3] https://docs.kernelci.org/kcidb/ >>>>> >>>>> Helen Koike (3): >>>>> kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing >>>>> kci-gitlab: Add documentation >>>>> kci-gitlab: docs: Add images >>>>> >>>>> Vignesh Raman (2): >>>>> MAINTAINERS: Add an entry for ci automated testing >>>>> kci-gitlab: Add drm scenario >>>> >>>> Hi Vignesh Raman, >>>> I am very happy to see this project going forward :) >>>> >>>> It's been a few years since I first thought on finding a good way of helping >>>> kernel developers testing their patches, while making use of the free runner >>>> minutes Gitlab offers. It can greatly simplify the testing for people who are >>>> new to kernel development, or students trying to understand it better. >>>> >>>> And this patchset allows that to happen :) >> >> Thanks for testing the patch and providing feedback. > > :) > >> >>>> >>>> Actually, I spoke to Helen last year, and to enable it to run on the free >>>> Gitlab-CI runners, there is a small extra patch which is needed: >>>> >>>> https://lore.kernel.org/all/20240327013055.139494-2-leobras@redhat.com/ >> >> Thanks for the patch. > > Thank you for most of the work, it sure will be useful :) > >> >> We can keep the variable name as KCI_CI_TAGS (like other variables) and >> add it in variables with default empty string. > > Just tested, if it's an empty string, it will just be ignored, as it won't match > anything (that is not an empty string) Ack. Adding it to variables will allow us pass KCI_CI_TAGS when manually creating a pipeline without changing the CI/CD settings, but we can ignore this for now. > >> >>> >>> Gitlab as an open-source software project (the community edition) is one >>> thing, but can we please avoid advertising specific proprietary services >>> in the kernel documentation ? >> >> Agree. We could remove the lines which mentions gitlab.com SaaS in >> commit message and documentation. > > Done :) > Added a generic reason for needing better instance > >> >>> >>>> Could you please apply it on top of your tree? >>>> Some stuff changed places, but I can send a v2 with that fix if you want. >> >> I will test your patch and we can add this for v3. > > Thanks! > That's the v2 with the required changes: > > https://lore.kernel.org/all/20250127185849.34131-1-leobras@redhat.com/T/#u > > Please let me know if any further change is needed. Thanks for the patch. We will pick this for v3. Regards, Vignesh > > Thanks! > Leo > >> >> Regards, >> Vignesh >> >>>> >>>> >>>> While I have yet to review this v2 patchset, I applied it on my repo for >>>> testing, and cherry-picked the patch on above link, triggering a Pipeline: >>>> >>>> https://gitlab.com/linux-kernel/linux/-/pipelines/1638955600 >>>> >>>> It seems to be working fine, please check it, as you may be more used to the >>>> results. >>>> >>>> Thanks! >>>> Leo >>> >
This patch series introduces "kci-gitlab," a GitLab CI pipeline specifically designed for kernel testing. It provides kernel developers with an integrated, efficient, and flexible testing framework using GitLab's CI/CD capabilities. This patch includes a .gitlab-ci file in the tools/ci/gitlab-ci/ folder, along with additional YAML and script files, to define a basic test pipeline triggered by code pushes to a GitLab-CI instance. The initial version implements: - Static checks: Includes checkpatch and smatch for code validation. - Build tests: Covers various architectures and configurations. - Boot tests: Utilizes virtme for basic boot testing. Additionally, it introduces a flexible "scenarios" mechanism to support subsystem-specific extensions. This series also introduces a drm scenario that adds a job to run IGT tests for vkms. This scenario includes helper scripts to build deqp-runner and IGT, leveraging approaches from the drm-ci/mesa-ci project. We are working towards creating a generic, upstream GitLab-CI pipeline (kci-gitlab) that will replace DRM-CI [1]. The proposed GitLab-CI pipeline is designed with a distributed infrastructure model, making it possible to run on any gitLab instance. We plan to leverage KernelCI [2] as the backend, utilizing its hardware, rootfs, test plans, and KCIDB [3] integration. For an example of a fully executed pipeline with drm scenario set, including documentation generation, see: https://gitlab.freedesktop.org/vigneshraman/kernel/-/pipelines/1350262 Please refer to the documentation included in the patch, or check the rendered version, here: https://vigneshraman.pages.freedesktop.org/-/kernel/-/jobs/69787927/artifacts/artifacts/Documentation-output/ci/gitlab-ci/gitlab-ci.html Differences from v1 to v2: - moved to tools/ci as suggested by Linus on the previous version - add arm64 containers for native compilation - added boot tests using virtme: this is the base structure for boot tests, next steps would be adding other tests such as kselftests/kunit tests - added DRM scenario testing on vkms: this should replace current vkms test in drm-ci. This work shows how a test scenario can be used by different subsystems to add their tests. - update documentation For more details on the motivation behind this work, please refer to the cover letter of v1: https://patchwork.kernel.org/project/linux-kselftest/cover/20240228225527.1052240-1-helen.koike@collabora.com/ [1] https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-pipeline-for-linux-kernel-testing/ [2] https://kernelci.org/ [3] https://docs.kernelci.org/kcidb/ Helen Koike (3): kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing kci-gitlab: Add documentation kci-gitlab: docs: Add images Vignesh Raman (2): MAINTAINERS: Add an entry for ci automated testing kci-gitlab: Add drm scenario Documentation/ci/gitlab-ci/gitlab-ci.rst | 471 ++++++++++ .../ci/gitlab-ci/images/drm-vkms.png | Bin 0 -> 73810 bytes .../ci/gitlab-ci/images/job-matrix.png | Bin 0 -> 20000 bytes .../gitlab-ci/images/new-project-runner.png | Bin 0 -> 607737 bytes .../ci/gitlab-ci/images/pipelines-on-push.png | Bin 0 -> 532143 bytes .../ci/gitlab-ci/images/the-pipeline.png | Bin 0 -> 62464 bytes .../ci/gitlab-ci/images/variables.png | Bin 0 -> 277518 bytes Documentation/index.rst | 7 + MAINTAINERS | 10 + tools/ci/gitlab-ci/arm_cross_compile.yml | 9 + tools/ci/gitlab-ci/arm_native_compile.yml | 20 + tools/ci/gitlab-ci/bootstrap-gitlab-runner.sh | 55 ++ tools/ci/gitlab-ci/build.yml | 43 + tools/ci/gitlab-ci/cache.yml | 24 + tools/ci/gitlab-ci/ci-scripts/build-docs.sh | 35 + tools/ci/gitlab-ci/ci-scripts/build-kernel.sh | 43 + .../ci/gitlab-ci/ci-scripts/ici-functions.sh | 106 +++ .../ci/gitlab-ci/ci-scripts/install-smatch.sh | 13 + .../ci-scripts/parse_commit_message.sh | 27 + .../ci/gitlab-ci/ci-scripts/run-checkpatch.sh | 20 + tools/ci/gitlab-ci/ci-scripts/run-smatch.sh | 45 + tools/ci/gitlab-ci/ci-scripts/run-virtme.sh | 52 ++ tools/ci/gitlab-ci/ci-scripts/test-boot.sh | 14 + tools/ci/gitlab-ci/container.yml | 114 +++ tools/ci/gitlab-ci/docker-compose.yaml | 18 + tools/ci/gitlab-ci/gitlab-ci.yml | 72 ++ tools/ci/gitlab-ci/scenarios.yml | 15 + .../scenarios/drm/build-deqp-runner.sh | 42 + tools/ci/gitlab-ci/scenarios/drm/build-igt.sh | 80 ++ .../ci/gitlab-ci/scenarios/drm/build-rust.sh | 42 + .../scenarios/drm/create-cross-file.sh | 65 ++ tools/ci/gitlab-ci/scenarios/drm/drm.yml | 44 + .../scenarios/drm/prepare-container.sh | 18 + tools/ci/gitlab-ci/scenarios/drm/run-igt.sh | 83 ++ tools/ci/gitlab-ci/scenarios/drm/test.yml | 32 + .../scenarios/drm/xfails/vkms-none-fails.txt | 22 + .../scenarios/drm/xfails/vkms-none-flakes.txt | 90 ++ .../scenarios/drm/xfails/vkms-none-skips.txt | 812 ++++++++++++++++++ .../scenarios/file-systems/file-systems.yml | 11 + tools/ci/gitlab-ci/scenarios/media/media.yml | 11 + .../gitlab-ci/scenarios/network/network.yml | 11 + tools/ci/gitlab-ci/static-checks.yml | 21 + tools/ci/gitlab-ci/test.yml | 16 + 43 files changed, 2613 insertions(+) create mode 100644 Documentation/ci/gitlab-ci/gitlab-ci.rst create mode 100644 Documentation/ci/gitlab-ci/images/drm-vkms.png create mode 100644 Documentation/ci/gitlab-ci/images/job-matrix.png create mode 100644 Documentation/ci/gitlab-ci/images/new-project-runner.png create mode 100644 Documentation/ci/gitlab-ci/images/pipelines-on-push.png create mode 100644 Documentation/ci/gitlab-ci/images/the-pipeline.png create mode 100644 Documentation/ci/gitlab-ci/images/variables.png create mode 100644 tools/ci/gitlab-ci/arm_cross_compile.yml create mode 100644 tools/ci/gitlab-ci/arm_native_compile.yml create mode 100755 tools/ci/gitlab-ci/bootstrap-gitlab-runner.sh create mode 100644 tools/ci/gitlab-ci/build.yml create mode 100644 tools/ci/gitlab-ci/cache.yml create mode 100755 tools/ci/gitlab-ci/ci-scripts/build-docs.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/build-kernel.sh create mode 100644 tools/ci/gitlab-ci/ci-scripts/ici-functions.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/install-smatch.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/parse_commit_message.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/run-checkpatch.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/run-smatch.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/run-virtme.sh create mode 100755 tools/ci/gitlab-ci/ci-scripts/test-boot.sh create mode 100644 tools/ci/gitlab-ci/container.yml create mode 100644 tools/ci/gitlab-ci/docker-compose.yaml create mode 100644 tools/ci/gitlab-ci/gitlab-ci.yml create mode 100644 tools/ci/gitlab-ci/scenarios.yml create mode 100755 tools/ci/gitlab-ci/scenarios/drm/build-deqp-runner.sh create mode 100755 tools/ci/gitlab-ci/scenarios/drm/build-igt.sh create mode 100755 tools/ci/gitlab-ci/scenarios/drm/build-rust.sh create mode 100755 tools/ci/gitlab-ci/scenarios/drm/create-cross-file.sh create mode 100644 tools/ci/gitlab-ci/scenarios/drm/drm.yml create mode 100755 tools/ci/gitlab-ci/scenarios/drm/prepare-container.sh create mode 100755 tools/ci/gitlab-ci/scenarios/drm/run-igt.sh create mode 100644 tools/ci/gitlab-ci/scenarios/drm/test.yml create mode 100644 tools/ci/gitlab-ci/scenarios/drm/xfails/vkms-none-fails.txt create mode 100644 tools/ci/gitlab-ci/scenarios/drm/xfails/vkms-none-flakes.txt create mode 100644 tools/ci/gitlab-ci/scenarios/drm/xfails/vkms-none-skips.txt create mode 100644 tools/ci/gitlab-ci/scenarios/file-systems/file-systems.yml create mode 100644 tools/ci/gitlab-ci/scenarios/media/media.yml create mode 100644 tools/ci/gitlab-ci/scenarios/network/network.yml create mode 100644 tools/ci/gitlab-ci/static-checks.yml create mode 100644 tools/ci/gitlab-ci/test.yml