| Message ID | 20231025142441.745947-1-mripard@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] drm/doc: ci: Require more context for flaky tests | expand |
On 25/10/2023 11:24, Maxime Ripard wrote: > Flaky tests can be very difficult to reproduce after the facts, which > will make it even harder to ever fix. > > Let's document the metadata we agreed on to provide more context to > anyone trying to address these fixes. > > Link: https://lore.kernel.org/dri-devel/CAPj87rPbJ1V1-R7WMTHkDat2A4nwSd61Df9mdGH2PR=ZzxaU=Q@mail.gmail.com/ > Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> > Signed-off-by: Maxime Ripard <mripard@kernel.org> Acked-by: Helen Koike <helen.koike@collabora.com> > > --- > Cc: Helen Koike <helen.koike@collabora.com> > > Changes from v1: > - Rephrase the sentence a bit to make it clearer we expect a bug report > by mail > --- > Documentation/gpu/automated_testing.rst | 13 +++++++++++++ > 1 file changed, 13 insertions(+) > > diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst > index 469b6fb65c30..792428a2bfdc 100644 > --- a/Documentation/gpu/automated_testing.rst > +++ b/Documentation/gpu/automated_testing.rst > @@ -67,6 +67,19 @@ Lists the tests that for a given driver on a specific hardware revision are > known to behave unreliably. These tests won't cause a job to fail regardless of > the result. They will still be run. > > +Each new flake entry must be associated with a link to the email reporting the > +bug to the author of the affected driver, the board name or Device Tree name of > +the board, the first kernel version affected, and an approximation of the > +failure rate. > + > +They should be provided under the following format:: > + > + # Bug Report: $LORE_OR_PATCHWORK_URL > + # Board Name: broken-board.dtb > + # Version: 6.6-rc1 > + # Failure Rate: 100 > + flaky-test > + > drivers/gpu/drm/ci/${DRIVER_NAME}-${HW_REVISION}-skips.txt > ----------------------------------------------------------- >
On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: > Flaky tests can be very difficult to reproduce after the facts, which > will make it even harder to ever fix. > > Let's document the metadata we agreed on to provide more context to > anyone trying to address these fixes. > > > [...] Applied to drm/drm-misc (drm-misc-next). Thanks! Maxime
On 26/10/2023 07:58, Maxime Ripard wrote: > On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: >> Flaky tests can be very difficult to reproduce after the facts, which >> will make it even harder to ever fix. >> >> Let's document the metadata we agreed on to provide more context to >> anyone trying to address these fixes. >> >> >> [...] > > Applied to drm/drm-misc (drm-misc-next). Thanks! Could you also apply https://patchwork.kernel.org/project/dri-devel/cover/20231024004525.169002-1-helen.koike@collabora.com/ (and the dependencies listed on it). Drm/ci in drm-misc is broken atm, there are some people asking me how to run it, and this unbreaks it. Thanks again Helen > > Thanks! > Maxime >
On 26/10/2023 09:01, Helen Koike wrote: > > > On 26/10/2023 07:58, Maxime Ripard wrote: >> On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: >>> Flaky tests can be very difficult to reproduce after the facts, which >>> will make it even harder to ever fix. >>> >>> Let's document the metadata we agreed on to provide more context to >>> anyone trying to address these fixes. >>> >>> >>> [...] >> >> Applied to drm/drm-misc (drm-misc-next). > > Thanks! > > Could you also apply > https://patchwork.kernel.org/project/dri-devel/cover/20231024004525.169002-1-helen.koike@collabora.com/ (and the dependencies listed on it). For some reason, commit message 7/10 (drm/ci: increase i915 job timeout to 1h30m) looks incomplete in patchwork, but it looks fine in my branch: https://gitlab.freedesktop.org/helen.fornazier/linux/-/commits/for-drm-misc-wip/ Let me know if you prefer that I send it again or if you could pull from the branch. Thanks Helen > > Drm/ci in drm-misc is broken atm, there are some people asking me how to > run it, and this unbreaks it. > > > Thanks again > Helen > >> >> Thanks! >> Maxime >>
On Thu, Oct 26, 2023 at 09:08:03AM -0300, Helen Koike wrote: > > > On 26/10/2023 09:01, Helen Koike wrote: > > > > > > On 26/10/2023 07:58, Maxime Ripard wrote: > > > On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: > > > > Flaky tests can be very difficult to reproduce after the facts, which > > > > will make it even harder to ever fix. > > > > > > > > Let's document the metadata we agreed on to provide more context to > > > > anyone trying to address these fixes. > > > > > > > > > > > > [...] > > > > > > Applied to drm/drm-misc (drm-misc-next). > > > > Thanks! > > > > Could you also apply https://patchwork.kernel.org/project/dri-devel/cover/20231024004525.169002-1-helen.koike@collabora.com/ > > (and the dependencies listed on it). > > For some reason, commit message 7/10 (drm/ci: increase i915 job timeout to > 1h30m) looks incomplete in patchwork, but it looks fine in my branch: > > https://gitlab.freedesktop.org/helen.fornazier/linux/-/commits/for-drm-misc-wip/ > > Let me know if you prefer that I send it again or if you could pull from the > branch. It was fine on lore.kernel.org and that's where I'm pulling from, so it all worked out :) Everything you asked for should be applied now Maxime
On 26/10/2023 10:27, Maxime Ripard wrote: > On Thu, Oct 26, 2023 at 09:08:03AM -0300, Helen Koike wrote: >> >> >> On 26/10/2023 09:01, Helen Koike wrote: >>> >>> >>> On 26/10/2023 07:58, Maxime Ripard wrote: >>>> On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: >>>>> Flaky tests can be very difficult to reproduce after the facts, which >>>>> will make it even harder to ever fix. >>>>> >>>>> Let's document the metadata we agreed on to provide more context to >>>>> anyone trying to address these fixes. >>>>> >>>>> >>>>> [...] >>>> >>>> Applied to drm/drm-misc (drm-misc-next). >>> >>> Thanks! >>> >>> Could you also apply https://patchwork.kernel.org/project/dri-devel/cover/20231024004525.169002-1-helen.koike@collabora.com/ >>> (and the dependencies listed on it). >> >> For some reason, commit message 7/10 (drm/ci: increase i915 job timeout to >> 1h30m) looks incomplete in patchwork, but it looks fine in my branch: >> >> https://gitlab.freedesktop.org/helen.fornazier/linux/-/commits/for-drm-misc-wip/ >> >> Let me know if you prefer that I send it again or if you could pull from the >> branch. > > It was fine on lore.kernel.org and that's where I'm pulling from, so it all worked out :) > > Everything you asked for should be applied now > > Maxime Awesome, thank you! Sorry, just another request, could you please pull this other one updating MAINTAINERS? https://patchwork.kernel.org/project/linux-arm-msm/patch/20230919182249.153499-1-helen.koike@collabora.com/ So get_maintainers.pl returns the right people and we don't forget to notify maintainers when flakes/fails/skips are updated. Thank again. Helen
Hi, On Thu, Oct 26, 2023 at 11:27:18AM -0300, Helen Koike wrote: > On 26/10/2023 10:27, Maxime Ripard wrote: > > On Thu, Oct 26, 2023 at 09:08:03AM -0300, Helen Koike wrote: > > > > > > > > > On 26/10/2023 09:01, Helen Koike wrote: > > > > > > > > > > > > On 26/10/2023 07:58, Maxime Ripard wrote: > > > > > On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: > > > > > > Flaky tests can be very difficult to reproduce after the facts, which > > > > > > will make it even harder to ever fix. > > > > > > > > > > > > Let's document the metadata we agreed on to provide more context to > > > > > > anyone trying to address these fixes. > > > > > > > > > > > > > > > > > > [...] > > > > > > > > > > Applied to drm/drm-misc (drm-misc-next). > > > > > > > > Thanks! > > > > > > > > Could you also apply https://patchwork.kernel.org/project/dri-devel/cover/20231024004525.169002-1-helen.koike@collabora.com/ > > > > (and the dependencies listed on it). > > > > > > For some reason, commit message 7/10 (drm/ci: increase i915 job timeout to > > > 1h30m) looks incomplete in patchwork, but it looks fine in my branch: > > > > > > https://gitlab.freedesktop.org/helen.fornazier/linux/-/commits/for-drm-misc-wip/ > > > > > > Let me know if you prefer that I send it again or if you could pull from the > > > branch. > > > > It was fine on lore.kernel.org and that's where I'm pulling from, so it all worked out :) > > > > Everything you asked for should be applied now > > > > Maxime > > Awesome, thank you! > > Sorry, just another request, could you please pull this other one updating > MAINTAINERS? > > https://patchwork.kernel.org/project/linux-arm-msm/patch/20230919182249.153499-1-helen.koike@collabora.com/ I don't mind, but the expectation (the one I had at least) was that you would do it :) If you don't have drm-misc access, please create an account, you have done way more than expected to get one already Maxime
On 26/10/2023 12:31, Maxime Ripard wrote: > Hi, > > On Thu, Oct 26, 2023 at 11:27:18AM -0300, Helen Koike wrote: >> On 26/10/2023 10:27, Maxime Ripard wrote: >>> On Thu, Oct 26, 2023 at 09:08:03AM -0300, Helen Koike wrote: >>>> >>>> >>>> On 26/10/2023 09:01, Helen Koike wrote: >>>>> >>>>> >>>>> On 26/10/2023 07:58, Maxime Ripard wrote: >>>>>> On Wed, 25 Oct 2023 16:24:41 +0200, Maxime Ripard wrote: >>>>>>> Flaky tests can be very difficult to reproduce after the facts, which >>>>>>> will make it even harder to ever fix. >>>>>>> >>>>>>> Let's document the metadata we agreed on to provide more context to >>>>>>> anyone trying to address these fixes. >>>>>>> >>>>>>> >>>>>>> [...] >>>>>> >>>>>> Applied to drm/drm-misc (drm-misc-next). >>>>> >>>>> Thanks! >>>>> >>>>> Could you also apply https://patchwork.kernel.org/project/dri-devel/cover/20231024004525.169002-1-helen.koike@collabora.com/ >>>>> (and the dependencies listed on it). >>>> >>>> For some reason, commit message 7/10 (drm/ci: increase i915 job timeout to >>>> 1h30m) looks incomplete in patchwork, but it looks fine in my branch: >>>> >>>> https://gitlab.freedesktop.org/helen.fornazier/linux/-/commits/for-drm-misc-wip/ >>>> >>>> Let me know if you prefer that I send it again or if you could pull from the >>>> branch. >>> >>> It was fine on lore.kernel.org and that's where I'm pulling from, so it all worked out :) >>> >>> Everything you asked for should be applied now >>> >>> Maxime >> >> Awesome, thank you! >> >> Sorry, just another request, could you please pull this other one updating >> MAINTAINERS? >> >> https://patchwork.kernel.org/project/linux-arm-msm/patch/20230919182249.153499-1-helen.koike@collabora.com/ > > I don't mind, but the expectation (the one I had at least) was that you would do it :) > > If you don't have drm-misc access, please create an account, you have > done way more than expected to get one already Nice! I just created this request https://gitlab.freedesktop.org/freedesktop/freedesktop/-/issues/983 Would you mind approving it? Thanks! Helen > > Maxime
diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst index 469b6fb65c30..792428a2bfdc 100644 --- a/Documentation/gpu/automated_testing.rst +++ b/Documentation/gpu/automated_testing.rst @@ -67,6 +67,19 @@ Lists the tests that for a given driver on a specific hardware revision are known to behave unreliably. These tests won't cause a job to fail regardless of the result. They will still be run. +Each new flake entry must be associated with a link to the email reporting the +bug to the author of the affected driver, the board name or Device Tree name of +the board, the first kernel version affected, and an approximation of the +failure rate. + +They should be provided under the following format:: + + # Bug Report: $LORE_OR_PATCHWORK_URL + # Board Name: broken-board.dtb + # Version: 6.6-rc1 + # Failure Rate: 100 + flaky-test + drivers/gpu/drm/ci/${DRIVER_NAME}-${HW_REVISION}-skips.txt -----------------------------------------------------------