drm/doc: Document uapi requirements in DRM

Commit Message

Daniel Vetter Aug. 22, 2016, 7:36 a.m. UTC

Everyone knows them, except all the new folks joining from the ARM
side haven't lived through all the pain of the past years and are
entirely surprised when I raise this. Definitely time to document
this.

Last time this was a big discussion was about 6 years ago, when qcom
tried to land a kernel driver without userspace. Dave Airlie made the
rules really clear:

http://airlied.livejournal.com/73115.html

This write-up here is essentially what I've put into a presentation a
while ago, which was also reviewed by Dave:

http://blog.ffwll.ch/2015/05/gfx-kernel-upstreaming-requirements.html

v2: Fix typos Eric&Rob spotted.

Cc: Dave Airlie <[e-mail]>
Cc: Oded Gabbay <[e-mail]>
Cc: Russell King <[e-mail]>
Cc: Tomi Valkeinen <[e-mail]>
Cc: Eric Anholt <[e-mail]>
Cc: Thomas Hellstrom <[e-mail]>
Cc: Sinclair Yeh <[e-mail]>
Cc: Lucas Stach <[e-mail]>
Cc: Benjamin Gaignard <[e-mail]>
Cc: Mark Yao <[e-mail]>
Cc: Laurent Pinchart <[e-mail]>
Cc: Ben Skeggs <[e-mail]>
Cc: Rob Clark <[e-mail]>
Cc: CK Hu <[e-mail]>
Cc: Xinliang Liu <[e-mail]>
Cc: Philipp Zabel <[e-mail]>
Cc: Stefan Agner <[e-mail]>
Cc: Inki Dae <[e-mail]>
Cc: Maxime Ripard  <[e-mail]>
Cc: Boris Brezillon <[e-mail]>
Cc: Jani Nikula <[e-mail]>
Cc: Daniel Vetter <[e-mail]>
Cc: Thierry Reding <[e-mail]>
Cc: Christian König <[e-mail]>
Cc: Alex Deucher <[e-mail]>
Cc: Gerd Hoffmann <[e-mail]>
Cc: Brian Starkey <[e-mail]>
Cc: Liviu Dudau <[e-mail]>
Cc: Alexey Brodkin <[e-mail]>
Acked-by: Dave Airlie <[e-mail]>
Reviewed-by: Rob Clark <[e-mail]>
Reviewed-by: Christian König <[e-mail]>
Reviewed-by: Eric Anholt <[e-mail]>
Signed-off-by: Daniel Vetter <[e-mail]>
---
 Documentation/gpu/drm-uapi.rst | 67 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

Comments

Jani Nikula Aug. 22, 2016, 8:10 a.m. UTC | #1

On Mon, 22 Aug 2016, Daniel Vetter <[e-mail]> wrote:
> Everyone knows them, except all the new folks joining from the ARM
> side haven't lived through all the pain of the past years and are
> entirely surprised when I raise this. Definitely time to document
> this.
>
> Last time this was a big discussion was about 6 years ago, when qcom
> tried to land a kernel driver without userspace. Dave Airlie made the
> rules really clear:
>
> http://airlied.livejournal.com/73115.html
>
> This write-up here is essentially what I've put into a presentation a
> while ago, which was also reviewed by Dave:
>
> http://blog.ffwll.ch/2015/05/gfx-kernel-upstreaming-requirements.html
>
> v2: Fix typos Eric&Rob spotted.
>
> Cc: Dave Airlie <[e-mail]>
> Cc: Oded Gabbay <[e-mail]>
> Cc: Russell King <[e-mail]>
> Cc: Tomi Valkeinen <[e-mail]>
> Cc: Eric Anholt <[e-mail]>
> Cc: Thomas Hellstrom <[e-mail]>
> Cc: Sinclair Yeh <[e-mail]>
> Cc: Lucas Stach <[e-mail]>
> Cc: Benjamin Gaignard <[e-mail]>
> Cc: Mark Yao <[e-mail]>
> Cc: Laurent Pinchart <[e-mail]>
> Cc: Ben Skeggs <[e-mail]>
> Cc: Rob Clark <[e-mail]>
> Cc: CK Hu <[e-mail]>
> Cc: Xinliang Liu <[e-mail]>
> Cc: Philipp Zabel <[e-mail]>
> Cc: Stefan Agner <[e-mail]>
> Cc: Inki Dae <[e-mail]>
> Cc: Maxime Ripard  <[e-mail]>
> Cc: Boris Brezillon <[e-mail]>
> Cc: Jani Nikula <[e-mail]>
> Cc: Daniel Vetter <[e-mail]>
> Cc: Thierry Reding <[e-mail]>
> Cc: Christian König <[e-mail]>
> Cc: Alex Deucher <[e-mail]>
> Cc: Gerd Hoffmann <[e-mail]>
> Cc: Brian Starkey <[e-mail]>
> Cc: Liviu Dudau <[e-mail]>
> Cc: Alexey Brodkin <[e-mail]>
> Acked-by: Dave Airlie <[e-mail]>
> Reviewed-by: Rob Clark <[e-mail]>
> Reviewed-by: Christian König <[e-mail]>
> Reviewed-by: Eric Anholt <[e-mail]>
> Signed-off-by: Daniel Vetter <[e-mail]>
> ---
>  Documentation/gpu/drm-uapi.rst | 67 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 67 insertions(+)
>
> diff --git a/Documentation/gpu/drm-uapi.rst b/Documentation/gpu/drm-uapi.rst
> index 94876938aef3..747b51f8c422 100644
> --- a/Documentation/gpu/drm-uapi.rst
> +++ b/Documentation/gpu/drm-uapi.rst
> @@ -36,6 +36,73 @@ Primary Nodes, DRM Master and Authentication
>  Open-Source Userspace Requirements
>  ==================================
>  
> +The DRM subsystem has stricter requirements on what the userspace side for new
> +uAPI needs to look like. This section here explains what exactly those
> +requirements are, and why they exist.

Nitpick, stricter requirements than what? I know what you mean, but the
comparative begs for the thing to compare against.

Otherwise,

Reviewed-by: Jani Nikula <[e-mail]>

> +
> +The short summary is that any addition of DRM uAPI requires corresponding
> +open-sourced userspace patches, and those patches must be reviewed and ready for
> +merging into a suitable and canonical upstream project.
> +
> +GFX devices (both display and render/GPU side) are really complex bits of
> +hardware, with userspace and kernel by necessity having to work together really
> +closely.  The interfaces, for rendering and modesetting, must be extremely wide
> +and flexible, and therefore it is almost always impossible to precisely define
> +them for every possible corner case. This in turn makes it really practically
> +infeasible to differentiate between behaviour that's required by userspace, and
> +which must not be changed to avoid regressions, and behaviour which is only an
> +accidental artifact of the current implementation.
> +
> +Without access to the full source code of all userspace users that means it
> +becomes impossible to change the implementation details, since userspace could
> +depend upon the accidental behaviour of the current implementation in minute
> +details. And debugging such regressions without access to source code is pretty
> +much impossible. As a consequence this means:
> +
> +- The Linux kernel's "no regression" policy holds in practice only for
> +  open-source userspace of the DRM subsystem. DRM developers are perfectly fine
> +  if closed-source blob drivers in userspace use the same uAPI as the open
> +  drivers, but they must do so in the exact same way as the open drivers.
> +  Creative (ab)use of the interfaces will, and in the past routinely has, lead
> +  to breakage.
> +
> +- Any new userspace interface must have an open-source implementation as
> +  demonstration vehicle.
> +
> +The other reason for requiring open-source userspace is uAPI review. Since the
> +kernel and userspace parts of a GFX stack must work together so closely, code
> +review can only assess whether a new interface achieves its goals by looking at
> +both sides. Making sure that the interface indeed covers the use-case fully
> +leads to a few additional requirements:
> +
> +- The open-source userspace must not be a toy/test application, but the real
> +  thing. Specifically it needs to handle all the usual error and corner cases.
> +  These are often the places where new uAPI falls apart and hence essential to
> +  assess the fitness of a proposed interface.
> +
> +- The userspace side must be fully reviewed and tested to the standards of that
> +  userspace project. For e.g. mesa this means piglit testcases and review on the
> +  mailing list. This is again to ensure that the new interface actually gets the
> +  job done.
> +
> +- The userspace patches must be against the canonical upstream, not some vendor
> +  fork. This is to make sure that no one cheats on the review and testing
> +  requirements by doing a quick fork.
> +
> +- The kernel patch can only be merged after all the above requirements are met,
> +  but it **must** be merged **before** the userspace patches land. uAPI always flows
> +  from the kernel, doing things the other way round risks divergence of the uAPI
> +  definitions and header files.
> +
> +These are fairly steep requirements, but have grown out from years of shared
> +pain and experience with uAPI added hastily, and almost always regretted about
> +just as fast. GFX devices change really fast, requiring a paradigm shift and
> +entire new set of uAPI interfaces every few years at least. Together with the
> +Linux kernel's guarantee to keep existing userspace running for 10+ years this
> +is already rather painful for the DRM subsystem, with multiple different uAPIs
> +for the same thing co-existing. If we add a few more complete mistakes into the
> +mix every year it would be entirely unmanageable.
> +
>  Render nodes
>  ============

Patch

diff --git a/Documentation/gpu/drm-uapi.rst b/Documentation/gpu/drm-uapi.rst
index 94876938aef3..747b51f8c422 100644
--- a/Documentation/gpu/drm-uapi.rst
+++ b/Documentation/gpu/drm-uapi.rst
@@ -36,6 +36,73 @@  Primary Nodes, DRM Master and Authentication
 Open-Source Userspace Requirements
 ==================================
 
+The DRM subsystem has stricter requirements on what the userspace side for new
+uAPI needs to look like. This section here explains what exactly those
+requirements are, and why they exist.
+
+The short summary is that any addition of DRM uAPI requires corresponding
+open-sourced userspace patches, and those patches must be reviewed and ready for
+merging into a suitable and canonical upstream project.
+
+GFX devices (both display and render/GPU side) are really complex bits of
+hardware, with userspace and kernel by necessity having to work together really
+closely.  The interfaces, for rendering and modesetting, must be extremely wide
+and flexible, and therefore it is almost always impossible to precisely define
+them for every possible corner case. This in turn makes it really practically
+infeasible to differentiate between behaviour that's required by userspace, and
+which must not be changed to avoid regressions, and behaviour which is only an
+accidental artifact of the current implementation.
+
+Without access to the full source code of all userspace users that means it
+becomes impossible to change the implementation details, since userspace could
+depend upon the accidental behaviour of the current implementation in minute
+details. And debugging such regressions without access to source code is pretty
+much impossible. As a consequence this means:
+
+- The Linux kernel's "no regression" policy holds in practice only for
+  open-source userspace of the DRM subsystem. DRM developers are perfectly fine
+  if closed-source blob drivers in userspace use the same uAPI as the open
+  drivers, but they must do so in the exact same way as the open drivers.
+  Creative (ab)use of the interfaces will, and in the past routinely has, lead
+  to breakage.
+
+- Any new userspace interface must have an open-source implementation as
+  demonstration vehicle.
+
+The other reason for requiring open-source userspace is uAPI review. Since the
+kernel and userspace parts of a GFX stack must work together so closely, code
+review can only assess whether a new interface achieves its goals by looking at
+both sides. Making sure that the interface indeed covers the use-case fully
+leads to a few additional requirements:
+
+- The open-source userspace must not be a toy/test application, but the real
+  thing. Specifically it needs to handle all the usual error and corner cases.
+  These are often the places where new uAPI falls apart and hence essential to
+  assess the fitness of a proposed interface.
+
+- The userspace side must be fully reviewed and tested to the standards of that
+  userspace project. For e.g. mesa this means piglit testcases and review on the
+  mailing list. This is again to ensure that the new interface actually gets the
+  job done.
+
+- The userspace patches must be against the canonical upstream, not some vendor
+  fork. This is to make sure that no one cheats on the review and testing
+  requirements by doing a quick fork.
+
+- The kernel patch can only be merged after all the above requirements are met,
+  but it **must** be merged **before** the userspace patches land. uAPI always flows
+  from the kernel, doing things the other way round risks divergence of the uAPI
+  definitions and header files.
+
+These are fairly steep requirements, but have grown out from years of shared
+pain and experience with uAPI added hastily, and almost always regretted about
+just as fast. GFX devices change really fast, requiring a paradigm shift and
+entire new set of uAPI interfaces every few years at least. Together with the
+Linux kernel's guarantee to keep existing userspace running for 10+ years this
+is already rather painful for the DRM subsystem, with multiple different uAPIs
+for the same thing co-existing. If we add a few more complete mistakes into the
+mix every year it would be entirely unmanageable.
+
 Render nodes
 ============