| Message ID | 20210906213904.27918-1-uma.shankar@intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add Support for Plane Color Lut and CSC features | expand |
On Tue, 7 Sep 2021 03:08:42 +0530 Uma Shankar <uma.shankar@intel.com> wrote: > This is how a typical display color hardware pipeline looks like: > +-------------------------------------------+ > | RAM | > | +------+ +---------+ +---------+ | > | | FB 1 | | FB 2 | | FB N | | > | +------+ +---------+ +---------+ | > +-------------------------------------------+ > | Plane Color Hardware Block | > +--------------------------------------------+ > | +---v-----+ +---v-------+ +---v------+ | > | | Plane A | | Plane B | | Plane N | | > | | DeGamma | | Degamma | | Degamma | | > | +---+-----+ +---+-------+ +---+------+ | > | | | | | > | +---v-----+ +---v-------+ +---v------+ | > | |Plane A | | Plane B | | Plane N | | > | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > | +---+-----+ +----+------+ +----+-----+ | > | | | | | > | +---v-----+ +----v------+ +----v-----+ | > | | Plane A | | Plane B | | Plane N | | > | | Gamma | | Gamma | | Gamma | | > | +---+-----+ +----+------+ +----+-----+ | > | | | | | > +--------------------------------------------+ > +------v--------------v---------------v-------| > || || > || Pipe Blender || > +--------------------+------------------------+ > | | | > | +-----------v----------+ | > | | Pipe DeGamma | | > | | | | > | +-----------+----------+ | > | | Pipe Color | > | +-----------v----------+ Hardware | > | | Pipe CSC/CTM | | > | | | | > | +-----------+----------+ | > | | | > | +-----------v----------+ | > | | Pipe Gamma | | > | | | | > | +-----------+----------+ | > | | | > +---------------------------------------------+ > | > v > Pipe Output > > This patch series adds properties for plane color features. It adds > properties for degamma used to linearize data and CSC used for gamut > conversion. It also includes Gamma support used to again non-linearize > data as per panel supported color space. These can be utilize by user > space to convert planes from one format to another, one color space to > another etc. > > Userspace can take smart blending decisions and utilize these hardware > supported plane color features to get accurate color profile. The same > can help in consistent color quality from source to panel taking > advantage of advanced color features in hardware. > > These patches add the property interfaces and enable helper functions. > This series adds Intel's XE_LPD hw specific plane gamma feature. We > can build up and add other platform/hardware specific implementation > on top of this series. > > Credits: Special mention and credits to Ville Syrjala for coming up > with a design for this feature and inputs. This series is based on > his original design and idea. > > Note: Userspace support for this new UAPI will be done on Chrome in > alignment with weston and general opensource community. > Discussion ongoing with Harry Wentland, Pekka and community on color > pipeline and UAPI design. Harry's RFC below: > https://patchwork.freedesktop.org/series/89506/ > We need to converge on a common UAPI interface which caters to > all the modern color hardware pipelines. > > ToDo: State readout for this feature will be added next. > > v2: Added UAPI description and added change in the rfc section of > kernel Documentation folder Hi, thank you for this. I do believe the KMS UAPI should expose what hardware can do (prescribed operations) rather than how they would be often used (to realize a conversion from one space description to another). This proposal fits quite nicely with what I have envisioned for Weston. I mainly went over the big picture by commenting in detail on the proposal document, and not looking too carefully at the other documentation or UAPI details at this time. Unfortunately I was unable to decipher how userspace is supposed to use the XE_LPD special gamma features. Thanks, pq > > Uma Shankar (22): > drm: RFC for Plane Color Hardware Pipeline > drm: Add Enhanced Gamma and color lut range attributes > drm: Add Plane Degamma Mode property > drm: Add Plane Degamma Lut property > drm/i915/xelpd: Define Degamma Lut range struct for HDR planes > drm/i915/xelpd: Add register definitions for Plane Degamma > drm/i915/xelpd: Enable plane color features > drm/i915/xelpd: Add color capabilities of SDR planes > drm/i915/xelpd: Program Plane Degamma Registers > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl > drm/i915/xelpd: Initialize plane color features > drm/i915/xelpd: Load plane color luts from atomic flip > drm: Add Plane CTM property > drm: Add helper to attach Plane ctm property > drm/i915/xelpd: Define Plane CSC Registers > drm/i915/xelpd: Enable Plane CSC > drm: Add Plane Gamma Mode property > drm: Add Plane Gamma Lut property > drm/i915/xelpd: Define and Initialize Plane Gamma Lut range > drm/i915/xelpd: Add register definitions for Plane Gamma > drm/i915/xelpd: Program Plane Gamma Registers > drm/i915/xelpd: Enable plane gamma > > Documentation/gpu/drm-kms.rst | 90 +++ > Documentation/gpu/rfc/drm_color_pipeline.rst | 167 ++++++ > drivers/gpu/drm/drm_atomic.c | 1 + > drivers/gpu/drm/drm_atomic_state_helper.c | 12 + > drivers/gpu/drm/drm_atomic_uapi.c | 38 ++ > drivers/gpu/drm/drm_color_mgmt.c | 177 +++++- > .../gpu/drm/i915/display/intel_atomic_plane.c | 6 + > .../gpu/drm/i915/display/intel_atomic_plane.h | 2 + > drivers/gpu/drm/i915/display/intel_color.c | 513 ++++++++++++++++++ > drivers/gpu/drm/i915/display/intel_color.h | 2 + > .../drm/i915/display/skl_universal_plane.c | 15 +- > drivers/gpu/drm/i915/i915_drv.h | 3 + > drivers/gpu/drm/i915/i915_reg.h | 176 +++++- > include/drm/drm_mode_object.h | 2 +- > include/drm/drm_plane.h | 81 +++ > include/uapi/drm/drm_mode.h | 58 ++ > 16 files changed, 1337 insertions(+), 6 deletions(-) > create mode 100644 Documentation/gpu/rfc/drm_color_pipeline.rst >
> -----Original Message----- > From: Pekka Paalanen <ppaalanen@gmail.com> > Sent: Tuesday, October 12, 2021 5:25 PM > To: Shankar, Uma <uma.shankar@intel.com> > Cc: intel-gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; > harry.wentland@amd.com; ville.syrjala@linux.intel.com; brian.starkey@arm.com; > sebastian@sebastianwick.net; Shashank.Sharma@amd.com > Subject: Re: [RFC v2 00/22] Add Support for Plane Color Lut and CSC features > > On Tue, 7 Sep 2021 03:08:42 +0530 > Uma Shankar <uma.shankar@intel.com> wrote: > > > This is how a typical display color hardware pipeline looks like: > > +-------------------------------------------+ > > | RAM | > > | +------+ +---------+ +---------+ | > > | | FB 1 | | FB 2 | | FB N | | > > | +------+ +---------+ +---------+ | > > +-------------------------------------------+ > > | Plane Color Hardware Block | > > +--------------------------------------------+ > > | +---v-----+ +---v-------+ +---v------+ | > > | | Plane A | | Plane B | | Plane N | | > > | | DeGamma | | Degamma | | Degamma | | > > | +---+-----+ +---+-------+ +---+------+ | > > | | | | | > > | +---v-----+ +---v-------+ +---v------+ | > > | |Plane A | | Plane B | | Plane N | | > > | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > > | +---+-----+ +----+------+ +----+-----+ | > > | | | | | > > | +---v-----+ +----v------+ +----v-----+ | > > | | Plane A | | Plane B | | Plane N | | > > | | Gamma | | Gamma | | Gamma | | > > | +---+-----+ +----+------+ +----+-----+ | > > | | | | | > > +--------------------------------------------+ > > +------v--------------v---------------v-------| > > || || > > || Pipe Blender || > > +--------------------+------------------------+ > > | | | > > | +-----------v----------+ | > > | | Pipe DeGamma | | > > | | | | > > | +-----------+----------+ | > > | | Pipe Color | > > | +-----------v----------+ Hardware | > > | | Pipe CSC/CTM | | > > | | | | > > | +-----------+----------+ | > > | | | > > | +-----------v----------+ | > > | | Pipe Gamma | | > > | | | | > > | +-----------+----------+ | > > | | | > > +---------------------------------------------+ > > | > > v > > Pipe Output > > > > This patch series adds properties for plane color features. It adds > > properties for degamma used to linearize data and CSC used for gamut > > conversion. It also includes Gamma support used to again non-linearize > > data as per panel supported color space. These can be utilize by user > > space to convert planes from one format to another, one color space to > > another etc. > > > > Userspace can take smart blending decisions and utilize these hardware > > supported plane color features to get accurate color profile. The same > > can help in consistent color quality from source to panel taking > > advantage of advanced color features in hardware. > > > > These patches add the property interfaces and enable helper functions. > > This series adds Intel's XE_LPD hw specific plane gamma feature. We > > can build up and add other platform/hardware specific implementation > > on top of this series. > > > > Credits: Special mention and credits to Ville Syrjala for coming up > > with a design for this feature and inputs. This series is based on his > > original design and idea. > > > > Note: Userspace support for this new UAPI will be done on Chrome in > > alignment with weston and general opensource community. > > Discussion ongoing with Harry Wentland, Pekka and community on color > > pipeline and UAPI design. Harry's RFC below: > > https://patchwork.freedesktop.org/series/89506/ > > We need to converge on a common UAPI interface which caters to all the > > modern color hardware pipelines. > > > > ToDo: State readout for this feature will be added next. > > > > v2: Added UAPI description and added change in the rfc section of > > kernel Documentation folder > > Hi, > > thank you for this. I do believe the KMS UAPI should expose what hardware can do > (prescribed operations) rather than how they would be often used (to realize a > conversion from one space description to another). This proposal fits quite nicely > with what I have envisioned for Weston. > I mainly went over the big picture by commenting in detail on the proposal > document, and not looking too carefully at the other documentation or UAPI details > at this time. Thanks Pekka for the feedback. > Unfortunately I was unable to decipher how userspace is supposed to use the > XE_LPD special gamma features. I will include the details on how userspace should actually get this through a sample IGT reference, that should help make this clear. Regards, Uma Shankar > > Thanks, > pq > > > > > Uma Shankar (22): > > drm: RFC for Plane Color Hardware Pipeline > > drm: Add Enhanced Gamma and color lut range attributes > > drm: Add Plane Degamma Mode property > > drm: Add Plane Degamma Lut property > > drm/i915/xelpd: Define Degamma Lut range struct for HDR planes > > drm/i915/xelpd: Add register definitions for Plane Degamma > > drm/i915/xelpd: Enable plane color features > > drm/i915/xelpd: Add color capabilities of SDR planes > > drm/i915/xelpd: Program Plane Degamma Registers > > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl > > drm/i915/xelpd: Initialize plane color features > > drm/i915/xelpd: Load plane color luts from atomic flip > > drm: Add Plane CTM property > > drm: Add helper to attach Plane ctm property > > drm/i915/xelpd: Define Plane CSC Registers > > drm/i915/xelpd: Enable Plane CSC > > drm: Add Plane Gamma Mode property > > drm: Add Plane Gamma Lut property > > drm/i915/xelpd: Define and Initialize Plane Gamma Lut range > > drm/i915/xelpd: Add register definitions for Plane Gamma > > drm/i915/xelpd: Program Plane Gamma Registers > > drm/i915/xelpd: Enable plane gamma > > > > Documentation/gpu/drm-kms.rst | 90 +++ > > Documentation/gpu/rfc/drm_color_pipeline.rst | 167 ++++++ > > drivers/gpu/drm/drm_atomic.c | 1 + > > drivers/gpu/drm/drm_atomic_state_helper.c | 12 + > > drivers/gpu/drm/drm_atomic_uapi.c | 38 ++ > > drivers/gpu/drm/drm_color_mgmt.c | 177 +++++- > > .../gpu/drm/i915/display/intel_atomic_plane.c | 6 + > > .../gpu/drm/i915/display/intel_atomic_plane.h | 2 + > > drivers/gpu/drm/i915/display/intel_color.c | 513 ++++++++++++++++++ > > drivers/gpu/drm/i915/display/intel_color.h | 2 + > > .../drm/i915/display/skl_universal_plane.c | 15 +- > > drivers/gpu/drm/i915/i915_drv.h | 3 + > > drivers/gpu/drm/i915/i915_reg.h | 176 +++++- > > include/drm/drm_mode_object.h | 2 +- > > include/drm/drm_plane.h | 81 +++ > > include/uapi/drm/drm_mode.h | 58 ++ > > 16 files changed, 1337 insertions(+), 6 deletions(-) create mode > > 100644 Documentation/gpu/rfc/drm_color_pipeline.rst > >
On 2021-10-12 17:01, Shankar, Uma wrote: > > >> -----Original Message----- >> From: Pekka Paalanen <ppaalanen@gmail.com> >> Sent: Tuesday, October 12, 2021 5:25 PM >> To: Shankar, Uma <uma.shankar@intel.com> >> Cc: intel-gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; >> harry.wentland@amd.com; ville.syrjala@linux.intel.com; brian.starkey@arm.com; >> sebastian@sebastianwick.net; Shashank.Sharma@amd.com >> Subject: Re: [RFC v2 00/22] Add Support for Plane Color Lut and CSC features >> >> On Tue, 7 Sep 2021 03:08:42 +0530 >> Uma Shankar <uma.shankar@intel.com> wrote: >> >>> This is how a typical display color hardware pipeline looks like: >>> +-------------------------------------------+ >>> | RAM | >>> | +------+ +---------+ +---------+ | >>> | | FB 1 | | FB 2 | | FB N | | >>> | +------+ +---------+ +---------+ | >>> +-------------------------------------------+ >>> | Plane Color Hardware Block | >>> +--------------------------------------------+ >>> | +---v-----+ +---v-------+ +---v------+ | >>> | | Plane A | | Plane B | | Plane N | | >>> | | DeGamma | | Degamma | | Degamma | | >>> | +---+-----+ +---+-------+ +---+------+ | >>> | | | | | >>> | +---v-----+ +---v-------+ +---v------+ | >>> | |Plane A | | Plane B | | Plane N | | >>> | |CSC/CTM | | CSC/CTM | | CSC/CTM | | >>> | +---+-----+ +----+------+ +----+-----+ | >>> | | | | | >>> | +---v-----+ +----v------+ +----v-----+ | >>> | | Plane A | | Plane B | | Plane N | | >>> | | Gamma | | Gamma | | Gamma | | >>> | +---+-----+ +----+------+ +----+-----+ | >>> | | | | | >>> +--------------------------------------------+ >>> +------v--------------v---------------v-------| >>> || || >>> || Pipe Blender || >>> +--------------------+------------------------+ >>> | | | >>> | +-----------v----------+ | >>> | | Pipe DeGamma | | >>> | | | | >>> | +-----------+----------+ | >>> | | Pipe Color | >>> | +-----------v----------+ Hardware | >>> | | Pipe CSC/CTM | | >>> | | | | >>> | +-----------+----------+ | >>> | | | >>> | +-----------v----------+ | >>> | | Pipe Gamma | | >>> | | | | >>> | +-----------+----------+ | >>> | | | >>> +---------------------------------------------+ >>> | >>> v >>> Pipe Output >>> >>> This patch series adds properties for plane color features. It adds >>> properties for degamma used to linearize data and CSC used for gamut >>> conversion. It also includes Gamma support used to again non-linearize >>> data as per panel supported color space. These can be utilize by user >>> space to convert planes from one format to another, one color space to >>> another etc. >>> >>> Userspace can take smart blending decisions and utilize these hardware >>> supported plane color features to get accurate color profile. The same >>> can help in consistent color quality from source to panel taking >>> advantage of advanced color features in hardware. >>> >>> These patches add the property interfaces and enable helper functions. >>> This series adds Intel's XE_LPD hw specific plane gamma feature. We >>> can build up and add other platform/hardware specific implementation >>> on top of this series. >>> >>> Credits: Special mention and credits to Ville Syrjala for coming up >>> with a design for this feature and inputs. This series is based on his >>> original design and idea. >>> >>> Note: Userspace support for this new UAPI will be done on Chrome in >>> alignment with weston and general opensource community. >>> Discussion ongoing with Harry Wentland, Pekka and community on color >>> pipeline and UAPI design. Harry's RFC below: >>> https://patchwork.freedesktop.org/series/89506/>>>> We need to converge on a common UAPI interface which caters to all the >>> modern color hardware pipelines. >>> >>> ToDo: State readout for this feature will be added next. >>> >>> v2: Added UAPI description and added change in the rfc section of >>> kernel Documentation folder >> >> Hi, >> >> thank you for this. I do believe the KMS UAPI should expose what hardware can do >> (prescribed operations) rather than how they would be often used (to realize a >> conversion from one space description to another). This proposal fits quite nicely >> with what I have envisioned for Weston. > It's taken me a while but I am starting to agree with the prescriptive approach to expose HW functionality. One thing we'll want to be careful of is to make sure this isn't tied to specific HW more than it needs to be. I'll comment in other places of this patchset to elaborate. What's making me come around, i.e. change from a prescriptive (these are the input/output/blending spaces/formats) to a descriptive (these are the LUTs and CTMs) approach? 1) The prescriptive way has no good way of dealing with gamut and tone mapping. To do so we would need explicit OOTFs and CTMs or 3D LUTs anyways. 2) The prescriptive way provides no semblance of guarantee that transforms are equivalent when the compositor uses shaders transforms and composition vs when the compositor uses KMS transforms and composition. 3) Policy about treatment of surfaces/planes and blending is best left with the compositor (for the above reasons). >> I mainly went over the big picture by commenting in detail on the proposal >> document, and not looking too carefully at the other documentation or UAPI details >> at this time. > > Thanks Pekka for the feedback. > >> Unfortunately I was unable to decipher how userspace is supposed to use the >> XE_LPD special gamma features. > > I will include the details on how userspace should actually get this through a sample > IGT reference, that should help make this clear. > It looks like with your current definitions each userspace compositor (Weston, kwin, mutter, wlroots, Chrome's compositor, Android's compositor, etc.) would need to learn how to program the XE_LPD LUTs as well as AMD LUTs. Would these definitions change in future Intel HW generations? Would this mean all compositors would need to learn again how to program the future LUT format? Other options would be to give userspace a generic LUT with 4k FP16 entries and then re-map that to the HW LUT in the kernel driver. I might be missing some of the nuances of the XE_LPD LUT but it seems to me that the main difference between different PWL implementations is the distribution of the points used to define the LUT. Maybe a more generic PWL implementation could have a kernel driver report one (or more) PWL point distributions. We could encode these as enums and pre-defined arrays in a UAPI header. That way the compositor could have a single, generic implementation of programming PWL in FP16 and the kernel driver would only need to remap the FP16 to the HW-internal format, which is a trivial conversion. Using this approach compositors would implement PWL support once and won't have to touch it again in the future. Is there anything that would make this approach a bad idea for Intel HW (or other HW)? (Credit for this idea goes to Vitaly) Harry > Regards, > Uma Shankar > >> >> Thanks, >> pq >> >>> >>> Uma Shankar (22): >>> drm: RFC for Plane Color Hardware Pipeline >>> drm: Add Enhanced Gamma and color lut range attributes >>> drm: Add Plane Degamma Mode property >>> drm: Add Plane Degamma Lut property >>> drm/i915/xelpd: Define Degamma Lut range struct for HDR planes >>> drm/i915/xelpd: Add register definitions for Plane Degamma >>> drm/i915/xelpd: Enable plane color features >>> drm/i915/xelpd: Add color capabilities of SDR planes >>> drm/i915/xelpd: Program Plane Degamma Registers >>> drm/i915/xelpd: Add plane color check to glk_plane_color_ctl >>> drm/i915/xelpd: Initialize plane color features >>> drm/i915/xelpd: Load plane color luts from atomic flip >>> drm: Add Plane CTM property >>> drm: Add helper to attach Plane ctm property >>> drm/i915/xelpd: Define Plane CSC Registers >>> drm/i915/xelpd: Enable Plane CSC >>> drm: Add Plane Gamma Mode property >>> drm: Add Plane Gamma Lut property >>> drm/i915/xelpd: Define and Initialize Plane Gamma Lut range >>> drm/i915/xelpd: Add register definitions for Plane Gamma >>> drm/i915/xelpd: Program Plane Gamma Registers >>> drm/i915/xelpd: Enable plane gamma >>> >>> Documentation/gpu/drm-kms.rst | 90 +++ >>> Documentation/gpu/rfc/drm_color_pipeline.rst | 167 ++++++ >>> drivers/gpu/drm/drm_atomic.c | 1 + >>> drivers/gpu/drm/drm_atomic_state_helper.c | 12 + >>> drivers/gpu/drm/drm_atomic_uapi.c | 38 ++ >>> drivers/gpu/drm/drm_color_mgmt.c | 177 +++++- >>> .../gpu/drm/i915/display/intel_atomic_plane.c | 6 + >>> .../gpu/drm/i915/display/intel_atomic_plane.h | 2 + >>> drivers/gpu/drm/i915/display/intel_color.c | 513 ++++++++++++++++++ >>> drivers/gpu/drm/i915/display/intel_color.h | 2 + >>> .../drm/i915/display/skl_universal_plane.c | 15 +- >>> drivers/gpu/drm/i915/i915_drv.h | 3 + >>> drivers/gpu/drm/i915/i915_reg.h | 176 +++++- >>> include/drm/drm_mode_object.h | 2 +- >>> include/drm/drm_plane.h | 81 +++ >>> include/uapi/drm/drm_mode.h | 58 ++ >>> 16 files changed, 1337 insertions(+), 6 deletions(-) create mode >>> 100644 Documentation/gpu/rfc/drm_color_pipeline.rst >>> >
On Tue, 26 Oct 2021 11:02:31 -0400 Harry Wentland <harry.wentland@amd.com> wrote: > On 2021-10-12 17:01, Shankar, Uma wrote: > > > > > >> -----Original Message----- > >> From: Pekka Paalanen <ppaalanen@gmail.com> > >> Sent: Tuesday, October 12, 2021 5:25 PM > >> To: Shankar, Uma <uma.shankar@intel.com> > >> Cc: intel-gfx@lists.freedesktop.org; dri-devel@lists.freedesktop.org; > >> harry.wentland@amd.com; ville.syrjala@linux.intel.com; brian.starkey@arm.com; > >> sebastian@sebastianwick.net; Shashank.Sharma@amd.com > >> Subject: Re: [RFC v2 00/22] Add Support for Plane Color Lut and CSC features > >> > >> On Tue, 7 Sep 2021 03:08:42 +0530 > >> Uma Shankar <uma.shankar@intel.com> wrote: > >> > >>> This is how a typical display color hardware pipeline looks like: > >>> +-------------------------------------------+ > >>> | RAM | > >>> | +------+ +---------+ +---------+ | > >>> | | FB 1 | | FB 2 | | FB N | | > >>> | +------+ +---------+ +---------+ | > >>> +-------------------------------------------+ > >>> | Plane Color Hardware Block | > >>> +--------------------------------------------+ > >>> | +---v-----+ +---v-------+ +---v------+ | > >>> | | Plane A | | Plane B | | Plane N | | > >>> | | DeGamma | | Degamma | | Degamma | | > >>> | +---+-----+ +---+-------+ +---+------+ | > >>> | | | | | > >>> | +---v-----+ +---v-------+ +---v------+ | > >>> | |Plane A | | Plane B | | Plane N | | > >>> | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > >>> | +---+-----+ +----+------+ +----+-----+ | > >>> | | | | | > >>> | +---v-----+ +----v------+ +----v-----+ | > >>> | | Plane A | | Plane B | | Plane N | | > >>> | | Gamma | | Gamma | | Gamma | | > >>> | +---+-----+ +----+------+ +----+-----+ | > >>> | | | | | > >>> +--------------------------------------------+ > >>> +------v--------------v---------------v-------| > >>> || || > >>> || Pipe Blender || > >>> +--------------------+------------------------+ > >>> | | | > >>> | +-----------v----------+ | > >>> | | Pipe DeGamma | | > >>> | | | | > >>> | +-----------+----------+ | > >>> | | Pipe Color | > >>> | +-----------v----------+ Hardware | > >>> | | Pipe CSC/CTM | | > >>> | | | | > >>> | +-----------+----------+ | > >>> | | | > >>> | +-----------v----------+ | > >>> | | Pipe Gamma | | > >>> | | | | > >>> | +-----------+----------+ | > >>> | | | > >>> +---------------------------------------------+ > >>> | > >>> v > >>> Pipe Output > >>> > >>> This patch series adds properties for plane color features. It adds > >>> properties for degamma used to linearize data and CSC used for gamut > >>> conversion. It also includes Gamma support used to again non-linearize > >>> data as per panel supported color space. These can be utilize by user > >>> space to convert planes from one format to another, one color space to > >>> another etc. > >>> > >>> Userspace can take smart blending decisions and utilize these hardware > >>> supported plane color features to get accurate color profile. The same > >>> can help in consistent color quality from source to panel taking > >>> advantage of advanced color features in hardware. > >>> > >>> These patches add the property interfaces and enable helper functions. > >>> This series adds Intel's XE_LPD hw specific plane gamma feature. We > >>> can build up and add other platform/hardware specific implementation > >>> on top of this series. > >>> > >>> Credits: Special mention and credits to Ville Syrjala for coming up > >>> with a design for this feature and inputs. This series is based on his > >>> original design and idea. > >>> > >>> Note: Userspace support for this new UAPI will be done on Chrome in > >>> alignment with weston and general opensource community. > >>> Discussion ongoing with Harry Wentland, Pekka and community on color > >>> pipeline and UAPI design. Harry's RFC below: > >>> https://patchwork.freedesktop.org/series/89506/>>>> We need to converge on a common UAPI interface which caters to all the > >>> modern color hardware pipelines. > >>> > >>> ToDo: State readout for this feature will be added next. > >>> > >>> v2: Added UAPI description and added change in the rfc section of > >>> kernel Documentation folder > >> > >> Hi, > >> > >> thank you for this. I do believe the KMS UAPI should expose what hardware can do > >> (prescribed operations) rather than how they would be often used (to realize a > >> conversion from one space description to another). This proposal fits quite nicely > >> with what I have envisioned for Weston. > > > > It's taken me a while but I am starting to agree with the prescriptive approach to > expose HW functionality. One thing we'll want to be careful of is to make sure > this isn't tied to specific HW more than it needs to be. I'll comment in other > places of this patchset to elaborate. > > What's making me come around, i.e. change from a prescriptive (these are the > input/output/blending spaces/formats) to a descriptive (these are the LUTs and > CTMs) approach? > > 1) The prescriptive way has no good way of dealing with gamut and tone mapping. > To do so we would need explicit OOTFs and CTMs or 3D LUTs anyways. > > 2) The prescriptive way provides no semblance of guarantee that transforms > are equivalent when the compositor uses shaders transforms and composition > vs when the compositor uses KMS transforms and composition. > > 3) Policy about treatment of surfaces/planes and blending is best left with > the compositor (for the above reasons). Hi Harry, I think we might have confusion there about prescriptive vs. descriptive, but I understand what you mean, and yes, those are my points too. > >> I mainly went over the big picture by commenting in detail on the proposal > >> document, and not looking too carefully at the other documentation or UAPI details > >> at this time. > > > > Thanks Pekka for the feedback. > > > >> Unfortunately I was unable to decipher how userspace is supposed to use the > >> XE_LPD special gamma features. > > > > I will include the details on how userspace should actually get this through a sample > > IGT reference, that should help make this clear. > > > > It looks like with your current definitions each userspace compositor (Weston, kwin, > mutter, wlroots, Chrome's compositor, Android's compositor, etc.) would need to learn > how to program the XE_LPD LUTs as well as AMD LUTs. Would these definitions change > in future Intel HW generations? Would this mean all compositors would need to learn > again how to program the future LUT format? > > Other options would be to give userspace a generic LUT with 4k FP16 entries and then > re-map that to the HW LUT in the kernel driver. Do you want to run the optimisation algorithm to map the generic LUT to hardware specific optimised representation in the kernel driver? Wouldn't that be too much? To me this seems like the perfect starting point for a libcamera equivalent to KMS. Feed the generic LUT to the lib, and get whatever the hardware-driver-specific KMS UAPI needs. Perhaps even with an error measure, so that a compositor can evaluate whether the mapping is accurate enough or if it needs to fall back. > I might be missing some of the nuances of the XE_LPD LUT but it seems to me that the > main difference between different PWL implementations is the distribution of the > points used to define the LUT. Maybe a more generic PWL implementation could have > a kernel driver report one (or more) PWL point distributions. We could encode these > as enums and pre-defined arrays in a UAPI header. That way the compositor could have > a single, generic implementation of programming PWL in FP16 and the kernel driver > would only need to remap the FP16 to the HW-internal format, which is a trivial > conversion. Using this approach compositors would implement PWL support once and won't > have to touch it again in the future. Is there anything that would make this approach > a bad idea for Intel HW (or other HW)? (Credit for this idea goes to Vitaly) I think that is also a workable solution. If that happens, it seems likely that a library for that optimisation will appear. It has just one shortcoming: the compositor has no idea of the precision as there is no way to get an error measure for how much the FP16->internal conversion loses precision. But maybe that is never significant? Or could we have some precision guarantees otherwise? I'm thinking of hard-core professional color users who wouldn't normally trust off-loading to KMS at all, at least not on-demand off-loading where you are never sure if you are looking at SW or KMS composition. Thanks, pq
On 2021-09-06 17:38, Uma Shankar wrote: > This is how a typical display color hardware pipeline looks like: > +-------------------------------------------+ > | RAM | > | +------+ +---------+ +---------+ | > | | FB 1 | | FB 2 | | FB N | | > | +------+ +---------+ +---------+ | > +-------------------------------------------+ > | Plane Color Hardware Block | > +--------------------------------------------+ > | +---v-----+ +---v-------+ +---v------+ | > | | Plane A | | Plane B | | Plane N | | > | | DeGamma | | Degamma | | Degamma | | > | +---+-----+ +---+-------+ +---+------+ | > | | | | | > | +---v-----+ +---v-------+ +---v------+ | > | |Plane A | | Plane B | | Plane N | | > | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > | +---+-----+ +----+------+ +----+-----+ | > | | | | | > | +---v-----+ +----v------+ +----v-----+ | > | | Plane A | | Plane B | | Plane N | | > | | Gamma | | Gamma | | Gamma | | > | +---+-----+ +----+------+ +----+-----+ | We've had a number of discussions on naming for these properties but I don't think we've arrived at a consensus. It has come up repeatedly, though, that gamma/degamma might be misleading terms. I've opened a ticket on gitlab to help track this item and would like it if we could discuss the merits of different naming schemes over there: https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/7 Uma, I tried to tag you but don't see you on gitlab.freedesktop.org. Harry > | | | | | > +--------------------------------------------+ > +------v--------------v---------------v-------| > || || > || Pipe Blender || > +--------------------+------------------------+ > | | | > | +-----------v----------+ | > | | Pipe DeGamma | | > | | | | > | +-----------+----------+ | > | | Pipe Color | > | +-----------v----------+ Hardware | > | | Pipe CSC/CTM | | > | | | | > | +-----------+----------+ | > | | | > | +-----------v----------+ | > | | Pipe Gamma | | > | | | | > | +-----------+----------+ | > | | | > +---------------------------------------------+ > | > v > Pipe Output > > This patch series adds properties for plane color features. It adds > properties for degamma used to linearize data and CSC used for gamut > conversion. It also includes Gamma support used to again non-linearize > data as per panel supported color space. These can be utilize by user > space to convert planes from one format to another, one color space to > another etc. > > Userspace can take smart blending decisions and utilize these hardware > supported plane color features to get accurate color profile. The same > can help in consistent color quality from source to panel taking > advantage of advanced color features in hardware. > > These patches add the property interfaces and enable helper functions. > This series adds Intel's XE_LPD hw specific plane gamma feature. We > can build up and add other platform/hardware specific implementation > on top of this series. > > Credits: Special mention and credits to Ville Syrjala for coming up > with a design for this feature and inputs. This series is based on > his original design and idea. > > Note: Userspace support for this new UAPI will be done on Chrome in > alignment with weston and general opensource community. > Discussion ongoing with Harry Wentland, Pekka and community on color > pipeline and UAPI design. Harry's RFC below: > https://patchwork.freedesktop.org/series/89506/ > We need to converge on a common UAPI interface which caters to > all the modern color hardware pipelines. > > ToDo: State readout for this feature will be added next. > > v2: Added UAPI description and added change in the rfc section of > kernel Documentation folder > > Uma Shankar (22): > drm: RFC for Plane Color Hardware Pipeline > drm: Add Enhanced Gamma and color lut range attributes > drm: Add Plane Degamma Mode property > drm: Add Plane Degamma Lut property > drm/i915/xelpd: Define Degamma Lut range struct for HDR planes > drm/i915/xelpd: Add register definitions for Plane Degamma > drm/i915/xelpd: Enable plane color features > drm/i915/xelpd: Add color capabilities of SDR planes > drm/i915/xelpd: Program Plane Degamma Registers > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl > drm/i915/xelpd: Initialize plane color features > drm/i915/xelpd: Load plane color luts from atomic flip > drm: Add Plane CTM property > drm: Add helper to attach Plane ctm property > drm/i915/xelpd: Define Plane CSC Registers > drm/i915/xelpd: Enable Plane CSC > drm: Add Plane Gamma Mode property > drm: Add Plane Gamma Lut property > drm/i915/xelpd: Define and Initialize Plane Gamma Lut range > drm/i915/xelpd: Add register definitions for Plane Gamma > drm/i915/xelpd: Program Plane Gamma Registers > drm/i915/xelpd: Enable plane gamma > > Documentation/gpu/drm-kms.rst | 90 +++ > Documentation/gpu/rfc/drm_color_pipeline.rst | 167 ++++++ > drivers/gpu/drm/drm_atomic.c | 1 + > drivers/gpu/drm/drm_atomic_state_helper.c | 12 + > drivers/gpu/drm/drm_atomic_uapi.c | 38 ++ > drivers/gpu/drm/drm_color_mgmt.c | 177 +++++- > .../gpu/drm/i915/display/intel_atomic_plane.c | 6 + > .../gpu/drm/i915/display/intel_atomic_plane.h | 2 + > drivers/gpu/drm/i915/display/intel_color.c | 513 ++++++++++++++++++ > drivers/gpu/drm/i915/display/intel_color.h | 2 + > .../drm/i915/display/skl_universal_plane.c | 15 +- > drivers/gpu/drm/i915/i915_drv.h | 3 + > drivers/gpu/drm/i915/i915_reg.h | 176 +++++- > include/drm/drm_mode_object.h | 2 +- > include/drm/drm_plane.h | 81 +++ > include/uapi/drm/drm_mode.h | 58 ++ > 16 files changed, 1337 insertions(+), 6 deletions(-) > create mode 100644 Documentation/gpu/rfc/drm_color_pipeline.rst >
> -----Original Message----- > From: dri-devel <dri-devel-bounces@lists.freedesktop.org> On Behalf Of Harry > Wentland > Sent: Wednesday, February 2, 2022 9:42 PM > To: Shankar, Uma <uma.shankar@intel.com>; intel-gfx@lists.freedesktop.org; dri- > devel@lists.freedesktop.org > Cc: sebastian@sebastianwick.net; Shashank.Sharma@amd.com > Subject: Re: [RFC v2 00/22] Add Support for Plane Color Lut and CSC features > > > > On 2021-09-06 17:38, Uma Shankar wrote: > > This is how a typical display color hardware pipeline looks like: > > +-------------------------------------------+ > > | RAM | > > | +------+ +---------+ +---------+ | > > | | FB 1 | | FB 2 | | FB N | | > > | +------+ +---------+ +---------+ | > > +-------------------------------------------+ > > | Plane Color Hardware Block | > > +--------------------------------------------+ > > | +---v-----+ +---v-------+ +---v------+ | > > | | Plane A | | Plane B | | Plane N | | > > | | DeGamma | | Degamma | | Degamma | | > > | +---+-----+ +---+-------+ +---+------+ | > > | | | | | > > | +---v-----+ +---v-------+ +---v------+ | > > | |Plane A | | Plane B | | Plane N | | > > | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > > | +---+-----+ +----+------+ +----+-----+ | > > | | | | | > > | +---v-----+ +----v------+ +----v-----+ | > > | | Plane A | | Plane B | | Plane N | | > > | | Gamma | | Gamma | | Gamma | | > > | +---+-----+ +----+------+ +----+-----+ | > > We've had a number of discussions on naming for these properties but I don't think > we've arrived at a consensus. It has come up repeatedly, though, that > gamma/degamma might be misleading terms. > > I've opened a ticket on gitlab to help track this item and would like it if we could > discuss the merits of different naming schemes over > there: > > https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/7 > > Uma, I tried to tag you but don't see you on gitlab.freedesktop.org. Thanks Harry for creating the issue. I have replied there and we can discuss and finalize the UAPI. Regards, Uma Shankar > Harry > > > | | | | | > > +--------------------------------------------+ > > +------v--------------v---------------v-------| > > || || > > || Pipe Blender || > > +--------------------+------------------------+ > > | | | > > | +-----------v----------+ | > > | | Pipe DeGamma | | > > | | | | > > | +-----------+----------+ | > > | | Pipe Color | > > | +-----------v----------+ Hardware | > > | | Pipe CSC/CTM | | > > | | | | > > | +-----------+----------+ | > > | | | > > | +-----------v----------+ | > > | | Pipe Gamma | | > > | | | | > > | +-----------+----------+ | > > | | | > > +---------------------------------------------+ > > | > > v > > Pipe Output > > > > This patch series adds properties for plane color features. It adds > > properties for degamma used to linearize data and CSC used for gamut > > conversion. It also includes Gamma support used to again non-linearize > > data as per panel supported color space. These can be utilize by user > > space to convert planes from one format to another, one color space to > > another etc. > > > > Userspace can take smart blending decisions and utilize these hardware > > supported plane color features to get accurate color profile. The same > > can help in consistent color quality from source to panel taking > > advantage of advanced color features in hardware. > > > > These patches add the property interfaces and enable helper functions. > > This series adds Intel's XE_LPD hw specific plane gamma feature. We > > can build up and add other platform/hardware specific implementation > > on top of this series. > > > > Credits: Special mention and credits to Ville Syrjala for coming up > > with a design for this feature and inputs. This series is based on his > > original design and idea. > > > > Note: Userspace support for this new UAPI will be done on Chrome in > > alignment with weston and general opensource community. > > Discussion ongoing with Harry Wentland, Pekka and community on color > > pipeline and UAPI design. Harry's RFC below: > > https://patchwork.freedesktop.org/series/89506/ > > We need to converge on a common UAPI interface which caters to all the > > modern color hardware pipelines. > > > > ToDo: State readout for this feature will be added next. > > > > v2: Added UAPI description and added change in the rfc section of > > kernel Documentation folder > > > > Uma Shankar (22): > > drm: RFC for Plane Color Hardware Pipeline > > drm: Add Enhanced Gamma and color lut range attributes > > drm: Add Plane Degamma Mode property > > drm: Add Plane Degamma Lut property > > drm/i915/xelpd: Define Degamma Lut range struct for HDR planes > > drm/i915/xelpd: Add register definitions for Plane Degamma > > drm/i915/xelpd: Enable plane color features > > drm/i915/xelpd: Add color capabilities of SDR planes > > drm/i915/xelpd: Program Plane Degamma Registers > > drm/i915/xelpd: Add plane color check to glk_plane_color_ctl > > drm/i915/xelpd: Initialize plane color features > > drm/i915/xelpd: Load plane color luts from atomic flip > > drm: Add Plane CTM property > > drm: Add helper to attach Plane ctm property > > drm/i915/xelpd: Define Plane CSC Registers > > drm/i915/xelpd: Enable Plane CSC > > drm: Add Plane Gamma Mode property > > drm: Add Plane Gamma Lut property > > drm/i915/xelpd: Define and Initialize Plane Gamma Lut range > > drm/i915/xelpd: Add register definitions for Plane Gamma > > drm/i915/xelpd: Program Plane Gamma Registers > > drm/i915/xelpd: Enable plane gamma > > > > Documentation/gpu/drm-kms.rst | 90 +++ > > Documentation/gpu/rfc/drm_color_pipeline.rst | 167 ++++++ > > drivers/gpu/drm/drm_atomic.c | 1 + > > drivers/gpu/drm/drm_atomic_state_helper.c | 12 + > > drivers/gpu/drm/drm_atomic_uapi.c | 38 ++ > > drivers/gpu/drm/drm_color_mgmt.c | 177 +++++- > > .../gpu/drm/i915/display/intel_atomic_plane.c | 6 + > > .../gpu/drm/i915/display/intel_atomic_plane.h | 2 + > > drivers/gpu/drm/i915/display/intel_color.c | 513 ++++++++++++++++++ > > drivers/gpu/drm/i915/display/intel_color.h | 2 + > > .../drm/i915/display/skl_universal_plane.c | 15 +- > > drivers/gpu/drm/i915/i915_drv.h | 3 + > > drivers/gpu/drm/i915/i915_reg.h | 176 +++++- > > include/drm/drm_mode_object.h | 2 +- > > include/drm/drm_plane.h | 81 +++ > > include/uapi/drm/drm_mode.h | 58 ++ > > 16 files changed, 1337 insertions(+), 6 deletions(-) create mode > > 100644 Documentation/gpu/rfc/drm_color_pipeline.rst > >
This is how a typical display color hardware pipeline looks like: +-------------------------------------------+ | RAM | | +------+ +---------+ +---------+ | | | FB 1 | | FB 2 | | FB N | | | +------+ +---------+ +---------+ | +-------------------------------------------+ | Plane Color Hardware Block | +--------------------------------------------+ | +---v-----+ +---v-------+ +---v------+ | | | Plane A | | Plane B | | Plane N | | | | DeGamma | | Degamma | | Degamma | | | +---+-----+ +---+-------+ +---+------+ | | | | | | | +---v-----+ +---v-------+ +---v------+ | | |Plane A | | Plane B | | Plane N | | | |CSC/CTM | | CSC/CTM | | CSC/CTM | | | +---+-----+ +----+------+ +----+-----+ | | | | | | | +---v-----+ +----v------+ +----v-----+ | | | Plane A | | Plane B | | Plane N | | | | Gamma | | Gamma | | Gamma | | | +---+-----+ +----+------+ +----+-----+ | | | | | | +--------------------------------------------+ +------v--------------v---------------v-------| || || || Pipe Blender || +--------------------+------------------------+ | | | | +-----------v----------+ | | | Pipe DeGamma | | | | | | | +-----------+----------+ | | | Pipe Color | | +-----------v----------+ Hardware | | | Pipe CSC/CTM | | | | | | | +-----------+----------+ | | | | | +-----------v----------+ | | | Pipe Gamma | | | | | | | +-----------+----------+ | | | | +---------------------------------------------+ | v Pipe Output This patch series adds properties for plane color features. It adds properties for degamma used to linearize data and CSC used for gamut conversion. It also includes Gamma support used to again non-linearize data as per panel supported color space. These can be utilize by user space to convert planes from one format to another, one color space to another etc. Userspace can take smart blending decisions and utilize these hardware supported plane color features to get accurate color profile. The same can help in consistent color quality from source to panel taking advantage of advanced color features in hardware. These patches add the property interfaces and enable helper functions. This series adds Intel's XE_LPD hw specific plane gamma feature. We can build up and add other platform/hardware specific implementation on top of this series. Credits: Special mention and credits to Ville Syrjala for coming up with a design for this feature and inputs. This series is based on his original design and idea. Note: Userspace support for this new UAPI will be done on Chrome in alignment with weston and general opensource community. Discussion ongoing with Harry Wentland, Pekka and community on color pipeline and UAPI design. Harry's RFC below: https://patchwork.freedesktop.org/series/89506/ We need to converge on a common UAPI interface which caters to all the modern color hardware pipelines. ToDo: State readout for this feature will be added next. v2: Added UAPI description and added change in the rfc section of kernel Documentation folder Uma Shankar (22): drm: RFC for Plane Color Hardware Pipeline drm: Add Enhanced Gamma and color lut range attributes drm: Add Plane Degamma Mode property drm: Add Plane Degamma Lut property drm/i915/xelpd: Define Degamma Lut range struct for HDR planes drm/i915/xelpd: Add register definitions for Plane Degamma drm/i915/xelpd: Enable plane color features drm/i915/xelpd: Add color capabilities of SDR planes drm/i915/xelpd: Program Plane Degamma Registers drm/i915/xelpd: Add plane color check to glk_plane_color_ctl drm/i915/xelpd: Initialize plane color features drm/i915/xelpd: Load plane color luts from atomic flip drm: Add Plane CTM property drm: Add helper to attach Plane ctm property drm/i915/xelpd: Define Plane CSC Registers drm/i915/xelpd: Enable Plane CSC drm: Add Plane Gamma Mode property drm: Add Plane Gamma Lut property drm/i915/xelpd: Define and Initialize Plane Gamma Lut range drm/i915/xelpd: Add register definitions for Plane Gamma drm/i915/xelpd: Program Plane Gamma Registers drm/i915/xelpd: Enable plane gamma Documentation/gpu/drm-kms.rst | 90 +++ Documentation/gpu/rfc/drm_color_pipeline.rst | 167 ++++++ drivers/gpu/drm/drm_atomic.c | 1 + drivers/gpu/drm/drm_atomic_state_helper.c | 12 + drivers/gpu/drm/drm_atomic_uapi.c | 38 ++ drivers/gpu/drm/drm_color_mgmt.c | 177 +++++- .../gpu/drm/i915/display/intel_atomic_plane.c | 6 + .../gpu/drm/i915/display/intel_atomic_plane.h | 2 + drivers/gpu/drm/i915/display/intel_color.c | 513 ++++++++++++++++++ drivers/gpu/drm/i915/display/intel_color.h | 2 + .../drm/i915/display/skl_universal_plane.c | 15 +- drivers/gpu/drm/i915/i915_drv.h | 3 + drivers/gpu/drm/i915/i915_reg.h | 176 +++++- include/drm/drm_mode_object.h | 2 +- include/drm/drm_plane.h | 81 +++ include/uapi/drm/drm_mode.h | 58 ++ 16 files changed, 1337 insertions(+), 6 deletions(-) create mode 100644 Documentation/gpu/rfc/drm_color_pipeline.rst