diff mbox series

[3/5] drm: fix drm_mode_addfb() on big endian machines.

Message ID 20180903105756.24912-4-kraxel@redhat.com (mailing list archive)
State New, archived
Headers show
Series drm: byteorder fixes | expand

Commit Message

Gerd Hoffmann Sept. 3, 2018, 10:57 a.m. UTC
Userspace on big endian machhines typically expects the ADDFB ioctl
returns a big endian framebuffer.  drm_mode_addfb() will call
drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
values though, which is wrong.  This patch fixes that.

Drivers (both kernel and xorg) have quirks in place to deal with the
broken drm_mode_addfb() behavior.  Because of this we can't just change
drm_mode_addfb() behavior for everybody without breaking things.  So add
a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
opt-in.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 include/drm/drm_drv.h             |  1 +
 drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
 2 files changed, 12 insertions(+)

Comments

Daniel Vetter Sept. 3, 2018, 4:45 p.m. UTC | #1
On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
> Userspace on big endian machhines typically expects the ADDFB ioctl
> returns a big endian framebuffer.  drm_mode_addfb() will call
> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
> values though, which is wrong.  This patch fixes that.
> 
> Drivers (both kernel and xorg) have quirks in place to deal with the
> broken drm_mode_addfb() behavior.  Because of this we can't just change
> drm_mode_addfb() behavior for everybody without breaking things.  So add
> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
> opt-in.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  include/drm/drm_drv.h             |  1 +
>  drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
>  2 files changed, 12 insertions(+)
> 
> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
> index 46a8009784..9cf12596cd 100644
> --- a/include/drm/drm_drv.h
> +++ b/include/drm/drm_drv.h
> @@ -57,6 +57,7 @@ struct drm_printer;
>  #define DRIVER_KMS_LEGACY_CONTEXT	0x20000
>  #define DRIVER_SYNCOBJ                  0x40000
>  #define DRIVER_PREFER_XBGR_30BPP        0x80000
> +#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000

Hm, not a huge fan of using driver_flags for random little quirks. I think
a boolean in sturct drm_mode_config would be much better. Bonus if you
also move the 30bpp hack over to that. Something like
mode_config.quirk_addfb_host_byte_order and
mode_config.quirk_addfb_prefer_xbgr_30bpp or whatever. That has the upside
of giving us a really nice place to put a huge comment about what this is
supposed to do.

I think otherwise this looks overall rather reasonable. I think the only
other driver that ever cared about big endian was radeon/amdgpu. Would be
good to get at least an ack from amd folks, or a "meh, stopped caring".
-Daniel

>  
>  /**
>   * struct drm_driver - DRM driver structure
> diff --git a/drivers/gpu/drm/drm_framebuffer.c b/drivers/gpu/drm/drm_framebuffer.c
> index 88758096d5..ccbda8a2e9 100644
> --- a/drivers/gpu/drm/drm_framebuffer.c
> +++ b/drivers/gpu/drm/drm_framebuffer.c
> @@ -124,6 +124,17 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or,
>  	    dev->driver->driver_features & DRIVER_PREFER_XBGR_30BPP)
>  		r.pixel_format = DRM_FORMAT_XBGR2101010;
>  
> +	if (dev->driver->driver_features & DRIVER_PREFER_HOST_BYTE_ORDER) {
> +		if (r.pixel_format == DRM_FORMAT_XRGB8888)
> +			r.pixel_format = DRM_FORMAT_HOST_XRGB8888;
> +		if (r.pixel_format == DRM_FORMAT_ARGB8888)
> +			r.pixel_format = DRM_FORMAT_HOST_ARGB8888;
> +		if (r.pixel_format == DRM_FORMAT_RGB565)
> +			r.pixel_format = DRM_FORMAT_HOST_RGB565;
> +		if (r.pixel_format == DRM_FORMAT_XRGB1555)
> +			r.pixel_format = DRM_FORMAT_HOST_XRGB1555;
> +	}
> +
>  	ret = drm_mode_addfb2(dev, &r, file_priv);
>  	if (ret)
>  		return ret;
> -- 
> 2.9.3
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
Michel Dänzer Sept. 3, 2018, 5:01 p.m. UTC | #2
On 2018-09-03 6:45 p.m., Daniel Vetter wrote:
> On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
>> Userspace on big endian machhines typically expects the ADDFB ioctl
>> returns a big endian framebuffer.  drm_mode_addfb() will call
>> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
>> values though, which is wrong.  This patch fixes that.
>>
>> Drivers (both kernel and xorg) have quirks in place to deal with the
>> broken drm_mode_addfb() behavior.  Because of this we can't just change
>> drm_mode_addfb() behavior for everybody without breaking things.  So add
>> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
>> opt-in.

Since the changes are opt-in now, they shouldn't affect drivers which
don't opt in; they should work as well (or as badly :) after these
changes as they did before. So no concerns from my side anymore.
Ilia Mirkin Sept. 3, 2018, 5:07 p.m. UTC | #3
On Mon, Sep 3, 2018 at 12:45 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
>> Userspace on big endian machhines typically expects the ADDFB ioctl
>> returns a big endian framebuffer.  drm_mode_addfb() will call
>> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
>> values though, which is wrong.  This patch fixes that.
>>
>> Drivers (both kernel and xorg) have quirks in place to deal with the
>> broken drm_mode_addfb() behavior.  Because of this we can't just change
>> drm_mode_addfb() behavior for everybody without breaking things.  So add
>> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
>> opt-in.
>>
>> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>> ---
>>  include/drm/drm_drv.h             |  1 +
>>  drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
>>  2 files changed, 12 insertions(+)
>>
>> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
>> index 46a8009784..9cf12596cd 100644
>> --- a/include/drm/drm_drv.h
>> +++ b/include/drm/drm_drv.h
>> @@ -57,6 +57,7 @@ struct drm_printer;
>>  #define DRIVER_KMS_LEGACY_CONTEXT    0x20000
>>  #define DRIVER_SYNCOBJ                  0x40000
>>  #define DRIVER_PREFER_XBGR_30BPP        0x80000
>> +#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000
>
> Hm, not a huge fan of using driver_flags for random little quirks. I think
> a boolean in sturct drm_mode_config would be much better. Bonus if you
> also move the 30bpp hack over to that. Something like
> mode_config.quirk_addfb_host_byte_order and
> mode_config.quirk_addfb_prefer_xbgr_30bpp or whatever. That has the upside
> of giving us a really nice place to put a huge comment about what this is
> supposed to do.
>
> I think otherwise this looks overall rather reasonable. I think the only
> other driver that ever cared about big endian was radeon/amdgpu. Would be
> good to get at least an ack from amd folks, or a "meh, stopped caring".

and nouveau.

I do believe that ADDFB should be made to always prefer host byte
order -- this is how all of the existing implementations work in
practice. However e.g. nouveau wants DRM_FORMAT_XRGB8888. But it still
treats it as host byte order. This will become more important in a
world where ADDFB2 is more common.

So, I think that this change should be applied, drivers (I suspect
just nouveau and radeon) fixed up to consume the "new" formats, and
then made to be the one-and-only way for ADDFB to function. That way
we'll no longer be lying about the DRM_FORMAT being used.

Cheers,

  -ilia
Michel Dänzer Sept. 4, 2018, 8 a.m. UTC | #4
On 2018-09-03 7:07 p.m., Ilia Mirkin wrote:
> On Mon, Sep 3, 2018 at 12:45 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
>> On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
>>> Userspace on big endian machhines typically expects the ADDFB ioctl
>>> returns a big endian framebuffer.  drm_mode_addfb() will call
>>> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
>>> values though, which is wrong.  This patch fixes that.
>>>
>>> Drivers (both kernel and xorg) have quirks in place to deal with the
>>> broken drm_mode_addfb() behavior.  Because of this we can't just change
>>> drm_mode_addfb() behavior for everybody without breaking things.  So add
>>> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
>>> opt-in.
>>>
>>> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>>> ---
>>>  include/drm/drm_drv.h             |  1 +
>>>  drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
>>>  2 files changed, 12 insertions(+)
>>>
>>> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
>>> index 46a8009784..9cf12596cd 100644
>>> --- a/include/drm/drm_drv.h
>>> +++ b/include/drm/drm_drv.h
>>> @@ -57,6 +57,7 @@ struct drm_printer;
>>>  #define DRIVER_KMS_LEGACY_CONTEXT    0x20000
>>>  #define DRIVER_SYNCOBJ                  0x40000
>>>  #define DRIVER_PREFER_XBGR_30BPP        0x80000
>>> +#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000
>>
>> Hm, not a huge fan of using driver_flags for random little quirks. I think
>> a boolean in sturct drm_mode_config would be much better. Bonus if you
>> also move the 30bpp hack over to that. Something like
>> mode_config.quirk_addfb_host_byte_order and
>> mode_config.quirk_addfb_prefer_xbgr_30bpp or whatever. That has the upside
>> of giving us a really nice place to put a huge comment about what this is
>> supposed to do.
>>
>> I think otherwise this looks overall rather reasonable. I think the only
>> other driver that ever cared about big endian was radeon/amdgpu. Would be
>> good to get at least an ack from amd folks, or a "meh, stopped caring".
> 
> and nouveau.
> 
> I do believe that ADDFB should be made to always prefer host byte
> order -- this is how all of the existing implementations work in
> practice. However e.g. nouveau wants DRM_FORMAT_XRGB8888. But it still
> treats it as host byte order. This will become more important in a
> world where ADDFB2 is more common.
> 
> So, I think that this change should be applied, drivers (I suspect
> just nouveau and radeon) fixed up to consume the "new" formats, [...]

As explained before, that would break radeon userspace on big endian hosts.
Ilia Mirkin Sept. 4, 2018, 1:05 p.m. UTC | #5
On Tue, Sep 4, 2018 at 4:00 AM, Michel Dänzer <michel@daenzer.net> wrote:
> On 2018-09-03 7:07 p.m., Ilia Mirkin wrote:
>> On Mon, Sep 3, 2018 at 12:45 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
>>> On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
>>>> Userspace on big endian machhines typically expects the ADDFB ioctl
>>>> returns a big endian framebuffer.  drm_mode_addfb() will call
>>>> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
>>>> values though, which is wrong.  This patch fixes that.
>>>>
>>>> Drivers (both kernel and xorg) have quirks in place to deal with the
>>>> broken drm_mode_addfb() behavior.  Because of this we can't just change
>>>> drm_mode_addfb() behavior for everybody without breaking things.  So add
>>>> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
>>>> opt-in.
>>>>
>>>> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>>>> ---
>>>>  include/drm/drm_drv.h             |  1 +
>>>>  drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
>>>>  2 files changed, 12 insertions(+)
>>>>
>>>> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
>>>> index 46a8009784..9cf12596cd 100644
>>>> --- a/include/drm/drm_drv.h
>>>> +++ b/include/drm/drm_drv.h
>>>> @@ -57,6 +57,7 @@ struct drm_printer;
>>>>  #define DRIVER_KMS_LEGACY_CONTEXT    0x20000
>>>>  #define DRIVER_SYNCOBJ                  0x40000
>>>>  #define DRIVER_PREFER_XBGR_30BPP        0x80000
>>>> +#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000
>>>
>>> Hm, not a huge fan of using driver_flags for random little quirks. I think
>>> a boolean in sturct drm_mode_config would be much better. Bonus if you
>>> also move the 30bpp hack over to that. Something like
>>> mode_config.quirk_addfb_host_byte_order and
>>> mode_config.quirk_addfb_prefer_xbgr_30bpp or whatever. That has the upside
>>> of giving us a really nice place to put a huge comment about what this is
>>> supposed to do.
>>>
>>> I think otherwise this looks overall rather reasonable. I think the only
>>> other driver that ever cared about big endian was radeon/amdgpu. Would be
>>> good to get at least an ack from amd folks, or a "meh, stopped caring".
>>
>> and nouveau.
>>
>> I do believe that ADDFB should be made to always prefer host byte
>> order -- this is how all of the existing implementations work in
>> practice. However e.g. nouveau wants DRM_FORMAT_XRGB8888. But it still
>> treats it as host byte order. This will become more important in a
>> world where ADDFB2 is more common.
>>
>> So, I think that this change should be applied, drivers (I suspect
>> just nouveau and radeon) fixed up to consume the "new" formats, [...]
>
> As explained before, that would break radeon userspace on big endian hosts.

We last discussed this about a year ago, so I hope you'll forgive my
lapse in memory...

There's userspace that uses ADDFB2 with DRM_FORMAT_XRGB8888 but
expects it to be host-endian? If so, that'll be a problem for nouveau
as well...

  -ilia
Michel Dänzer Sept. 4, 2018, 3:02 p.m. UTC | #6
On 2018-09-04 3:05 p.m., Ilia Mirkin wrote:
> On Tue, Sep 4, 2018 at 4:00 AM, Michel Dänzer <michel@daenzer.net> wrote:
>> On 2018-09-03 7:07 p.m., Ilia Mirkin wrote:
>>> On Mon, Sep 3, 2018 at 12:45 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
>>>> On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
>>>>> Userspace on big endian machhines typically expects the ADDFB ioctl
>>>>> returns a big endian framebuffer.  drm_mode_addfb() will call
>>>>> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
>>>>> values though, which is wrong.  This patch fixes that.
>>>>>
>>>>> Drivers (both kernel and xorg) have quirks in place to deal with the
>>>>> broken drm_mode_addfb() behavior.  Because of this we can't just change
>>>>> drm_mode_addfb() behavior for everybody without breaking things.  So add
>>>>> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
>>>>> opt-in.
>>>>>
>>>>> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>>>>> ---
>>>>>  include/drm/drm_drv.h             |  1 +
>>>>>  drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
>>>>>  2 files changed, 12 insertions(+)
>>>>>
>>>>> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
>>>>> index 46a8009784..9cf12596cd 100644
>>>>> --- a/include/drm/drm_drv.h
>>>>> +++ b/include/drm/drm_drv.h
>>>>> @@ -57,6 +57,7 @@ struct drm_printer;
>>>>>  #define DRIVER_KMS_LEGACY_CONTEXT    0x20000
>>>>>  #define DRIVER_SYNCOBJ                  0x40000
>>>>>  #define DRIVER_PREFER_XBGR_30BPP        0x80000
>>>>> +#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000
>>>>
>>>> Hm, not a huge fan of using driver_flags for random little quirks. I think
>>>> a boolean in sturct drm_mode_config would be much better. Bonus if you
>>>> also move the 30bpp hack over to that. Something like
>>>> mode_config.quirk_addfb_host_byte_order and
>>>> mode_config.quirk_addfb_prefer_xbgr_30bpp or whatever. That has the upside
>>>> of giving us a really nice place to put a huge comment about what this is
>>>> supposed to do.
>>>>
>>>> I think otherwise this looks overall rather reasonable. I think the only
>>>> other driver that ever cared about big endian was radeon/amdgpu. Would be
>>>> good to get at least an ack from amd folks, or a "meh, stopped caring".
>>>
>>> and nouveau.
>>>
>>> I do believe that ADDFB should be made to always prefer host byte
>>> order -- this is how all of the existing implementations work in
>>> practice. However e.g. nouveau wants DRM_FORMAT_XRGB8888. But it still
>>> treats it as host byte order. This will become more important in a
>>> world where ADDFB2 is more common.
>>>
>>> So, I think that this change should be applied, drivers (I suspect
>>> just nouveau and radeon) fixed up to consume the "new" formats, [...]
>>
>> As explained before, that would break radeon userspace on big endian hosts.
> 
> We last discussed this about a year ago, so I hope you'll forgive my
> lapse in memory...
> 
> There's userspace that uses ADDFB2 with DRM_FORMAT_XRGB8888 but
> expects it to be host-endian?

ADDFB, not ADDFB2. The latter probably didn't even exist yet when this
was made to work. :)
Ilia Mirkin Sept. 4, 2018, 3:15 p.m. UTC | #7
On Tue, Sep 4, 2018 at 11:02 AM, Michel Dänzer <michel@daenzer.net> wrote:
> On 2018-09-04 3:05 p.m., Ilia Mirkin wrote:
>> On Tue, Sep 4, 2018 at 4:00 AM, Michel Dänzer <michel@daenzer.net> wrote:
>>> On 2018-09-03 7:07 p.m., Ilia Mirkin wrote:
>>>> On Mon, Sep 3, 2018 at 12:45 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
>>>>> On Mon, Sep 03, 2018 at 12:57:54PM +0200, Gerd Hoffmann wrote:
>>>>>> Userspace on big endian machhines typically expects the ADDFB ioctl
>>>>>> returns a big endian framebuffer.  drm_mode_addfb() will call
>>>>>> drm_mode_addfb2() unconditionally with little endian DRM_FORMAT_*
>>>>>> values though, which is wrong.  This patch fixes that.
>>>>>>
>>>>>> Drivers (both kernel and xorg) have quirks in place to deal with the
>>>>>> broken drm_mode_addfb() behavior.  Because of this we can't just change
>>>>>> drm_mode_addfb() behavior for everybody without breaking things.  So add
>>>>>> a new driver feature flag DRIVER_PREFER_HOST_BYTE_ORDER, so drivers can
>>>>>> opt-in.
>>>>>>
>>>>>> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>>>>>> ---
>>>>>>  include/drm/drm_drv.h             |  1 +
>>>>>>  drivers/gpu/drm/drm_framebuffer.c | 11 +++++++++++
>>>>>>  2 files changed, 12 insertions(+)
>>>>>>
>>>>>> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
>>>>>> index 46a8009784..9cf12596cd 100644
>>>>>> --- a/include/drm/drm_drv.h
>>>>>> +++ b/include/drm/drm_drv.h
>>>>>> @@ -57,6 +57,7 @@ struct drm_printer;
>>>>>>  #define DRIVER_KMS_LEGACY_CONTEXT    0x20000
>>>>>>  #define DRIVER_SYNCOBJ                  0x40000
>>>>>>  #define DRIVER_PREFER_XBGR_30BPP        0x80000
>>>>>> +#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000
>>>>>
>>>>> Hm, not a huge fan of using driver_flags for random little quirks. I think
>>>>> a boolean in sturct drm_mode_config would be much better. Bonus if you
>>>>> also move the 30bpp hack over to that. Something like
>>>>> mode_config.quirk_addfb_host_byte_order and
>>>>> mode_config.quirk_addfb_prefer_xbgr_30bpp or whatever. That has the upside
>>>>> of giving us a really nice place to put a huge comment about what this is
>>>>> supposed to do.
>>>>>
>>>>> I think otherwise this looks overall rather reasonable. I think the only
>>>>> other driver that ever cared about big endian was radeon/amdgpu. Would be
>>>>> good to get at least an ack from amd folks, or a "meh, stopped caring".
>>>>
>>>> and nouveau.
>>>>
>>>> I do believe that ADDFB should be made to always prefer host byte
>>>> order -- this is how all of the existing implementations work in
>>>> practice. However e.g. nouveau wants DRM_FORMAT_XRGB8888. But it still
>>>> treats it as host byte order. This will become more important in a
>>>> world where ADDFB2 is more common.
>>>>
>>>> So, I think that this change should be applied, drivers (I suspect
>>>> just nouveau and radeon) fixed up to consume the "new" formats, [...]
>>>
>>> As explained before, that would break radeon userspace on big endian hosts.
>>
>> We last discussed this about a year ago, so I hope you'll forgive my
>> lapse in memory...
>>
>> There's userspace that uses ADDFB2 with DRM_FORMAT_XRGB8888 but
>> expects it to be host-endian?
>
> ADDFB, not ADDFB2. The latter probably didn't even exist yet when this
> was made to work. :)

Right, but ADDFB doesn't know or care about DRM_FORMAT_*. That's what
I'm saying -- keep ADDFB working, and fix up the DRM_FORMAT_*
underneath it both in the conversion and in the driver. Gerd's patch
allows us to do this incrementally, eventually truing up the
DRM_FORMAT_* in the driver, enabling ADDFB2 to work as expected.

  -ilia
Gerd Hoffmann Sept. 5, 2018, 6:10 a.m. UTC | #8
Hi,

> >>> As explained before, that would break radeon userspace on big endian hosts.
> >>
> >> We last discussed this about a year ago, so I hope you'll forgive my
> >> lapse in memory...
> >>
> >> There's userspace that uses ADDFB2 with DRM_FORMAT_XRGB8888 but
> >> expects it to be host-endian?
> >
> > ADDFB, not ADDFB2. The latter probably didn't even exist yet when this
> > was made to work. :)
> 
> Right, but ADDFB doesn't know or care about DRM_FORMAT_*. That's what
> I'm saying -- keep ADDFB working, and fix up the DRM_FORMAT_*
> underneath it both in the conversion and in the driver. Gerd's patch
> allows us to do this incrementally, eventually truing up the
> DRM_FORMAT_* in the driver, enabling ADDFB2 to work as expected.

If it is that simple then yes, we should be able to fix the radeon kms
driver, then drop the quirk once all kms drivers are fixed.

But IIRC there are some radeon-sepcific calls used by the radeon xorg
driver affected too (thats why the commit message says "... both xorg and
kernel drivers ..."), so fixing it for radeon isn't that easy ...

cheers,
  Gerd
diff mbox series

Patch

diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
index 46a8009784..9cf12596cd 100644
--- a/include/drm/drm_drv.h
+++ b/include/drm/drm_drv.h
@@ -57,6 +57,7 @@  struct drm_printer;
 #define DRIVER_KMS_LEGACY_CONTEXT	0x20000
 #define DRIVER_SYNCOBJ                  0x40000
 #define DRIVER_PREFER_XBGR_30BPP        0x80000
+#define DRIVER_PREFER_HOST_BYTE_ORDER   0x100000
 
 /**
  * struct drm_driver - DRM driver structure
diff --git a/drivers/gpu/drm/drm_framebuffer.c b/drivers/gpu/drm/drm_framebuffer.c
index 88758096d5..ccbda8a2e9 100644
--- a/drivers/gpu/drm/drm_framebuffer.c
+++ b/drivers/gpu/drm/drm_framebuffer.c
@@ -124,6 +124,17 @@  int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or,
 	    dev->driver->driver_features & DRIVER_PREFER_XBGR_30BPP)
 		r.pixel_format = DRM_FORMAT_XBGR2101010;
 
+	if (dev->driver->driver_features & DRIVER_PREFER_HOST_BYTE_ORDER) {
+		if (r.pixel_format == DRM_FORMAT_XRGB8888)
+			r.pixel_format = DRM_FORMAT_HOST_XRGB8888;
+		if (r.pixel_format == DRM_FORMAT_ARGB8888)
+			r.pixel_format = DRM_FORMAT_HOST_ARGB8888;
+		if (r.pixel_format == DRM_FORMAT_RGB565)
+			r.pixel_format = DRM_FORMAT_HOST_RGB565;
+		if (r.pixel_format == DRM_FORMAT_XRGB1555)
+			r.pixel_format = DRM_FORMAT_HOST_XRGB1555;
+	}
+
 	ret = drm_mode_addfb2(dev, &r, file_priv);
 	if (ret)
 		return ret;