mbox series

[00/13] media: rockchip: rga: add support for multi-planar formats

Message ID 20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de (mailing list archive)
Headers show
Series media: rockchip: rga: add support for multi-planar formats | expand

Message

Michael Tretter Sept. 14, 2023, 12:40 p.m. UTC
This series adds support for the V4L2 multi-planar API to the Rockchip
RGA driver. Once the RGA driver supports the multi-planar API, it is
easier to share buffers with other V4L2 drivers that also support the
multi-planar API and may not expose planar formats with contiguous
planes.

With non-contiguous planes, the U and V planes may not start at the same
offset as with the continuous planes. Therefore, the RGA driver cannot
rely on its calculation of the plane offsets based on the format and
frame size anymore, but must remember the offsets when it created the
mapping. Therefore, I also reworked how the DMA mapping is handled.

As a bonus, the RGA driver should now work correctly on devices with
more than 4 GB of memory. Video buffers should now be allocated within
the 4 GB boundary and an import of buffers that have higher addresses
into the driver should fail.

Patch 1 fixes the swizzling of RGA formats. While testing all formats of
the driver on rk3568, I discovered that the color channels of the RGB
formats are wrong when converting to NV12. I didn't test this on other
SoCs with an RGA and I am not sure, if they behave differently regarding
the color channels. Please report, if this breaks the color conversion
on other SoCs, and I will make this SoC-specific.

Patches 2 to 6 are the rework the DMA descriptor handling for the RGA
MMU. The patches clean up, how the driver uses the DMA API, and make the
creation of the descriptor list more explicit. Furthermore, the driver
is changed to keep the mapping per video buffer instead of using a
single mapping that is updated with every buffer.

Patches 7 to 11 prepare the driver for the multi-planar API including a
cleanup of the format handling in the buffer, and finally switch to the
multi-planar API.

Patch 12 updates the code that creates the DMA-descriptor mapping to
correctly handle buffers with multiple planes. The driver has to iterate
all planes and make them the continuous for the RGA.

Patch 13 enables the NV12M format, which is the multi-planar variant of
the NV12 format.

Michael

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
Michael Tretter (13):
      media: rockchip: rga: fix swizzling for RGB formats
      media: rockchip: rga: extract helper to fill descriptors
      media: rockchip: rga: allocate DMA descriptors per buffer
      media: rockchip: rga: split src and dst buffer setup
      media: rockchip: rga: pre-calculate plane offsets
      media: rockchip: rga: set dma mask to 32 bits
      media: rockchip: rga: use clamp() to clamp size to limits
      media: rockchip: rga: use pixelformat to find format
      media: rockchip: rga: add local variable for pix_format
      media: rockchip: rga: use macros for testing buffer type
      media: rockchip: rga: switch to multi-planar API
      media: rockchip: rga: rework buffer handling for multi-planar formats
      media: rockchip: rga: add NV12M support

 drivers/media/platform/rockchip/rga/rga-buf.c | 166 ++++++++++++++++-------
 drivers/media/platform/rockchip/rga/rga-hw.c  | 143 +++++++++++++-------
 drivers/media/platform/rockchip/rga/rga.c     | 187 ++++++++++++--------------
 drivers/media/platform/rockchip/rga/rga.h     |  35 ++++-
 4 files changed, 329 insertions(+), 202 deletions(-)
---
base-commit: 0bb80ecc33a8fb5a682236443c1e740d5c917d1d
change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93

Best regards,

Comments

Hans Verkuil Sept. 25, 2023, 7:45 a.m. UTC | #1
On 14/09/2023 14:40, Michael Tretter wrote:
> This series adds support for the V4L2 multi-planar API to the Rockchip
> RGA driver. Once the RGA driver supports the multi-planar API, it is
> easier to share buffers with other V4L2 drivers that also support the
> multi-planar API and may not expose planar formats with contiguous
> planes.
> 
> With non-contiguous planes, the U and V planes may not start at the same
> offset as with the continuous planes. Therefore, the RGA driver cannot
> rely on its calculation of the plane offsets based on the format and
> frame size anymore, but must remember the offsets when it created the
> mapping. Therefore, I also reworked how the DMA mapping is handled.
> 
> As a bonus, the RGA driver should now work correctly on devices with
> more than 4 GB of memory. Video buffers should now be allocated within
> the 4 GB boundary and an import of buffers that have higher addresses
> into the driver should fail.
> 
> Patch 1 fixes the swizzling of RGA formats. While testing all formats of
> the driver on rk3568, I discovered that the color channels of the RGB
> formats are wrong when converting to NV12. I didn't test this on other
> SoCs with an RGA and I am not sure, if they behave differently regarding
> the color channels. Please report, if this breaks the color conversion
> on other SoCs, and I will make this SoC-specific.
> 
> Patches 2 to 6 are the rework the DMA descriptor handling for the RGA
> MMU. The patches clean up, how the driver uses the DMA API, and make the
> creation of the descriptor list more explicit. Furthermore, the driver
> is changed to keep the mapping per video buffer instead of using a
> single mapping that is updated with every buffer.
> 
> Patches 7 to 11 prepare the driver for the multi-planar API including a
> cleanup of the format handling in the buffer, and finally switch to the
> multi-planar API.
> 
> Patch 12 updates the code that creates the DMA-descriptor mapping to
> correctly handle buffers with multiple planes. The driver has to iterate
> all planes and make them the continuous for the RGA.
> 
> Patch 13 enables the NV12M format, which is the multi-planar variant of
> the NV12 format.

I get these smatch warnings for this series:

drivers/media/platform/rockchip/rga/rga-hw.c: In function 'rga_get_addr_offset':
drivers/media/platform/rockchip/rga/rga-hw.c:46:65: warning: variable 'uv_factor' set but not used [-Wunused-but-set-variable]
   46 |                      y_div = 0, uv_stride = 0, pixel_width = 0, uv_factor = 0;
      |                                                                 ^~~~~~~~~
drivers/media/platform/rockchip/rga/rga-buf.c:129 rga_buf_prepare() warn: unsigned 'ret' is never less than zero.

Regards,

	Hans

> 
> Michael
> 
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
> ---
> Michael Tretter (13):
>       media: rockchip: rga: fix swizzling for RGB formats
>       media: rockchip: rga: extract helper to fill descriptors
>       media: rockchip: rga: allocate DMA descriptors per buffer
>       media: rockchip: rga: split src and dst buffer setup
>       media: rockchip: rga: pre-calculate plane offsets
>       media: rockchip: rga: set dma mask to 32 bits
>       media: rockchip: rga: use clamp() to clamp size to limits
>       media: rockchip: rga: use pixelformat to find format
>       media: rockchip: rga: add local variable for pix_format
>       media: rockchip: rga: use macros for testing buffer type
>       media: rockchip: rga: switch to multi-planar API
>       media: rockchip: rga: rework buffer handling for multi-planar formats
>       media: rockchip: rga: add NV12M support
> 
>  drivers/media/platform/rockchip/rga/rga-buf.c | 166 ++++++++++++++++-------
>  drivers/media/platform/rockchip/rga/rga-hw.c  | 143 +++++++++++++-------
>  drivers/media/platform/rockchip/rga/rga.c     | 187 ++++++++++++--------------
>  drivers/media/platform/rockchip/rga/rga.h     |  35 ++++-
>  4 files changed, 329 insertions(+), 202 deletions(-)
> ---
> base-commit: 0bb80ecc33a8fb5a682236443c1e740d5c917d1d
> change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93
> 
> Best regards,
Michael Tretter Oct. 4, 2023, 2:08 p.m. UTC | #2
On Mon, 25 Sep 2023 09:45:10 +0200, Hans Verkuil wrote:
> On 14/09/2023 14:40, Michael Tretter wrote:
> > This series adds support for the V4L2 multi-planar API to the Rockchip
> > RGA driver. Once the RGA driver supports the multi-planar API, it is
> > easier to share buffers with other V4L2 drivers that also support the
> > multi-planar API and may not expose planar formats with contiguous
> > planes.
> > 
> > With non-contiguous planes, the U and V planes may not start at the same
> > offset as with the continuous planes. Therefore, the RGA driver cannot
> > rely on its calculation of the plane offsets based on the format and
> > frame size anymore, but must remember the offsets when it created the
> > mapping. Therefore, I also reworked how the DMA mapping is handled.
> > 
> > As a bonus, the RGA driver should now work correctly on devices with
> > more than 4 GB of memory. Video buffers should now be allocated within
> > the 4 GB boundary and an import of buffers that have higher addresses
> > into the driver should fail.
> > 
> > Patch 1 fixes the swizzling of RGA formats. While testing all formats of
> > the driver on rk3568, I discovered that the color channels of the RGB
> > formats are wrong when converting to NV12. I didn't test this on other
> > SoCs with an RGA and I am not sure, if they behave differently regarding
> > the color channels. Please report, if this breaks the color conversion
> > on other SoCs, and I will make this SoC-specific.
> > 
> > Patches 2 to 6 are the rework the DMA descriptor handling for the RGA
> > MMU. The patches clean up, how the driver uses the DMA API, and make the
> > creation of the descriptor list more explicit. Furthermore, the driver
> > is changed to keep the mapping per video buffer instead of using a
> > single mapping that is updated with every buffer.
> > 
> > Patches 7 to 11 prepare the driver for the multi-planar API including a
> > cleanup of the format handling in the buffer, and finally switch to the
> > multi-planar API.
> > 
> > Patch 12 updates the code that creates the DMA-descriptor mapping to
> > correctly handle buffers with multiple planes. The driver has to iterate
> > all planes and make them the continuous for the RGA.
> > 
> > Patch 13 enables the NV12M format, which is the multi-planar variant of
> > the NV12 format.
> 
> I get these smatch warnings for this series:
> 
> drivers/media/platform/rockchip/rga/rga-hw.c: In function 'rga_get_addr_offset':
> drivers/media/platform/rockchip/rga/rga-hw.c:46:65: warning: variable 'uv_factor' set but not used [-Wunused-but-set-variable]
>    46 |                      y_div = 0, uv_stride = 0, pixel_width = 0, uv_factor = 0;
>       |                                                                 ^~~~~~~~~
> drivers/media/platform/rockchip/rga/rga-buf.c:129 rga_buf_prepare() warn: unsigned 'ret' is never less than zero.

Thanks, I will run smatch on the series and send a v2.

Michael

> 
> Regards,
> 
> 	Hans
> 
> > 
> > Michael
> > 
> > Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
> > ---
> > Michael Tretter (13):
> >       media: rockchip: rga: fix swizzling for RGB formats
> >       media: rockchip: rga: extract helper to fill descriptors
> >       media: rockchip: rga: allocate DMA descriptors per buffer
> >       media: rockchip: rga: split src and dst buffer setup
> >       media: rockchip: rga: pre-calculate plane offsets
> >       media: rockchip: rga: set dma mask to 32 bits
> >       media: rockchip: rga: use clamp() to clamp size to limits
> >       media: rockchip: rga: use pixelformat to find format
> >       media: rockchip: rga: add local variable for pix_format
> >       media: rockchip: rga: use macros for testing buffer type
> >       media: rockchip: rga: switch to multi-planar API
> >       media: rockchip: rga: rework buffer handling for multi-planar formats
> >       media: rockchip: rga: add NV12M support
> > 
> >  drivers/media/platform/rockchip/rga/rga-buf.c | 166 ++++++++++++++++-------
> >  drivers/media/platform/rockchip/rga/rga-hw.c  | 143 +++++++++++++-------
> >  drivers/media/platform/rockchip/rga/rga.c     | 187 ++++++++++++--------------
> >  drivers/media/platform/rockchip/rga/rga.h     |  35 ++++-
> >  4 files changed, 329 insertions(+), 202 deletions(-)
> > ---
> > base-commit: 0bb80ecc33a8fb5a682236443c1e740d5c917d1d
> > change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93
> > 
> > Best regards,
> 
>