| Message ID | 20230914-rockchip-rga-multiplanar-v2-0-bbfa6abf8bbf@pengutronix.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | media: rockchip: rga: add support for multi-planar formats | expand |
On Fri, 13 Oct 2023 13:00:21 +0200, Michael Tretter wrote: > This is v2 of the series that adds support for the V4L2 multi-planar API > to the Rockchip RGA driver. Once the RGA driver supports the > multi-planar API, it is easier to share buffers with other V4L2 drivers > that also support the multi-planar API and may not expose planar formats > with contiguous planes. > > v2 fixes the smatch warnings and compile errors of v1. Furthermore, now > the DMA mask is set to 32 bits for coherent, too, and the gfp_flags are > configured to ensure that buffers are allocated in the lower 4GB area. Gentle Ping. Michael > > With non-contiguous planes, the U and V planes may not start at the same > offset as with the continuous planes. Therefore, the RGA driver cannot > rely on its calculation of the plane offsets based on the format and > frame size anymore, but must remember the offsets when it created the > mapping. Therefore, I also reworked how the DMA mapping is handled. > > As a bonus, the RGA driver should now work correctly on devices with > more than 4 GB of memory. Video buffers should now be allocated within > the 4 GB boundary and an import of buffers that have higher addresses > into the driver should fail. > > Patch 1 fixes the swizzling of RGA formats. While testing all formats of > the driver on rk3568, I discovered that the color channels of the RGB > formats are wrong when converting to NV12. I didn't test this on other > SoCs with an RGA and I am not sure, if they behave differently regarding > the color channels. Please report, if this breaks the color conversion > on other SoCs, and I will make this SoC-specific. > > Patches 2 to 6 are the rework the DMA descriptor handling for the RGA > MMU. The patches clean up, how the driver uses the DMA API, and make the > creation of the descriptor list more explicit. Furthermore, the driver > is changed to keep the mapping per video buffer instead of using a > single mapping that is updated with every buffer. > > Patches 7 to 11 prepare the driver for the multi-planar API including a > cleanup of the format handling in the buffer, and finally switch to the > multi-planar API. > > Patch 12 updates the code that creates the DMA-descriptor mapping to > correctly handle buffers with multiple planes. The driver has to iterate > all planes and make them the continuous for the RGA. > > Patch 13 enables the NV12M format, which is the multi-planar variant of > the NV12 format. > > Michael > > Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> > --- > Changes in v2: > - Fix smatch warnings > - Fix cast to dst_mmu_pages/src_mmu_pages to fix compile error in Patch 2 > - Remove check for upper_32_bits when filling the DMA descriptor table > - Remove useless dma_sync_single_for_device() > - Set DMA mask for DMA coherent > - Set gfp_flags to __GFP_DMA32 > - Link to v1: https://lore.kernel.org/r/20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de > > --- > Michael Tretter (13): > media: rockchip: rga: fix swizzling for RGB formats > media: rockchip: rga: extract helper to fill descriptors > media: rockchip: rga: allocate DMA descriptors per buffer > media: rockchip: rga: split src and dst buffer setup > media: rockchip: rga: pre-calculate plane offsets > media: rockchip: rga: set dma mask to 32 bits > media: rockchip: rga: use clamp() to clamp size to limits > media: rockchip: rga: use pixelformat to find format > media: rockchip: rga: add local variable for pix_format > media: rockchip: rga: use macros for testing buffer type > media: rockchip: rga: switch to multi-planar API > media: rockchip: rga: rework buffer handling for multi-planar formats > media: rockchip: rga: add NV12M support > > drivers/media/platform/rockchip/rga/rga-buf.c | 162 ++++++++++++++++------ > drivers/media/platform/rockchip/rga/rga-hw.c | 146 ++++++++++++-------- > drivers/media/platform/rockchip/rga/rga.c | 189 ++++++++++++-------------- > drivers/media/platform/rockchip/rga/rga.h | 35 ++++- > 4 files changed, 328 insertions(+), 204 deletions(-) > --- > base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 > change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93 > > Best regards, > -- > Michael Tretter <m.tretter@pengutronix.de> > >
Hi Michael, On 13/10/2023 13:00, Michael Tretter wrote: > This is v2 of the series that adds support for the V4L2 multi-planar API > to the Rockchip RGA driver. Once the RGA driver supports the > multi-planar API, it is easier to share buffers with other V4L2 drivers > that also support the multi-planar API and may not expose planar formats > with contiguous planes. > > v2 fixes the smatch warnings and compile errors of v1. Furthermore, now > the DMA mask is set to 32 bits for coherent, too, and the gfp_flags are > configured to ensure that buffers are allocated in the lower 4GB area. > > With non-contiguous planes, the U and V planes may not start at the same > offset as with the continuous planes. Therefore, the RGA driver cannot > rely on its calculation of the plane offsets based on the format and > frame size anymore, but must remember the offsets when it created the > mapping. Therefore, I also reworked how the DMA mapping is handled. > > As a bonus, the RGA driver should now work correctly on devices with > more than 4 GB of memory. Video buffers should now be allocated within > the 4 GB boundary and an import of buffers that have higher addresses > into the driver should fail. > > Patch 1 fixes the swizzling of RGA formats. While testing all formats of > the driver on rk3568, I discovered that the color channels of the RGB > formats are wrong when converting to NV12. I didn't test this on other > SoCs with an RGA and I am not sure, if they behave differently regarding > the color channels. Please report, if this breaks the color conversion > on other SoCs, and I will make this SoC-specific. > > Patches 2 to 6 are the rework the DMA descriptor handling for the RGA > MMU. The patches clean up, how the driver uses the DMA API, and make the > creation of the descriptor list more explicit. Furthermore, the driver > is changed to keep the mapping per video buffer instead of using a > single mapping that is updated with every buffer. > > Patches 7 to 11 prepare the driver for the multi-planar API including a > cleanup of the format handling in the buffer, and finally switch to the > multi-planar API. Switching over to the multi-planar API is always a bit scary. How did you test this? Using gstreamer? Other apps? Regards, Hans > > Patch 12 updates the code that creates the DMA-descriptor mapping to > correctly handle buffers with multiple planes. The driver has to iterate > all planes and make them the continuous for the RGA. > > Patch 13 enables the NV12M format, which is the multi-planar variant of > the NV12 format. > > Michael > > Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> > --- > Changes in v2: > - Fix smatch warnings > - Fix cast to dst_mmu_pages/src_mmu_pages to fix compile error in Patch 2 > - Remove check for upper_32_bits when filling the DMA descriptor table > - Remove useless dma_sync_single_for_device() > - Set DMA mask for DMA coherent > - Set gfp_flags to __GFP_DMA32 > - Link to v1: https://lore.kernel.org/r/20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de > > --- > Michael Tretter (13): > media: rockchip: rga: fix swizzling for RGB formats > media: rockchip: rga: extract helper to fill descriptors > media: rockchip: rga: allocate DMA descriptors per buffer > media: rockchip: rga: split src and dst buffer setup > media: rockchip: rga: pre-calculate plane offsets > media: rockchip: rga: set dma mask to 32 bits > media: rockchip: rga: use clamp() to clamp size to limits > media: rockchip: rga: use pixelformat to find format > media: rockchip: rga: add local variable for pix_format > media: rockchip: rga: use macros for testing buffer type > media: rockchip: rga: switch to multi-planar API > media: rockchip: rga: rework buffer handling for multi-planar formats > media: rockchip: rga: add NV12M support > > drivers/media/platform/rockchip/rga/rga-buf.c | 162 ++++++++++++++++------ > drivers/media/platform/rockchip/rga/rga-hw.c | 146 ++++++++++++-------- > drivers/media/platform/rockchip/rga/rga.c | 189 ++++++++++++-------------- > drivers/media/platform/rockchip/rga/rga.h | 35 ++++- > 4 files changed, 328 insertions(+), 204 deletions(-) > --- > base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 > change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93 > > Best regards,
Hi Hans, On Fri, 24 Nov 2023 14:39:37 +0100, Hans Verkuil wrote: > On 13/10/2023 13:00, Michael Tretter wrote: > > This is v2 of the series that adds support for the V4L2 multi-planar API > > to the Rockchip RGA driver. Once the RGA driver supports the > > multi-planar API, it is easier to share buffers with other V4L2 drivers > > that also support the multi-planar API and may not expose planar formats > > with contiguous planes. > > > > v2 fixes the smatch warnings and compile errors of v1. Furthermore, now > > the DMA mask is set to 32 bits for coherent, too, and the gfp_flags are > > configured to ensure that buffers are allocated in the lower 4GB area. > > > > With non-contiguous planes, the U and V planes may not start at the same > > offset as with the continuous planes. Therefore, the RGA driver cannot > > rely on its calculation of the plane offsets based on the format and > > frame size anymore, but must remember the offsets when it created the > > mapping. Therefore, I also reworked how the DMA mapping is handled. > > > > As a bonus, the RGA driver should now work correctly on devices with > > more than 4 GB of memory. Video buffers should now be allocated within > > the 4 GB boundary and an import of buffers that have higher addresses > > into the driver should fail. > > > > Patch 1 fixes the swizzling of RGA formats. While testing all formats of > > the driver on rk3568, I discovered that the color channels of the RGB > > formats are wrong when converting to NV12. I didn't test this on other > > SoCs with an RGA and I am not sure, if they behave differently regarding > > the color channels. Please report, if this breaks the color conversion > > on other SoCs, and I will make this SoC-specific. > > > > Patches 2 to 6 are the rework the DMA descriptor handling for the RGA > > MMU. The patches clean up, how the driver uses the DMA API, and make the > > creation of the descriptor list more explicit. Furthermore, the driver > > is changed to keep the mapping per video buffer instead of using a > > single mapping that is updated with every buffer. > > > > Patches 7 to 11 prepare the driver for the multi-planar API including a > > cleanup of the format handling in the buffer, and finally switch to the > > multi-planar API. > > Switching over to the multi-planar API is always a bit scary. How did you > test this? Using gstreamer? Other apps? I tested with GStreamer. My use case is using the RGA to convert RGB Buffers to NV12 to feed them to a Hantro JPEG encoder. I also tried to separate the preparatory work, the switch to the multi-planar API, and the addition of actually multi-planar formats into separate patches. Is there anything that I should take special care of or explicitly test, when switching the driver to the multi-planar API? Michael > > Regards, > > Hans > > > > > Patch 12 updates the code that creates the DMA-descriptor mapping to > > correctly handle buffers with multiple planes. The driver has to iterate > > all planes and make them the continuous for the RGA. > > > > Patch 13 enables the NV12M format, which is the multi-planar variant of > > the NV12 format. > > > > Michael > > > > Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> > > --- > > Changes in v2: > > - Fix smatch warnings > > - Fix cast to dst_mmu_pages/src_mmu_pages to fix compile error in Patch 2 > > - Remove check for upper_32_bits when filling the DMA descriptor table > > - Remove useless dma_sync_single_for_device() > > - Set DMA mask for DMA coherent > > - Set gfp_flags to __GFP_DMA32 > > - Link to v1: https://lore.kernel.org/r/20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de > > > > --- > > Michael Tretter (13): > > media: rockchip: rga: fix swizzling for RGB formats > > media: rockchip: rga: extract helper to fill descriptors > > media: rockchip: rga: allocate DMA descriptors per buffer > > media: rockchip: rga: split src and dst buffer setup > > media: rockchip: rga: pre-calculate plane offsets > > media: rockchip: rga: set dma mask to 32 bits > > media: rockchip: rga: use clamp() to clamp size to limits > > media: rockchip: rga: use pixelformat to find format > > media: rockchip: rga: add local variable for pix_format > > media: rockchip: rga: use macros for testing buffer type > > media: rockchip: rga: switch to multi-planar API > > media: rockchip: rga: rework buffer handling for multi-planar formats > > media: rockchip: rga: add NV12M support > > > > drivers/media/platform/rockchip/rga/rga-buf.c | 162 ++++++++++++++++------ > > drivers/media/platform/rockchip/rga/rga-hw.c | 146 ++++++++++++-------- > > drivers/media/platform/rockchip/rga/rga.c | 189 ++++++++++++-------------- > > drivers/media/platform/rockchip/rga/rga.h | 35 ++++- > > 4 files changed, 328 insertions(+), 204 deletions(-) > > --- > > base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 > > change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93 > > > > Best regards, > >
On 24/11/2023 16:31, Michael Tretter wrote: > Hi Hans, > > On Fri, 24 Nov 2023 14:39:37 +0100, Hans Verkuil wrote: >> On 13/10/2023 13:00, Michael Tretter wrote: >>> This is v2 of the series that adds support for the V4L2 multi-planar API >>> to the Rockchip RGA driver. Once the RGA driver supports the >>> multi-planar API, it is easier to share buffers with other V4L2 drivers >>> that also support the multi-planar API and may not expose planar formats >>> with contiguous planes. >>> >>> v2 fixes the smatch warnings and compile errors of v1. Furthermore, now >>> the DMA mask is set to 32 bits for coherent, too, and the gfp_flags are >>> configured to ensure that buffers are allocated in the lower 4GB area. >>> >>> With non-contiguous planes, the U and V planes may not start at the same >>> offset as with the continuous planes. Therefore, the RGA driver cannot >>> rely on its calculation of the plane offsets based on the format and >>> frame size anymore, but must remember the offsets when it created the >>> mapping. Therefore, I also reworked how the DMA mapping is handled. >>> >>> As a bonus, the RGA driver should now work correctly on devices with >>> more than 4 GB of memory. Video buffers should now be allocated within >>> the 4 GB boundary and an import of buffers that have higher addresses >>> into the driver should fail. >>> >>> Patch 1 fixes the swizzling of RGA formats. While testing all formats of >>> the driver on rk3568, I discovered that the color channels of the RGB >>> formats are wrong when converting to NV12. I didn't test this on other >>> SoCs with an RGA and I am not sure, if they behave differently regarding >>> the color channels. Please report, if this breaks the color conversion >>> on other SoCs, and I will make this SoC-specific. >>> >>> Patches 2 to 6 are the rework the DMA descriptor handling for the RGA >>> MMU. The patches clean up, how the driver uses the DMA API, and make the >>> creation of the descriptor list more explicit. Furthermore, the driver >>> is changed to keep the mapping per video buffer instead of using a >>> single mapping that is updated with every buffer. >>> >>> Patches 7 to 11 prepare the driver for the multi-planar API including a >>> cleanup of the format handling in the buffer, and finally switch to the >>> multi-planar API. >> >> Switching over to the multi-planar API is always a bit scary. How did you >> test this? Using gstreamer? Other apps? > > I tested with GStreamer. My use case is using the RGA to convert RGB Buffers > to NV12 to feed them to a Hantro JPEG encoder. I also tried to separate the > preparatory work, the switch to the multi-planar API, and the addition of > actually multi-planar formats into separate patches. > > Is there anything that I should take special care of or explicitly test, when > switching the driver to the multi-planar API? No, I just wanted to have a better overview of how this driver is used and how you tested it. Regards, Hans > > Michael > >> >> Regards, >> >> Hans >> >>> >>> Patch 12 updates the code that creates the DMA-descriptor mapping to >>> correctly handle buffers with multiple planes. The driver has to iterate >>> all planes and make them the continuous for the RGA. >>> >>> Patch 13 enables the NV12M format, which is the multi-planar variant of >>> the NV12 format. >>> >>> Michael >>> >>> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> >>> --- >>> Changes in v2: >>> - Fix smatch warnings >>> - Fix cast to dst_mmu_pages/src_mmu_pages to fix compile error in Patch 2 >>> - Remove check for upper_32_bits when filling the DMA descriptor table >>> - Remove useless dma_sync_single_for_device() >>> - Set DMA mask for DMA coherent >>> - Set gfp_flags to __GFP_DMA32 >>> - Link to v1: https://lore.kernel.org/r/20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de >>> >>> --- >>> Michael Tretter (13): >>> media: rockchip: rga: fix swizzling for RGB formats >>> media: rockchip: rga: extract helper to fill descriptors >>> media: rockchip: rga: allocate DMA descriptors per buffer >>> media: rockchip: rga: split src and dst buffer setup >>> media: rockchip: rga: pre-calculate plane offsets >>> media: rockchip: rga: set dma mask to 32 bits >>> media: rockchip: rga: use clamp() to clamp size to limits >>> media: rockchip: rga: use pixelformat to find format >>> media: rockchip: rga: add local variable for pix_format >>> media: rockchip: rga: use macros for testing buffer type >>> media: rockchip: rga: switch to multi-planar API >>> media: rockchip: rga: rework buffer handling for multi-planar formats >>> media: rockchip: rga: add NV12M support >>> >>> drivers/media/platform/rockchip/rga/rga-buf.c | 162 ++++++++++++++++------ >>> drivers/media/platform/rockchip/rga/rga-hw.c | 146 ++++++++++++-------- >>> drivers/media/platform/rockchip/rga/rga.c | 189 ++++++++++++-------------- >>> drivers/media/platform/rockchip/rga/rga.h | 35 ++++- >>> 4 files changed, 328 insertions(+), 204 deletions(-) >>> --- >>> base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 >>> change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93 >>> >>> Best regards, >> >>
This is v2 of the series that adds support for the V4L2 multi-planar API to the Rockchip RGA driver. Once the RGA driver supports the multi-planar API, it is easier to share buffers with other V4L2 drivers that also support the multi-planar API and may not expose planar formats with contiguous planes. v2 fixes the smatch warnings and compile errors of v1. Furthermore, now the DMA mask is set to 32 bits for coherent, too, and the gfp_flags are configured to ensure that buffers are allocated in the lower 4GB area. With non-contiguous planes, the U and V planes may not start at the same offset as with the continuous planes. Therefore, the RGA driver cannot rely on its calculation of the plane offsets based on the format and frame size anymore, but must remember the offsets when it created the mapping. Therefore, I also reworked how the DMA mapping is handled. As a bonus, the RGA driver should now work correctly on devices with more than 4 GB of memory. Video buffers should now be allocated within the 4 GB boundary and an import of buffers that have higher addresses into the driver should fail. Patch 1 fixes the swizzling of RGA formats. While testing all formats of the driver on rk3568, I discovered that the color channels of the RGB formats are wrong when converting to NV12. I didn't test this on other SoCs with an RGA and I am not sure, if they behave differently regarding the color channels. Please report, if this breaks the color conversion on other SoCs, and I will make this SoC-specific. Patches 2 to 6 are the rework the DMA descriptor handling for the RGA MMU. The patches clean up, how the driver uses the DMA API, and make the creation of the descriptor list more explicit. Furthermore, the driver is changed to keep the mapping per video buffer instead of using a single mapping that is updated with every buffer. Patches 7 to 11 prepare the driver for the multi-planar API including a cleanup of the format handling in the buffer, and finally switch to the multi-planar API. Patch 12 updates the code that creates the DMA-descriptor mapping to correctly handle buffers with multiple planes. The driver has to iterate all planes and make them the continuous for the RGA. Patch 13 enables the NV12M format, which is the multi-planar variant of the NV12 format. Michael Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> --- Changes in v2: - Fix smatch warnings - Fix cast to dst_mmu_pages/src_mmu_pages to fix compile error in Patch 2 - Remove check for upper_32_bits when filling the DMA descriptor table - Remove useless dma_sync_single_for_device() - Set DMA mask for DMA coherent - Set gfp_flags to __GFP_DMA32 - Link to v1: https://lore.kernel.org/r/20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de --- Michael Tretter (13): media: rockchip: rga: fix swizzling for RGB formats media: rockchip: rga: extract helper to fill descriptors media: rockchip: rga: allocate DMA descriptors per buffer media: rockchip: rga: split src and dst buffer setup media: rockchip: rga: pre-calculate plane offsets media: rockchip: rga: set dma mask to 32 bits media: rockchip: rga: use clamp() to clamp size to limits media: rockchip: rga: use pixelformat to find format media: rockchip: rga: add local variable for pix_format media: rockchip: rga: use macros for testing buffer type media: rockchip: rga: switch to multi-planar API media: rockchip: rga: rework buffer handling for multi-planar formats media: rockchip: rga: add NV12M support drivers/media/platform/rockchip/rga/rga-buf.c | 162 ++++++++++++++++------ drivers/media/platform/rockchip/rga/rga-hw.c | 146 ++++++++++++-------- drivers/media/platform/rockchip/rga/rga.c | 189 ++++++++++++-------------- drivers/media/platform/rockchip/rga/rga.h | 35 ++++- 4 files changed, 328 insertions(+), 204 deletions(-) --- base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93 Best regards,