mbox series

[v7,00/12] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

Message ID 20250225-rkvdec_h264_high10_and_422_support-v7-0-7992a68a4910@collabora.com (mailing list archive)
Headers show
Series media: rkvdec: Add H.264 High 10 and 4:2:2 profile support | expand

Message

Sebastian Fricke Feb. 25, 2025, 9:40 a.m. UTC
This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
Video Decoder driver.

Patch 1 add helpers for calculating plane bytesperline and sizeimage.
Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.

Patch 3 change to use bytesperline and buffer height to configure strides.
Patch 4 change to use values from SPS/PPS control to configure the HW.

Patch 5-9 refactor code to support filtering of CAPUTRE formats based
on the image format returned from a get_image_fmt ops.

Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.

Patch 11 Limit minimum profile to constrained baseline due to
unsupported features in the full baseline profile

Patch 12 add a fix for enumerated frame sizes returned to userspace.

Tested with Fluster on a ROCK Pi 4 (RK3399)

129/135 on JVT-AVC_V1
44/69 on JVT-FR-EXT

Changes in v7:
- Split out the change with the minimum profile
- s/v4l2_format_plane_width/v4l2_format_plane_stride/
- Move V4L2_PIX_FMT_NV15/V4L2_PIX_FMT_NV20 documentation as suggested
- Change return value from int to bool on rkvdec_image_fmt_match
- Add reviewed-by tags
Link to v6: https://lore.kernel.org/linux-media/20240909192522.1076704-1-jonas@kwiboo.se/

Changes in v6:
- Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
- Add patch to fix enumerated frame sizes returned to userspace (Alex)
- Fluster test score is same as v4 and v5, see [4] and [5]
Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/

Changes in v5:
- Drop Remove SPS validation at streaming start patch
- Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
- Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
- Collect r-b and t-b tags
- Fluster test score is same as v4, see [4] and [5]
Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/

Changes in v4:
- Fix failed v4l2-compliance tests related to CAPTURE queue
- Rework CAPTURE format filter anv validate to use an image format
- Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/

Changes in v3:
- Drop merged patches
- Use bpp and bpp_div instead of prior misuse of block_w/block_h
- New patch to use values from SPS/PPS control to configure the HW
- New patch to remove an unnecessary call to validate sps at streaming start
- Reworked pixel format validation
Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/

Changes in v2:
- Collect r-b tags
- SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
- New patch to not override output buffer sizeimage
- Reworked pixel format validation
- Only align decoded buffer instead of changing frmsize step_width
Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/

To fully runtime test this series you may need FFmpeg patches from [1]
and fluster patches from [2], this series is also available at [3].

[1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
[2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
[3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
[4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
[5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf

Regards,
Jonas

Alex Bee (1):
  media: rkvdec: h264: Don't hardcode SPS/PPS parameters

Jonas Karlman (10):
  media: v4l2-common: Add helpers to calculate bytesperline and
    sizeimage
  media: v4l2: Add NV15 and NV20 pixel formats
  media: rkvdec: h264: Use bytesperline and buffer height as virstride
  media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
  media: rkvdec: Move rkvdec_reset_decoded_fmt helper
  media: rkvdec: Extract decoded format enumeration into helper
  media: rkvdec: Add image format concept
  media: rkvdec: Add get_image_fmt ops
  media: rkvdec: h264: Support High 10 and 4:2:2 profiles
  media: rkvdec: Fix enumerate frame sizes

 .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
 drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
 drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
 drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
 include/uapi/linux/videodev2.h                |   2 +
 7 files changed, 410 insertions(+), 123 deletions(-)

--
2.46.0

---
Alex Bee (1):
      media: rkvdec: h264: Don't hardcode SPS/PPS parameters

Jonas Karlman (10):
      media: v4l2-common: Add helpers to calculate bytesperline and sizeimage
      media: v4l2: Add NV15 and NV20 pixel formats
      media: rkvdec: h264: Use bytesperline and buffer height as virstride
      media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
      media: rkvdec: Move rkvdec_reset_decoded_fmt helper
      media: rkvdec: Extract decoded format enumeration into helper
      media: rkvdec: Add image format concept
      media: rkvdec: Add get_image_fmt ops
      media: rkvdec: h264: Support High 10 and 4:2:2 profiles
      media: rkvdec: Fix frame size enumeration

Sebastian Fricke (1):
      media: rkvdec: h264: Limit minimum profile to constrained baseline

 .../userspace-api/media/v4l/pixfmt-yuv-planar.rst  | 128 +++++++++++
 drivers/media/v4l2-core/v4l2-common.c              |  80 +++----
 drivers/media/v4l2-core/v4l2-ioctl.c               |   2 +
 drivers/staging/media/rkvdec/rkvdec-h264.c         |  64 ++++--
 drivers/staging/media/rkvdec/rkvdec.c              | 239 +++++++++++++++------
 drivers/staging/media/rkvdec/rkvdec.h              |  18 +-
 include/uapi/linux/videodev2.h                     |   2 +
 7 files changed, 410 insertions(+), 123 deletions(-)
---
base-commit: c2b96a6818159fba8a3bcc38262da9e77f9b3ec7
change-id: 20250116-rkvdec_h264_high10_and_422_support-a8f2da8a09e6

Best regards,

Comments

Sebastian Fricke Feb. 25, 2025, 12:46 p.m. UTC | #1
Hey,

sorry about missing this in the patch series, if you don't like the fixup path, then I can send a new patch series as well.
I just thought the change was minor enough and addressed the final comments.

Regards,
Sebastian

 ---- On Tue, 25 Feb 2025 13:40:08 +0100  Sebastian Fricke <sebastian.fricke@collabora.com> wrote --- 
 > ---
 >  drivers/media/v4l2-core/v4l2-common.c | 2 +-
 >  1 file changed, 1 insertion(+), 1 deletion(-)
 > 
 > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
 > index 07a999f75755..aa86b8c6aa75 100644
 > --- a/drivers/media/v4l2-core/v4l2-common.c
 > +++ b/drivers/media/v4l2-core/v4l2-common.c
 > @@ -360,7 +360,7 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
 >  }
 >  
 >  static inline unsigned int v4l2_format_plane_stride(const struct v4l2_format_info *info, int plane,
 > -                           unsigned int width)
 > +                            unsigned int width)
 >  {
 >      unsigned int hdiv = plane ? info->hdiv : 1;
 >      unsigned int aligned_width =
 > -- 
 > 2.25.1
 > 
 >
Hans Verkuil Feb. 25, 2025, 12:50 p.m. UTC | #2
On 2/25/25 13:46, Sebastian Fricke wrote:
> Hey,
> 
> sorry about missing this in the patch series, if you don't like the fixup path, then I can send a new patch series as well.
> I just thought the change was minor enough and addressed the final comments.

Ah, this relates to patch 01/12 of this patch series:

https://patchwork.linuxtv.org/project/linux-media/list/?series=14577

Next time it might be better to just reply to the offending patch with the fixup.

That way it is clear for the maintainer what the fixup is for.

Regards,

	Hans

> 
> Regards,
> Sebastian
> 
>  ---- On Tue, 25 Feb 2025 13:40:08 +0100  Sebastian Fricke <sebastian.fricke@collabora.com> wrote --- 
>  > ---
>  >  drivers/media/v4l2-core/v4l2-common.c | 2 +-
>  >  1 file changed, 1 insertion(+), 1 deletion(-)
>  > 
>  > diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
>  > index 07a999f75755..aa86b8c6aa75 100644
>  > --- a/drivers/media/v4l2-core/v4l2-common.c
>  > +++ b/drivers/media/v4l2-core/v4l2-common.c
>  > @@ -360,7 +360,7 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
>  >  }
>  >  
>  >  static inline unsigned int v4l2_format_plane_stride(const struct v4l2_format_info *info, int plane,
>  > -                           unsigned int width)
>  > +                            unsigned int width)
>  >  {
>  >      unsigned int hdiv = plane ? info->hdiv : 1;
>  >      unsigned int aligned_width =
>  > -- 
>  > 2.25.1
>  > 
>  > 
>
Nicolas Dufresne Feb. 25, 2025, 2:05 p.m. UTC | #3
Le mardi 25 février 2025 à 13:50 +0100, Hans Verkuil a écrit :
> On 2/25/25 13:46, Sebastian Fricke wrote:
> > Hey,
> > 
> > sorry about missing this in the patch series, if you don't like the
> > fixup path, then I can send a new patch series as well.
> > I just thought the change was minor enough and addressed the final
> > comments.
> 
> Ah, this relates to patch 01/12 of this patch series:
> 
> https://patchwork.linuxtv.org/project/linux-media/list/?series=14577
> 
> Next time it might be better to just reply to the offending patch
> with the fixup.
> 
> That way it is clear for the maintainer what the fixup is for.

I believe you could have used:

  git send-email -in-reply-to=9db356bd-4d71-4975-91c6-7435bee8aef3@xs4all.nl <somepatch>

To make that a reply.

cheers,
Nicolas
> 
> Regards,
> 
> 	Hans
> 
> > 
> > Regards,
> > Sebastian
> > 
> >  ---- On Tue, 25 Feb 2025 13:40:08 +0100  Sebastian Fricke
> > <sebastian.fricke@collabora.com> wrote --- 
> >  > ---
> >  >  drivers/media/v4l2-core/v4l2-common.c | 2 +-
> >  >  1 file changed, 1 insertion(+), 1 deletion(-)
> >  > 
> >  > diff --git a/drivers/media/v4l2-core/v4l2-common.c
> > b/drivers/media/v4l2-core/v4l2-common.c
> >  > index 07a999f75755..aa86b8c6aa75 100644
> >  > --- a/drivers/media/v4l2-core/v4l2-common.c
> >  > +++ b/drivers/media/v4l2-core/v4l2-common.c
> >  > @@ -360,7 +360,7 @@ static inline unsigned int
> > v4l2_format_block_height(const struct v4l2_format_inf
> >  >  }
> >  >  
> >  >  static inline unsigned int v4l2_format_plane_stride(const
> > struct v4l2_format_info *info, int plane,
> >  > -                           unsigned int width)
> >  > +                            unsigned int width)
> >  >  {
> >  >      unsigned int hdiv = plane ? info->hdiv : 1;
> >  >      unsigned int aligned_width =
> >  > -- 
> >  > 2.25.1
> >  > 
> >  > 
> > 
>