[v2,00/14] Clean H264 stateless uAPI

Message

Ezequiel Garcia Aug. 6, 2020, 3:12 p.m. UTC

Here's a new round for the H.264 uAPI cleanup, which as discussed
aims at being stabilized and promoted as a first-class public uAPI soon.

It should be noted that there is already GStreamer native
support for this interface, which will be part of 1.18,
once it's released.

I have pushed a branch porting GStreamer to
support these interface changes:

https://gitlab.freedesktop.org/ezequielgarcia/gst-plugins-bad/-/commits/for_h264_uapi_v3

As was discussed the SLICE_PARAMS control is now clarified
to work for one-slice-per-request operation, using CAPTURE
buffer holding features. This is how Cedrus driver is implemented.

The other drivers currently supported Hantro and Rockchip VDEC,
as well as the MTK stateless decoder posted by Alex Courbot
operate in frame-based mode.

These "frame-based" devices parse the slice headers in hardware,
and therefore shall not support SLICE_PARAMS. To that extent,
the redundant bitstream fields are now part of the DECODE_PARAMS
control.

Hopefully now the specification documentation is clear enough.
GStreamer, Chromium and FFmpeg (which I'm sure will be implemented
as soon as we stabilize the API) should serve as reference examples
on how the API is consumed.

Changelog:

v1->v2:
* Clean SLICE_PARAMS documentation, which we don't
  expect to be part of an array anymore. 
* Clarify how frame-based and slice-based modes
  are expected to work.
* Add Cedrus patches to fix field references,
  as requested by Jernej.
* Fix wrongly removed SPS in rkvdec.
* Fix rkvdec DPB reference implementation.
* Fix missing Cedrus and missing control member,
  for prediction weight table control.
* Say "raster scan" instead of "matrix" in the docs.
* Drop duplicated macros and use v4l2_h264_dpb_reference
  for the DPB reference signalling.

RFC->v1: 
* Split prediction weight table to a separate control.
* Increase size of first_mb_in_slice field.
* Cleanup DPB entry interface, to support field coding.
* Increase of DPB entry pic_num field.
* Move slice invariant fields to the per-frame control.

Ezequiel Garcia (10):
  media: uapi: h264: Further clarify scaling lists order
  media: uapi: h264: Split prediction weight parameters
  media: uapi: h264: Increase size of 'first_mb_in_slice' field
  media: uapi: h264: Clean DPB entry interface
  media: uapi: h264: Increase size of DPB entry pic_num
  media: uapi: h264: Drop SLICE_PARAMS 'size' field
  media: uapi: h264: Clarify SLICE_BASED mode
  media: uapi: h264: Clean slice invariants syntax elements
  media: hantro: Don't require unneeded H264_SLICE_PARAMS
  media: rkvdec: Don't require unneeded H264_SLICE_PARAMS

Jernej Skrabec (3):
  media: uapi: h264: Update reference lists
  media: cedrus: h264: Properly configure reference field
  media: cedrus: h264: Fix frame list construction

Philipp Zabel (1):
  media: uapi: h264: Clarify pic_order_cnt_bit_size field

 .../media/v4l/ext-ctrls-codec.rst             | 224 ++++++++++--------
 drivers/media/v4l2-core/v4l2-ctrls.c          |  28 +++
 drivers/media/v4l2-core/v4l2-h264.c           |  12 +-
 drivers/staging/media/hantro/hantro_drv.c     |   5 -
 .../staging/media/hantro/hantro_g1_h264_dec.c |  21 +-
 drivers/staging/media/hantro/hantro_h264.c    |   8 +-
 drivers/staging/media/hantro/hantro_hw.h      |   2 -
 drivers/staging/media/rkvdec/rkvdec-h264.c    |  27 +--
 drivers/staging/media/rkvdec/rkvdec.c         |   5 -
 drivers/staging/media/sunxi/cedrus/cedrus.c   |   7 +
 drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
 .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
 .../staging/media/sunxi/cedrus/cedrus_h264.c  |  49 ++--
 include/media/h264-ctrls.h                    |  80 ++++---
 include/media/v4l2-ctrls.h                    |   2 +
 include/media/v4l2-h264.h                     |   3 +-
 16 files changed, 257 insertions(+), 219 deletions(-)

Comments

Jernej Škrabec Aug. 11, 2020, 7:16 p.m. UTC | #1

Hi!

Dne četrtek, 06. avgust 2020 ob 17:12:56 CEST je Ezequiel Garcia napisal(a):
> Here's a new round for the H.264 uAPI cleanup, which as discussed
> aims at being stabilized and promoted as a first-class public uAPI soon.
> 
> It should be noted that there is already GStreamer native
> support for this interface, which will be part of 1.18,
> once it's released.
> 
> I have pushed a branch porting GStreamer to
> support these interface changes:
> 
> https://gitlab.freedesktop.org/ezequielgarcia/gst-plugins-bad/-/commits/for_
> h264_uapi_v3
> 
> As was discussed the SLICE_PARAMS control is now clarified
> to work for one-slice-per-request operation, using CAPTURE
> buffer holding features. This is how Cedrus driver is implemented.
> 
> The other drivers currently supported Hantro and Rockchip VDEC,
> as well as the MTK stateless decoder posted by Alex Courbot
> operate in frame-based mode.
> 
> These "frame-based" devices parse the slice headers in hardware,
> and therefore shall not support SLICE_PARAMS. To that extent,
> the redundant bitstream fields are now part of the DECODE_PARAMS
> control.
> 
> Hopefully now the specification documentation is clear enough.
> GStreamer, Chromium and FFmpeg (which I'm sure will be implemented
> as soon as we stabilize the API) should serve as reference examples
> on how the API is consumed.
> 

I tested this series using https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-hwaccel-4.3.1 on Cedrus (Allwinner H6) using additional patch 
contained in discussion around patch 3 and I couldn't find any issue.

You can add to whole series:
Tested-by: Jernej Skrabec <[e-mail]>

Best regards,
Jernej