| Message ID | 20190628130002.24293-1-stanimir.varbanov@linaro.org (mailing list archive) |
|---|---|
| State | Not Applicable, archived |
| Headers | show |
On 6/28/19 2:59 PM, Stanimir Varbanov wrote: > Hello, > > Here is v2 of the Venus transition to stateful codec API > compliance. The v2 can be found at [1]. > > Changes since v1: > * codec_state is now enum > * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called() > * corrected g_fmt and reconfig logic > * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers > * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper > * various fixes to make v4l2-compliance pass the streaming test > > To test the streaming with --stream-from-hdr v4l2-compliance option I have > to make the following hack (it is needed because the size of decoder input > buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver > default resolution is 64x64 but the h264 stream is 320x240): > > diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp > index c71dcf65b721..dc0fcf20d3e4 100644 > --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp > +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp > @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode) > fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); > } > fail_on_test(q.create_bufs(node, 1, &fmt)); > + > + for (unsigned p = 0; p < fmt.g_num_planes(); p++) > + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); > + node->s_fmt(fmt); > + > fail_on_test(q.reqbufs(node, 2)); > } > if (v4l_type_is_output(type)) Does the venus driver set sizeimage based on the given output resolution? E.g. if v4l2-compliance would first set the output resolution to 320x240, is the returned sizeimage value OK in that case? And this also means that the venus driver requires each buffer to have a single compressed frame, right? I.e. it can't be spread over multiple OUTPUT buffers. We really need to let userspace know about such restrictions. Stanimir, can you list the restrictions of the decoder for the various codecs? Regards, Hans
Hi Hans, On 6/28/19 4:37 PM, Hans Verkuil wrote: > On 6/28/19 2:59 PM, Stanimir Varbanov wrote: >> Hello, >> >> Here is v2 of the Venus transition to stateful codec API >> compliance. The v2 can be found at [1]. >> >> Changes since v1: >> * codec_state is now enum >> * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called() >> * corrected g_fmt and reconfig logic >> * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers >> * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper >> * various fixes to make v4l2-compliance pass the streaming test >> >> To test the streaming with --stream-from-hdr v4l2-compliance option I have >> to make the following hack (it is needed because the size of decoder input >> buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver >> default resolution is 64x64 but the h264 stream is 320x240): >> >> diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp >> index c71dcf65b721..dc0fcf20d3e4 100644 >> --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp >> +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp >> @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode) >> fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); >> } >> fail_on_test(q.create_bufs(node, 1, &fmt)); >> + >> + for (unsigned p = 0; p < fmt.g_num_planes(); p++) >> + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); >> + node->s_fmt(fmt); >> + >> fail_on_test(q.reqbufs(node, 2)); >> } >> if (v4l_type_is_output(type)) > > Does the venus driver set sizeimage based on the given output resolution? Yes. > > E.g. if v4l2-compliance would first set the output resolution to 320x240, > is the returned sizeimage value OK in that case? Yes. Here are few options to me: - set the correct resolution - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if the bitstream is 4K it will not be enough if the bitrate is huge. - invent some mechanism to trigger reconfiguration on the OUTPUT queue as well (similar to the CAPTURE queue) > > And this also means that the venus driver requires each buffer to have > a single compressed frame, right? I.e. it can't be spread over multiple > OUTPUT buffers. I cannot say for sure but that is how all downstream cases uses it i.e. one compressed frame per input buffer. I wonder if you fill input decoder buffer with many compressed frames in one input decoder buffer how you pass the timestamp for every packet? > > We really need to let userspace know about such restrictions. > > Stanimir, can you list the restrictions of the decoder for the various > codecs? What you mean? Restrictions like "one compressed frame per input buffer"?
On 6/28/19 4:23 PM, Stanimir Varbanov wrote: > Hi Hans, > > On 6/28/19 4:37 PM, Hans Verkuil wrote: >> On 6/28/19 2:59 PM, Stanimir Varbanov wrote: >>> Hello, >>> >>> Here is v2 of the Venus transition to stateful codec API >>> compliance. The v2 can be found at [1]. >>> >>> Changes since v1: >>> * codec_state is now enum >>> * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called() >>> * corrected g_fmt and reconfig logic >>> * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers >>> * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper >>> * various fixes to make v4l2-compliance pass the streaming test >>> >>> To test the streaming with --stream-from-hdr v4l2-compliance option I have >>> to make the following hack (it is needed because the size of decoder input >>> buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver >>> default resolution is 64x64 but the h264 stream is 320x240): >>> >>> diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp >>> index c71dcf65b721..dc0fcf20d3e4 100644 >>> --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp >>> +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp >>> @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode) >>> fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); >>> } >>> fail_on_test(q.create_bufs(node, 1, &fmt)); >>> + >>> + for (unsigned p = 0; p < fmt.g_num_planes(); p++) >>> + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); >>> + node->s_fmt(fmt); >>> + >>> fail_on_test(q.reqbufs(node, 2)); >>> } >>> if (v4l_type_is_output(type)) >> >> Does the venus driver set sizeimage based on the given output resolution? > > Yes. > >> >> E.g. if v4l2-compliance would first set the output resolution to 320x240, >> is the returned sizeimage value OK in that case? > > Yes. > > Here are few options to me: > - set the correct resolution > - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if > the bitstream is 4K it will not be enough if the bitrate is huge. > - invent some mechanism to trigger reconfiguration on the OUTPUT queue > as well (similar to the CAPTURE queue) > >> >> And this also means that the venus driver requires each buffer to have >> a single compressed frame, right? I.e. it can't be spread over multiple >> OUTPUT buffers. > > I cannot say for sure but that is how all downstream cases uses it i.e. > one compressed frame per input buffer. I wonder if you fill input > decoder buffer with many compressed frames in one input decoder buffer > how you pass the timestamp for every packet? > >> >> We really need to let userspace know about such restrictions. >> >> Stanimir, can you list the restrictions of the decoder for the various >> codecs? > > What you mean? Restrictions like "one compressed frame per input buffer"? > Yes :-) Regards, Hans
Le vendredi 28 juin 2019 à 16:25 +0200, Hans Verkuil a écrit : > On 6/28/19 4:23 PM, Stanimir Varbanov wrote: > > Hi Hans, > > > > On 6/28/19 4:37 PM, Hans Verkuil wrote: > > > On 6/28/19 2:59 PM, Stanimir Varbanov wrote: > > > > Hello, > > > > > > > > Here is v2 of the Venus transition to stateful codec API > > > > compliance. The v2 can be found at [1]. > > > > > > > > Changes since v1: > > > > * codec_state is now enum > > > > * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called() > > > > * corrected g_fmt and reconfig logic > > > > * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers > > > > * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper > > > > * various fixes to make v4l2-compliance pass the streaming test > > > > > > > > To test the streaming with --stream-from-hdr v4l2-compliance option I have > > > > to make the following hack (it is needed because the size of decoder input > > > > buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver > > > > default resolution is 64x64 but the h264 stream is 320x240): > > > > > > > > diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp > > > > index c71dcf65b721..dc0fcf20d3e4 100644 > > > > --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp > > > > +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp > > > > @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode) > > > > fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); > > > > } > > > > fail_on_test(q.create_bufs(node, 1, &fmt)); > > > > + > > > > + for (unsigned p = 0; p < fmt.g_num_planes(); p++) > > > > + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); > > > > + node->s_fmt(fmt); > > > > + > > > > fail_on_test(q.reqbufs(node, 2)); > > > > } > > > > if (v4l_type_is_output(type)) > > > > > > Does the venus driver set sizeimage based on the given output resolution? > > > > Yes. > > > > > E.g. if v4l2-compliance would first set the output resolution to 320x240, > > > is the returned sizeimage value OK in that case? > > > > Yes. > > > > Here are few options to me: > > - set the correct resolution > > - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if > > the bitstream is 4K it will not be enough if the bitrate is huge. > > - invent some mechanism to trigger reconfiguration on the OUTPUT queue > > as well (similar to the CAPTURE queue) > > > > > And this also means that the venus driver requires each buffer to have > > > a single compressed frame, right? I.e. it can't be spread over multiple > > > OUTPUT buffers. > > > > I cannot say for sure but that is how all downstream cases uses it i.e. > > one compressed frame per input buffer. I wonder if you fill input > > decoder buffer with many compressed frames in one input decoder buffer > > how you pass the timestamp for every packet? > > > > > We really need to let userspace know about such restrictions. > > > > > > Stanimir, can you list the restrictions of the decoder for the various > > > codecs? > > > > What you mean? Restrictions like "one compressed frame per input buffer"? > > > > Yes :-) I think I just had the same discussions through some RPi patches reviews. All the stateless codec drivers we have so far assumes full frames and some wording iirc specified this in the spec at some point. That removes ambiguity for timestamps application. Even though, timestamps in V4L2 are useless for B-Frame enabled streams (ffmpeg and GStreamer will ignore them). Nicolas
Le vendredi 28 juin 2019 à 10:31 -0400, Nicolas Dufresne a écrit : > Le vendredi 28 juin 2019 à 16:25 +0200, Hans Verkuil a écrit : > > On 6/28/19 4:23 PM, Stanimir Varbanov wrote: > > > Hi Hans, > > > > > > On 6/28/19 4:37 PM, Hans Verkuil wrote: > > > > On 6/28/19 2:59 PM, Stanimir Varbanov wrote: > > > > > Hello, > > > > > > > > > > Here is v2 of the Venus transition to stateful codec API > > > > > compliance. The v2 can be found at [1]. > > > > > > > > > > Changes since v1: > > > > > * codec_state is now enum > > > > > * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called() > > > > > * corrected g_fmt and reconfig logic > > > > > * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers > > > > > * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper > > > > > * various fixes to make v4l2-compliance pass the streaming test > > > > > > > > > > To test the streaming with --stream-from-hdr v4l2-compliance option I have > > > > > to make the following hack (it is needed because the size of decoder input > > > > > buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver > > > > > default resolution is 64x64 but the h264 stream is 320x240): > > > > > > > > > > diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp > > > > > index c71dcf65b721..dc0fcf20d3e4 100644 > > > > > --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp > > > > > +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp > > > > > @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode) > > > > > fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); > > > > > } > > > > > fail_on_test(q.create_bufs(node, 1, &fmt)); > > > > > + > > > > > + for (unsigned p = 0; p < fmt.g_num_planes(); p++) > > > > > + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); > > > > > + node->s_fmt(fmt); > > > > > + > > > > > fail_on_test(q.reqbufs(node, 2)); > > > > > } > > > > > if (v4l_type_is_output(type)) > > > > > > > > Does the venus driver set sizeimage based on the given output resolution? > > > > > > Yes. > > > > > > > E.g. if v4l2-compliance would first set the output resolution to 320x240, > > > > is the returned sizeimage value OK in that case? > > > > > > Yes. > > > > > > Here are few options to me: > > > - set the correct resolution > > > - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if > > > the bitstream is 4K it will not be enough if the bitrate is huge. > > > - invent some mechanism to trigger reconfiguration on the OUTPUT queue > > > as well (similar to the CAPTURE queue) > > > > > > > And this also means that the venus driver requires each buffer to have > > > > a single compressed frame, right? I.e. it can't be spread over multiple > > > > OUTPUT buffers. > > > > > > I cannot say for sure but that is how all downstream cases uses it i.e. > > > one compressed frame per input buffer. I wonder if you fill input > > > decoder buffer with many compressed frames in one input decoder buffer > > > how you pass the timestamp for every packet? > > > > > > > We really need to let userspace know about such restrictions. > > > > > > > > Stanimir, can you list the restrictions of the decoder for the various > > > > codecs? > > > > > > What you mean? Restrictions like "one compressed frame per input buffer"? > > > > > > > Yes :-) > > I think I just had the same discussions through some RPi patches > reviews. All the stateless codec drivers we have so far assumes full > frames and some wording iirc specified this in the spec at some point. > That removes ambiguity for timestamps application. Even though, > timestamps in V4L2 are useless for B-Frame enabled streams (ffmpeg and > GStreamer will ignore them). Sorry, I mean they are not used as timestamp, only for frame matching. > > Nicolas
diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp index c71dcf65b721..dc0fcf20d3e4 100644 --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode) fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); } fail_on_test(q.create_bufs(node, 1, &fmt)); + + for (unsigned p = 0; p < fmt.g_num_planes(); p++) + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p); + node->s_fmt(fmt); + fail_on_test(q.reqbufs(node, 2)); } if (v4l_type_is_output(type))
Hello, Here is v2 of the Venus transition to stateful codec API compliance. The v2 can be found at [1]. Changes since v1: * codec_state is now enum * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called() * corrected g_fmt and reconfig logic * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper * various fixes to make v4l2-compliance pass the streaming test To test the streaming with --stream-from-hdr v4l2-compliance option I have to make the following hack (it is needed because the size of decoder input buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver default resolution is 64x64 but the h264 stream is 320x240): ... and with it the output of the test is: dragonboard-410c:~/v4l-utils$ v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250 -d /dev/video1 v4l2-compliance SHA: 08fed4d0edb1492b91d9d1054c36fed95c372eaa, 64 bits Compliance test for qcom-venus device /dev/video1: Driver Info: Driver name : qcom-venus Card type : Qualcomm Venus video decoder Bus info : platform:qcom-venus Driver version : 5.2.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Detected Stateful Decoder Required ioctls: test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video1 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK test VIDIOC_G/S_CTRL: OK test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 9 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK test Composing: OK test Scaling: OK Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK test Requests: OK (Not Supported) Test input 0: Streaming ioctls: test read/write: OK (Not Supported) test blocking wait: OK Video Capture Multiplanar: Captured 251 buffers test MMAP (select): OK Video Capture Multiplanar: Captured 251 buffers test MMAP (epoll): OK test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device Total for qcom-venus device /dev/video1: 49, Succeeded: 49, Failed: 0, Warnings: 0 regards, Stan [1] https://lore.kernel.org/lkml/0fc547f46d2ff90a2a8adf957cca7297@codeaurora.org/T/ Malathi Gottam (1): venus: venc: amend buffer size for bitstream plane Stanimir Varbanov (10): venus: helpers: export few helper functions venus: hfi: add type argument to hfi flush function venus: hfi: export few HFI functions venus: hfi: return an error if session_init is already called venus: helpers: add three more helper functions venus: vdec_ctrls: get real minimum buffers for capture venus: vdec: allow bigger sizeimage set by clients venus: make decoder compliant with stateful codec API venus: helpers: handle correctly vbuf field venus: dec: populate properly timestamps and flags for capture buffers drivers/media/platform/qcom/venus/core.h | 32 +- drivers/media/platform/qcom/venus/helpers.c | 199 ++++++- drivers/media/platform/qcom/venus/helpers.h | 12 + drivers/media/platform/qcom/venus/hfi.c | 11 +- drivers/media/platform/qcom/venus/hfi.h | 2 +- drivers/media/platform/qcom/venus/vdec.c | 537 ++++++++++++++---- .../media/platform/qcom/venus/vdec_ctrls.c | 7 +- drivers/media/platform/qcom/venus/venc.c | 13 +- 8 files changed, 656 insertions(+), 157 deletions(-)