mbox series

[0/5] Add V4L2 driver for i.MX8 JPEG Encoder/Decoder

Message ID 1573053633-21437-1-git-send-email-mirela.rabulea@nxp.com (mailing list archive)
Headers show
Series Add V4L2 driver for i.MX8 JPEG Encoder/Decoder | expand

Message

Mirela Rabulea Nov. 6, 2019, 3:20 p.m. UTC
This patch set adds the V4L2 driver for i.MX8QXP/QM JPEG encoder/decoder
and it's dependencies.
The driver was tested on i.MX8QXP, using a unit test application and
the v4l2-compliance tool, which passes when run on the encoder/decoder dev
node, without any other parameters:
v4l2-compliance -d /dev/video0
v4l2-compliance -d /dev/video1
v4l2-compliance SHA: 2ff1e6b3d67dbbdde212c8cf0de603f9f52078ed, 64 bits

The first patch "Add packed YUV444 24bpp pixel format" was already reviewed
upstream, but not accepted because there was no driver using this format,
until now. This pixel format, YUV444 24bpp, is now used and needed by the
i.MX8QXP/QM JPEG encoder/decoder driver.

Mirela Rabulea (5):
  media: v4l: Add packed YUV444 24bpp pixel format
  firmware: imx: scu-pd: Add power domains for imx-jpeg
  media: dt-bindings: Add bindings for i.MX8QXP/QM JPEG driver
  media: imx-jpeg: Add V4L2 driver for i.MX8 JPEG Encoder/Decoder
  arm64: dts: imx8qxp: Add jpeg encoder/decoder nodes

 .../devicetree/bindings/media/imx8-jpeg.yaml       |   83 +
 Documentation/media/uapi/v4l/pixfmt-packed-yuv.rst |   37 +-
 arch/arm64/boot/dts/freescale/imx8qxp-mek.dts      |    8 +
 arch/arm64/boot/dts/freescale/imx8qxp.dtsi         |   37 +
 drivers/firmware/imx/scu-pd.c                      |    6 +
 drivers/media/platform/Kconfig                     |    2 +
 drivers/media/platform/Makefile                    |    1 +
 drivers/media/platform/imx-jpeg/Kconfig            |   10 +
 drivers/media/platform/imx-jpeg/Makefile           |    3 +
 drivers/media/platform/imx-jpeg/mxc-jpeg-hw.c      |  168 ++
 drivers/media/platform/imx-jpeg/mxc-jpeg-hw.h      |  140 ++
 drivers/media/platform/imx-jpeg/mxc-jpeg.c         | 2266 ++++++++++++++++++++
 drivers/media/platform/imx-jpeg/mxc-jpeg.h         |  187 ++
 drivers/media/v4l2-core/v4l2-ioctl.c               |    1 +
 include/uapi/linux/videodev2.h                     |    1 +
 15 files changed, 2949 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/imx8-jpeg.yaml
 create mode 100644 drivers/media/platform/imx-jpeg/Kconfig
 create mode 100644 drivers/media/platform/imx-jpeg/Makefile
 create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg-hw.c
 create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg-hw.h
 create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg.c
 create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg.h

Comments

Hans Verkuil Nov. 6, 2019, 3:37 p.m. UTC | #1
Hi Mirela,

On 11/6/19 4:20 PM, Mirela Rabulea wrote:
> This patch set adds the V4L2 driver for i.MX8QXP/QM JPEG encoder/decoder
> and it's dependencies.
> The driver was tested on i.MX8QXP, using a unit test application and
> the v4l2-compliance tool, which passes when run on the encoder/decoder dev
> node, without any other parameters:
> v4l2-compliance -d /dev/video0
> v4l2-compliance -d /dev/video1
> v4l2-compliance SHA: 2ff1e6b3d67dbbdde212c8cf0de603f9f52078ed, 64 bits

That's too old. Always test with the latest version from the v4l-utils git
repo (git://linuxtv.org/v4l-utils.git). v4l2-compliance is continually
improved, so driver testing should always be done with the latest version.

For the JPEG encoder you can add the -s option to v4l2-compliance. Unfortunately,
the streaming tests for JPEG decoders in v4l2-compliance are not well supported
at the moment, so you can't use -s there.

Please provide the full output of the v4l2-compliance command as well, I'd
like to see the actual output.

Thank you very much for working on this, always nice to see new HW support.

Regards,

	Hans

> 
> The first patch "Add packed YUV444 24bpp pixel format" was already reviewed
> upstream, but not accepted because there was no driver using this format,
> until now. This pixel format, YUV444 24bpp, is now used and needed by the
> i.MX8QXP/QM JPEG encoder/decoder driver.
> 
> Mirela Rabulea (5):
>   media: v4l: Add packed YUV444 24bpp pixel format
>   firmware: imx: scu-pd: Add power domains for imx-jpeg
>   media: dt-bindings: Add bindings for i.MX8QXP/QM JPEG driver
>   media: imx-jpeg: Add V4L2 driver for i.MX8 JPEG Encoder/Decoder
>   arm64: dts: imx8qxp: Add jpeg encoder/decoder nodes
> 
>  .../devicetree/bindings/media/imx8-jpeg.yaml       |   83 +
>  Documentation/media/uapi/v4l/pixfmt-packed-yuv.rst |   37 +-
>  arch/arm64/boot/dts/freescale/imx8qxp-mek.dts      |    8 +
>  arch/arm64/boot/dts/freescale/imx8qxp.dtsi         |   37 +
>  drivers/firmware/imx/scu-pd.c                      |    6 +
>  drivers/media/platform/Kconfig                     |    2 +
>  drivers/media/platform/Makefile                    |    1 +
>  drivers/media/platform/imx-jpeg/Kconfig            |   10 +
>  drivers/media/platform/imx-jpeg/Makefile           |    3 +
>  drivers/media/platform/imx-jpeg/mxc-jpeg-hw.c      |  168 ++
>  drivers/media/platform/imx-jpeg/mxc-jpeg-hw.h      |  140 ++
>  drivers/media/platform/imx-jpeg/mxc-jpeg.c         | 2266 ++++++++++++++++++++
>  drivers/media/platform/imx-jpeg/mxc-jpeg.h         |  187 ++
>  drivers/media/v4l2-core/v4l2-ioctl.c               |    1 +
>  include/uapi/linux/videodev2.h                     |    1 +
>  15 files changed, 2949 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/devicetree/bindings/media/imx8-jpeg.yaml
>  create mode 100644 drivers/media/platform/imx-jpeg/Kconfig
>  create mode 100644 drivers/media/platform/imx-jpeg/Makefile
>  create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg-hw.c
>  create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg-hw.h
>  create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg.c
>  create mode 100644 drivers/media/platform/imx-jpeg/mxc-jpeg.h
>
Mirela Rabulea Nov. 6, 2019, 4:45 p.m. UTC | #2
Hi Hans,

On Mi, 2019-11-06 at 16:37 +0100, Hans Verkuil wrote:
> That's too old. Always test with the latest version from the v4l-
> utils git
> repo (git://linuxtv.org/v4l-utils.git). v4l2-compliance is
> continually
> improved, so driver testing should always be done with the latest
> version.
> 
> For the JPEG encoder you can add the -s option to v4l2-compliance.
> Unfortunately,
> the streaming tests for JPEG decoders in v4l2-compliance are not well
> supported
> at the moment, so you can't use -s there.
> 
> Please provide the full output of the v4l2-compliance command as
> well, I'd
> like to see the actual output.

Thanks for the reply. I updated to the latest v4l-utils, basic tests
pass, but I have issues with the streaming tests (-s). I'll work on
those, it might take a while.
For the reference, here's the actual output, for both the encoder &
decoder.


root@imx8qxpmek:/unit_tests/JPEG# ./v4l2-compliance -d /dev/video1 -s
v4l2-compliance SHA: 6c415a11fceb32067cdb5c2e33f90dbf018182a4, 64 bits

Compliance test for mxc-jpeg decode device /dev/video1:

Driver Info:
	Driver name      : mxc-jpeg decode
	Card type        : mxc-jpeg decoder
	Bus info         : platform:58450000.jpegenc
	Driver version   : 5.4.0
	Capabilities     : 0x84204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
	Detected JPEG Encoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video1 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
	test read/write: OK (Not Supported)
	test blocking wait: OK
		fail: v4l2-test-buffers.cpp(254): g_field() ==
V4L2_FIELD_ANY
		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
	test MMAP (no poll): FAIL
		fail: v4l2-test-buffers.cpp(254): g_field() ==
V4L2_FIELD_ANY
		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
	test MMAP (select): FAIL
		fail: v4l2-test-buffers.cpp(254): g_field() ==
V4L2_FIELD_ANY
		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
	test MMAP (epoll): FAIL
		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
q)
	test USERPTR (no poll): FAIL
		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
q)
	test USERPTR (select): FAIL
	test DMABUF: Cannot test, specify --expbuf-device

Total for mxc-jpeg decode device /dev/video1: 51, Succeeded: 46,
Failed: 5, Warnings: 0



root@imx8qxpmek:/unit_tests/JPEG# ./v4l2-compliance -d /dev/video0 -s
v4l2-compliance SHA: 6c415a11fceb32067cdb5c2e33f90dbf018182a4, 64 bits

Compliance test for mxc-jpeg decode device /dev/video0:

Driver Info:
	Driver name      : mxc-jpeg decode
	Card type        : mxc-jpeg decoder
	Bus info         : platform:58400000.jpegdec
	Driver version   : 5.4.0
	Capabilities     : 0x84204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
	Detected JPEG Decoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK (Not Supported)
	test VIDIOC_G/S_CTRL: OK (Not Supported)
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
	test read/write: OK (Not Supported)
	test blocking wait: OK
		fail: v4l2-test-buffers.cpp(254): g_field() ==
V4L2_FIELD_ANY
		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
	test MMAP (no poll): FAIL
		fail: v4l2-test-buffers.cpp(254): g_field() ==
V4L2_FIELD_ANY
		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
	test MMAP (select): FAIL
		fail: v4l2-test-buffers.cpp(254): g_field() ==
V4L2_FIELD_ANY
		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
	test MMAP (epoll): FAIL
		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
q)
	test USERPTR (no poll): FAIL
		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
q)
	test USERPTR (select): FAIL
	test DMABUF: Cannot test, specify --expbuf-device

Total for mxc-jpeg decode device /dev/video0: 51, Succeeded: 46,
Failed: 5, Warnings: 0
Hans Verkuil Nov. 6, 2019, 4:52 p.m. UTC | #3
On 11/6/19 5:45 PM, Mirela Rabulea wrote:
> Hi Hans,
> 
> On Mi, 2019-11-06 at 16:37 +0100, Hans Verkuil wrote:
>> That's too old. Always test with the latest version from the v4l-
>> utils git
>> repo (git://linuxtv.org/v4l-utils.git). v4l2-compliance is
>> continually
>> improved, so driver testing should always be done with the latest
>> version.
>>
>> For the JPEG encoder you can add the -s option to v4l2-compliance.
>> Unfortunately,
>> the streaming tests for JPEG decoders in v4l2-compliance are not well
>> supported
>> at the moment, so you can't use -s there.
>>
>> Please provide the full output of the v4l2-compliance command as
>> well, I'd
>> like to see the actual output.
> 
> Thanks for the reply. I updated to the latest v4l-utils, basic tests
> pass, but I have issues with the streaming tests (-s). I'll work on
> those, it might take a while.
> For the reference, here's the actual output, for both the encoder &
> decoder.
> 
> 
> root@imx8qxpmek:/unit_tests/JPEG# ./v4l2-compliance -d /dev/video1 -s
> v4l2-compliance SHA: 6c415a11fceb32067cdb5c2e33f90dbf018182a4, 64 bits
> 
> Compliance test for mxc-jpeg decode device /dev/video1:
> 
> Driver Info:
> 	Driver name      : mxc-jpeg decode
> 	Card type        : mxc-jpeg decoder
> 	Bus info         : platform:58450000.jpegenc
> 	Driver version   : 5.4.0
> 	Capabilities     : 0x84204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 	Detected JPEG Encoder
> 
> Required ioctls:
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video1 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK

The presence of a scaler is suspicious: is the encoder indeed
capable of scaling? I suspect this is a bug.

> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK

The presence of this...

> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK

...and this is also strange for a JPEG codec. These ioctls are typically only
needed for MPEG/H264/etc. codecs, and not for a simple JPEG codec.

The same issues are found for the JPEG decoder.

> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK (Not Supported)
> 
> Test input 0:
> 
> Streaming ioctls:
> 	test read/write: OK (Not Supported)
> 	test blocking wait: OK
> 		fail: v4l2-test-buffers.cpp(254): g_field() ==
> V4L2_FIELD_ANY

The driver shall never return FIELD_ANY. This needs to be FIELD_NONE.

Regards,

	Hans

> 		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
> 	test MMAP (no poll): FAIL
> 		fail: v4l2-test-buffers.cpp(254): g_field() ==
> V4L2_FIELD_ANY
> 		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
> 	test MMAP (select): FAIL
> 		fail: v4l2-test-buffers.cpp(254): g_field() ==
> V4L2_FIELD_ANY
> 		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
> 	test MMAP (epoll): FAIL
> 		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
> 		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
> q)
> 	test USERPTR (no poll): FAIL
> 		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
> 		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
> q)
> 	test USERPTR (select): FAIL
> 	test DMABUF: Cannot test, specify --expbuf-device
> 
> Total for mxc-jpeg decode device /dev/video1: 51, Succeeded: 46,
> Failed: 5, Warnings: 0
> 
> 
> 
> root@imx8qxpmek:/unit_tests/JPEG# ./v4l2-compliance -d /dev/video0 -s
> v4l2-compliance SHA: 6c415a11fceb32067cdb5c2e33f90dbf018182a4, 64 bits
> 
> Compliance test for mxc-jpeg decode device /dev/video0:
> 
> Driver Info:
> 	Driver name      : mxc-jpeg decode
> 	Card type        : mxc-jpeg decoder
> 	Bus info         : platform:58400000.jpegdec
> 	Driver version   : 5.4.0
> 	Capabilities     : 0x84204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 	Detected JPEG Decoder
> 
> Required ioctls:
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video0 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> 	test VIDIOC_QUERYCTRL: OK (Not Supported)
> 	test VIDIOC_G/S_CTRL: OK (Not Supported)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 0 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK (Not Supported)
> 
> Test input 0:
> 
> Streaming ioctls:
> 	test read/write: OK (Not Supported)
> 	test blocking wait: OK
> 		fail: v4l2-test-buffers.cpp(254): g_field() ==
> V4L2_FIELD_ANY
> 		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
> 	test MMAP (no poll): FAIL
> 		fail: v4l2-test-buffers.cpp(254): g_field() ==
> V4L2_FIELD_ANY
> 		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
> 	test MMAP (select): FAIL
> 		fail: v4l2-test-buffers.cpp(254): g_field() ==
> V4L2_FIELD_ANY
> 		fail: v4l2-test-buffers.cpp(1266): buf.qbuf(node)
> 	test MMAP (epoll): FAIL
> 		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
> 		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
> q)
> 	test USERPTR (no poll): FAIL
> 		fail: v4l2-test-buffers.cpp(1492): ret && ret != ENOTTY
> 		fail: v4l2-test-buffers.cpp(1580): setupUserPtr(node,
> q)
> 	test USERPTR (select): FAIL
> 	test DMABUF: Cannot test, specify --expbuf-device
> 
> Total for mxc-jpeg decode device /dev/video0: 51, Succeeded: 46,
> Failed: 5, Warnings: 0
> 
> 
>
Mirela Rabulea Nov. 8, 2019, 1:36 p.m. UTC | #4
Hi Hans,

On Mi, 2019-11-06 at 17:52 +0100, Hans Verkuil wrote:
>       test Scaling: OK
> The presence of a scaler is suspicious: is the encoder indeed
> capable of scaling? I suspect this is a bug.

No, it's not capable of scaling. You suspect a bug in the driver or the
tests?

> Codec ioctls:
> >       test VIDIOC_(TRY_)ENCODER_CMD: OK
> The presence of this...
> 
> > 
> >       test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> >       test VIDIOC_(TRY_)DECODER_CMD: OK
> ...and this is also strange for a JPEG codec. These ioctls are
> typically only
> needed for MPEG/H264/etc. codecs, and not for a simple JPEG codec.
> 
> The same issues are found for the JPEG decoder.

I implemented the CMD_STOP for both encoder & decoder, because it was
requested by our developer for gstreamer plugin for this codec.
The context in which this was requested was for playing MJPEG videos (a
concatenation of JPEG frames).

> Streaming ioctls:
> >       test read/write: OK (Not Supported)
> >       test blocking wait: OK
> >               fail: v4l2-test-buffers.cpp(254): g_field() ==
> > V4L2_FIELD_ANY
> The driver shall never return FIELD_ANY. This needs to be FIELD_NONE.

Is there a "good example" of a v4l m2m driver that passes these
streaming tests? That would save some time on my side.
For the FIELD_ANY issue, I got inspired from your commit:
ab7afaf3 media: vim2m: add buf_out_validate callback
But there's a lot more to go...

Thanks,
Mirela
Hans Verkuil Nov. 8, 2019, 2:06 p.m. UTC | #5
On 11/8/19 2:36 PM, Mirela Rabulea wrote:
> Hi Hans,
> 
> On Mi, 2019-11-06 at 17:52 +0100, Hans Verkuil wrote:
>>       test Scaling: OK
>> The presence of a scaler is suspicious: is the encoder indeed
>> capable of scaling? I suspect this is a bug.
> 
> No, it's not capable of scaling. You suspect a bug in the driver or the
> tests?

Actually, I think that's an outstanding bug in v4l-utils. It doesn't
correctly handle the m2m case with respect to scaling. I think. I'll
look into this a bit more.

> 
>> Codec ioctls:
>>>       test VIDIOC_(TRY_)ENCODER_CMD: OK
>> The presence of this...
>>
>>>
>>>       test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>>       test VIDIOC_(TRY_)DECODER_CMD: OK
>> ...and this is also strange for a JPEG codec. These ioctls are
>> typically only
>> needed for MPEG/H264/etc. codecs, and not for a simple JPEG codec.
>>
>> The same issues are found for the JPEG decoder.
> 
> I implemented the CMD_STOP for both encoder & decoder, because it was
> requested by our developer for gstreamer plugin for this codec.
> The context in which this was requested was for playing MJPEG videos (a
> concatenation of JPEG frames).

This ioctl makes no sense for JPEG codecs, and in fact jpeg drivers like
s5p-jpeg or mtk-jpeg do not implement this. This sounds like a gstreamer bug.
Nicolas, do you know anything about this?

> 
>> Streaming ioctls:
>>>       test read/write: OK (Not Supported)
>>>       test blocking wait: OK
>>>               fail: v4l2-test-buffers.cpp(254): g_field() ==
>>> V4L2_FIELD_ANY
>> The driver shall never return FIELD_ANY. This needs to be FIELD_NONE.
> 
> Is there a "good example" of a v4l m2m driver that passes these

vim2m. Also drivers/media/platform/mtk-jpeg/ (although I'm not sure when
it was last tested with v4l2-compliance, so it might be a bit out of date).

> streaming tests? That would save some time on my side.
> For the FIELD_ANY issue, I got inspired from your commit:
> ab7afaf3 media: vim2m: add buf_out_validate callback
> But there's a lot more to go...
> 
> Thanks,
> Mirela
> 

Regards,

	Hans
Nicolas Dufresne Nov. 11, 2019, 7:27 p.m. UTC | #6
Le vendredi 08 novembre 2019 à 15:06 +0100, Hans Verkuil a écrit :
> On 11/8/19 2:36 PM, Mirela Rabulea wrote:
> > Hi Hans,
> > 
> > On Mi, 2019-11-06 at 17:52 +0100, Hans Verkuil wrote:
> > >       test Scaling: OK
> > > The presence of a scaler is suspicious: is the encoder indeed
> > > capable of scaling? I suspect this is a bug.
> > 
> > No, it's not capable of scaling. You suspect a bug in the driver or the
> > tests?
> 
> Actually, I think that's an outstanding bug in v4l-utils. It doesn't
> correctly handle the m2m case with respect to scaling. I think. I'll
> look into this a bit more.
> 
> > > Codec ioctls:
> > > >       test VIDIOC_(TRY_)ENCODER_CMD: OK
> > > The presence of this...
> > > 
> > > >       test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > >       test VIDIOC_(TRY_)DECODER_CMD: OK
> > > ...and this is also strange for a JPEG codec. These ioctls are
> > > typically only
> > > needed for MPEG/H264/etc. codecs, and not for a simple JPEG codec.
> > > 
> > > The same issues are found for the JPEG decoder.
> > 
> > I implemented the CMD_STOP for both encoder & decoder, because it was
> > requested by our developer for gstreamer plugin for this codec.
> > The context in which this was requested was for playing MJPEG videos (a
> > concatenation of JPEG frames).
> 
> This ioctl makes no sense for JPEG codecs, and in fact jpeg drivers like
> s5p-jpeg or mtk-jpeg do not implement this. This sounds like a gstreamer bug.
> Nicolas, do you know anything about this?

This code was added in generic bits of GStreamer. So we just always do
CMD_STOP to drain the remaining buffers and expect EPIPE later on when
done. We can special case this, but at the same time why ? This could
be buried into the m2m helpers with a default implementation.

If you feel like this is completely wrong and userspace should do more
case-by-case code path for different CODEC handling, then someone will
have to update GStreamer accordingly. This is all moving target at the
spec just got merged.

> 
> > > Streaming ioctls:
> > > >       test read/write: OK (Not Supported)
> > > >       test blocking wait: OK
> > > >               fail: v4l2-test-buffers.cpp(254): g_field() ==
> > > > V4L2_FIELD_ANY
> > > The driver shall never return FIELD_ANY. This needs to be FIELD_NONE.
> > 
> > Is there a "good example" of a v4l m2m driver that passes these
> 
> vim2m. Also drivers/media/platform/mtk-jpeg/ (although I'm not sure when
> it was last tested with v4l2-compliance, so it might be a bit out of date).
> 
> > streaming tests? That would save some time on my side.
> > For the FIELD_ANY issue, I got inspired from your commit:
> > ab7afaf3 media: vim2m: add buf_out_validate callback
> > But there's a lot more to go...
> > 
> > Thanks,
> > Mirela
> > 
> 
> Regards,
> 
> 	Hans
Ezequiel Garcia Nov. 12, 2019, 4:17 p.m. UTC | #7
Hello Mirela,

Thanks for the patch.

On Wed, 2019-11-06 at 17:20 +0200, Mirela Rabulea wrote:
> This patch set adds the V4L2 driver for i.MX8QXP/QM JPEG encoder/decoder
> and it's dependencies.
> The driver was tested on i.MX8QXP, using a unit test application and

What's the relatioship between this hardware and the JPEG hardware support that
is available on i.MX 8M?

Thanks,
Ezequiel
Mirela Rabulea Nov. 13, 2019, 12:52 p.m. UTC | #8
Hi Ezequiel,

On Ma, 2019-11-12 at 13:17 -0300, Ezequiel Garcia wrote:
> This patch set adds the V4L2 driver for i.MX8QXP/QM JPEG
> > encoder/decoder
> > and it's dependencies.
> > The driver was tested on i.MX8QXP, using a unit test application
> > and
> What's the relatioship between this hardware and the JPEG hardware
> support that
> is available on i.MX 8M?

If you are referring to the VPU, the relation is none, it's a different
IP. The VPU is also capable of MJPEG decoding, but no encoding.

The IPs for which this driver is written are the JPEGENC and JPEGDEC,
and it's present only on 8QM & 8QXP (which also have the VPU).

Regards,
Mirela
Mirela Rabulea Nov. 15, 2019, 12:34 p.m. UTC | #9
Hi Hans,

On Vi, 2019-11-08 at 15:06 +0100, Hans Verkuil wrote:
> > Is there a "good example" of a v4l m2m driver that passes these
> vim2m. Also drivers/media/platform/mtk-jpeg/ (although I'm not sure
> when
> it was last tested with v4l2-compliance, so it might be a bit out of
> date).
> 
> > 
> > streaming tests? That would save some time on my side.

Thanks a lot for the examples.
I'm sending _v2 of the patch, with fixes for streaming tests.
One mention here, I dropped the support for USERPTR buffers, based on
this:
https://lwn.net/Articles/447435/
"VB2_USERPTR: buffers allocated in user space. Normally, only devices
which can do scatter/gather I/O can deal with user-space buffers.
Interestingly, videobuf2 supports contiguous buffers allocated by user
space; the only way to get those, though, is to use some sort of
special mechanism like the out-of-tree Android "pmem" driver.
Contiguous I/O to huge pages is not supported."

I wonder if VB2_USERPTR make sense together
with &vb2_dma_contig_memops? I see at least 1 driver with this
combination (s5p-jpeg), I would be surprised if that passes the
streaming tests.

Regards,
Mirela