mbox series

[RFC,V0,0/2] media: v4l2: extend framework to support advanced feature of Mediatek Camsys driver

Message ID 1620356017-26486-1-git-send-email-john.wei@mediatek.com (mailing list archive)
Headers show
Series media: v4l2: extend framework to support advanced feature of Mediatek Camsys driver | expand

Message

john.wei@mediatek.com May 7, 2021, 2:53 a.m. UTC
Hello,

This is the first version of the RFC patch series to v4l2 and media framework
to support virtual channel (an advanced feature of MIPI CSI). Virtual channel
was used commonly among modern image sensor. It was used to transfer
metadata of an image such as statistical data of AE or AF.Some advanced
features of camera such as HDR (High dynamic range), PDAF (Phase Detection
Auto Focus) were achieved by virtual channel. Medaitek made some modification
to frame descriptor based on https://patchwork.kernel.org/patch/10875875/ and
https://patchwork.kernel.org/patch/10875875/.We extend the struct
v4l2_mbus_frame_desc_entry by add enable, hsize, vsize and user_data_desc.
With this modification, frame descriptor are now more powerful to meet all
kinds of need among different applications. Here is an example that we use
frame descriptor to describe all data streams in a frame output by a
 stagger sensor.

static struct v4l2_mbus_frame_desc_entry frame_desc_cus1[] = {
        {
                .bus.csi2 = {
                        .channel = 0,
                        .data_type = 0x2b,
                        .enable = 1,
                        .hsize = 0xF00,
                        .vsize = 0x870,
                        .user_data_desc = V4L2_MBUS_CSI2_USER_DEFINED_DATA_DESC_HDR_LE,
                },
        },
        {
                .bus.csi2 = {
                        .channel = 1,
                        .data_type = 0x2b,
                        .enable = 1,
                        .hsize = 0xF00,
                        .vsize = 0x870,
                        .user_data_desc = V4L2_MBUS_CSI2_USER_DEFINED_DATA_DESC_HDR_SE,
                },
        },
};

The first data stream, data for long exposure was transfer by channel 0 with
data type 0x2b and the resolution was 0xF00 by horizontal and 0x870 by vertical.
The second data stream, data for short exposure was transfer by channel 1
with data type 0x2b and the resolution was also 0xF00 by horizontal and 0x870
by vertical. Both long exposure and short exposure data will be fused in ISP
stage to output a HDR image.

  media: v4l2: Add fields to frame descriptors
  media: v4l2-ctrl: Add user defined base for ISP user control

 include/media/v4l2-subdev.h        |   25 ++++++++++++++++++++++++-
 include/uapi/linux/v4l2-controls.h |   10 ++++++++++
 2 files changed, 34 insertions(+), 1 deletion(-)

Comments

Sakari Ailus May 7, 2021, 9:38 a.m. UTC | #1
Hi John,

On Fri, May 07, 2021 at 10:53:35AM +0800, john.wei@mediatek.com wrote:
> Hello,
> 
> This is the first version of the RFC patch series to v4l2 and media framework
> to support virtual channel (an advanced feature of MIPI CSI). Virtual channel
> was used commonly among modern image sensor. It was used to transfer
> metadata of an image such as statistical data of AE or AF.Some advanced
> features of camera such as HDR (High dynamic range), PDAF (Phase Detection
> Auto Focus) were achieved by virtual channel. Medaitek made some modification
> to frame descriptor based on https://patchwork.kernel.org/patch/10875875/ and
> https://patchwork.kernel.org/patch/10875875/.We extend the struct
> v4l2_mbus_frame_desc_entry by add enable, hsize, vsize and user_data_desc.
> With this modification, frame descriptor are now more powerful to meet all
> kinds of need among different applications. Here is an example that we use
> frame descriptor to describe all data streams in a frame output by a
>  stagger sensor.

Thanks for your e-mail.

Work has been done to address the problem area previously but it hasn't
reached upstream yet.

Tomi Valkeinen has posted a set here:

<https://lore.kernel.org/linux-media/20210427124523.990938-1-tomi.valkeinen@ideasonboard.com/>

Cc Tomi as well.

> 
> static struct v4l2_mbus_frame_desc_entry frame_desc_cus1[] = {
>         {
>                 .bus.csi2 = {
>                         .channel = 0,
>                         .data_type = 0x2b,
>                         .enable = 1,
>                         .hsize = 0xF00,
>                         .vsize = 0x870,
>                         .user_data_desc = V4L2_MBUS_CSI2_USER_DEFINED_DATA_DESC_HDR_LE,
>                 },
>         },
>         {
>                 .bus.csi2 = {
>                         .channel = 1,
>                         .data_type = 0x2b,
>                         .enable = 1,
>                         .hsize = 0xF00,
>                         .vsize = 0x870,
>                         .user_data_desc = V4L2_MBUS_CSI2_USER_DEFINED_DATA_DESC_HDR_SE,
>                 },
>         },
> };
> 
> The first data stream, data for long exposure was transfer by channel 0 with
> data type 0x2b and the resolution was 0xF00 by horizontal and 0x870 by vertical.
> The second data stream, data for short exposure was transfer by channel 1
> with data type 0x2b and the resolution was also 0xF00 by horizontal and 0x870
> by vertical. Both long exposure and short exposure data will be fused in ISP
> stage to output a HDR image.
> 
>   media: v4l2: Add fields to frame descriptors
>   media: v4l2-ctrl: Add user defined base for ISP user control
> 
>  include/media/v4l2-subdev.h        |   25 ++++++++++++++++++++++++-
>  include/uapi/linux/v4l2-controls.h |   10 ++++++++++
>  2 files changed, 34 insertions(+), 1 deletion(-)
>