| Message ID | 20210114180738.1758707-3-helen.koike@collabora.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | media: v4l2: Add extended fmt and buffer ioctls | expand |
Hi, Am 14.01.21 um 19:07 schrieb Helen Koike: > This is part of the multiplanar and singleplanar unification process. > v4l2_ext_pix_format is supposed to work for both cases. > > We also add the concept of modifiers already employed in DRM to expose > HW-specific formats (like tiled or compressed formats) and allow > exchanging this information with the DRM subsystem in a consistent way. > > Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in > v4l2_ext_format, other types will be rejected if you use the > {G,S,TRY}_EXT_PIX_FMT ioctls. > > New hooks have been added to v4l2_ioctl_ops to support those new ioctls > in drivers, but, in the meantime, the core takes care of converting > {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can > still work if the userspace app/lib uses the new ioctls. > > The conversion is also done the other around to allow userspace > apps/libs using {S,G,TRY}_FMT to work with drivers implementing the > _ext_ hooks. > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> > Signed-off-by: Helen Koike <helen.koike@collabora.com> > --- > > Changes in v6: > The main change here was fixing the conversion, so planes reflects color planes, > and to implement this properly I made major refactors compared to the previous > version. > - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz) > - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) > - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) > - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) > - refactor conversion functions, so planes are color planes (Tomasz) > - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz) > - Use "ef" for extended formats in the framework for consistency (Tomasz) > - Handle xfer_func field in conversions (Tomasz) > - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz) > - Refactor format functions to use v4l_fmt_ioctl_via_ext() > - Several fixes/refactoring/changes > - Remove EXT API for touch devices > > Changes in v5: > - change sizes and reorder fields to avoid holes in the struct and make > it the same for 32 and 64 bits > - removed __attribute__ ((packed)) from uapi structs > - Fix doc warning from make htmldocs > - Updated commit message with EXT_PIX prefix for the ioctls. > > Changes in v4: > - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, > making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. > - Add reserved fields > - Removed num_planes from struct v4l2_ext_pix_format > - Removed flag field from struct v4l2_ext_pix_format, since the only > defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, > where we can use modifiers, or add it back later through the reserved > bits. > - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && > != MOD_INVALID > - Fix type assignment in v4l_g_fmt_ext_pix() > - Rebased on top of media/master (post 5.8-rc1) > > Changes in v3: > - Rebased on top of media/master (post 5.4-rc1) > > Changes in v2: > - Move the modifier in v4l2_ext_format (was formerly placed in > v4l2_ext_plane) > - Fix a few bugs in the converters and add a strict parameter to > allow conversion of uninitialized/mis-initialized objects > --- > drivers/media/v4l2-core/v4l2-dev.c | 27 +- > drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- > include/media/v4l2-ioctl.h | 28 ++ > include/uapi/linux/videodev2.h | 41 ++ > 4 files changed, 602 insertions(+), 32 deletions(-) > > diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c > index f9cff033d0dc..5add58cb6d45 100644 > --- a/drivers/media/v4l2-core/v4l2-dev.c > +++ b/drivers/media/v4l2-core/v4l2-dev.c > @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct video_device *vdev) > ops->vidioc_enum_fmt_vid_overlay)) || > (is_tx && ops->vidioc_enum_fmt_vid_out)) > set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); > + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || > + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) > + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > if ((is_rx && (ops->vidioc_g_fmt_vid_cap || > ops->vidioc_g_fmt_vid_cap_mplane || > - ops->vidioc_g_fmt_vid_overlay)) || > + ops->vidioc_g_ext_pix_fmt_vid_cap)) || > (is_tx && (ops->vidioc_g_fmt_vid_out || > ops->vidioc_g_fmt_vid_out_mplane || > - ops->vidioc_g_fmt_vid_out_overlay))) > + ops->vidioc_g_ext_pix_fmt_vid_out))) { > set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); > + } > + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || > + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) > + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > if ((is_rx && (ops->vidioc_s_fmt_vid_cap || > ops->vidioc_s_fmt_vid_cap_mplane || > - ops->vidioc_s_fmt_vid_overlay)) || > + ops->vidioc_s_ext_pix_fmt_vid_cap)) || > (is_tx && (ops->vidioc_s_fmt_vid_out || > ops->vidioc_s_fmt_vid_out_mplane || > - ops->vidioc_s_fmt_vid_out_overlay))) > + ops->vidioc_s_ext_pix_fmt_vid_out))) { > set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); > + } > + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || > + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) > + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > if ((is_rx && (ops->vidioc_try_fmt_vid_cap || > ops->vidioc_try_fmt_vid_cap_mplane || > - ops->vidioc_try_fmt_vid_overlay)) || > + ops->vidioc_try_ext_pix_fmt_vid_cap)) || > (is_tx && (ops->vidioc_try_fmt_vid_out || > ops->vidioc_try_fmt_vid_out_mplane || > - ops->vidioc_try_fmt_vid_out_overlay))) > + ops->vidioc_try_ext_pix_fmt_vid_out))) { > set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); > + } > SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); > SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); > SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > index 848286a284f6..a9c07c0a73ec 100644 > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > @@ -18,6 +18,8 @@ > > #include <linux/videodev2.h> > > +#include <drm/drm_fourcc.h> > + > #include <media/v4l2-common.h> > #include <media/v4l2-ioctl.h> > #include <media/v4l2-ctrls.h> > @@ -38,6 +40,11 @@ > > #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls) > > +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ > + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ > + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ > + V4L2_CAP_VIDEO_M2M_MPLANE)) > + > struct std_descr { > v4l2_std_id std; > const char *descr; > @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, bool write_only) > } > } > > +static void v4l_print_ext_pix_format(const void *arg, bool write_only) > +{ > + const struct v4l2_ext_pix_format *ef = arg; > + unsigned int i; > + > + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n", > + prt_names(ef->type, v4l2_type_names), > + ef->width, ef->height, > + (ef->pixelformat & 0xff), > + (ef->pixelformat >> 8) & 0xff, > + (ef->pixelformat >> 16) & 0xff, > + (ef->pixelformat >> 24) & 0xff, > + ef->modifier, prt_names(ef->field, v4l2_field_names), > + ef->colorspace, ef->ycbcr_enc, > + ef->quantization, ef->xfer_func); > + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", > + i, ef->plane_fmt[i].bytesperline, > + ef->plane_fmt[i].sizeimage); > +} > + > static void v4l_print_framebuffer(const void *arg, bool write_only) > { > const struct v4l2_framebuffer *p = arg; > @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) > switch (type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > if ((is_vid || is_tch) && is_rx && > - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane)) > + (ops->vidioc_g_fmt_vid_cap || > + ops->vidioc_g_ext_pix_fmt_vid_cap || > + ops->vidioc_g_fmt_vid_cap_mplane)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - if ((is_vid || is_tch) && is_rx && ops->vidioc_g_fmt_vid_cap_mplane) > + if ((is_vid || is_tch) && is_rx && > + (ops->vidioc_g_fmt_vid_cap_mplane || > + ops->vidioc_g_ext_pix_fmt_vid_cap)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) > break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > if (is_vid && is_tx && > - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane)) > + (ops->vidioc_g_fmt_vid_out || > + ops->vidioc_g_ext_pix_fmt_vid_out || > + ops->vidioc_g_fmt_vid_out_mplane)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) > + if (is_vid && is_tx && > + (ops->vidioc_g_ext_pix_fmt_vid_out || > + ops->vidioc_g_fmt_vid_out_mplane)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct v4l2_format *fmt) > sizeof(fmt->fmt.pix) - offset); > } > > +static void > +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, > + struct v4l2_pix_format *pix) > +{ > + unsigned int i; > + > + pix->width = ef->width; > + pix->height = ef->height; > + pix->field = ef->field; > + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > + pix->colorspace = ef->colorspace; > + pix->ycbcr_enc = ef->ycbcr_enc; > + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; > + pix->quantization = ef->quantization; > + pix->pixelformat = ef->pixelformat; > + pix->bytesperline = ef->plane_fmt[0].bytesperline; > + pix->sizeimage = ef->plane_fmt[0].sizeimage; > + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pix->sizeimage += ef->plane_fmt[i].sizeimage; > +} > + > +static void > +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef, > + struct v4l2_pix_format_mplane *pix_mp) > +{ > + const struct v4l2_format_info *info = > + v4l2_format_info(ef->pixelformat); > + unsigned int i; > + > + pix_mp->width = ef->width; > + pix_mp->height = ef->height; > + pix_mp->field = ef->field; > + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > + pix_mp->colorspace = ef->colorspace; > + pix_mp->ycbcr_enc = ef->ycbcr_enc; > + pix_mp->quantization = ef->quantization; > + pix_mp->pixelformat = ef->pixelformat; > + > + /* This is true when converting to non-M-variant */ > + if (info && info->mem_planes == 1) { > + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; > + pix_mp->num_planes = 1; > + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage; > + > + return; > + } > + > + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; > + pix_mp->num_planes = i; > +} > + > +/* > + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format > + * > + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. > + * @f: A pointer to struct v4l2_format to be filled. > + * @is_mplane: Bool indicating if multiplanar API should be used in @f. > + * > + * If pixelformat should be converted to M-variant, change ef->pixelformat > + * to the M-variant before calling this function. > + */ > +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef, > + struct v4l2_format *f, bool is_mplane) > +{ > + memset(f, 0, sizeof(*f)); > + > + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && > + ef->modifier != DRM_FORMAT_MOD_INVALID) > + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n", > + ef->modifier); > + > + if (!is_mplane) { > + f->type = ef->type; > + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); > + return; > + } > + > + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) > + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; > + else > + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; > + > + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); > +} > + > +static void > +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, > + struct v4l2_ext_pix_format *ef) > +{ > + const struct v4l2_format_info *info = > + v4l2_format_info(pix->pixelformat); > + unsigned int i; > + > + ef->width = pix->width; > + ef->height = pix->height; > + ef->field = pix->field; > + ef->colorspace = pix->colorspace; > + ef->ycbcr_enc = pix->ycbcr_enc; > + ef->quantization = pix->quantization; > + ef->xfer_func = pix->xfer_func; > + if (pix->flags) > + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); > + > + /* We assume M-variants won't be used in this function */ > + ef->pixelformat = pix->pixelformat; > + > + ef->plane_fmt[0].bytesperline = pix->bytesperline; > + ef->plane_fmt[0].sizeimage = pix->sizeimage; > + > + if (!info) > + return; What if info is null? It looks like we have a wrong conversion in that case. Many parts of this patch rellay on 'info' information, maybe we should extend v4l2_format_info to support all formats? > + > + for (i = 1; i < info->comp_planes; i++) { > + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; > + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * > + ef->height / info->vdiv; > + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > + } > +} > + > +static void > +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp, > + struct v4l2_ext_pix_format *ef) > +{ > + const struct v4l2_format_info *info = > + v4l2_format_info(pix_mp->pixelformat); > + unsigned int i; > + > + ef->width = pix_mp->width; > + ef->height = pix_mp->height; > + ef->field = pix_mp->field; > + ef->colorspace = pix_mp->colorspace; > + ef->ycbcr_enc = pix_mp->ycbcr_enc; > + ef->quantization = pix_mp->quantization; > + ef->xfer_func = pix_mp->xfer_func; > + if (pix_mp->flags) > + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); > + > + if (!info) > + return; > + > + ef->pixelformat = info && info->norm ? The code above this line already return if !info so you can replace s/info && info->norm ?/info->norm/ > + info->norm : pix_mp->pixelformat; > + > + if (info->comp_planes == info->mem_planes) { > + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++) > + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; > + > + return; > + } > + > + /* case where mem_planes is 1 and comp_planes > 1 */ > + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; > + for (i = 1; i < info->comp_planes; i++) { > + ef->plane_fmt[i].bytesperline = > + pix_mp->plane_fmt[0].bytesperline / info->hdiv; > + ef->plane_fmt[i].sizeimage = > + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; > + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > + } > +} > + > +/* > + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format > + * > + * @f: A pointer to struct v4l2_format to be converted. > + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. > + * > + * This method normalize the pixelformat to non-M variant. > + */ > +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, > + struct v4l2_ext_pix_format *ef) > +{ > + memset(ef, 0, sizeof(*ef)); > + > + switch (f->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + ef->type = f->type; > + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > + break; > + default: > + WARN("Converting to Ext Pix Format with wrong buffer type %s\n", > + prt_names(f->type, v4l2_type_names)); > + break; > + } > +} > + > static int v4l_querycap(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct v4l2_pix_format *p) > p->xfer_func = 0; > } > > +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, > + struct v4l2_format *f, > + unsigned int ioctl) > +{ > + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); > + struct video_device *vdev = video_devdata(file); > + struct v4l2_ext_pix_format ef = {0}; > + u32 original_pixfmt = 0; > + u32 cap_mask; > + int ret; > + > + if (ioctl != VIDIOC_G_FMT) { > + /* > + * If CSC attributes are read only, set them to DEFAULT > + * to avoid changes by the driver.> + */ > + if (is_multiplanar) { > + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; > + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; > + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; > + } > + /* Unset the flag to avoid warning in the convertion */ which warning? > + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > + > + /* Save pixelformat in case M-variant is being used */ > + original_pixfmt = f->fmt.pix_mp.pixelformat; > + } else { > + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; > + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; > + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; > + } > + /* Unset the flag to avoid warning in the convertion */ > + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > + } I don't think there is a need to do this CSC adjustment. If the CSC flag is not set then drivers should ignore the colorspace,ycbcr_enc,quantizaion,xfer_func anyway. > + v4l2_format_to_ext_pix_format(f, &ef); > + } > + > + switch (f->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | > + V4L2_CAP_VIDEO_M2M_MPLANE; > + if (!!(vdev->device_caps & cap_mask) != > + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) This function is called if a driver supports the Ext API right? In that case shouldn't "vdev->device_caps & cap_mask" always be non-zero here? > + return -EINVAL; > + > + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > + if (ioctl == VIDIOC_G_FMT) > + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); > + else if (ioctl == VIDIOC_S_FMT) > + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); > + else > + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > + &ef); > + break; > + > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | > + V4L2_CAP_VIDEO_M2M_MPLANE; > + if (!!(vdev->device_caps & cap_mask) != dito > + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) > + return -EINVAL; > + > + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > + if (ioctl == VIDIOC_G_FMT) > + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); > + else if (ioctl == VIDIOC_S_FMT) > + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); > + else > + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > + &ef); > + break; > + > + default: > + return -EINVAL; > + } > + > + if (ret) > + return ret; > + > + if (original_pixfmt != ef.pixelformat && > + v4l2_format_info(original_pixfmt)) > + ef.pixelformat = original_pixfmt; original_pixfmt is set only for multiplaner, so it looks like this line might set ef.pixelformat to 0. > + > + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); > + return 0; > +} > + > static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > > switch (p->type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) > + if (unlikely(!ops->vidioc_g_fmt_vid_cap && > + !ops->vidioc_g_ext_pix_fmt_vid_cap)) > break; > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); > + ret = ops->vidioc_g_fmt_vid_cap ? > + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_G_FMT);> /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > if (vfd->vfl_type == VFL_TYPE_TOUCH) > v4l_pix_format_touch(&p->fmt.pix); > return ret; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > + if (ops->vidioc_g_fmt_vid_cap_mplane) > + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) > + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_G_FMT); > + break; > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); > case V4L2_BUF_TYPE_VBI_CAPTURE: > @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: > return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > - if (unlikely(!ops->vidioc_g_fmt_vid_out)) > + if (unlikely(!ops->vidioc_g_fmt_vid_out && > + !ops->vidioc_g_ext_pix_fmt_vid_out)) > break; > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); > + ret = ops->vidioc_g_fmt_vid_out ? > + ops->vidioc_g_fmt_vid_out(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > return ret; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > + if (ops->vidioc_g_fmt_vid_out_mplane) > + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > + else if (ops->vidioc_g_ext_pix_fmt_vid_out) > + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_G_FMT); > + break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); > case V4L2_BUF_TYPE_VBI_OUTPUT: > @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > return -EINVAL; > } > > +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, void *arg) > +{ > + struct v4l2_ext_pix_format *ef = arg; > + struct v4l2_format f = { > + .type = ef->type, > + }; > + int ret = check_fmt(file, ef->type); > + > + if (ret) > + return ret; > + > + memset(ef, 0, sizeof(*ef)); > + ef->type = f.type; > + > + switch (f.type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + if (ops->vidioc_g_ext_pix_fmt_vid_cap) > + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + if (ops->vidioc_g_ext_pix_fmt_vid_out) > + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); > + break; > + default: > + return -EINVAL; > + } > + > + ret = v4l_g_fmt(ops, file, fh, &f); > + if (ret) > + return ret; > + > + v4l2_format_to_ext_pix_format(&f, ef); > + return 0; > +} > + > static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > > switch (p->type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) > + if (unlikely(!ops->vidioc_s_fmt_vid_cap && > + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); > + ret = ops->vidioc_s_fmt_vid_cap ? > + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > if (vfd->vfl_type == VFL_TYPE_TOUCH) > v4l_pix_format_touch(&p->fmt.pix); > return ret; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) > + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && > + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); > + return ops->vidioc_s_fmt_vid_cap_mplane ? > + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) > break; > @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > - if (unlikely(!ops->vidioc_s_fmt_vid_out)) > + if (unlikely(!ops->vidioc_s_fmt_vid_out && > + !ops->vidioc_s_ext_pix_fmt_vid_out)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); > + ret = ops->vidioc_s_fmt_vid_out ? > + ops->vidioc_s_fmt_vid_out(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > return ret; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) > + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && > + !ops->vidioc_s_ext_pix_fmt_vid_out)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); > + return ops->vidioc_s_fmt_vid_out_mplane ? > + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) > break; > @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > return -EINVAL; > } > > +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, void *arg) > +{ > + struct video_device *vfd = video_devdata(file); > + struct v4l2_ext_pix_format *ef = arg; > + struct v4l2_format f; > + int ret = check_fmt(file, ef->type); > + > + if (ret) > + return ret; > + > + memset(ef->reserved, 0, sizeof(ef->reserved)); > + > + switch (ef->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + if (ops->vidioc_s_ext_pix_fmt_vid_cap) > + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + if (ops->vidioc_s_ext_pix_fmt_vid_out) > + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); > + break; > + default: > + return -EINVAL; > + } > + > + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); > + > + ret = v4l_s_fmt(ops, file, fh, &f); > + if (ret) > + /* TODO: retry with M-variant of ef->pixelformat? */ I think not, ef->pixelformat is a paramater that comes from userspace right? Then userspace should set it according to what the driver supports. > + return ret; > + > + v4l2_format_to_ext_pix_format(&f, ef); > + return 0; > +} > + > static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > > switch (p->type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) > + if (unlikely(!ops->vidioc_try_fmt_vid_cap && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); > + ret = ops->vidioc_try_fmt_vid_cap ? > + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > if (vfd->vfl_type == VFL_TYPE_TOUCH) > v4l_pix_format_touch(&p->fmt.pix); > return ret; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) > + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); > + return ops->vidioc_try_fmt_vid_cap_mplane ? > + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_TRY_FMT); > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) > break; > @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > - if (unlikely(!ops->vidioc_try_fmt_vid_out)) > + if (unlikely(!ops->vidioc_try_fmt_vid_out && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); > + ret = ops->vidioc_try_fmt_vid_out ? > + ops->vidioc_try_fmt_vid_out(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > return ret; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) > + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); > + return ops->vidioc_try_fmt_vid_out_mplane ? > + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_TRY_FMT); > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) > break; > @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > return -EINVAL; > } > > +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, void *arg) > +{ > + struct video_device *vfd = video_devdata(file); > + struct v4l2_ext_pix_format *ef = arg; > + struct v4l2_format f; > + int ret = check_fmt(file, ef->type); > + > + if (ret) > + return ret; > + > + memset(ef->reserved, 0, sizeof(ef->reserved)); > + > + switch (ef->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + if (ops->vidioc_try_ext_pix_fmt_vid_cap) > + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > + ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + if (ops->vidioc_try_ext_pix_fmt_vid_out) > + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > + ef); > + break; > + default: > + return -EINVAL; > + } > + > + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); > + > + ret = v4l_try_fmt(ops, file, fh, &f); > + if (ret) > + /* TODO: retry with M-variant of ef->pixelformat? */ dito > + return ret; > + > + v4l2_format_to_ext_pix_format(&f, ef); > + return 0; > +} > + > static int v4l_streamon(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { > IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0), > IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), > IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), > + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0), > + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO), > + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0), > }; > #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) > > diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h > index edb733f21604..c44708dc9355 100644 > --- a/include/media/v4l2-ioctl.h > +++ b/include/media/v4l2-ioctl.h > @@ -48,11 +48,17 @@ struct v4l2_fh; > * @vidioc_g_fmt_vid_cap: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture > * in single plane mode > + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements > + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video > + * capture > * @vidioc_g_fmt_vid_overlay: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * @vidioc_g_fmt_vid_out: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out > * in single plane mode > + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements > + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video > + * out > * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output > * @vidioc_g_fmt_vbi_cap: pointer to the function that implements > @@ -82,11 +88,16 @@ struct v4l2_fh; > * @vidioc_s_fmt_vid_cap: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture > * in single plane mode > + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements > + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video > + * capture > * @vidioc_s_fmt_vid_overlay: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * @vidioc_s_fmt_vid_out: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out > * in single plane mode > + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements > + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out s/vidioc_g_fmt/vidioc_g_ext_pix_fmt/ ? > * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output > * @vidioc_s_fmt_vbi_cap: pointer to the function that implements > @@ -116,11 +127,16 @@ struct v4l2_fh; > * @vidioc_try_fmt_vid_cap: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture > * in single plane mode > + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements > + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for > + video capture > * @vidioc_try_fmt_vid_overlay: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * @vidioc_try_fmt_vid_out: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out > * in single plane mode > + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements > + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out similar dito Thanks, Dafna > * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * output > @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { > /* VIDIOC_G_FMT handlers */ > int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, > @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { > /* VIDIOC_S_FMT handlers */ > int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, > @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { > /* VIDIOC_TRY_FMT handlers */ > int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index d9b7c9177605..a2d850513708 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { > __u8 reserved[7]; > } __attribute__ ((packed)); > > +/** > + * struct v4l2_ext_pix_format - extended single/multiplanar format definition > + * @type: type of the data stream; V4L2_BUF_TYPE_VIDEO_CAPTURE or > + * V4L2_BUF_TYPE_VIDEO_OUTPUT > + * @width: image width in pixels > + * @height: image height in pixels > + * @field: enum v4l2_field; field order (for interlaced video) > + * @plane_fmt: per-plane information > + * @pixelformat: little endian four character code (fourcc) > + * @modifier: modifier applied to the format (used for tiled formats > + * and other kind of HW-specific formats, like compressed > + * formats) as defined in drm_fourcc.h > + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat > + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function > + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding > + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding > + * @quantization: enum v4l2_quantization, colorspace quantization > + * @reserved: extra space reserved for future fields, must be set to 0 > + */ > +struct v4l2_ext_pix_format { > + __u32 type; > + __u32 width; > + __u32 height; > + __u32 field; > + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; > + __u32 pixelformat; > + __u64 modifier; > + __u32 colorspace; > + __u32 xfer_func; > + union { > + __u32 ycbcr_enc; > + __u32 hsv_enc; > + }; > + __u32 quantization; > + __u32 reserved[9]; > +}; > + > /** > * struct v4l2_sdr_format - SDR format definition > * @pixelformat: little endian four character code (fourcc) > @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { > > #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) > > +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format) > +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format) > +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format) > + > /* Reminder: when adding new ioctls please add support for them to > drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ > >
Hi Helen, On 14/01/2021 19:07, Helen Koike wrote: > This is part of the multiplanar and singleplanar unification process. > v4l2_ext_pix_format is supposed to work for both cases. > > We also add the concept of modifiers already employed in DRM to expose > HW-specific formats (like tiled or compressed formats) and allow > exchanging this information with the DRM subsystem in a consistent way. > > Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in > v4l2_ext_format, other types will be rejected if you use the > {G,S,TRY}_EXT_PIX_FMT ioctls. > > New hooks have been added to v4l2_ioctl_ops to support those new ioctls > in drivers, but, in the meantime, the core takes care of converting > {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can > still work if the userspace app/lib uses the new ioctls. > > The conversion is also done the other around to allow userspace > apps/libs using {S,G,TRY}_FMT to work with drivers implementing the > _ext_ hooks. I have some small comments below, but also one high level comment: Regarding M variants of pixelformats: this patch 'normalizes' them to regular pixelformats in the extended API. This makes life complicated, and I wonder if this is the right approach. Currently there are two reasons for a driver to support e.g. NV12M: either luma and chroma need to be in two memory banks, or the luma and chroma planes cannot be contiguous due to alignment requirements. The first requirement is still valid for drivers that support the extended API. The second requirement is no longer a reason to support NV12M. But I don't think we should just drop NV12M support if it was already supported before the conversion to this extended API. Since NV12M allocates two buffers instead of one, it is still different from a regular NV12. I would prefer that such drivers support both NV12 and NV12M, so no automatic conversion. A related question is how to handle pixelformat enumeration: with the extended API an NV12 format might work, but not with the old API (e.g. due to memory alignment requirements). I wonder if a VIDIOC_ENUM_EXT_PIX_FMT isn't needed. VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while VIDIOC_ENUM_FMT would just report NV12M. > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> > Signed-off-by: Helen Koike <helen.koike@collabora.com> > --- > > Changes in v6: > The main change here was fixing the conversion, so planes reflects color planes, > and to implement this properly I made major refactors compared to the previous > version. > - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz) > - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) > - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) > - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) > - refactor conversion functions, so planes are color planes (Tomasz) > - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz) > - Use "ef" for extended formats in the framework for consistency (Tomasz) > - Handle xfer_func field in conversions (Tomasz) > - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz) > - Refactor format functions to use v4l_fmt_ioctl_via_ext() > - Several fixes/refactoring/changes > - Remove EXT API for touch devices > > Changes in v5: > - change sizes and reorder fields to avoid holes in the struct and make > it the same for 32 and 64 bits > - removed __attribute__ ((packed)) from uapi structs > - Fix doc warning from make htmldocs > - Updated commit message with EXT_PIX prefix for the ioctls. > > Changes in v4: > - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, > making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. > - Add reserved fields > - Removed num_planes from struct v4l2_ext_pix_format > - Removed flag field from struct v4l2_ext_pix_format, since the only > defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, > where we can use modifiers, or add it back later through the reserved > bits. > - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && > != MOD_INVALID > - Fix type assignment in v4l_g_fmt_ext_pix() > - Rebased on top of media/master (post 5.8-rc1) > > Changes in v3: > - Rebased on top of media/master (post 5.4-rc1) > > Changes in v2: > - Move the modifier in v4l2_ext_format (was formerly placed in > v4l2_ext_plane) > - Fix a few bugs in the converters and add a strict parameter to > allow conversion of uninitialized/mis-initialized objects > --- > drivers/media/v4l2-core/v4l2-dev.c | 27 +- > drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- > include/media/v4l2-ioctl.h | 28 ++ > include/uapi/linux/videodev2.h | 41 ++ > 4 files changed, 602 insertions(+), 32 deletions(-) > > diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c > index f9cff033d0dc..5add58cb6d45 100644 > --- a/drivers/media/v4l2-core/v4l2-dev.c > +++ b/drivers/media/v4l2-core/v4l2-dev.c > @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct video_device *vdev) > ops->vidioc_enum_fmt_vid_overlay)) || > (is_tx && ops->vidioc_enum_fmt_vid_out)) > set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); > + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || > + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) > + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > if ((is_rx && (ops->vidioc_g_fmt_vid_cap || > ops->vidioc_g_fmt_vid_cap_mplane || > - ops->vidioc_g_fmt_vid_overlay)) || > + ops->vidioc_g_ext_pix_fmt_vid_cap)) || > (is_tx && (ops->vidioc_g_fmt_vid_out || > ops->vidioc_g_fmt_vid_out_mplane || > - ops->vidioc_g_fmt_vid_out_overlay))) > + ops->vidioc_g_ext_pix_fmt_vid_out))) { > set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); > + } > + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || > + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) > + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > if ((is_rx && (ops->vidioc_s_fmt_vid_cap || > ops->vidioc_s_fmt_vid_cap_mplane || > - ops->vidioc_s_fmt_vid_overlay)) || > + ops->vidioc_s_ext_pix_fmt_vid_cap)) || > (is_tx && (ops->vidioc_s_fmt_vid_out || > ops->vidioc_s_fmt_vid_out_mplane || > - ops->vidioc_s_fmt_vid_out_overlay))) > + ops->vidioc_s_ext_pix_fmt_vid_out))) { > set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); > + } > + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || > + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) > + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > if ((is_rx && (ops->vidioc_try_fmt_vid_cap || > ops->vidioc_try_fmt_vid_cap_mplane || > - ops->vidioc_try_fmt_vid_overlay)) || > + ops->vidioc_try_ext_pix_fmt_vid_cap)) || > (is_tx && (ops->vidioc_try_fmt_vid_out || > ops->vidioc_try_fmt_vid_out_mplane || > - ops->vidioc_try_fmt_vid_out_overlay))) > + ops->vidioc_try_ext_pix_fmt_vid_out))) { > set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); > + } > SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); > SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); > SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > index 848286a284f6..a9c07c0a73ec 100644 > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > @@ -18,6 +18,8 @@ > > #include <linux/videodev2.h> > > +#include <drm/drm_fourcc.h> > + > #include <media/v4l2-common.h> > #include <media/v4l2-ioctl.h> > #include <media/v4l2-ctrls.h> > @@ -38,6 +40,11 @@ > > #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls) > > +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ > + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ > + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ > + V4L2_CAP_VIDEO_M2M_MPLANE)) > + > struct std_descr { > v4l2_std_id std; > const char *descr; > @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, bool write_only) > } > } > > +static void v4l_print_ext_pix_format(const void *arg, bool write_only) > +{ > + const struct v4l2_ext_pix_format *ef = arg; > + unsigned int i; > + > + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n", > + prt_names(ef->type, v4l2_type_names), > + ef->width, ef->height, > + (ef->pixelformat & 0xff), > + (ef->pixelformat >> 8) & 0xff, > + (ef->pixelformat >> 16) & 0xff, > + (ef->pixelformat >> 24) & 0xff, > + ef->modifier, prt_names(ef->field, v4l2_field_names), > + ef->colorspace, ef->ycbcr_enc, > + ef->quantization, ef->xfer_func); > + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", > + i, ef->plane_fmt[i].bytesperline, > + ef->plane_fmt[i].sizeimage); > +} > + > static void v4l_print_framebuffer(const void *arg, bool write_only) > { > const struct v4l2_framebuffer *p = arg; > @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) > switch (type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > if ((is_vid || is_tch) && is_rx && > - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane)) > + (ops->vidioc_g_fmt_vid_cap || > + ops->vidioc_g_ext_pix_fmt_vid_cap || > + ops->vidioc_g_fmt_vid_cap_mplane)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - if ((is_vid || is_tch) && is_rx && ops->vidioc_g_fmt_vid_cap_mplane) > + if ((is_vid || is_tch) && is_rx && > + (ops->vidioc_g_fmt_vid_cap_mplane || > + ops->vidioc_g_ext_pix_fmt_vid_cap)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) > break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > if (is_vid && is_tx && > - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane)) > + (ops->vidioc_g_fmt_vid_out || > + ops->vidioc_g_ext_pix_fmt_vid_out || > + ops->vidioc_g_fmt_vid_out_mplane)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) > + if (is_vid && is_tx && > + (ops->vidioc_g_ext_pix_fmt_vid_out || > + ops->vidioc_g_fmt_vid_out_mplane)) > return 0; > break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct v4l2_format *fmt) > sizeof(fmt->fmt.pix) - offset); > } > > +static void > +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, > + struct v4l2_pix_format *pix) > +{ > + unsigned int i; > + > + pix->width = ef->width; > + pix->height = ef->height; > + pix->field = ef->field; > + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > + pix->colorspace = ef->colorspace; > + pix->ycbcr_enc = ef->ycbcr_enc; > + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; > + pix->quantization = ef->quantization; > + pix->pixelformat = ef->pixelformat; > + pix->bytesperline = ef->plane_fmt[0].bytesperline; > + pix->sizeimage = ef->plane_fmt[0].sizeimage; > + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pix->sizeimage += ef->plane_fmt[i].sizeimage; > +} > + > +static void > +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef, > + struct v4l2_pix_format_mplane *pix_mp) > +{ > + const struct v4l2_format_info *info = > + v4l2_format_info(ef->pixelformat); > + unsigned int i; > + > + pix_mp->width = ef->width; > + pix_mp->height = ef->height; > + pix_mp->field = ef->field; > + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > + pix_mp->colorspace = ef->colorspace; > + pix_mp->ycbcr_enc = ef->ycbcr_enc; > + pix_mp->quantization = ef->quantization; > + pix_mp->pixelformat = ef->pixelformat; > + > + /* This is true when converting to non-M-variant */ > + if (info && info->mem_planes == 1) { > + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; > + pix_mp->num_planes = 1; > + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage; > + > + return; > + } > + > + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) > + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; > + pix_mp->num_planes = i; > +} > + > +/* > + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format > + * > + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. > + * @f: A pointer to struct v4l2_format to be filled. > + * @is_mplane: Bool indicating if multiplanar API should be used in @f. > + * > + * If pixelformat should be converted to M-variant, change ef->pixelformat > + * to the M-variant before calling this function. > + */ > +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef, > + struct v4l2_format *f, bool is_mplane) > +{ > + memset(f, 0, sizeof(*f)); > + > + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && > + ef->modifier != DRM_FORMAT_MOD_INVALID) > + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n", > + ef->modifier); > + > + if (!is_mplane) { > + f->type = ef->type; > + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); > + return; > + } > + > + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) > + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; > + else > + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; > + > + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); > +} > + > +static void > +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, > + struct v4l2_ext_pix_format *ef) > +{ > + const struct v4l2_format_info *info = > + v4l2_format_info(pix->pixelformat); > + unsigned int i; > + > + ef->width = pix->width; > + ef->height = pix->height; > + ef->field = pix->field; > + ef->colorspace = pix->colorspace; > + ef->ycbcr_enc = pix->ycbcr_enc; > + ef->quantization = pix->quantization; > + ef->xfer_func = pix->xfer_func; > + if (pix->flags) > + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); > + > + /* We assume M-variants won't be used in this function */ > + ef->pixelformat = pix->pixelformat; > + > + ef->plane_fmt[0].bytesperline = pix->bytesperline; > + ef->plane_fmt[0].sizeimage = pix->sizeimage; > + > + if (!info) > + return; > + > + for (i = 1; i < info->comp_planes; i++) { > + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; > + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * > + ef->height / info->vdiv; > + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > + } > +} > + > +static void > +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp, > + struct v4l2_ext_pix_format *ef) > +{ > + const struct v4l2_format_info *info = > + v4l2_format_info(pix_mp->pixelformat); > + unsigned int i; > + > + ef->width = pix_mp->width; > + ef->height = pix_mp->height; > + ef->field = pix_mp->field; > + ef->colorspace = pix_mp->colorspace; > + ef->ycbcr_enc = pix_mp->ycbcr_enc; > + ef->quantization = pix_mp->quantization; > + ef->xfer_func = pix_mp->xfer_func; > + if (pix_mp->flags) > + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); > + > + if (!info) > + return; > + > + ef->pixelformat = info && info->norm ? 'info &&' can be dropped, info is always non-NULL here. > + info->norm : pix_mp->pixelformat; > + > + if (info->comp_planes == info->mem_planes) { > + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++) > + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; > + > + return; > + } > + > + /* case where mem_planes is 1 and comp_planes > 1 */ > + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; > + for (i = 1; i < info->comp_planes; i++) { > + ef->plane_fmt[i].bytesperline = > + pix_mp->plane_fmt[0].bytesperline / info->hdiv; > + ef->plane_fmt[i].sizeimage = > + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; > + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > + } > +} > + > +/* > + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format > + * > + * @f: A pointer to struct v4l2_format to be converted. > + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. > + * > + * This method normalize the pixelformat to non-M variant. > + */ > +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, > + struct v4l2_ext_pix_format *ef) > +{ > + memset(ef, 0, sizeof(*ef)); > + > + switch (f->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + ef->type = f->type; > + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > + break; > + default: > + WARN("Converting to Ext Pix Format with wrong buffer type %s\n", > + prt_names(f->type, v4l2_type_names)); > + break; > + } > +} > + > static int v4l_querycap(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct v4l2_pix_format *p) > p->xfer_func = 0; > } > > +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, > + struct v4l2_format *f, > + unsigned int ioctl) > +{ > + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); > + struct video_device *vdev = video_devdata(file); > + struct v4l2_ext_pix_format ef = {0}; > + u32 original_pixfmt = 0; > + u32 cap_mask; > + int ret; > + > + if (ioctl != VIDIOC_G_FMT) { > + /* > + * If CSC attributes are read only, set them to DEFAULT > + * to avoid changes by the driver. > + */ > + if (is_multiplanar) { > + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; > + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; > + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; > + } > + /* Unset the flag to avoid warning in the convertion */ > + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > + > + /* Save pixelformat in case M-variant is being used */ > + original_pixfmt = f->fmt.pix_mp.pixelformat; > + } else { > + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; > + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; > + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; > + } > + /* Unset the flag to avoid warning in the convertion */ > + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > + } > + v4l2_format_to_ext_pix_format(f, &ef); > + } > + > + switch (f->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | > + V4L2_CAP_VIDEO_M2M_MPLANE; > + if (!!(vdev->device_caps & cap_mask) != > + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) > + return -EINVAL; > + > + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > + if (ioctl == VIDIOC_G_FMT) > + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); > + else if (ioctl == VIDIOC_S_FMT) > + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); > + else > + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > + &ef); > + break; > + > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | > + V4L2_CAP_VIDEO_M2M_MPLANE; > + if (!!(vdev->device_caps & cap_mask) != > + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) > + return -EINVAL; > + > + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > + if (ioctl == VIDIOC_G_FMT) > + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); > + else if (ioctl == VIDIOC_S_FMT) > + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); > + else > + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > + &ef); > + break; > + > + default: > + return -EINVAL; > + } > + > + if (ret) > + return ret; > + > + if (original_pixfmt != ef.pixelformat && > + v4l2_format_info(original_pixfmt)) Could this test be simplified to: 'if (original_pixfmt)'? I.e., if the original pixfmt was saved, then restore it here. > + ef.pixelformat = original_pixfmt; > + > + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); > + return 0; > +} > + > static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > > switch (p->type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) > + if (unlikely(!ops->vidioc_g_fmt_vid_cap && > + !ops->vidioc_g_ext_pix_fmt_vid_cap)) > break; > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); > + ret = ops->vidioc_g_fmt_vid_cap ? > + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_G_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > if (vfd->vfl_type == VFL_TYPE_TOUCH) > v4l_pix_format_touch(&p->fmt.pix); > return ret; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > + if (ops->vidioc_g_fmt_vid_cap_mplane) > + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) 'else' can be dropped. > + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_G_FMT); > + break; > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); > case V4L2_BUF_TYPE_VBI_CAPTURE: > @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: > return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > - if (unlikely(!ops->vidioc_g_fmt_vid_out)) > + if (unlikely(!ops->vidioc_g_fmt_vid_out && > + !ops->vidioc_g_ext_pix_fmt_vid_out)) > break; > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); > + ret = ops->vidioc_g_fmt_vid_out ? > + ops->vidioc_g_fmt_vid_out(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > return ret; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > + if (ops->vidioc_g_fmt_vid_out_mplane) > + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > + else if (ops->vidioc_g_ext_pix_fmt_vid_out) Ditto. > + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_G_FMT); > + break; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); > case V4L2_BUF_TYPE_VBI_OUTPUT: > @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > return -EINVAL; > } > > +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, void *arg) > +{ > + struct v4l2_ext_pix_format *ef = arg; > + struct v4l2_format f = { > + .type = ef->type, > + }; > + int ret = check_fmt(file, ef->type); > + > + if (ret) > + return ret; > + > + memset(ef, 0, sizeof(*ef)); > + ef->type = f.type; > + > + switch (f.type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + if (ops->vidioc_g_ext_pix_fmt_vid_cap) > + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + if (ops->vidioc_g_ext_pix_fmt_vid_out) > + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); > + break; > + default: > + return -EINVAL; > + } > + > + ret = v4l_g_fmt(ops, file, fh, &f); > + if (ret) > + return ret; > + > + v4l2_format_to_ext_pix_format(&f, ef); > + return 0; > +} > + > static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > > switch (p->type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) > + if (unlikely(!ops->vidioc_s_fmt_vid_cap && > + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); > + ret = ops->vidioc_s_fmt_vid_cap ? > + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > if (vfd->vfl_type == VFL_TYPE_TOUCH) > v4l_pix_format_touch(&p->fmt.pix); > return ret; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) > + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && > + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); > + return ops->vidioc_s_fmt_vid_cap_mplane ? > + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) > break; > @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > - if (unlikely(!ops->vidioc_s_fmt_vid_out)) > + if (unlikely(!ops->vidioc_s_fmt_vid_out && > + !ops->vidioc_s_ext_pix_fmt_vid_out)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); > + ret = ops->vidioc_s_fmt_vid_out ? > + ops->vidioc_s_fmt_vid_out(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > return ret; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) > + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && > + !ops->vidioc_s_ext_pix_fmt_vid_out)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); > + return ops->vidioc_s_fmt_vid_out_mplane ? > + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) > break; > @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > return -EINVAL; > } > > +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, void *arg) > +{ > + struct video_device *vfd = video_devdata(file); > + struct v4l2_ext_pix_format *ef = arg; > + struct v4l2_format f; > + int ret = check_fmt(file, ef->type); > + > + if (ret) > + return ret; > + > + memset(ef->reserved, 0, sizeof(ef->reserved)); > + > + switch (ef->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + if (ops->vidioc_s_ext_pix_fmt_vid_cap) > + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + if (ops->vidioc_s_ext_pix_fmt_vid_out) > + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); > + break; > + default: > + return -EINVAL; > + } > + > + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); > + > + ret = v4l_s_fmt(ops, file, fh, &f); > + if (ret) > + /* TODO: retry with M-variant of ef->pixelformat? */ See my comments on this at the top. > + return ret; > + > + v4l2_format_to_ext_pix_format(&f, ef); > + return 0; > +} > + > static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > > switch (p->type) { > case V4L2_BUF_TYPE_VIDEO_CAPTURE: > - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) > + if (unlikely(!ops->vidioc_try_fmt_vid_cap && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); > + ret = ops->vidioc_try_fmt_vid_cap ? > + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > if (vfd->vfl_type == VFL_TYPE_TOUCH) > v4l_pix_format_touch(&p->fmt.pix); > return ret; > case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) > + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); > + return ops->vidioc_try_fmt_vid_cap_mplane ? > + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_TRY_FMT); > case V4L2_BUF_TYPE_VIDEO_OVERLAY: > if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) > break; > @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); > case V4L2_BUF_TYPE_VIDEO_OUTPUT: > - if (unlikely(!ops->vidioc_try_fmt_vid_out)) > + if (unlikely(!ops->vidioc_try_fmt_vid_out && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix); > - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); > + ret = ops->vidioc_try_fmt_vid_out ? > + ops->vidioc_try_fmt_vid_out(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); > /* just in case the driver zeroed it again */ > p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > return ret; > case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) > + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && > + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > break; > CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > bytesperline); > - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); > + return ops->vidioc_try_fmt_vid_out_mplane ? > + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : > + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > + VIDIOC_TRY_FMT); > case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) > break; > @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > return -EINVAL; > } > > +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > + struct file *file, void *fh, void *arg) > +{ > + struct video_device *vfd = video_devdata(file); > + struct v4l2_ext_pix_format *ef = arg; > + struct v4l2_format f; > + int ret = check_fmt(file, ef->type); > + > + if (ret) > + return ret; > + > + memset(ef->reserved, 0, sizeof(ef->reserved)); > + > + switch (ef->type) { > + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > + if (ops->vidioc_try_ext_pix_fmt_vid_cap) > + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > + ef); > + break; > + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > + if (ops->vidioc_try_ext_pix_fmt_vid_out) > + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > + ef); > + break; > + default: > + return -EINVAL; > + } > + > + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); > + > + ret = v4l_try_fmt(ops, file, fh, &f); > + if (ret) > + /* TODO: retry with M-variant of ef->pixelformat? */ > + return ret; > + > + v4l2_format_to_ext_pix_format(&f, ef); > + return 0; > +} > + > static int v4l_streamon(const struct v4l2_ioctl_ops *ops, > struct file *file, void *fh, void *arg) > { > @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { > IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0), > IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), > IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), > + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0), > + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO), > + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0), > }; > #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) > > diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h > index edb733f21604..c44708dc9355 100644 > --- a/include/media/v4l2-ioctl.h > +++ b/include/media/v4l2-ioctl.h > @@ -48,11 +48,17 @@ struct v4l2_fh; > * @vidioc_g_fmt_vid_cap: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture > * in single plane mode > + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements > + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video > + * capture > * @vidioc_g_fmt_vid_overlay: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * @vidioc_g_fmt_vid_out: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out > * in single plane mode > + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements > + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video > + * out > * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements > * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output > * @vidioc_g_fmt_vbi_cap: pointer to the function that implements > @@ -82,11 +88,16 @@ struct v4l2_fh; > * @vidioc_s_fmt_vid_cap: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture > * in single plane mode > + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements > + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video > + * capture > * @vidioc_s_fmt_vid_overlay: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * @vidioc_s_fmt_vid_out: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out > * in single plane mode > + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements > + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out > * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements > * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output > * @vidioc_s_fmt_vbi_cap: pointer to the function that implements > @@ -116,11 +127,16 @@ struct v4l2_fh; > * @vidioc_try_fmt_vid_cap: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture > * in single plane mode > + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements > + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for > + video capture > * @vidioc_try_fmt_vid_overlay: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * @vidioc_try_fmt_vid_out: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out > * in single plane mode > + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements > + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out > * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements > * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay > * output > @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { > /* VIDIOC_G_FMT handlers */ > int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, > @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { > /* VIDIOC_S_FMT handlers */ > int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, > @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { > /* VIDIOC_TRY_FMT handlers */ > int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, > struct v4l2_format *f); > + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, > + struct v4l2_ext_pix_format *ef); > int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, > struct v4l2_format *f); > int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index d9b7c9177605..a2d850513708 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { > __u8 reserved[7]; > } __attribute__ ((packed)); > > +/** > + * struct v4l2_ext_pix_format - extended single/multiplanar format definition > + * @type: type of the data stream; V4L2_BUF_TYPE_VIDEO_CAPTURE or > + * V4L2_BUF_TYPE_VIDEO_OUTPUT > + * @width: image width in pixels > + * @height: image height in pixels > + * @field: enum v4l2_field; field order (for interlaced video) > + * @plane_fmt: per-plane information > + * @pixelformat: little endian four character code (fourcc) > + * @modifier: modifier applied to the format (used for tiled formats > + * and other kind of HW-specific formats, like compressed > + * formats) as defined in drm_fourcc.h > + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat > + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function > + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding > + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding > + * @quantization: enum v4l2_quantization, colorspace quantization > + * @reserved: extra space reserved for future fields, must be set to 0 > + */ > +struct v4l2_ext_pix_format { > + __u32 type; > + __u32 width; > + __u32 height; > + __u32 field; > + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; > + __u32 pixelformat; > + __u64 modifier; > + __u32 colorspace; > + __u32 xfer_func; > + union { > + __u32 ycbcr_enc; > + __u32 hsv_enc; > + }; > + __u32 quantization; > + __u32 reserved[9]; > +}; > + > /** > * struct v4l2_sdr_format - SDR format definition > * @pixelformat: little endian four character code (fourcc) > @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { > > #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) > > +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format) > +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format) > +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format) > + > /* Reminder: when adding new ioctls please add support for them to > drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ > > Regards, Hans
Hi Hans, Thank you for your comment, please see my reply below. On 2/23/21 9:35 AM, Hans Verkuil wrote: > Hi Helen, > > On 14/01/2021 19:07, Helen Koike wrote: >> This is part of the multiplanar and singleplanar unification process. >> v4l2_ext_pix_format is supposed to work for both cases. >> >> We also add the concept of modifiers already employed in DRM to expose >> HW-specific formats (like tiled or compressed formats) and allow >> exchanging this information with the DRM subsystem in a consistent way. >> >> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in >> v4l2_ext_format, other types will be rejected if you use the >> {G,S,TRY}_EXT_PIX_FMT ioctls. >> >> New hooks have been added to v4l2_ioctl_ops to support those new ioctls >> in drivers, but, in the meantime, the core takes care of converting >> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can >> still work if the userspace app/lib uses the new ioctls. >> >> The conversion is also done the other around to allow userspace >> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the >> _ext_ hooks. > > I have some small comments below, but also one high level comment: > > Regarding M variants of pixelformats: this patch 'normalizes' them to > regular pixelformats in the extended API. This makes life complicated, > and I wonder if this is the right approach. > > Currently there are two reasons for a driver to support e.g. NV12M: > either luma and chroma need to be in two memory banks, or the luma > and chroma planes cannot be contiguous due to alignment requirements. > > The first requirement is still valid for drivers that support the extended API. > The second requirement is no longer a reason to support NV12M. But I > don't think we should just drop NV12M support if it was already supported > before the conversion to this extended API. Since NV12M allocates two buffers > instead of one, it is still different from a regular NV12. I don't see what would be the difference when using NV12 and NV12M in Ext API. NV12 could be used for both requirements. It would even allow the second requirement with a single memory buffer. > > I would prefer that such drivers support both NV12 and NV12M, so no > automatic conversion. > > A related question is how to handle pixelformat enumeration: with the > extended API an NV12 format might work, but not with the old API (e.g. > due to memory alignment requirements). I wonder if a VIDIOC_ENUM_EXT_PIX_FMT > isn't needed. We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. If the driver reports NV12M, userspace can use it and the framework normilizes it. > > VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while VIDIOC_ENUM_FMT > would just report NV12M. If NV12 and NV12M are equivalent in Ext API, I don't see why we would report both (unless I'm missing something, which is probably the case). The idea was to deprecate the M-variants one day. Regards, Helen > >> >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> >> Signed-off-by: Helen Koike <helen.koike@collabora.com> >> --- >> >> Changes in v6: >> The main change here was fixing the conversion, so planes reflects color planes, >> and to implement this properly I made major refactors compared to the previous >> version. >> - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz) >> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) >> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) >> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) >> - refactor conversion functions, so planes are color planes (Tomasz) >> - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz) >> - Use "ef" for extended formats in the framework for consistency (Tomasz) >> - Handle xfer_func field in conversions (Tomasz) >> - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz) >> - Refactor format functions to use v4l_fmt_ioctl_via_ext() >> - Several fixes/refactoring/changes >> - Remove EXT API for touch devices >> >> Changes in v5: >> - change sizes and reorder fields to avoid holes in the struct and make >> it the same for 32 and 64 bits >> - removed __attribute__ ((packed)) from uapi structs >> - Fix doc warning from make htmldocs >> - Updated commit message with EXT_PIX prefix for the ioctls. >> >> Changes in v4: >> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, >> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >> - Add reserved fields >> - Removed num_planes from struct v4l2_ext_pix_format >> - Removed flag field from struct v4l2_ext_pix_format, since the only >> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, >> where we can use modifiers, or add it back later through the reserved >> bits. >> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && >> != MOD_INVALID >> - Fix type assignment in v4l_g_fmt_ext_pix() >> - Rebased on top of media/master (post 5.8-rc1) >> >> Changes in v3: >> - Rebased on top of media/master (post 5.4-rc1) >> >> Changes in v2: >> - Move the modifier in v4l2_ext_format (was formerly placed in >> v4l2_ext_plane) >> - Fix a few bugs in the converters and add a strict parameter to >> allow conversion of uninitialized/mis-initialized objects >> --- >> drivers/media/v4l2-core/v4l2-dev.c | 27 +- >> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- >> include/media/v4l2-ioctl.h | 28 ++ >> include/uapi/linux/videodev2.h | 41 ++ >> 4 files changed, 602 insertions(+), 32 deletions(-) >> >> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c >> index f9cff033d0dc..5add58cb6d45 100644 >> --- a/drivers/media/v4l2-core/v4l2-dev.c >> +++ b/drivers/media/v4l2-core/v4l2-dev.c >> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct video_device *vdev) >> ops->vidioc_enum_fmt_vid_overlay)) || >> (is_tx && ops->vidioc_enum_fmt_vid_out)) >> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); >> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || >> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) >> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || >> ops->vidioc_g_fmt_vid_cap_mplane || >> - ops->vidioc_g_fmt_vid_overlay)) || >> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || >> (is_tx && (ops->vidioc_g_fmt_vid_out || >> ops->vidioc_g_fmt_vid_out_mplane || >> - ops->vidioc_g_fmt_vid_out_overlay))) >> + ops->vidioc_g_ext_pix_fmt_vid_out))) { >> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); >> + } >> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || >> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) >> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || >> ops->vidioc_s_fmt_vid_cap_mplane || >> - ops->vidioc_s_fmt_vid_overlay)) || >> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || >> (is_tx && (ops->vidioc_s_fmt_vid_out || >> ops->vidioc_s_fmt_vid_out_mplane || >> - ops->vidioc_s_fmt_vid_out_overlay))) >> + ops->vidioc_s_ext_pix_fmt_vid_out))) { >> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); >> + } >> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || >> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) >> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || >> ops->vidioc_try_fmt_vid_cap_mplane || >> - ops->vidioc_try_fmt_vid_overlay)) || >> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || >> (is_tx && (ops->vidioc_try_fmt_vid_out || >> ops->vidioc_try_fmt_vid_out_mplane || >> - ops->vidioc_try_fmt_vid_out_overlay))) >> + ops->vidioc_try_ext_pix_fmt_vid_out))) { >> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); >> + } >> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); >> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); >> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); >> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c >> index 848286a284f6..a9c07c0a73ec 100644 >> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >> @@ -18,6 +18,8 @@ >> >> #include <linux/videodev2.h> >> >> +#include <drm/drm_fourcc.h> >> + >> #include <media/v4l2-common.h> >> #include <media/v4l2-ioctl.h> >> #include <media/v4l2-ctrls.h> >> @@ -38,6 +40,11 @@ >> >> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls) >> >> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ >> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ >> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ >> + V4L2_CAP_VIDEO_M2M_MPLANE)) >> + >> struct std_descr { >> v4l2_std_id std; >> const char *descr; >> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, bool write_only) >> } >> } >> >> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) >> +{ >> + const struct v4l2_ext_pix_format *ef = arg; >> + unsigned int i; >> + >> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n", >> + prt_names(ef->type, v4l2_type_names), >> + ef->width, ef->height, >> + (ef->pixelformat & 0xff), >> + (ef->pixelformat >> 8) & 0xff, >> + (ef->pixelformat >> 16) & 0xff, >> + (ef->pixelformat >> 24) & 0xff, >> + ef->modifier, prt_names(ef->field, v4l2_field_names), >> + ef->colorspace, ef->ycbcr_enc, >> + ef->quantization, ef->xfer_func); >> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", >> + i, ef->plane_fmt[i].bytesperline, >> + ef->plane_fmt[i].sizeimage); >> +} >> + >> static void v4l_print_framebuffer(const void *arg, bool write_only) >> { >> const struct v4l2_framebuffer *p = arg; >> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) >> switch (type) { >> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> if ((is_vid || is_tch) && is_rx && >> - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane)) >> + (ops->vidioc_g_fmt_vid_cap || >> + ops->vidioc_g_ext_pix_fmt_vid_cap || >> + ops->vidioc_g_fmt_vid_cap_mplane)) >> return 0; >> break; >> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >> - if ((is_vid || is_tch) && is_rx && ops->vidioc_g_fmt_vid_cap_mplane) >> + if ((is_vid || is_tch) && is_rx && >> + (ops->vidioc_g_fmt_vid_cap_mplane || >> + ops->vidioc_g_ext_pix_fmt_vid_cap)) >> return 0; >> break; >> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) >> break; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> if (is_vid && is_tx && >> - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane)) >> + (ops->vidioc_g_fmt_vid_out || >> + ops->vidioc_g_ext_pix_fmt_vid_out || >> + ops->vidioc_g_fmt_vid_out_mplane)) >> return 0; >> break; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) >> + if (is_vid && is_tx && >> + (ops->vidioc_g_ext_pix_fmt_vid_out || >> + ops->vidioc_g_fmt_vid_out_mplane)) >> return 0; >> break; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct v4l2_format *fmt) >> sizeof(fmt->fmt.pix) - offset); >> } >> >> +static void >> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, >> + struct v4l2_pix_format *pix) >> +{ >> + unsigned int i; >> + >> + pix->width = ef->width; >> + pix->height = ef->height; >> + pix->field = ef->field; >> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >> + pix->colorspace = ef->colorspace; >> + pix->ycbcr_enc = ef->ycbcr_enc; >> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; >> + pix->quantization = ef->quantization; >> + pix->pixelformat = ef->pixelformat; >> + pix->bytesperline = ef->plane_fmt[0].bytesperline; >> + pix->sizeimage = ef->plane_fmt[0].sizeimage; >> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >> + pix->sizeimage += ef->plane_fmt[i].sizeimage; >> +} >> + >> +static void >> +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef, >> + struct v4l2_pix_format_mplane *pix_mp) >> +{ >> + const struct v4l2_format_info *info = >> + v4l2_format_info(ef->pixelformat); >> + unsigned int i; >> + >> + pix_mp->width = ef->width; >> + pix_mp->height = ef->height; >> + pix_mp->field = ef->field; >> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >> + pix_mp->colorspace = ef->colorspace; >> + pix_mp->ycbcr_enc = ef->ycbcr_enc; >> + pix_mp->quantization = ef->quantization; >> + pix_mp->pixelformat = ef->pixelformat; >> + >> + /* This is true when converting to non-M-variant */ >> + if (info && info->mem_planes == 1) { >> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; >> + pix_mp->num_planes = 1; >> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >> + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage; >> + >> + return; >> + } >> + >> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; >> + pix_mp->num_planes = i; >> +} >> + >> +/* >> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format >> + * >> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. >> + * @f: A pointer to struct v4l2_format to be filled. >> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. >> + * >> + * If pixelformat should be converted to M-variant, change ef->pixelformat >> + * to the M-variant before calling this function. >> + */ >> +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef, >> + struct v4l2_format *f, bool is_mplane) >> +{ >> + memset(f, 0, sizeof(*f)); >> + >> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && >> + ef->modifier != DRM_FORMAT_MOD_INVALID) >> + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n", >> + ef->modifier); >> + >> + if (!is_mplane) { >> + f->type = ef->type; >> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); >> + return; >> + } >> + >> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) >> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; >> + else >> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; >> + >> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); >> +} >> + >> +static void >> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, >> + struct v4l2_ext_pix_format *ef) >> +{ >> + const struct v4l2_format_info *info = >> + v4l2_format_info(pix->pixelformat); >> + unsigned int i; >> + >> + ef->width = pix->width; >> + ef->height = pix->height; >> + ef->field = pix->field; >> + ef->colorspace = pix->colorspace; >> + ef->ycbcr_enc = pix->ycbcr_enc; >> + ef->quantization = pix->quantization; >> + ef->xfer_func = pix->xfer_func; >> + if (pix->flags) >> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); >> + >> + /* We assume M-variants won't be used in this function */ >> + ef->pixelformat = pix->pixelformat; >> + >> + ef->plane_fmt[0].bytesperline = pix->bytesperline; >> + ef->plane_fmt[0].sizeimage = pix->sizeimage; >> + >> + if (!info) >> + return; >> + >> + for (i = 1; i < info->comp_planes; i++) { >> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; >> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * >> + ef->height / info->vdiv; >> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >> + } >> +} >> + >> +static void >> +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp, >> + struct v4l2_ext_pix_format *ef) >> +{ >> + const struct v4l2_format_info *info = >> + v4l2_format_info(pix_mp->pixelformat); >> + unsigned int i; >> + >> + ef->width = pix_mp->width; >> + ef->height = pix_mp->height; >> + ef->field = pix_mp->field; >> + ef->colorspace = pix_mp->colorspace; >> + ef->ycbcr_enc = pix_mp->ycbcr_enc; >> + ef->quantization = pix_mp->quantization; >> + ef->xfer_func = pix_mp->xfer_func; >> + if (pix_mp->flags) >> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); >> + >> + if (!info) >> + return; >> + >> + ef->pixelformat = info && info->norm ? > > 'info &&' can be dropped, info is always non-NULL here. > >> + info->norm : pix_mp->pixelformat; >> + >> + if (info->comp_planes == info->mem_planes) { >> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++) >> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; >> + >> + return; >> + } >> + >> + /* case where mem_planes is 1 and comp_planes > 1 */ >> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; >> + for (i = 1; i < info->comp_planes; i++) { >> + ef->plane_fmt[i].bytesperline = >> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; >> + ef->plane_fmt[i].sizeimage = >> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; >> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >> + } >> +} >> + >> +/* >> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format >> + * >> + * @f: A pointer to struct v4l2_format to be converted. >> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. >> + * >> + * This method normalize the pixelformat to non-M variant. >> + */ >> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, >> + struct v4l2_ext_pix_format *ef) >> +{ >> + memset(ef, 0, sizeof(*ef)); >> + >> + switch (f->type) { >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> + ef->type = f->type; >> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); >> + break; >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >> + break; >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >> + break; >> + default: >> + WARN("Converting to Ext Pix Format with wrong buffer type %s\n", >> + prt_names(f->type, v4l2_type_names)); >> + break; >> + } >> +} >> + >> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, >> struct file *file, void *fh, void *arg) >> { >> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct v4l2_pix_format *p) >> p->xfer_func = 0; >> } >> >> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, >> + struct file *file, void *fh, >> + struct v4l2_format *f, >> + unsigned int ioctl) >> +{ >> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); >> + struct video_device *vdev = video_devdata(file); >> + struct v4l2_ext_pix_format ef = {0}; >> + u32 original_pixfmt = 0; >> + u32 cap_mask; >> + int ret; >> + >> + if (ioctl != VIDIOC_G_FMT) { >> + /* >> + * If CSC attributes are read only, set them to DEFAULT >> + * to avoid changes by the driver. >> + */ >> + if (is_multiplanar) { >> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; >> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; >> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; >> + } >> + /* Unset the flag to avoid warning in the convertion */ >> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >> + >> + /* Save pixelformat in case M-variant is being used */ >> + original_pixfmt = f->fmt.pix_mp.pixelformat; >> + } else { >> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; >> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; >> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; >> + } >> + /* Unset the flag to avoid warning in the convertion */ >> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >> + } >> + v4l2_format_to_ext_pix_format(f, &ef); >> + } >> + >> + switch (f->type) { >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | >> + V4L2_CAP_VIDEO_M2M_MPLANE; >> + if (!!(vdev->device_caps & cap_mask) != >> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) >> + return -EINVAL; >> + >> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >> + if (ioctl == VIDIOC_G_FMT) >> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); >> + else if (ioctl == VIDIOC_S_FMT) >> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); >> + else >> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >> + &ef); >> + break; >> + >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | >> + V4L2_CAP_VIDEO_M2M_MPLANE; >> + if (!!(vdev->device_caps & cap_mask) != >> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) >> + return -EINVAL; >> + >> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >> + if (ioctl == VIDIOC_G_FMT) >> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); >> + else if (ioctl == VIDIOC_S_FMT) >> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); >> + else >> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >> + &ef); >> + break; >> + >> + default: >> + return -EINVAL; >> + } >> + >> + if (ret) >> + return ret; >> + >> + if (original_pixfmt != ef.pixelformat && >> + v4l2_format_info(original_pixfmt)) > > Could this test be simplified to: 'if (original_pixfmt)'? > > I.e., if the original pixfmt was saved, then restore it here. > >> + ef.pixelformat = original_pixfmt; >> + >> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); >> + return 0; >> +} >> + >> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >> struct file *file, void *fh, void *arg) >> { >> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >> >> switch (p->type) { >> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) >> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && >> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) >> break; >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); >> + ret = ops->vidioc_g_fmt_vid_cap ? >> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >> + VIDIOC_G_FMT); >> /* just in case the driver zeroed it again */ >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> if (vfd->vfl_type == VFL_TYPE_TOUCH) >> v4l_pix_format_touch(&p->fmt.pix); >> return ret; >> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >> + if (ops->vidioc_g_fmt_vid_cap_mplane) >> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) > > 'else' can be dropped. > >> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >> + VIDIOC_G_FMT); >> + break; >> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); >> case V4L2_BUF_TYPE_VBI_CAPTURE: >> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: >> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); >> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) >> + if (unlikely(!ops->vidioc_g_fmt_vid_out && >> + !ops->vidioc_g_ext_pix_fmt_vid_out)) >> break; >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); >> + ret = ops->vidioc_g_fmt_vid_out ? >> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); >> /* just in case the driver zeroed it again */ >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> return ret; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >> + if (ops->vidioc_g_fmt_vid_out_mplane) >> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) > > Ditto. > >> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >> + VIDIOC_G_FMT); >> + break; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); >> case V4L2_BUF_TYPE_VBI_OUTPUT: >> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >> return -EINVAL; >> } >> >> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >> + struct file *file, void *fh, void *arg) >> +{ >> + struct v4l2_ext_pix_format *ef = arg; >> + struct v4l2_format f = { >> + .type = ef->type, >> + }; >> + int ret = check_fmt(file, ef->type); >> + >> + if (ret) >> + return ret; >> + >> + memset(ef, 0, sizeof(*ef)); >> + ef->type = f.type; >> + >> + switch (f.type) { >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) >> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); >> + break; >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> + if (ops->vidioc_g_ext_pix_fmt_vid_out) >> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); >> + break; >> + default: >> + return -EINVAL; >> + } >> + >> + ret = v4l_g_fmt(ops, file, fh, &f); >> + if (ret) >> + return ret; >> + >> + v4l2_format_to_ext_pix_format(&f, ef); >> + return 0; >> +} >> + >> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >> struct file *file, void *fh, void *arg) >> { >> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >> >> switch (p->type) { >> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) >> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && >> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix); >> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); >> + ret = ops->vidioc_s_fmt_vid_cap ? >> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >> /* just in case the driver zeroed it again */ >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> if (vfd->vfl_type == VFL_TYPE_TOUCH) >> v4l_pix_format_touch(&p->fmt.pix); >> return ret; >> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) >> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && >> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >> bytesperline); >> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); >> + return ops->vidioc_s_fmt_vid_cap_mplane ? >> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) >> break; >> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); >> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) >> + if (unlikely(!ops->vidioc_s_fmt_vid_out && >> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix); >> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); >> + ret = ops->vidioc_s_fmt_vid_out ? >> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >> /* just in case the driver zeroed it again */ >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> return ret; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) >> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && >> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >> bytesperline); >> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); >> + return ops->vidioc_s_fmt_vid_out_mplane ? >> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) >> break; >> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >> return -EINVAL; >> } >> >> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >> + struct file *file, void *fh, void *arg) >> +{ >> + struct video_device *vfd = video_devdata(file); >> + struct v4l2_ext_pix_format *ef = arg; >> + struct v4l2_format f; >> + int ret = check_fmt(file, ef->type); >> + >> + if (ret) >> + return ret; >> + >> + memset(ef->reserved, 0, sizeof(ef->reserved)); >> + >> + switch (ef->type) { >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) >> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); >> + break; >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> + if (ops->vidioc_s_ext_pix_fmt_vid_out) >> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); >> + break; >> + default: >> + return -EINVAL; >> + } >> + >> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); >> + >> + ret = v4l_s_fmt(ops, file, fh, &f); >> + if (ret) >> + /* TODO: retry with M-variant of ef->pixelformat? */ > > See my comments on this at the top. > >> + return ret; >> + >> + v4l2_format_to_ext_pix_format(&f, ef); >> + return 0; >> +} >> + >> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >> struct file *file, void *fh, void *arg) >> { >> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >> >> switch (p->type) { >> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) >> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && >> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix); >> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); >> + ret = ops->vidioc_try_fmt_vid_cap ? >> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); >> /* just in case the driver zeroed it again */ >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> if (vfd->vfl_type == VFL_TYPE_TOUCH) >> v4l_pix_format_touch(&p->fmt.pix); >> return ret; >> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) >> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && >> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >> bytesperline); >> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); >> + return ops->vidioc_try_fmt_vid_cap_mplane ? >> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >> + VIDIOC_TRY_FMT); >> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) >> break; >> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); >> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) >> + if (unlikely(!ops->vidioc_try_fmt_vid_out && >> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix); >> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); >> + ret = ops->vidioc_try_fmt_vid_out ? >> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); >> /* just in case the driver zeroed it again */ >> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >> return ret; >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) >> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && >> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >> break; >> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >> bytesperline); >> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); >> + return ops->vidioc_try_fmt_vid_out_mplane ? >> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : >> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >> + VIDIOC_TRY_FMT); >> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) >> break; >> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >> return -EINVAL; >> } >> >> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >> + struct file *file, void *fh, void *arg) >> +{ >> + struct video_device *vfd = video_devdata(file); >> + struct v4l2_ext_pix_format *ef = arg; >> + struct v4l2_format f; >> + int ret = check_fmt(file, ef->type); >> + >> + if (ret) >> + return ret; >> + >> + memset(ef->reserved, 0, sizeof(ef->reserved)); >> + >> + switch (ef->type) { >> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) >> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >> + ef); >> + break; >> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >> + if (ops->vidioc_try_ext_pix_fmt_vid_out) >> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >> + ef); >> + break; >> + default: >> + return -EINVAL; >> + } >> + >> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); >> + >> + ret = v4l_try_fmt(ops, file, fh, &f); >> + if (ret) >> + /* TODO: retry with M-variant of ef->pixelformat? */ >> + return ret; >> + >> + v4l2_format_to_ext_pix_format(&f, ef); >> + return 0; >> +} >> + >> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, >> struct file *file, void *fh, void *arg) >> { >> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { >> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0), >> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), >> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), >> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0), >> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO), >> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0), >> }; >> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) >> >> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h >> index edb733f21604..c44708dc9355 100644 >> --- a/include/media/v4l2-ioctl.h >> +++ b/include/media/v4l2-ioctl.h >> @@ -48,11 +48,17 @@ struct v4l2_fh; >> * @vidioc_g_fmt_vid_cap: pointer to the function that implements >> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture >> * in single plane mode >> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements >> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video >> + * capture >> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements >> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay >> * @vidioc_g_fmt_vid_out: pointer to the function that implements >> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out >> * in single plane mode >> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements >> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video >> + * out >> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements >> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output >> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements >> @@ -82,11 +88,16 @@ struct v4l2_fh; >> * @vidioc_s_fmt_vid_cap: pointer to the function that implements >> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture >> * in single plane mode >> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements >> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video >> + * capture >> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements >> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay >> * @vidioc_s_fmt_vid_out: pointer to the function that implements >> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out >> * in single plane mode >> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements >> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out >> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements >> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output >> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements >> @@ -116,11 +127,16 @@ struct v4l2_fh; >> * @vidioc_try_fmt_vid_cap: pointer to the function that implements >> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture >> * in single plane mode >> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements >> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for >> + video capture >> * @vidioc_try_fmt_vid_overlay: pointer to the function that implements >> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay >> * @vidioc_try_fmt_vid_out: pointer to the function that implements >> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out >> * in single plane mode >> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements >> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out >> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements >> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay >> * output >> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { >> /* VIDIOC_G_FMT handlers */ >> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, >> struct v4l2_format *f); >> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >> + struct v4l2_ext_pix_format *ef); >> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, >> struct v4l2_format *f); >> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, >> struct v4l2_format *f); >> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, >> + struct v4l2_ext_pix_format *ef); >> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, >> struct v4l2_format *f); >> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, >> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { >> /* VIDIOC_S_FMT handlers */ >> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, >> struct v4l2_format *f); >> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >> + struct v4l2_ext_pix_format *ef); >> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, >> struct v4l2_format *f); >> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, >> struct v4l2_format *f); >> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, >> + struct v4l2_ext_pix_format *ef); >> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, >> struct v4l2_format *f); >> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, >> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { >> /* VIDIOC_TRY_FMT handlers */ >> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, >> struct v4l2_format *f); >> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >> + struct v4l2_ext_pix_format *ef); >> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, >> struct v4l2_format *f); >> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, >> struct v4l2_format *f); >> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, >> + struct v4l2_ext_pix_format *ef); >> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, >> struct v4l2_format *f); >> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >> index d9b7c9177605..a2d850513708 100644 >> --- a/include/uapi/linux/videodev2.h >> +++ b/include/uapi/linux/videodev2.h >> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { >> __u8 reserved[7]; >> } __attribute__ ((packed)); >> >> +/** >> + * struct v4l2_ext_pix_format - extended single/multiplanar format definition >> + * @type: type of the data stream; V4L2_BUF_TYPE_VIDEO_CAPTURE or >> + * V4L2_BUF_TYPE_VIDEO_OUTPUT >> + * @width: image width in pixels >> + * @height: image height in pixels >> + * @field: enum v4l2_field; field order (for interlaced video) >> + * @plane_fmt: per-plane information >> + * @pixelformat: little endian four character code (fourcc) >> + * @modifier: modifier applied to the format (used for tiled formats >> + * and other kind of HW-specific formats, like compressed >> + * formats) as defined in drm_fourcc.h >> + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat >> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function >> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding >> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding >> + * @quantization: enum v4l2_quantization, colorspace quantization >> + * @reserved: extra space reserved for future fields, must be set to 0 >> + */ >> +struct v4l2_ext_pix_format { >> + __u32 type; >> + __u32 width; >> + __u32 height; >> + __u32 field; >> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; >> + __u32 pixelformat; >> + __u64 modifier; >> + __u32 colorspace; >> + __u32 xfer_func; >> + union { >> + __u32 ycbcr_enc; >> + __u32 hsv_enc; >> + }; >> + __u32 quantization; >> + __u32 reserved[9]; >> +}; >> + >> /** >> * struct v4l2_sdr_format - SDR format definition >> * @pixelformat: little endian four character code (fourcc) >> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { >> >> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) >> >> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format) >> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format) >> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format) >> + >> /* Reminder: when adding new ioctls please add support for them to >> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ >> >> > > Regards, > > Hans >
Hello Helen I didn't see any updates from V6 and V7-WIP in your repo. That is what I need to for our complex tile formats in our platform. Any future plane here? Besides I have some ideas on these patches. On 2/24/21 23:12, Helen Koike wrote: > Hi Hans, > > Thank you for your comment, please see my reply below. > > On 2/23/21 9:35 AM, Hans Verkuil wrote: >> Hi Helen, >> >> On 14/01/2021 19:07, Helen Koike wrote: >>> This is part of the multiplanar and singleplanar unification process. >>> v4l2_ext_pix_format is supposed to work for both cases. >>> >>> We also add the concept of modifiers already employed in DRM to expose >>> HW-specific formats (like tiled or compressed formats) and allow >>> exchanging this information with the DRM subsystem in a consistent way. >>> >>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in >>> v4l2_ext_format, other types will be rejected if you use the >>> {G,S,TRY}_EXT_PIX_FMT ioctls. >>> >>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls >>> in drivers, but, in the meantime, the core takes care of converting >>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers >>> can >>> still work if the userspace app/lib uses the new ioctls. >>> >>> The conversion is also done the other around to allow userspace >>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the >>> _ext_ hooks. >> >> I have some small comments below, but also one high level comment: >> >> Regarding M variants of pixelformats: this patch 'normalizes' them to >> regular pixelformats in the extended API. This makes life complicated, >> and I wonder if this is the right approach. >> >> Currently there are two reasons for a driver to support e.g. NV12M: >> either luma and chroma need to be in two memory banks, or the luma >> and chroma planes cannot be contiguous due to alignment requirements. >> >> The first requirement is still valid for drivers that support the >> extended API. >> The second requirement is no longer a reason to support NV12M. But I >> don't think we should just drop NV12M support if it was already supported >> before the conversion to this extended API. Since NV12M allocates two >> buffers >> instead of one, it is still different from a regular NV12. > > I don't see what would be the difference when using NV12 and NV12M in > Ext API. > NV12 could be used for both requirements. It would even allow the second > requirement with a single memory buffer. > Although I don't have problem here to support both single and multiple planes in our hardware. But using the single plane format here is not convience for us. The tile format in our platform is little complex, you can't calculate the stride and sizeimage easily with just the modifier, width and height. Then we don't have a good place to record the offset here. Besides, the luma and chroma planes would always need their own compression meta data planes. If we use NV12 here, that would make a very strange pixel format, one plane for the pixel datas and two planes for the meta data. >> >> I would prefer that such drivers support both NV12 and NV12M, so no >> automatic conversion. >> >> A related question is how to handle pixelformat enumeration: with the >> extended API an NV12 format might work, but not with the old API (e.g. >> due to memory alignment requirements). I wonder if a >> VIDIOC_ENUM_EXT_PIX_FMT >> isn't needed. > > We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. > If the driver reports NV12M, userspace can use it and the framework > normilizes it. > >> >> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while >> VIDIOC_ENUM_FMT >> would just report NV12M. > > If NV12 and NV12M are equivalent in Ext API, I don't see why we would > report both (unless I'm missing something, which is probably the case). > > The idea was to deprecate the M-variants one day. I was thinking the way in DRM API is better, always assuming it would always in a multiple planes. The only problem is we don't have a way to let the allocator that allocate contiguous memory for planes when we need to do that. > > Regards, > Helen > >> >>> >>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> >>> Signed-off-by: Helen Koike <helen.koike@collabora.com> >>> --- >>> >>> Changes in v6: >>> The main change here was fixing the conversion, so planes reflects >>> color planes, >>> and to implement this properly I made major refactors compared to >>> the previous >>> version. >>> - struct v4l2_plane_ext_pix_format removed, using struct >>> v4l2_plane_pix_format instead (Tomasz) >>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) >>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) >>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) >>> - refactor conversion functions, so planes are color planes (Tomasz) >>> - Don't explicitly check for e->modifier != 0 in >>> v4l2_ext_pix_format_to_format() (Tomasz) >>> - Use "ef" for extended formats in the framework for consistency >>> (Tomasz) >>> - Handle xfer_func field in conversions (Tomasz) >>> - Zero reserved fields in v4l_s_ext_pix_fmt() and >>> v4l_try_ext_pix_fmt() (Tomasz) >>> - Refactor format functions to use v4l_fmt_ioctl_via_ext() >>> - Several fixes/refactoring/changes >>> - Remove EXT API for touch devices >>> >>> Changes in v5: >>> - change sizes and reorder fields to avoid holes in the struct and make >>> it the same for 32 and 64 bits >>> - removed __attribute__ ((packed)) from uapi structs >>> - Fix doc warning from make htmldocs >>> - Updated commit message with EXT_PIX prefix for the ioctls. >>> >>> Changes in v4: >>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, >>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >>> - Add reserved fields >>> - Removed num_planes from struct v4l2_ext_pix_format >>> - Removed flag field from struct v4l2_ext_pix_format, since the only >>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, >>> where we can use modifiers, or add it back later through the reserved >>> bits. >>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && >>> != MOD_INVALID >>> - Fix type assignment in v4l_g_fmt_ext_pix() >>> - Rebased on top of media/master (post 5.8-rc1) >>> >>> Changes in v3: >>> - Rebased on top of media/master (post 5.4-rc1) >>> >>> Changes in v2: >>> - Move the modifier in v4l2_ext_format (was formerly placed in >>> v4l2_ext_plane) >>> - Fix a few bugs in the converters and add a strict parameter to >>> allow conversion of uninitialized/mis-initialized objects >>> --- >>> drivers/media/v4l2-core/v4l2-dev.c | 27 +- >>> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- >>> include/media/v4l2-ioctl.h | 28 ++ >>> include/uapi/linux/videodev2.h | 41 ++ >>> 4 files changed, 602 insertions(+), 32 deletions(-) >>> >>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c >>> b/drivers/media/v4l2-core/v4l2-dev.c >>> index f9cff033d0dc..5add58cb6d45 100644 >>> --- a/drivers/media/v4l2-core/v4l2-dev.c >>> +++ b/drivers/media/v4l2-core/v4l2-dev.c >>> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct >>> video_device *vdev) >>> ops->vidioc_enum_fmt_vid_overlay)) || >>> (is_tx && ops->vidioc_enum_fmt_vid_out)) >>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); >>> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || >>> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) >>> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || >>> ops->vidioc_g_fmt_vid_cap_mplane || >>> - ops->vidioc_g_fmt_vid_overlay)) || >>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || >>> (is_tx && (ops->vidioc_g_fmt_vid_out || >>> ops->vidioc_g_fmt_vid_out_mplane || >>> - ops->vidioc_g_fmt_vid_out_overlay))) >>> + ops->vidioc_g_ext_pix_fmt_vid_out))) { >>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); >>> + } >>> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || >>> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) >>> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || >>> ops->vidioc_s_fmt_vid_cap_mplane || >>> - ops->vidioc_s_fmt_vid_overlay)) || >>> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || >>> (is_tx && (ops->vidioc_s_fmt_vid_out || >>> ops->vidioc_s_fmt_vid_out_mplane || >>> - ops->vidioc_s_fmt_vid_out_overlay))) >>> + ops->vidioc_s_ext_pix_fmt_vid_out))) { >>> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); >>> + } >>> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || >>> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) >>> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || >>> ops->vidioc_try_fmt_vid_cap_mplane || >>> - ops->vidioc_try_fmt_vid_overlay)) || >>> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || >>> (is_tx && (ops->vidioc_try_fmt_vid_out || >>> ops->vidioc_try_fmt_vid_out_mplane || >>> - ops->vidioc_try_fmt_vid_out_overlay))) >>> + ops->vidioc_try_ext_pix_fmt_vid_out))) { >>> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); >>> + } >>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); >>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); >>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c >>> b/drivers/media/v4l2-core/v4l2-ioctl.c >>> index 848286a284f6..a9c07c0a73ec 100644 >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >>> @@ -18,6 +18,8 @@ >>> #include <linux/videodev2.h> >>> +#include <drm/drm_fourcc.h> >>> + >>> #include <media/v4l2-common.h> >>> #include <media/v4l2-ioctl.h> >>> #include <media/v4l2-ctrls.h> >>> @@ -38,6 +40,11 @@ >>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), >>> (vfd)->valid_ioctls) >>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ >>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ >>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ >>> + V4L2_CAP_VIDEO_M2M_MPLANE)) >>> + >>> struct std_descr { >>> v4l2_std_id std; >>> const char *descr; >>> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, >>> bool write_only) >>> } >>> } >>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) >>> +{ >>> + const struct v4l2_ext_pix_format *ef = arg; >>> + unsigned int i; >>> + >>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier >>> %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, >>> xfer_func=%u\n", >>> + prt_names(ef->type, v4l2_type_names), >>> + ef->width, ef->height, >>> + (ef->pixelformat & 0xff), >>> + (ef->pixelformat >> 8) & 0xff, >>> + (ef->pixelformat >> 16) & 0xff, >>> + (ef->pixelformat >> 24) & 0xff, >>> + ef->modifier, prt_names(ef->field, v4l2_field_names), >>> + ef->colorspace, ef->ycbcr_enc, >>> + ef->quantization, ef->xfer_func); >>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>> i++) >>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", >>> + i, ef->plane_fmt[i].bytesperline, >>> + ef->plane_fmt[i].sizeimage); >>> +} >>> + >>> static void v4l_print_framebuffer(const void *arg, bool write_only) >>> { >>> const struct v4l2_framebuffer *p = arg; >>> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum >>> v4l2_buf_type type) >>> switch (type) { >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> if ((is_vid || is_tch) && is_rx && >>> - (ops->vidioc_g_fmt_vid_cap || >>> ops->vidioc_g_fmt_vid_cap_mplane)) >>> + (ops->vidioc_g_fmt_vid_cap || >>> + ops->vidioc_g_ext_pix_fmt_vid_cap || >>> + ops->vidioc_g_fmt_vid_cap_mplane)) >>> return 0; >>> break; >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>> - if ((is_vid || is_tch) && is_rx && >>> ops->vidioc_g_fmt_vid_cap_mplane) >>> + if ((is_vid || is_tch) && is_rx && >>> + (ops->vidioc_g_fmt_vid_cap_mplane || >>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) >>> return 0; >>> break; >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum >>> v4l2_buf_type type) >>> break; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> if (is_vid && is_tx && >>> - (ops->vidioc_g_fmt_vid_out || >>> ops->vidioc_g_fmt_vid_out_mplane)) >>> + (ops->vidioc_g_fmt_vid_out || >>> + ops->vidioc_g_ext_pix_fmt_vid_out || >>> + ops->vidioc_g_fmt_vid_out_mplane)) >>> return 0; >>> break; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) >>> + if (is_vid && is_tx && >>> + (ops->vidioc_g_ext_pix_fmt_vid_out || >>> + ops->vidioc_g_fmt_vid_out_mplane)) >>> return 0; >>> break; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct >>> v4l2_format *fmt) >>> sizeof(fmt->fmt.pix) - offset); >>> } >>> +static void >>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, >>> + struct v4l2_pix_format *pix) >>> +{ >>> + unsigned int i; >>> + >>> + pix->width = ef->width; >>> + pix->height = ef->height; >>> + pix->field = ef->field; >>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>> + pix->colorspace = ef->colorspace; >>> + pix->ycbcr_enc = ef->ycbcr_enc; >>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> + pix->quantization = ef->quantization; >>> + pix->pixelformat = ef->pixelformat; >>> + pix->bytesperline = ef->plane_fmt[0].bytesperline; >>> + pix->sizeimage = ef->plane_fmt[0].sizeimage; >>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>> i++) >>> + pix->sizeimage += ef->plane_fmt[i].sizeimage; >>> +} >>> + >>> +static void >>> +v4l2_ext_pix_format_to_pix_mp_format(const struct >>> v4l2_ext_pix_format *ef, >>> + struct v4l2_pix_format_mplane *pix_mp) >>> +{ >>> + const struct v4l2_format_info *info = >>> + v4l2_format_info(ef->pixelformat); >>> + unsigned int i; >>> + >>> + pix_mp->width = ef->width; >>> + pix_mp->height = ef->height; >>> + pix_mp->field = ef->field; >>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>> + pix_mp->colorspace = ef->colorspace; >>> + pix_mp->ycbcr_enc = ef->ycbcr_enc; >>> + pix_mp->quantization = ef->quantization; >>> + pix_mp->pixelformat = ef->pixelformat; >>> + >>> + /* This is true when converting to non-M-variant */ >>> + if (info && info->mem_planes == 1) { >>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; >>> + pix_mp->num_planes = 1; >>> + for (i = 1; i < VIDEO_MAX_PLANES && >>> ef->plane_fmt[i].sizeimage; i++) >>> + pix_mp->plane_fmt[0].sizeimage += >>> ef->plane_fmt[i].sizeimage; >>> + >>> + return; >>> + } >>> + >>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>> i++) >>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; >>> + pix_mp->num_planes = i; >>> +} >>> + >>> +/* >>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to >>> v4l2_format >>> + * >>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. >>> + * @f: A pointer to struct v4l2_format to be filled. >>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. >>> + * >>> + * If pixelformat should be converted to M-variant, change >>> ef->pixelformat >>> + * to the M-variant before calling this function. >>> + */ >>> +static void v4l2_ext_pix_format_to_format(const struct >>> v4l2_ext_pix_format *ef, >>> + struct v4l2_format *f, bool is_mplane) >>> +{ >>> + memset(f, 0, sizeof(*f)); >>> + >>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && >>> + ef->modifier != DRM_FORMAT_MOD_INVALID) >>> + pr_warn("Modifiers are not supported in v4l2_format, >>> ignoring %llx\n", >>> + ef->modifier); >>> + >>> + if (!is_mplane) { >>> + f->type = ef->type; >>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); >>> + return; >>> + } >>> + >>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) >>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; >>> + else >>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; >>> + >>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); >>> +} >>> + >>> +static void >>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, >>> + struct v4l2_ext_pix_format *ef) >>> +{ >>> + const struct v4l2_format_info *info = >>> + v4l2_format_info(pix->pixelformat); >>> + unsigned int i; >>> + >>> + ef->width = pix->width; >>> + ef->height = pix->height; >>> + ef->field = pix->field; >>> + ef->colorspace = pix->colorspace; >>> + ef->ycbcr_enc = pix->ycbcr_enc; >>> + ef->quantization = pix->quantization; >>> + ef->xfer_func = pix->xfer_func; >>> + if (pix->flags) >>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); >>> + >>> + /* We assume M-variants won't be used in this function */ >>> + ef->pixelformat = pix->pixelformat; >>> + >>> + ef->plane_fmt[0].bytesperline = pix->bytesperline; >>> + ef->plane_fmt[0].sizeimage = pix->sizeimage; >>> + >>> + if (!info) >>> + return; >>> + >>> + for (i = 1; i < info->comp_planes; i++) { >>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; >>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * >>> + ef->height / info->vdiv; >>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>> + } >>> +} >>> + >>> +static void >>> +v4l2_pix_mp_format_to_ext_pix_format(const struct >>> v4l2_pix_format_mplane *pix_mp, >>> + struct v4l2_ext_pix_format *ef) >>> +{ >>> + const struct v4l2_format_info *info = >>> + v4l2_format_info(pix_mp->pixelformat); >>> + unsigned int i; >>> + >>> + ef->width = pix_mp->width; >>> + ef->height = pix_mp->height; >>> + ef->field = pix_mp->field; >>> + ef->colorspace = pix_mp->colorspace; >>> + ef->ycbcr_enc = pix_mp->ycbcr_enc; >>> + ef->quantization = pix_mp->quantization; >>> + ef->xfer_func = pix_mp->xfer_func; >>> + if (pix_mp->flags) >>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); >>> + >>> + if (!info) >>> + return; >>> + >>> + ef->pixelformat = info && info->norm ? >> >> 'info &&' can be dropped, info is always non-NULL here. >> >>> + info->norm : pix_mp->pixelformat; >>> + >>> + if (info->comp_planes == info->mem_planes) { >>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; >>> i++) >>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; >>> + >>> + return; >>> + } >>> + >>> + /* case where mem_planes is 1 and comp_planes > 1 */ >>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; >>> + for (i = 1; i < info->comp_planes; i++) { >>> + ef->plane_fmt[i].bytesperline = >>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; >>> + ef->plane_fmt[i].sizeimage = >>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; >>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>> + } >>> +} >>> + >>> +/* >>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to >>> v4l2_ext_pix_format >>> + * >>> + * @f: A pointer to struct v4l2_format to be converted. >>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. >>> + * >>> + * This method normalize the pixelformat to non-M variant. >>> + */ >>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, >>> + struct v4l2_ext_pix_format *ef) >>> +{ >>> + memset(ef, 0, sizeof(*ef)); >>> + >>> + switch (f->type) { >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> + ef->type = f->type; >>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); >>> + break; >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>> + break; >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>> + break; >>> + default: >>> + WARN("Converting to Ext Pix Format with wrong buffer type >>> %s\n", >>> + prt_names(f->type, v4l2_type_names)); >>> + break; >>> + } >>> +} >>> + >>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, >>> struct file *file, void *fh, void *arg) >>> { >>> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct >>> v4l2_pix_format *p) >>> p->xfer_func = 0; >>> } >>> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, >>> + struct file *file, void *fh, >>> + struct v4l2_format *f, >>> + unsigned int ioctl) >>> +{ >>> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); >>> + struct video_device *vdev = video_devdata(file); >>> + struct v4l2_ext_pix_format ef = {0}; >>> + u32 original_pixfmt = 0; >>> + u32 cap_mask; >>> + int ret; >>> + >>> + if (ioctl != VIDIOC_G_FMT) { >>> + /* >>> + * If CSC attributes are read only, set them to DEFAULT >>> + * to avoid changes by the driver. >>> + */ >>> + if (is_multiplanar) { >>> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; >>> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; >>> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>> + } >>> + /* Unset the flag to avoid warning in the convertion */ >>> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>> + >>> + /* Save pixelformat in case M-variant is being used */ >>> + original_pixfmt = f->fmt.pix_mp.pixelformat; >>> + } else { >>> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; >>> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; >>> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>> + } >>> + /* Unset the flag to avoid warning in the convertion */ >>> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>> + } >>> + v4l2_format_to_ext_pix_format(f, &ef); >>> + } >>> + >>> + switch (f->type) { >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | >>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>> + if (!!(vdev->device_caps & cap_mask) != >>> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) >>> + return -EINVAL; >>> + >>> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>> + if (ioctl == VIDIOC_G_FMT) >>> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); >>> + else if (ioctl == VIDIOC_S_FMT) >>> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); >>> + else >>> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>> + &ef); >>> + break; >>> + >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | >>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>> + if (!!(vdev->device_caps & cap_mask) != >>> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) >>> + return -EINVAL; >>> + >>> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>> + if (ioctl == VIDIOC_G_FMT) >>> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); >>> + else if (ioctl == VIDIOC_S_FMT) >>> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); >>> + else >>> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>> + &ef); >>> + break; >>> + >>> + default: >>> + return -EINVAL; >>> + } >>> + >>> + if (ret) >>> + return ret; >>> + >>> + if (original_pixfmt != ef.pixelformat && >>> + v4l2_format_info(original_pixfmt)) >> >> Could this test be simplified to: 'if (original_pixfmt)'? >> >> I.e., if the original pixfmt was saved, then restore it here. >> >>> + ef.pixelformat = original_pixfmt; >>> + >>> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); >>> + return 0; >>> +} >>> + >>> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>> struct file *file, void *fh, void *arg) >>> { >>> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> switch (p->type) { >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) >>> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && >>> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) >>> break; >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); >>> + ret = ops->vidioc_g_fmt_vid_cap ? >>> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> + VIDIOC_G_FMT); >>> /* just in case the driver zeroed it again */ >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>> v4l_pix_format_touch(&p->fmt.pix); >>> return ret; >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>> + if (ops->vidioc_g_fmt_vid_cap_mplane) >>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) >> >> 'else' can be dropped. >> >>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> + VIDIOC_G_FMT); >>> + break; >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); >>> case V4L2_BUF_TYPE_VBI_CAPTURE: >>> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: >>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) >>> + if (unlikely(!ops->vidioc_g_fmt_vid_out && >>> + !ops->vidioc_g_ext_pix_fmt_vid_out)) >>> break; >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); >>> + ret = ops->vidioc_g_fmt_vid_out ? >>> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); >>> /* just in case the driver zeroed it again */ >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> return ret; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>> + if (ops->vidioc_g_fmt_vid_out_mplane) >>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) >> >> Ditto. >> >>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> + VIDIOC_G_FMT); >>> + break; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); >>> case V4L2_BUF_TYPE_VBI_OUTPUT: >>> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> return -EINVAL; >>> } >>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>> + struct file *file, void *fh, void *arg) >>> +{ >>> + struct v4l2_ext_pix_format *ef = arg; >>> + struct v4l2_format f = { >>> + .type = ef->type, >>> + }; >>> + int ret = check_fmt(file, ef->type); >>> + >>> + if (ret) >>> + return ret; >>> + >>> + memset(ef, 0, sizeof(*ef)); >>> + ef->type = f.type; >>> + >>> + switch (f.type) { >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); >>> + break; >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> + if (ops->vidioc_g_ext_pix_fmt_vid_out) >>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); >>> + break; >>> + default: >>> + return -EINVAL; >>> + } >>> + >>> + ret = v4l_g_fmt(ops, file, fh, &f); >>> + if (ret) >>> + return ret; >>> + >>> + v4l2_format_to_ext_pix_format(&f, ef); >>> + return 0; >>> +} >>> + >>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>> struct file *file, void *fh, void *arg) >>> { >>> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> switch (p->type) { >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) >>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && >>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix); >>> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); >>> + ret = ops->vidioc_s_fmt_vid_cap ? >>> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>> /* just in case the driver zeroed it again */ >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>> v4l_pix_format_touch(&p->fmt.pix); >>> return ret; >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) >>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && >>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>> bytesperline); >>> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); >>> + return ops->vidioc_s_fmt_vid_cap_mplane ? >>> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> VIDIOC_S_FMT); >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) >>> break; >>> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) >>> + if (unlikely(!ops->vidioc_s_fmt_vid_out && >>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix); >>> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); >>> + ret = ops->vidioc_s_fmt_vid_out ? >>> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>> /* just in case the driver zeroed it again */ >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> return ret; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) >>> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && >>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>> bytesperline); >>> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); >>> + return ops->vidioc_s_fmt_vid_out_mplane ? >>> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) >>> break; >>> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> return -EINVAL; >>> } >>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>> + struct file *file, void *fh, void *arg) >>> +{ >>> + struct video_device *vfd = video_devdata(file); >>> + struct v4l2_ext_pix_format *ef = arg; >>> + struct v4l2_format f; >>> + int ret = check_fmt(file, ef->type); >>> + >>> + if (ret) >>> + return ret; >>> + >>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>> + >>> + switch (ef->type) { >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) >>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); >>> + break; >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> + if (ops->vidioc_s_ext_pix_fmt_vid_out) >>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); >>> + break; >>> + default: >>> + return -EINVAL; >>> + } >>> + >>> + v4l2_ext_pix_format_to_format(ef, &f, >>> V4L2_IS_CAP_MULTIPLANAR(vfd)); >>> + >>> + ret = v4l_s_fmt(ops, file, fh, &f); >>> + if (ret) >>> + /* TODO: retry with M-variant of ef->pixelformat? */ >> >> See my comments on this at the top. >> >>> + return ret; >>> + >>> + v4l2_format_to_ext_pix_format(&f, ef); >>> + return 0; >>> +} >>> + >>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>> struct file *file, void *fh, void *arg) >>> { >>> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> switch (p->type) { >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) >>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix); >>> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); >>> + ret = ops->vidioc_try_fmt_vid_cap ? >>> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> VIDIOC_TRY_FMT); >>> /* just in case the driver zeroed it again */ >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>> v4l_pix_format_touch(&p->fmt.pix); >>> return ret; >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) >>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>> bytesperline); >>> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); >>> + return ops->vidioc_try_fmt_vid_cap_mplane ? >>> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> + VIDIOC_TRY_FMT); >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) >>> break; >>> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) >>> + if (unlikely(!ops->vidioc_try_fmt_vid_out && >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix); >>> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); >>> + ret = ops->vidioc_try_fmt_vid_out ? >>> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> VIDIOC_TRY_FMT); >>> /* just in case the driver zeroed it again */ >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>> return ret; >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) >>> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>> break; >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>> bytesperline); >>> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); >>> + return ops->vidioc_try_fmt_vid_out_mplane ? >>> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>> + VIDIOC_TRY_FMT); >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) >>> break; >>> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct >>> v4l2_ioctl_ops *ops, >>> return -EINVAL; >>> } >>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>> + struct file *file, void *fh, void *arg) >>> +{ >>> + struct video_device *vfd = video_devdata(file); >>> + struct v4l2_ext_pix_format *ef = arg; >>> + struct v4l2_format f; >>> + int ret = check_fmt(file, ef->type); >>> + >>> + if (ret) >>> + return ret; >>> + >>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>> + >>> + switch (ef->type) { >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) >>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>> + ef); >>> + break; >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>> + if (ops->vidioc_try_ext_pix_fmt_vid_out) >>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>> + ef); >>> + break; >>> + default: >>> + return -EINVAL; >>> + } >>> + >>> + v4l2_ext_pix_format_to_format(ef, &f, >>> V4L2_IS_CAP_MULTIPLANAR(vfd)); >>> + >>> + ret = v4l_try_fmt(ops, file, fh, &f); >>> + if (ret) >>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>> + return ret; >>> + >>> + v4l2_format_to_ext_pix_format(&f, ef); >>> + return 0; >>> +} >>> + >>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, >>> struct file *file, void *fh, void *arg) >>> { >>> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info >>> v4l2_ioctls[] = { >>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, >>> v4l_print_freq_band, 0), >>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, >>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), >>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, >>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | >>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), >>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, >>> v4l_print_ext_pix_format, 0), >>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, >>> v4l_print_ext_pix_format, INFO_FL_PRIO), >>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, >>> v4l_print_ext_pix_format, 0), >>> }; >>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) >>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h >>> index edb733f21604..c44708dc9355 100644 >>> --- a/include/media/v4l2-ioctl.h >>> +++ b/include/media/v4l2-ioctl.h >>> @@ -48,11 +48,17 @@ struct v4l2_fh; >>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture >>> * in single plane mode >>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that >>> implements >>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>> for video >>> + * capture >>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>> * @vidioc_g_fmt_vid_out: pointer to the function that implements >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out >>> * in single plane mode >>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that >>> implements >>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>> for video >>> + * out >>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that >>> implements >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video >>> overlay output >>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements >>> @@ -82,11 +88,16 @@ struct v4l2_fh; >>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture >>> * in single plane mode >>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that >>> implements >>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>> for video >>> + * capture >>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>> * @vidioc_s_fmt_vid_out: pointer to the function that implements >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out >>> * in single plane mode >>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that >>> implements >>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for >>> video out >>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that >>> implements >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video >>> overlay output >>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements >>> @@ -116,11 +127,16 @@ struct v4l2_fh; >>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>> capture >>> * in single plane mode >>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that >>> implements >>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl >>> logic for >>> + video capture >>> * @vidioc_try_fmt_vid_overlay: pointer to the function that >>> implements >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>> overlay >>> * @vidioc_try_fmt_vid_out: pointer to the function that implements >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out >>> * in single plane mode >>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that >>> implements >>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for >>> video out >>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that >>> implements >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>> overlay >>> * output >>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { >>> /* VIDIOC_G_FMT handlers */ >>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>> + struct v4l2_ext_pix_format *ef); >>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>> + struct v4l2_ext_pix_format *ef); >>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, >>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { >>> /* VIDIOC_S_FMT handlers */ >>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>> + struct v4l2_ext_pix_format *ef); >>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>> + struct v4l2_ext_pix_format *ef); >>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, >>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { >>> /* VIDIOC_TRY_FMT handlers */ >>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>> + struct v4l2_ext_pix_format *ef); >>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>> + struct v4l2_ext_pix_format *ef); >>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, >>> struct v4l2_format *f); >>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, >>> diff --git a/include/uapi/linux/videodev2.h >>> b/include/uapi/linux/videodev2.h >>> index d9b7c9177605..a2d850513708 100644 >>> --- a/include/uapi/linux/videodev2.h >>> +++ b/include/uapi/linux/videodev2.h >>> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { >>> __u8 reserved[7]; >>> } __attribute__ ((packed)); >>> +/** >>> + * struct v4l2_ext_pix_format - extended single/multiplanar format >>> definition >>> + * @type: type of the data stream; >>> V4L2_BUF_TYPE_VIDEO_CAPTURE or >>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT >>> + * @width: image width in pixels >>> + * @height: image height in pixels >>> + * @field: enum v4l2_field; field order (for interlaced video) >>> + * @plane_fmt: per-plane information >>> + * @pixelformat: little endian four character code (fourcc) >>> + * @modifier: modifier applied to the format (used for tiled >>> formats >>> + * and other kind of HW-specific formats, like compressed >>> + * formats) as defined in drm_fourcc.h >>> + * @colorspace: enum v4l2_colorspace; supplemental to >>> pixelformat >>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function >>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding >>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding >>> + * @quantization: enum v4l2_quantization, colorspace quantization >>> + * @reserved: extra space reserved for future fields, must be >>> set to 0 >>> + */ >>> +struct v4l2_ext_pix_format { >>> + __u32 type; >>> + __u32 width; >>> + __u32 height; >>> + __u32 field; >>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; >>> + __u32 pixelformat; >>> + __u64 modifier; >>> + __u32 colorspace; >>> + __u32 xfer_func; >>> + union { >>> + __u32 ycbcr_enc; >>> + __u32 hsv_enc; >>> + }; >>> + __u32 quantization; >>> + __u32 reserved[9]; >>> +}; >>> + >>> /** >>> * struct v4l2_sdr_format - SDR format definition >>> * @pixelformat: little endian four character code (fourcc) >>> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { >>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct >>> v4l2_query_ext_ctrl) >>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct >>> v4l2_ext_pix_format) >>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct >>> v4l2_ext_pix_format) >>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct >>> v4l2_ext_pix_format) >>> + >>> /* Reminder: when adding new ioctls please add support for them to >>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ >>> >> >> Regards, >> >> Hans >> >
Hi Hsia-Jun, On Sat, Nov 05, 2022 at 11:19:10PM +0800, Hsia-Jun Li wrote: > Hello Helen > > I didn't see any updates from V6 and V7-WIP in your repo. That is what I > need to for our complex tile formats in our platform. > > Any future plane here? > > Besides I have some ideas on these patches. > > On 2/24/21 23:12, Helen Koike wrote: > > Hi Hans, > > > > Thank you for your comment, please see my reply below. > > > > On 2/23/21 9:35 AM, Hans Verkuil wrote: > >> Hi Helen, > >> > >> On 14/01/2021 19:07, Helen Koike wrote: > >>> This is part of the multiplanar and singleplanar unification process. > >>> v4l2_ext_pix_format is supposed to work for both cases. > >>> > >>> We also add the concept of modifiers already employed in DRM to expose > >>> HW-specific formats (like tiled or compressed formats) and allow > >>> exchanging this information with the DRM subsystem in a consistent way. > >>> > >>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in > >>> v4l2_ext_format, other types will be rejected if you use the > >>> {G,S,TRY}_EXT_PIX_FMT ioctls. > >>> > >>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls > >>> in drivers, but, in the meantime, the core takes care of converting > >>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers > >>> can > >>> still work if the userspace app/lib uses the new ioctls. > >>> > >>> The conversion is also done the other around to allow userspace > >>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the > >>> _ext_ hooks. > >> > >> I have some small comments below, but also one high level comment: > >> > >> Regarding M variants of pixelformats: this patch 'normalizes' them to > >> regular pixelformats in the extended API. This makes life complicated, > >> and I wonder if this is the right approach. > >> > >> Currently there are two reasons for a driver to support e.g. NV12M: > >> either luma and chroma need to be in two memory banks, or the luma > >> and chroma planes cannot be contiguous due to alignment requirements. > >> > >> The first requirement is still valid for drivers that support the > >> extended API. > >> The second requirement is no longer a reason to support NV12M. But I > >> don't think we should just drop NV12M support if it was already supported > >> before the conversion to this extended API. Since NV12M allocates two > >> buffers > >> instead of one, it is still different from a regular NV12. > > > > I don't see what would be the difference when using NV12 and NV12M in > > Ext API. > > NV12 could be used for both requirements. It would even allow the second > > requirement with a single memory buffer. > > > Although I don't have problem here to support both single and multiple > planes in our hardware. But using the single plane format here is not > convience for us. > > The tile format in our platform is little complex, you can't calculate > the stride and sizeimage easily with just the modifier, width and > height. Then we don't have a good place to record the offset here. What else is needed to compute those values ? Can they be computed by the kernel driver, or do they have computed by userspace ? > Besides, the luma and chroma planes would always need their own > compression meta data planes. If we use NV12 here, that would make a > very strange pixel format, one plane for the pixel datas and two planes > for the meta data. > > >> I would prefer that such drivers support both NV12 and NV12M, so no > >> automatic conversion. > >> > >> A related question is how to handle pixelformat enumeration: with the > >> extended API an NV12 format might work, but not with the old API (e.g. > >> due to memory alignment requirements). I wonder if a > >> VIDIOC_ENUM_EXT_PIX_FMT > >> isn't needed. > > > > We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. > > If the driver reports NV12M, userspace can use it and the framework > > normilizes it. > > > >> > >> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > >> VIDIOC_ENUM_FMT > >> would just report NV12M. > > > > If NV12 and NV12M are equivalent in Ext API, I don't see why we would > > report both (unless I'm missing something, which is probably the case). > > > > The idea was to deprecate the M-variants one day. > > I was thinking the way in DRM API is better, always assuming it would > always in a multiple planes. The only problem is we don't have a way to > let the allocator that allocate contiguous memory for planes when we > need to do that. > > >>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> > >>> Signed-off-by: Helen Koike <helen.koike@collabora.com> > >>> --- > >>> > >>> Changes in v6: > >>> The main change here was fixing the conversion, so planes reflects > >>> color planes, > >>> and to implement this properly I made major refactors compared to > >>> the previous > >>> version. > >>> - struct v4l2_plane_ext_pix_format removed, using struct > >>> v4l2_plane_pix_format instead (Tomasz) > >>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) > >>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) > >>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) > >>> - refactor conversion functions, so planes are color planes (Tomasz) > >>> - Don't explicitly check for e->modifier != 0 in > >>> v4l2_ext_pix_format_to_format() (Tomasz) > >>> - Use "ef" for extended formats in the framework for consistency > >>> (Tomasz) > >>> - Handle xfer_func field in conversions (Tomasz) > >>> - Zero reserved fields in v4l_s_ext_pix_fmt() and > >>> v4l_try_ext_pix_fmt() (Tomasz) > >>> - Refactor format functions to use v4l_fmt_ioctl_via_ext() > >>> - Several fixes/refactoring/changes > >>> - Remove EXT API for touch devices > >>> > >>> Changes in v5: > >>> - change sizes and reorder fields to avoid holes in the struct and make > >>> it the same for 32 and 64 bits > >>> - removed __attribute__ ((packed)) from uapi structs > >>> - Fix doc warning from make htmldocs > >>> - Updated commit message with EXT_PIX prefix for the ioctls. > >>> > >>> Changes in v4: > >>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, > >>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. > >>> - Add reserved fields > >>> - Removed num_planes from struct v4l2_ext_pix_format > >>> - Removed flag field from struct v4l2_ext_pix_format, since the only > >>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, > >>> where we can use modifiers, or add it back later through the reserved > >>> bits. > >>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && > >>> != MOD_INVALID > >>> - Fix type assignment in v4l_g_fmt_ext_pix() > >>> - Rebased on top of media/master (post 5.8-rc1) > >>> > >>> Changes in v3: > >>> - Rebased on top of media/master (post 5.4-rc1) > >>> > >>> Changes in v2: > >>> - Move the modifier in v4l2_ext_format (was formerly placed in > >>> v4l2_ext_plane) > >>> - Fix a few bugs in the converters and add a strict parameter to > >>> allow conversion of uninitialized/mis-initialized objects > >>> --- > >>> drivers/media/v4l2-core/v4l2-dev.c | 27 +- > >>> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- > >>> include/media/v4l2-ioctl.h | 28 ++ > >>> include/uapi/linux/videodev2.h | 41 ++ > >>> 4 files changed, 602 insertions(+), 32 deletions(-) > >>> > >>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c > >>> b/drivers/media/v4l2-core/v4l2-dev.c > >>> index f9cff033d0dc..5add58cb6d45 100644 > >>> --- a/drivers/media/v4l2-core/v4l2-dev.c > >>> +++ b/drivers/media/v4l2-core/v4l2-dev.c > >>> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct > >>> video_device *vdev) > >>> ops->vidioc_enum_fmt_vid_overlay)) || > >>> (is_tx && ops->vidioc_enum_fmt_vid_out)) > >>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); > >>> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || > >>> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) > >>> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > >>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || > >>> ops->vidioc_g_fmt_vid_cap_mplane || > >>> - ops->vidioc_g_fmt_vid_overlay)) || > >>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || > >>> (is_tx && (ops->vidioc_g_fmt_vid_out || > >>> ops->vidioc_g_fmt_vid_out_mplane || > >>> - ops->vidioc_g_fmt_vid_out_overlay))) > >>> + ops->vidioc_g_ext_pix_fmt_vid_out))) { > >>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > >>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); > >>> + } > >>> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || > >>> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) > >>> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > >>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || > >>> ops->vidioc_s_fmt_vid_cap_mplane || > >>> - ops->vidioc_s_fmt_vid_overlay)) || > >>> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || > >>> (is_tx && (ops->vidioc_s_fmt_vid_out || > >>> ops->vidioc_s_fmt_vid_out_mplane || > >>> - ops->vidioc_s_fmt_vid_out_overlay))) > >>> + ops->vidioc_s_ext_pix_fmt_vid_out))) { > >>> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > >>> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); > >>> + } > >>> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || > >>> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) > >>> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > >>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || > >>> ops->vidioc_try_fmt_vid_cap_mplane || > >>> - ops->vidioc_try_fmt_vid_overlay)) || > >>> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || > >>> (is_tx && (ops->vidioc_try_fmt_vid_out || > >>> ops->vidioc_try_fmt_vid_out_mplane || > >>> - ops->vidioc_try_fmt_vid_out_overlay))) > >>> + ops->vidioc_try_ext_pix_fmt_vid_out))) { > >>> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > >>> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); > >>> + } > >>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); > >>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); > >>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); > >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c > >>> b/drivers/media/v4l2-core/v4l2-ioctl.c > >>> index 848286a284f6..a9c07c0a73ec 100644 > >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c > >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > >>> @@ -18,6 +18,8 @@ > >>> #include <linux/videodev2.h> > >>> +#include <drm/drm_fourcc.h> > >>> + > >>> #include <media/v4l2-common.h> > >>> #include <media/v4l2-ioctl.h> > >>> #include <media/v4l2-ctrls.h> > >>> @@ -38,6 +40,11 @@ > >>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), > >>> (vfd)->valid_ioctls) > >>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ > >>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ > >>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ > >>> + V4L2_CAP_VIDEO_M2M_MPLANE)) > >>> + > >>> struct std_descr { > >>> v4l2_std_id std; > >>> const char *descr; > >>> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, > >>> bool write_only) > >>> } > >>> } > >>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) > >>> +{ > >>> + const struct v4l2_ext_pix_format *ef = arg; > >>> + unsigned int i; > >>> + > >>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier > >>> %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, > >>> xfer_func=%u\n", > >>> + prt_names(ef->type, v4l2_type_names), > >>> + ef->width, ef->height, > >>> + (ef->pixelformat & 0xff), > >>> + (ef->pixelformat >> 8) & 0xff, > >>> + (ef->pixelformat >> 16) & 0xff, > >>> + (ef->pixelformat >> 24) & 0xff, > >>> + ef->modifier, prt_names(ef->field, v4l2_field_names), > >>> + ef->colorspace, ef->ycbcr_enc, > >>> + ef->quantization, ef->xfer_func); > >>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; > >>> i++) > >>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", > >>> + i, ef->plane_fmt[i].bytesperline, > >>> + ef->plane_fmt[i].sizeimage); > >>> +} > >>> + > >>> static void v4l_print_framebuffer(const void *arg, bool write_only) > >>> { > >>> const struct v4l2_framebuffer *p = arg; > >>> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum > >>> v4l2_buf_type type) > >>> switch (type) { > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> if ((is_vid || is_tch) && is_rx && > >>> - (ops->vidioc_g_fmt_vid_cap || > >>> ops->vidioc_g_fmt_vid_cap_mplane)) > >>> + (ops->vidioc_g_fmt_vid_cap || > >>> + ops->vidioc_g_ext_pix_fmt_vid_cap || > >>> + ops->vidioc_g_fmt_vid_cap_mplane)) > >>> return 0; > >>> break; > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>> - if ((is_vid || is_tch) && is_rx && > >>> ops->vidioc_g_fmt_vid_cap_mplane) > >>> + if ((is_vid || is_tch) && is_rx && > >>> + (ops->vidioc_g_fmt_vid_cap_mplane || > >>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) > >>> return 0; > >>> break; > >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum > >>> v4l2_buf_type type) > >>> break; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> if (is_vid && is_tx && > >>> - (ops->vidioc_g_fmt_vid_out || > >>> ops->vidioc_g_fmt_vid_out_mplane)) > >>> + (ops->vidioc_g_fmt_vid_out || > >>> + ops->vidioc_g_ext_pix_fmt_vid_out || > >>> + ops->vidioc_g_fmt_vid_out_mplane)) > >>> return 0; > >>> break; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) > >>> + if (is_vid && is_tx && > >>> + (ops->vidioc_g_ext_pix_fmt_vid_out || > >>> + ops->vidioc_g_fmt_vid_out_mplane)) > >>> return 0; > >>> break; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct > >>> v4l2_format *fmt) > >>> sizeof(fmt->fmt.pix) - offset); > >>> } > >>> +static void > >>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, > >>> + struct v4l2_pix_format *pix) > >>> +{ > >>> + unsigned int i; > >>> + > >>> + pix->width = ef->width; > >>> + pix->height = ef->height; > >>> + pix->field = ef->field; > >>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > >>> + pix->colorspace = ef->colorspace; > >>> + pix->ycbcr_enc = ef->ycbcr_enc; > >>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> + pix->quantization = ef->quantization; > >>> + pix->pixelformat = ef->pixelformat; > >>> + pix->bytesperline = ef->plane_fmt[0].bytesperline; > >>> + pix->sizeimage = ef->plane_fmt[0].sizeimage; > >>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; > >>> i++) > >>> + pix->sizeimage += ef->plane_fmt[i].sizeimage; > >>> +} > >>> + > >>> +static void > >>> +v4l2_ext_pix_format_to_pix_mp_format(const struct > >>> v4l2_ext_pix_format *ef, > >>> + struct v4l2_pix_format_mplane *pix_mp) > >>> +{ > >>> + const struct v4l2_format_info *info = > >>> + v4l2_format_info(ef->pixelformat); > >>> + unsigned int i; > >>> + > >>> + pix_mp->width = ef->width; > >>> + pix_mp->height = ef->height; > >>> + pix_mp->field = ef->field; > >>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > >>> + pix_mp->colorspace = ef->colorspace; > >>> + pix_mp->ycbcr_enc = ef->ycbcr_enc; > >>> + pix_mp->quantization = ef->quantization; > >>> + pix_mp->pixelformat = ef->pixelformat; > >>> + > >>> + /* This is true when converting to non-M-variant */ > >>> + if (info && info->mem_planes == 1) { > >>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; > >>> + pix_mp->num_planes = 1; > >>> + for (i = 1; i < VIDEO_MAX_PLANES && > >>> ef->plane_fmt[i].sizeimage; i++) > >>> + pix_mp->plane_fmt[0].sizeimage += > >>> ef->plane_fmt[i].sizeimage; > >>> + > >>> + return; > >>> + } > >>> + > >>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; > >>> i++) > >>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; > >>> + pix_mp->num_planes = i; > >>> +} > >>> + > >>> +/* > >>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to > >>> v4l2_format > >>> + * > >>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. > >>> + * @f: A pointer to struct v4l2_format to be filled. > >>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. > >>> + * > >>> + * If pixelformat should be converted to M-variant, change > >>> ef->pixelformat > >>> + * to the M-variant before calling this function. > >>> + */ > >>> +static void v4l2_ext_pix_format_to_format(const struct > >>> v4l2_ext_pix_format *ef, > >>> + struct v4l2_format *f, bool is_mplane) > >>> +{ > >>> + memset(f, 0, sizeof(*f)); > >>> + > >>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && > >>> + ef->modifier != DRM_FORMAT_MOD_INVALID) > >>> + pr_warn("Modifiers are not supported in v4l2_format, > >>> ignoring %llx\n", > >>> + ef->modifier); > >>> + > >>> + if (!is_mplane) { > >>> + f->type = ef->type; > >>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); > >>> + return; > >>> + } > >>> + > >>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) > >>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; > >>> + else > >>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; > >>> + > >>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); > >>> +} > >>> + > >>> +static void > >>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, > >>> + struct v4l2_ext_pix_format *ef) > >>> +{ > >>> + const struct v4l2_format_info *info = > >>> + v4l2_format_info(pix->pixelformat); > >>> + unsigned int i; > >>> + > >>> + ef->width = pix->width; > >>> + ef->height = pix->height; > >>> + ef->field = pix->field; > >>> + ef->colorspace = pix->colorspace; > >>> + ef->ycbcr_enc = pix->ycbcr_enc; > >>> + ef->quantization = pix->quantization; > >>> + ef->xfer_func = pix->xfer_func; > >>> + if (pix->flags) > >>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); > >>> + > >>> + /* We assume M-variants won't be used in this function */ > >>> + ef->pixelformat = pix->pixelformat; > >>> + > >>> + ef->plane_fmt[0].bytesperline = pix->bytesperline; > >>> + ef->plane_fmt[0].sizeimage = pix->sizeimage; > >>> + > >>> + if (!info) > >>> + return; > >>> + > >>> + for (i = 1; i < info->comp_planes; i++) { > >>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; > >>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * > >>> + ef->height / info->vdiv; > >>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > >>> + } > >>> +} > >>> + > >>> +static void > >>> +v4l2_pix_mp_format_to_ext_pix_format(const struct > >>> v4l2_pix_format_mplane *pix_mp, > >>> + struct v4l2_ext_pix_format *ef) > >>> +{ > >>> + const struct v4l2_format_info *info = > >>> + v4l2_format_info(pix_mp->pixelformat); > >>> + unsigned int i; > >>> + > >>> + ef->width = pix_mp->width; > >>> + ef->height = pix_mp->height; > >>> + ef->field = pix_mp->field; > >>> + ef->colorspace = pix_mp->colorspace; > >>> + ef->ycbcr_enc = pix_mp->ycbcr_enc; > >>> + ef->quantization = pix_mp->quantization; > >>> + ef->xfer_func = pix_mp->xfer_func; > >>> + if (pix_mp->flags) > >>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); > >>> + > >>> + if (!info) > >>> + return; > >>> + > >>> + ef->pixelformat = info && info->norm ? > >> > >> 'info &&' can be dropped, info is always non-NULL here. > >> > >>> + info->norm : pix_mp->pixelformat; > >>> + > >>> + if (info->comp_planes == info->mem_planes) { > >>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; > >>> i++) > >>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; > >>> + > >>> + return; > >>> + } > >>> + > >>> + /* case where mem_planes is 1 and comp_planes > 1 */ > >>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; > >>> + for (i = 1; i < info->comp_planes; i++) { > >>> + ef->plane_fmt[i].bytesperline = > >>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; > >>> + ef->plane_fmt[i].sizeimage = > >>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; > >>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > >>> + } > >>> +} > >>> + > >>> +/* > >>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to > >>> v4l2_ext_pix_format > >>> + * > >>> + * @f: A pointer to struct v4l2_format to be converted. > >>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. > >>> + * > >>> + * This method normalize the pixelformat to non-M variant. > >>> + */ > >>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, > >>> + struct v4l2_ext_pix_format *ef) > >>> +{ > >>> + memset(ef, 0, sizeof(*ef)); > >>> + > >>> + switch (f->type) { > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> + ef->type = f->type; > >>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); > >>> + break; > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > >>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > >>> + break; > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > >>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > >>> + break; > >>> + default: > >>> + WARN("Converting to Ext Pix Format with wrong buffer type > >>> %s\n", > >>> + prt_names(f->type, v4l2_type_names)); > >>> + break; > >>> + } > >>> +} > >>> + > >>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, > >>> struct file *file, void *fh, void *arg) > >>> { > >>> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct > >>> v4l2_pix_format *p) > >>> p->xfer_func = 0; > >>> } > >>> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, > >>> + struct file *file, void *fh, > >>> + struct v4l2_format *f, > >>> + unsigned int ioctl) > >>> +{ > >>> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); > >>> + struct video_device *vdev = video_devdata(file); > >>> + struct v4l2_ext_pix_format ef = {0}; > >>> + u32 original_pixfmt = 0; > >>> + u32 cap_mask; > >>> + int ret; > >>> + > >>> + if (ioctl != VIDIOC_G_FMT) { > >>> + /* > >>> + * If CSC attributes are read only, set them to DEFAULT > >>> + * to avoid changes by the driver. > >>> + */ > >>> + if (is_multiplanar) { > >>> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > >>> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; > >>> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > >>> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; > >>> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; > >>> + } > >>> + /* Unset the flag to avoid warning in the convertion */ > >>> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > >>> + > >>> + /* Save pixelformat in case M-variant is being used */ > >>> + original_pixfmt = f->fmt.pix_mp.pixelformat; > >>> + } else { > >>> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > >>> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; > >>> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > >>> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; > >>> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; > >>> + } > >>> + /* Unset the flag to avoid warning in the convertion */ > >>> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > >>> + } > >>> + v4l2_format_to_ext_pix_format(f, &ef); > >>> + } > >>> + > >>> + switch (f->type) { > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | > >>> + V4L2_CAP_VIDEO_M2M_MPLANE; > >>> + if (!!(vdev->device_caps & cap_mask) != > >>> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) > >>> + return -EINVAL; > >>> + > >>> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > >>> + if (ioctl == VIDIOC_G_FMT) > >>> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); > >>> + else if (ioctl == VIDIOC_S_FMT) > >>> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); > >>> + else > >>> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > >>> + &ef); > >>> + break; > >>> + > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | > >>> + V4L2_CAP_VIDEO_M2M_MPLANE; > >>> + if (!!(vdev->device_caps & cap_mask) != > >>> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) > >>> + return -EINVAL; > >>> + > >>> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > >>> + if (ioctl == VIDIOC_G_FMT) > >>> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); > >>> + else if (ioctl == VIDIOC_S_FMT) > >>> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); > >>> + else > >>> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > >>> + &ef); > >>> + break; > >>> + > >>> + default: > >>> + return -EINVAL; > >>> + } > >>> + > >>> + if (ret) > >>> + return ret; > >>> + > >>> + if (original_pixfmt != ef.pixelformat && > >>> + v4l2_format_info(original_pixfmt)) > >> > >> Could this test be simplified to: 'if (original_pixfmt)'? > >> > >> I.e., if the original pixfmt was saved, then restore it here. > >> > >>> + ef.pixelformat = original_pixfmt; > >>> + > >>> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); > >>> + return 0; > >>> +} > >>> + > >>> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > >>> struct file *file, void *fh, void *arg) > >>> { > >>> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> switch (p->type) { > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) > >>> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && > >>> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) > >>> break; > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); > >>> + ret = ops->vidioc_g_fmt_vid_cap ? > >>> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> + VIDIOC_G_FMT); > >>> /* just in case the driver zeroed it again */ > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> if (vfd->vfl_type == VFL_TYPE_TOUCH) > >>> v4l_pix_format_touch(&p->fmt.pix); > >>> return ret; > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > >>> + if (ops->vidioc_g_fmt_vid_cap_mplane) > >>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > >>> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) > >> > >> 'else' can be dropped. > >> > >>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> + VIDIOC_G_FMT); > >>> + break; > >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); > >>> case V4L2_BUF_TYPE_VBI_CAPTURE: > >>> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: > >>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) > >>> + if (unlikely(!ops->vidioc_g_fmt_vid_out && > >>> + !ops->vidioc_g_ext_pix_fmt_vid_out)) > >>> break; > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); > >>> + ret = ops->vidioc_g_fmt_vid_out ? > >>> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); > >>> /* just in case the driver zeroed it again */ > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> return ret; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > >>> + if (ops->vidioc_g_fmt_vid_out_mplane) > >>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > >>> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) > >> > >> Ditto. > >> > >>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> + VIDIOC_G_FMT); > >>> + break; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); > >>> case V4L2_BUF_TYPE_VBI_OUTPUT: > >>> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> return -EINVAL; > >>> } > >>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > >>> + struct file *file, void *fh, void *arg) > >>> +{ > >>> + struct v4l2_ext_pix_format *ef = arg; > >>> + struct v4l2_format f = { > >>> + .type = ef->type, > >>> + }; > >>> + int ret = check_fmt(file, ef->type); > >>> + > >>> + if (ret) > >>> + return ret; > >>> + > >>> + memset(ef, 0, sizeof(*ef)); > >>> + ef->type = f.type; > >>> + > >>> + switch (f.type) { > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) > >>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); > >>> + break; > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> + if (ops->vidioc_g_ext_pix_fmt_vid_out) > >>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); > >>> + break; > >>> + default: > >>> + return -EINVAL; > >>> + } > >>> + > >>> + ret = v4l_g_fmt(ops, file, fh, &f); > >>> + if (ret) > >>> + return ret; > >>> + > >>> + v4l2_format_to_ext_pix_format(&f, ef); > >>> + return 0; > >>> +} > >>> + > >>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > >>> struct file *file, void *fh, void *arg) > >>> { > >>> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> switch (p->type) { > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) > >>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && > >>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); > >>> + ret = ops->vidioc_s_fmt_vid_cap ? > >>> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > >>> /* just in case the driver zeroed it again */ > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> if (vfd->vfl_type == VFL_TYPE_TOUCH) > >>> v4l_pix_format_touch(&p->fmt.pix); > >>> return ret; > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) > >>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && > >>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>> bytesperline); > >>> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); > >>> + return ops->vidioc_s_fmt_vid_cap_mplane ? > >>> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> VIDIOC_S_FMT); > >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) > >>> break; > >>> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > >>> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) > >>> + if (unlikely(!ops->vidioc_s_fmt_vid_out && > >>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); > >>> + ret = ops->vidioc_s_fmt_vid_out ? > >>> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > >>> /* just in case the driver zeroed it again */ > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> return ret; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) > >>> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && > >>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>> bytesperline); > >>> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); > >>> + return ops->vidioc_s_fmt_vid_out_mplane ? > >>> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) > >>> break; > >>> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> return -EINVAL; > >>> } > >>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > >>> + struct file *file, void *fh, void *arg) > >>> +{ > >>> + struct video_device *vfd = video_devdata(file); > >>> + struct v4l2_ext_pix_format *ef = arg; > >>> + struct v4l2_format f; > >>> + int ret = check_fmt(file, ef->type); > >>> + > >>> + if (ret) > >>> + return ret; > >>> + > >>> + memset(ef->reserved, 0, sizeof(ef->reserved)); > >>> + > >>> + switch (ef->type) { > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) > >>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); > >>> + break; > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> + if (ops->vidioc_s_ext_pix_fmt_vid_out) > >>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); > >>> + break; > >>> + default: > >>> + return -EINVAL; > >>> + } > >>> + > >>> + v4l2_ext_pix_format_to_format(ef, &f, > >>> V4L2_IS_CAP_MULTIPLANAR(vfd)); > >>> + > >>> + ret = v4l_s_fmt(ops, file, fh, &f); > >>> + if (ret) > >>> + /* TODO: retry with M-variant of ef->pixelformat? */ > >> > >> See my comments on this at the top. > >> > >>> + return ret; > >>> + > >>> + v4l2_format_to_ext_pix_format(&f, ef); > >>> + return 0; > >>> +} > >>> + > >>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > >>> struct file *file, void *fh, void *arg) > >>> { > >>> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> switch (p->type) { > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) > >>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && > >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); > >>> + ret = ops->vidioc_try_fmt_vid_cap ? > >>> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> VIDIOC_TRY_FMT); > >>> /* just in case the driver zeroed it again */ > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> if (vfd->vfl_type == VFL_TYPE_TOUCH) > >>> v4l_pix_format_touch(&p->fmt.pix); > >>> return ret; > >>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) > >>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && > >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>> bytesperline); > >>> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); > >>> + return ops->vidioc_try_fmt_vid_cap_mplane ? > >>> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> + VIDIOC_TRY_FMT); > >>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) > >>> break; > >>> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > >>> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) > >>> + if (unlikely(!ops->vidioc_try_fmt_vid_out && > >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); > >>> + ret = ops->vidioc_try_fmt_vid_out ? > >>> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> VIDIOC_TRY_FMT); > >>> /* just in case the driver zeroed it again */ > >>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>> return ret; > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) > >>> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && > >>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>> break; > >>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>> bytesperline); > >>> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); > >>> + return ops->vidioc_try_fmt_vid_out_mplane ? > >>> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : > >>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>> + VIDIOC_TRY_FMT); > >>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) > >>> break; > >>> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct > >>> v4l2_ioctl_ops *ops, > >>> return -EINVAL; > >>> } > >>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > >>> + struct file *file, void *fh, void *arg) > >>> +{ > >>> + struct video_device *vfd = video_devdata(file); > >>> + struct v4l2_ext_pix_format *ef = arg; > >>> + struct v4l2_format f; > >>> + int ret = check_fmt(file, ef->type); > >>> + > >>> + if (ret) > >>> + return ret; > >>> + > >>> + memset(ef->reserved, 0, sizeof(ef->reserved)); > >>> + > >>> + switch (ef->type) { > >>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) > >>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > >>> + ef); > >>> + break; > >>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>> + if (ops->vidioc_try_ext_pix_fmt_vid_out) > >>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > >>> + ef); > >>> + break; > >>> + default: > >>> + return -EINVAL; > >>> + } > >>> + > >>> + v4l2_ext_pix_format_to_format(ef, &f, > >>> V4L2_IS_CAP_MULTIPLANAR(vfd)); > >>> + > >>> + ret = v4l_try_fmt(ops, file, fh, &f); > >>> + if (ret) > >>> + /* TODO: retry with M-variant of ef->pixelformat? */ > >>> + return ret; > >>> + > >>> + v4l2_format_to_ext_pix_format(&f, ef); > >>> + return 0; > >>> +} > >>> + > >>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, > >>> struct file *file, void *fh, void *arg) > >>> { > >>> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info > >>> v4l2_ioctls[] = { > >>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, > >>> v4l_print_freq_band, 0), > >>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, > >>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), > >>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, > >>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | > >>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), > >>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, > >>> v4l_print_ext_pix_format, 0), > >>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, > >>> v4l_print_ext_pix_format, INFO_FL_PRIO), > >>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, > >>> v4l_print_ext_pix_format, 0), > >>> }; > >>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) > >>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h > >>> index edb733f21604..c44708dc9355 100644 > >>> --- a/include/media/v4l2-ioctl.h > >>> +++ b/include/media/v4l2-ioctl.h > >>> @@ -48,11 +48,17 @@ struct v4l2_fh; > >>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements > >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture > >>> * in single plane mode > >>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that > >>> implements > >>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic > >>> for video > >>> + * capture > >>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements > >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay > >>> * @vidioc_g_fmt_vid_out: pointer to the function that implements > >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out > >>> * in single plane mode > >>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that > >>> implements > >>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic > >>> for video > >>> + * out > >>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that > >>> implements > >>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video > >>> overlay output > >>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements > >>> @@ -82,11 +88,16 @@ struct v4l2_fh; > >>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements > >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture > >>> * in single plane mode > >>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that > >>> implements > >>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic > >>> for video > >>> + * capture > >>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements > >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay > >>> * @vidioc_s_fmt_vid_out: pointer to the function that implements > >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out > >>> * in single plane mode > >>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that > >>> implements > >>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for > >>> video out > >>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that > >>> implements > >>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video > >>> overlay output > >>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements > >>> @@ -116,11 +127,16 @@ struct v4l2_fh; > >>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements > >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video > >>> capture > >>> * in single plane mode > >>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that > >>> implements > >>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl > >>> logic for > >>> + video capture > >>> * @vidioc_try_fmt_vid_overlay: pointer to the function that > >>> implements > >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video > >>> overlay > >>> * @vidioc_try_fmt_vid_out: pointer to the function that implements > >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out > >>> * in single plane mode > >>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that > >>> implements > >>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for > >>> video out > >>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that > >>> implements > >>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video > >>> overlay > >>> * output > >>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { > >>> /* VIDIOC_G_FMT handlers */ > >>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > >>> + struct v4l2_ext_pix_format *ef); > >>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, > >>> + struct v4l2_ext_pix_format *ef); > >>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, > >>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { > >>> /* VIDIOC_S_FMT handlers */ > >>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > >>> + struct v4l2_ext_pix_format *ef); > >>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, > >>> + struct v4l2_ext_pix_format *ef); > >>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, > >>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { > >>> /* VIDIOC_TRY_FMT handlers */ > >>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > >>> + struct v4l2_ext_pix_format *ef); > >>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, > >>> + struct v4l2_ext_pix_format *ef); > >>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, > >>> struct v4l2_format *f); > >>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, > >>> diff --git a/include/uapi/linux/videodev2.h > >>> b/include/uapi/linux/videodev2.h > >>> index d9b7c9177605..a2d850513708 100644 > >>> --- a/include/uapi/linux/videodev2.h > >>> +++ b/include/uapi/linux/videodev2.h > >>> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { > >>> __u8 reserved[7]; > >>> } __attribute__ ((packed)); > >>> +/** > >>> + * struct v4l2_ext_pix_format - extended single/multiplanar format > >>> definition > >>> + * @type: type of the data stream; > >>> V4L2_BUF_TYPE_VIDEO_CAPTURE or > >>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT > >>> + * @width: image width in pixels > >>> + * @height: image height in pixels > >>> + * @field: enum v4l2_field; field order (for interlaced video) > >>> + * @plane_fmt: per-plane information > >>> + * @pixelformat: little endian four character code (fourcc) > >>> + * @modifier: modifier applied to the format (used for tiled > >>> formats > >>> + * and other kind of HW-specific formats, like compressed > >>> + * formats) as defined in drm_fourcc.h > >>> + * @colorspace: enum v4l2_colorspace; supplemental to > >>> pixelformat > >>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function > >>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding > >>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding > >>> + * @quantization: enum v4l2_quantization, colorspace quantization > >>> + * @reserved: extra space reserved for future fields, must be > >>> set to 0 > >>> + */ > >>> +struct v4l2_ext_pix_format { > >>> + __u32 type; > >>> + __u32 width; > >>> + __u32 height; > >>> + __u32 field; > >>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; > >>> + __u32 pixelformat; > >>> + __u64 modifier; > >>> + __u32 colorspace; > >>> + __u32 xfer_func; > >>> + union { > >>> + __u32 ycbcr_enc; > >>> + __u32 hsv_enc; > >>> + }; > >>> + __u32 quantization; > >>> + __u32 reserved[9]; > >>> +}; > >>> + > >>> /** > >>> * struct v4l2_sdr_format - SDR format definition > >>> * @pixelformat: little endian four character code (fourcc) > >>> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { > >>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct > >>> v4l2_query_ext_ctrl) > >>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct > >>> v4l2_ext_pix_format) > >>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct > >>> v4l2_ext_pix_format) > >>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct > >>> v4l2_ext_pix_format) > >>> + > >>> /* Reminder: when adding new ioctls please add support for them to > >>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ > >>>
On 11/5/22 18:19, Hsia-Jun Li wrote: > Hello Helen > > I didn't see any updates from V6 and V7-WIP in your repo. That is what I > need to for our complex tile formats in our platform. > > Any future plane here? > > Besides I have some ideas on these patches. I was looking into updating this patchset few months ago and the biggest blocker was the absence of immediate upstream user for this new UAPI. What your platform is? Is the driver stack completely opensource?
On 11/7/22 03:24, Laurent Pinchart wrote: > > Hi Hsia-Jun, > > On Sat, Nov 05, 2022 at 11:19:10PM +0800, Hsia-Jun Li wrote: >> Hello Helen >> >> I didn't see any updates from V6 and V7-WIP in your repo. That is what I >> need to for our complex tile formats in our platform. >> >> Any future plane here? >> >> Besides I have some ideas on these patches. >> >> On 2/24/21 23:12, Helen Koike wrote: >>> Hi Hans, >>> >>> Thank you for your comment, please see my reply below. >>> >>> On 2/23/21 9:35 AM, Hans Verkuil wrote: >>>> Hi Helen, >>>> >>>> On 14/01/2021 19:07, Helen Koike wrote: >>>>> This is part of the multiplanar and singleplanar unification process. >>>>> v4l2_ext_pix_format is supposed to work for both cases. >>>>> >>>>> We also add the concept of modifiers already employed in DRM to expose >>>>> HW-specific formats (like tiled or compressed formats) and allow >>>>> exchanging this information with the DRM subsystem in a consistent way. >>>>> >>>>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in >>>>> v4l2_ext_format, other types will be rejected if you use the >>>>> {G,S,TRY}_EXT_PIX_FMT ioctls. >>>>> >>>>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls >>>>> in drivers, but, in the meantime, the core takes care of converting >>>>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers >>>>> can >>>>> still work if the userspace app/lib uses the new ioctls. >>>>> >>>>> The conversion is also done the other around to allow userspace >>>>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the >>>>> _ext_ hooks. >>>> >>>> I have some small comments below, but also one high level comment: >>>> >>>> Regarding M variants of pixelformats: this patch 'normalizes' them to >>>> regular pixelformats in the extended API. This makes life complicated, >>>> and I wonder if this is the right approach. >>>> >>>> Currently there are two reasons for a driver to support e.g. NV12M: >>>> either luma and chroma need to be in two memory banks, or the luma >>>> and chroma planes cannot be contiguous due to alignment requirements. >>>> >>>> The first requirement is still valid for drivers that support the >>>> extended API. >>>> The second requirement is no longer a reason to support NV12M. But I >>>> don't think we should just drop NV12M support if it was already supported >>>> before the conversion to this extended API. Since NV12M allocates two >>>> buffers >>>> instead of one, it is still different from a regular NV12. >>> >>> I don't see what would be the difference when using NV12 and NV12M in >>> Ext API. >>> NV12 could be used for both requirements. It would even allow the second >>> requirement with a single memory buffer. >>> >> Although I don't have problem here to support both single and multiple >> planes in our hardware. But using the single plane format here is not >> convience for us. >> >> The tile format in our platform is little complex, you can't calculate >> the stride and sizeimage easily with just the modifier, width and >> height. Then we don't have a good place to record the offset here. > > What else is needed to compute those values ? Can they be computed by > the kernel driver, or do they have computed by userspace ? > I could calculate the stride(bytesperline) and sizeimage in the kernel driver. But it would be better to let the firmware do that. Our driver would depend on the V4L2_EVENT_SOURCE_CHANGE event. >> Besides, the luma and chroma planes would always need their own >> compression meta data planes. If we use NV12 here, that would make a >> very strange pixel format, one plane for the pixel datas and two planes >> for the meta data. >> >>>> I would prefer that such drivers support both NV12 and NV12M, so no >>>> automatic conversion. >>>> >>>> A related question is how to handle pixelformat enumeration: with the >>>> extended API an NV12 format might work, but not with the old API (e.g. >>>> due to memory alignment requirements). I wonder if a >>>> VIDIOC_ENUM_EXT_PIX_FMT >>>> isn't needed. >>> >>> We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. >>> If the driver reports NV12M, userspace can use it and the framework >>> normilizes it. >>> >>>> >>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while >>>> VIDIOC_ENUM_FMT >>>> would just report NV12M. >>> >>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would >>> report both (unless I'm missing something, which is probably the case). >>> >>> The idea was to deprecate the M-variants one day. >> >> I was thinking the way in DRM API is better, always assuming it would >> always in a multiple planes. The only problem is we don't have a way to >> let the allocator that allocate contiguous memory for planes when we >> need to do that. >> >>>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> >>>>> Signed-off-by: Helen Koike <helen.koike@collabora.com> >>>>> --- >>>>> >>>>> Changes in v6: >>>>> The main change here was fixing the conversion, so planes reflects >>>>> color planes, >>>>> and to implement this properly I made major refactors compared to >>>>> the previous >>>>> version. >>>>> - struct v4l2_plane_ext_pix_format removed, using struct >>>>> v4l2_plane_pix_format instead (Tomasz) >>>>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) >>>>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) >>>>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) >>>>> - refactor conversion functions, so planes are color planes (Tomasz) >>>>> - Don't explicitly check for e->modifier != 0 in >>>>> v4l2_ext_pix_format_to_format() (Tomasz) >>>>> - Use "ef" for extended formats in the framework for consistency >>>>> (Tomasz) >>>>> - Handle xfer_func field in conversions (Tomasz) >>>>> - Zero reserved fields in v4l_s_ext_pix_fmt() and >>>>> v4l_try_ext_pix_fmt() (Tomasz) >>>>> - Refactor format functions to use v4l_fmt_ioctl_via_ext() >>>>> - Several fixes/refactoring/changes >>>>> - Remove EXT API for touch devices >>>>> >>>>> Changes in v5: >>>>> - change sizes and reorder fields to avoid holes in the struct and make >>>>> it the same for 32 and 64 bits >>>>> - removed __attribute__ ((packed)) from uapi structs >>>>> - Fix doc warning from make htmldocs >>>>> - Updated commit message with EXT_PIX prefix for the ioctls. >>>>> >>>>> Changes in v4: >>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, >>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >>>>> - Add reserved fields >>>>> - Removed num_planes from struct v4l2_ext_pix_format >>>>> - Removed flag field from struct v4l2_ext_pix_format, since the only >>>>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, >>>>> where we can use modifiers, or add it back later through the reserved >>>>> bits. >>>>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && >>>>> != MOD_INVALID >>>>> - Fix type assignment in v4l_g_fmt_ext_pix() >>>>> - Rebased on top of media/master (post 5.8-rc1) >>>>> >>>>> Changes in v3: >>>>> - Rebased on top of media/master (post 5.4-rc1) >>>>> >>>>> Changes in v2: >>>>> - Move the modifier in v4l2_ext_format (was formerly placed in >>>>> v4l2_ext_plane) >>>>> - Fix a few bugs in the converters and add a strict parameter to >>>>> allow conversion of uninitialized/mis-initialized objects >>>>> --- >>>>> drivers/media/v4l2-core/v4l2-dev.c | 27 +- >>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- >>>>> include/media/v4l2-ioctl.h | 28 ++ >>>>> include/uapi/linux/videodev2.h | 41 ++ >>>>> 4 files changed, 602 insertions(+), 32 deletions(-) >>>>> >>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c >>>>> b/drivers/media/v4l2-core/v4l2-dev.c >>>>> index f9cff033d0dc..5add58cb6d45 100644 >>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c >>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c >>>>> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct >>>>> video_device *vdev) >>>>> ops->vidioc_enum_fmt_vid_overlay)) || >>>>> (is_tx && ops->vidioc_enum_fmt_vid_out)) >>>>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); >>>>> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || >>>>> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) >>>>> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>>>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || >>>>> ops->vidioc_g_fmt_vid_cap_mplane || >>>>> - ops->vidioc_g_fmt_vid_overlay)) || >>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || >>>>> (is_tx && (ops->vidioc_g_fmt_vid_out || >>>>> ops->vidioc_g_fmt_vid_out_mplane || >>>>> - ops->vidioc_g_fmt_vid_out_overlay))) >>>>> + ops->vidioc_g_ext_pix_fmt_vid_out))) { >>>>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>>>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); >>>>> + } >>>>> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || >>>>> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) >>>>> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>>>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || >>>>> ops->vidioc_s_fmt_vid_cap_mplane || >>>>> - ops->vidioc_s_fmt_vid_overlay)) || >>>>> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || >>>>> (is_tx && (ops->vidioc_s_fmt_vid_out || >>>>> ops->vidioc_s_fmt_vid_out_mplane || >>>>> - ops->vidioc_s_fmt_vid_out_overlay))) >>>>> + ops->vidioc_s_ext_pix_fmt_vid_out))) { >>>>> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>>>> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); >>>>> + } >>>>> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || >>>>> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) >>>>> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>>>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || >>>>> ops->vidioc_try_fmt_vid_cap_mplane || >>>>> - ops->vidioc_try_fmt_vid_overlay)) || >>>>> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || >>>>> (is_tx && (ops->vidioc_try_fmt_vid_out || >>>>> ops->vidioc_try_fmt_vid_out_mplane || >>>>> - ops->vidioc_try_fmt_vid_out_overlay))) >>>>> + ops->vidioc_try_ext_pix_fmt_vid_out))) { >>>>> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>>>> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); >>>>> + } >>>>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); >>>>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); >>>>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); >>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> index 848286a284f6..a9c07c0a73ec 100644 >>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> @@ -18,6 +18,8 @@ >>>>> #include <linux/videodev2.h> >>>>> +#include <drm/drm_fourcc.h> >>>>> + >>>>> #include <media/v4l2-common.h> >>>>> #include <media/v4l2-ioctl.h> >>>>> #include <media/v4l2-ctrls.h> >>>>> @@ -38,6 +40,11 @@ >>>>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), >>>>> (vfd)->valid_ioctls) >>>>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ >>>>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ >>>>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ >>>>> + V4L2_CAP_VIDEO_M2M_MPLANE)) >>>>> + >>>>> struct std_descr { >>>>> v4l2_std_id std; >>>>> const char *descr; >>>>> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, >>>>> bool write_only) >>>>> } >>>>> } >>>>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) >>>>> +{ >>>>> + const struct v4l2_ext_pix_format *ef = arg; >>>>> + unsigned int i; >>>>> + >>>>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier >>>>> %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, >>>>> xfer_func=%u\n", >>>>> + prt_names(ef->type, v4l2_type_names), >>>>> + ef->width, ef->height, >>>>> + (ef->pixelformat & 0xff), >>>>> + (ef->pixelformat >> 8) & 0xff, >>>>> + (ef->pixelformat >> 16) & 0xff, >>>>> + (ef->pixelformat >> 24) & 0xff, >>>>> + ef->modifier, prt_names(ef->field, v4l2_field_names), >>>>> + ef->colorspace, ef->ycbcr_enc, >>>>> + ef->quantization, ef->xfer_func); >>>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>>>> i++) >>>>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", >>>>> + i, ef->plane_fmt[i].bytesperline, >>>>> + ef->plane_fmt[i].sizeimage); >>>>> +} >>>>> + >>>>> static void v4l_print_framebuffer(const void *arg, bool write_only) >>>>> { >>>>> const struct v4l2_framebuffer *p = arg; >>>>> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum >>>>> v4l2_buf_type type) >>>>> switch (type) { >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> if ((is_vid || is_tch) && is_rx && >>>>> - (ops->vidioc_g_fmt_vid_cap || >>>>> ops->vidioc_g_fmt_vid_cap_mplane)) >>>>> + (ops->vidioc_g_fmt_vid_cap || >>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap || >>>>> + ops->vidioc_g_fmt_vid_cap_mplane)) >>>>> return 0; >>>>> break; >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>> - if ((is_vid || is_tch) && is_rx && >>>>> ops->vidioc_g_fmt_vid_cap_mplane) >>>>> + if ((is_vid || is_tch) && is_rx && >>>>> + (ops->vidioc_g_fmt_vid_cap_mplane || >>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) >>>>> return 0; >>>>> break; >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum >>>>> v4l2_buf_type type) >>>>> break; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> if (is_vid && is_tx && >>>>> - (ops->vidioc_g_fmt_vid_out || >>>>> ops->vidioc_g_fmt_vid_out_mplane)) >>>>> + (ops->vidioc_g_fmt_vid_out || >>>>> + ops->vidioc_g_ext_pix_fmt_vid_out || >>>>> + ops->vidioc_g_fmt_vid_out_mplane)) >>>>> return 0; >>>>> break; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) >>>>> + if (is_vid && is_tx && >>>>> + (ops->vidioc_g_ext_pix_fmt_vid_out || >>>>> + ops->vidioc_g_fmt_vid_out_mplane)) >>>>> return 0; >>>>> break; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct >>>>> v4l2_format *fmt) >>>>> sizeof(fmt->fmt.pix) - offset); >>>>> } >>>>> +static void >>>>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, >>>>> + struct v4l2_pix_format *pix) >>>>> +{ >>>>> + unsigned int i; >>>>> + >>>>> + pix->width = ef->width; >>>>> + pix->height = ef->height; >>>>> + pix->field = ef->field; >>>>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>>>> + pix->colorspace = ef->colorspace; >>>>> + pix->ycbcr_enc = ef->ycbcr_enc; >>>>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> + pix->quantization = ef->quantization; >>>>> + pix->pixelformat = ef->pixelformat; >>>>> + pix->bytesperline = ef->plane_fmt[0].bytesperline; >>>>> + pix->sizeimage = ef->plane_fmt[0].sizeimage; >>>>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>>>> i++) >>>>> + pix->sizeimage += ef->plane_fmt[i].sizeimage; >>>>> +} >>>>> + >>>>> +static void >>>>> +v4l2_ext_pix_format_to_pix_mp_format(const struct >>>>> v4l2_ext_pix_format *ef, >>>>> + struct v4l2_pix_format_mplane *pix_mp) >>>>> +{ >>>>> + const struct v4l2_format_info *info = >>>>> + v4l2_format_info(ef->pixelformat); >>>>> + unsigned int i; >>>>> + >>>>> + pix_mp->width = ef->width; >>>>> + pix_mp->height = ef->height; >>>>> + pix_mp->field = ef->field; >>>>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>>>> + pix_mp->colorspace = ef->colorspace; >>>>> + pix_mp->ycbcr_enc = ef->ycbcr_enc; >>>>> + pix_mp->quantization = ef->quantization; >>>>> + pix_mp->pixelformat = ef->pixelformat; >>>>> + >>>>> + /* This is true when converting to non-M-variant */ >>>>> + if (info && info->mem_planes == 1) { >>>>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; >>>>> + pix_mp->num_planes = 1; >>>>> + for (i = 1; i < VIDEO_MAX_PLANES && >>>>> ef->plane_fmt[i].sizeimage; i++) >>>>> + pix_mp->plane_fmt[0].sizeimage += >>>>> ef->plane_fmt[i].sizeimage; >>>>> + >>>>> + return; >>>>> + } >>>>> + >>>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>>>> i++) >>>>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; >>>>> + pix_mp->num_planes = i; >>>>> +} >>>>> + >>>>> +/* >>>>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to >>>>> v4l2_format >>>>> + * >>>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. >>>>> + * @f: A pointer to struct v4l2_format to be filled. >>>>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. >>>>> + * >>>>> + * If pixelformat should be converted to M-variant, change >>>>> ef->pixelformat >>>>> + * to the M-variant before calling this function. >>>>> + */ >>>>> +static void v4l2_ext_pix_format_to_format(const struct >>>>> v4l2_ext_pix_format *ef, >>>>> + struct v4l2_format *f, bool is_mplane) >>>>> +{ >>>>> + memset(f, 0, sizeof(*f)); >>>>> + >>>>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && >>>>> + ef->modifier != DRM_FORMAT_MOD_INVALID) >>>>> + pr_warn("Modifiers are not supported in v4l2_format, >>>>> ignoring %llx\n", >>>>> + ef->modifier); >>>>> + >>>>> + if (!is_mplane) { >>>>> + f->type = ef->type; >>>>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); >>>>> + return; >>>>> + } >>>>> + >>>>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) >>>>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; >>>>> + else >>>>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; >>>>> + >>>>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); >>>>> +} >>>>> + >>>>> +static void >>>>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, >>>>> + struct v4l2_ext_pix_format *ef) >>>>> +{ >>>>> + const struct v4l2_format_info *info = >>>>> + v4l2_format_info(pix->pixelformat); >>>>> + unsigned int i; >>>>> + >>>>> + ef->width = pix->width; >>>>> + ef->height = pix->height; >>>>> + ef->field = pix->field; >>>>> + ef->colorspace = pix->colorspace; >>>>> + ef->ycbcr_enc = pix->ycbcr_enc; >>>>> + ef->quantization = pix->quantization; >>>>> + ef->xfer_func = pix->xfer_func; >>>>> + if (pix->flags) >>>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); >>>>> + >>>>> + /* We assume M-variants won't be used in this function */ >>>>> + ef->pixelformat = pix->pixelformat; >>>>> + >>>>> + ef->plane_fmt[0].bytesperline = pix->bytesperline; >>>>> + ef->plane_fmt[0].sizeimage = pix->sizeimage; >>>>> + >>>>> + if (!info) >>>>> + return; >>>>> + >>>>> + for (i = 1; i < info->comp_planes; i++) { >>>>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; >>>>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * >>>>> + ef->height / info->vdiv; >>>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>>>> + } >>>>> +} >>>>> + >>>>> +static void >>>>> +v4l2_pix_mp_format_to_ext_pix_format(const struct >>>>> v4l2_pix_format_mplane *pix_mp, >>>>> + struct v4l2_ext_pix_format *ef) >>>>> +{ >>>>> + const struct v4l2_format_info *info = >>>>> + v4l2_format_info(pix_mp->pixelformat); >>>>> + unsigned int i; >>>>> + >>>>> + ef->width = pix_mp->width; >>>>> + ef->height = pix_mp->height; >>>>> + ef->field = pix_mp->field; >>>>> + ef->colorspace = pix_mp->colorspace; >>>>> + ef->ycbcr_enc = pix_mp->ycbcr_enc; >>>>> + ef->quantization = pix_mp->quantization; >>>>> + ef->xfer_func = pix_mp->xfer_func; >>>>> + if (pix_mp->flags) >>>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); >>>>> + >>>>> + if (!info) >>>>> + return; >>>>> + >>>>> + ef->pixelformat = info && info->norm ? >>>> >>>> 'info &&' can be dropped, info is always non-NULL here. >>>> >>>>> + info->norm : pix_mp->pixelformat; >>>>> + >>>>> + if (info->comp_planes == info->mem_planes) { >>>>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; >>>>> i++) >>>>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; >>>>> + >>>>> + return; >>>>> + } >>>>> + >>>>> + /* case where mem_planes is 1 and comp_planes > 1 */ >>>>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; >>>>> + for (i = 1; i < info->comp_planes; i++) { >>>>> + ef->plane_fmt[i].bytesperline = >>>>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; >>>>> + ef->plane_fmt[i].sizeimage = >>>>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; >>>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>>>> + } >>>>> +} >>>>> + >>>>> +/* >>>>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to >>>>> v4l2_ext_pix_format >>>>> + * >>>>> + * @f: A pointer to struct v4l2_format to be converted. >>>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. >>>>> + * >>>>> + * This method normalize the pixelformat to non-M variant. >>>>> + */ >>>>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, >>>>> + struct v4l2_ext_pix_format *ef) >>>>> +{ >>>>> + memset(ef, 0, sizeof(*ef)); >>>>> + >>>>> + switch (f->type) { >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> + ef->type = f->type; >>>>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); >>>>> + break; >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>>>> + break; >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>>>> + break; >>>>> + default: >>>>> + WARN("Converting to Ext Pix Format with wrong buffer type >>>>> %s\n", >>>>> + prt_names(f->type, v4l2_type_names)); >>>>> + break; >>>>> + } >>>>> +} >>>>> + >>>>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, >>>>> struct file *file, void *fh, void *arg) >>>>> { >>>>> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct >>>>> v4l2_pix_format *p) >>>>> p->xfer_func = 0; >>>>> } >>>>> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, >>>>> + struct file *file, void *fh, >>>>> + struct v4l2_format *f, >>>>> + unsigned int ioctl) >>>>> +{ >>>>> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); >>>>> + struct video_device *vdev = video_devdata(file); >>>>> + struct v4l2_ext_pix_format ef = {0}; >>>>> + u32 original_pixfmt = 0; >>>>> + u32 cap_mask; >>>>> + int ret; >>>>> + >>>>> + if (ioctl != VIDIOC_G_FMT) { >>>>> + /* >>>>> + * If CSC attributes are read only, set them to DEFAULT >>>>> + * to avoid changes by the driver. >>>>> + */ >>>>> + if (is_multiplanar) { >>>>> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>>>> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; >>>>> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>>>> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; >>>>> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>>>> + } >>>>> + /* Unset the flag to avoid warning in the convertion */ >>>>> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>>>> + >>>>> + /* Save pixelformat in case M-variant is being used */ >>>>> + original_pixfmt = f->fmt.pix_mp.pixelformat; >>>>> + } else { >>>>> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>>>> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; >>>>> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>>>> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; >>>>> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>>>> + } >>>>> + /* Unset the flag to avoid warning in the convertion */ >>>>> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>>>> + } >>>>> + v4l2_format_to_ext_pix_format(f, &ef); >>>>> + } >>>>> + >>>>> + switch (f->type) { >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | >>>>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>>>> + if (!!(vdev->device_caps & cap_mask) != >>>>> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) >>>>> + return -EINVAL; >>>>> + >>>>> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>>>> + if (ioctl == VIDIOC_G_FMT) >>>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); >>>>> + else if (ioctl == VIDIOC_S_FMT) >>>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); >>>>> + else >>>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>>>> + &ef); >>>>> + break; >>>>> + >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | >>>>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>>>> + if (!!(vdev->device_caps & cap_mask) != >>>>> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) >>>>> + return -EINVAL; >>>>> + >>>>> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>>>> + if (ioctl == VIDIOC_G_FMT) >>>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); >>>>> + else if (ioctl == VIDIOC_S_FMT) >>>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); >>>>> + else >>>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>>>> + &ef); >>>>> + break; >>>>> + >>>>> + default: >>>>> + return -EINVAL; >>>>> + } >>>>> + >>>>> + if (ret) >>>>> + return ret; >>>>> + >>>>> + if (original_pixfmt != ef.pixelformat && >>>>> + v4l2_format_info(original_pixfmt)) >>>> >>>> Could this test be simplified to: 'if (original_pixfmt)'? >>>> >>>> I.e., if the original pixfmt was saved, then restore it here. >>>> >>>>> + ef.pixelformat = original_pixfmt; >>>>> + >>>>> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); >>>>> + return 0; >>>>> +} >>>>> + >>>>> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>>>> struct file *file, void *fh, void *arg) >>>>> { >>>>> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> switch (p->type) { >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) >>>>> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && >>>>> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); >>>>> + ret = ops->vidioc_g_fmt_vid_cap ? >>>>> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> + VIDIOC_G_FMT); >>>>> /* just in case the driver zeroed it again */ >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>>> v4l_pix_format_touch(&p->fmt.pix); >>>>> return ret; >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>>>> + if (ops->vidioc_g_fmt_vid_cap_mplane) >>>>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>>> >>>> 'else' can be dropped. >>>> >>>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> + VIDIOC_G_FMT); >>>>> + break; >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); >>>>> case V4L2_BUF_TYPE_VBI_CAPTURE: >>>>> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: >>>>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) >>>>> + if (unlikely(!ops->vidioc_g_fmt_vid_out && >>>>> + !ops->vidioc_g_ext_pix_fmt_vid_out)) >>>>> break; >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); >>>>> + ret = ops->vidioc_g_fmt_vid_out ? >>>>> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); >>>>> /* just in case the driver zeroed it again */ >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> return ret; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>>>> + if (ops->vidioc_g_fmt_vid_out_mplane) >>>>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) >>>> >>>> Ditto. >>>> >>>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> + VIDIOC_G_FMT); >>>>> + break; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); >>>>> case V4L2_BUF_TYPE_VBI_OUTPUT: >>>>> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> return -EINVAL; >>>>> } >>>>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>>> + struct file *file, void *fh, void *arg) >>>>> +{ >>>>> + struct v4l2_ext_pix_format *ef = arg; >>>>> + struct v4l2_format f = { >>>>> + .type = ef->type, >>>>> + }; >>>>> + int ret = check_fmt(file, ef->type); >>>>> + >>>>> + if (ret) >>>>> + return ret; >>>>> + >>>>> + memset(ef, 0, sizeof(*ef)); >>>>> + ef->type = f.type; >>>>> + >>>>> + switch (f.type) { >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>>>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); >>>>> + break; >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> + if (ops->vidioc_g_ext_pix_fmt_vid_out) >>>>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); >>>>> + break; >>>>> + default: >>>>> + return -EINVAL; >>>>> + } >>>>> + >>>>> + ret = v4l_g_fmt(ops, file, fh, &f); >>>>> + if (ret) >>>>> + return ret; >>>>> + >>>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>>> + return 0; >>>>> +} >>>>> + >>>>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>>>> struct file *file, void *fh, void *arg) >>>>> { >>>>> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> switch (p->type) { >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); >>>>> + ret = ops->vidioc_s_fmt_vid_cap ? >>>>> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>>> /* just in case the driver zeroed it again */ >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>>> v4l_pix_format_touch(&p->fmt.pix); >>>>> return ret; >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>> bytesperline); >>>>> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); >>>>> + return ops->vidioc_s_fmt_vid_cap_mplane ? >>>>> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> VIDIOC_S_FMT); >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) >>>>> break; >>>>> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>>>> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out && >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); >>>>> + ret = ops->vidioc_s_fmt_vid_out ? >>>>> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>>> /* just in case the driver zeroed it again */ >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> return ret; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>> bytesperline); >>>>> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); >>>>> + return ops->vidioc_s_fmt_vid_out_mplane ? >>>>> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) >>>>> break; >>>>> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> return -EINVAL; >>>>> } >>>>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>>> + struct file *file, void *fh, void *arg) >>>>> +{ >>>>> + struct video_device *vfd = video_devdata(file); >>>>> + struct v4l2_ext_pix_format *ef = arg; >>>>> + struct v4l2_format f; >>>>> + int ret = check_fmt(file, ef->type); >>>>> + >>>>> + if (ret) >>>>> + return ret; >>>>> + >>>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>>>> + >>>>> + switch (ef->type) { >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) >>>>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); >>>>> + break; >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> + if (ops->vidioc_s_ext_pix_fmt_vid_out) >>>>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); >>>>> + break; >>>>> + default: >>>>> + return -EINVAL; >>>>> + } >>>>> + >>>>> + v4l2_ext_pix_format_to_format(ef, &f, >>>>> V4L2_IS_CAP_MULTIPLANAR(vfd)); >>>>> + >>>>> + ret = v4l_s_fmt(ops, file, fh, &f); >>>>> + if (ret) >>>>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>>> >>>> See my comments on this at the top. >>>> >>>>> + return ret; >>>>> + >>>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>>> + return 0; >>>>> +} >>>>> + >>>>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>>>> struct file *file, void *fh, void *arg) >>>>> { >>>>> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> switch (p->type) { >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); >>>>> + ret = ops->vidioc_try_fmt_vid_cap ? >>>>> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> VIDIOC_TRY_FMT); >>>>> /* just in case the driver zeroed it again */ >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>>> v4l_pix_format_touch(&p->fmt.pix); >>>>> return ret; >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>> bytesperline); >>>>> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); >>>>> + return ops->vidioc_try_fmt_vid_cap_mplane ? >>>>> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> + VIDIOC_TRY_FMT); >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) >>>>> break; >>>>> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>>>> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out && >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); >>>>> + ret = ops->vidioc_try_fmt_vid_out ? >>>>> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> VIDIOC_TRY_FMT); >>>>> /* just in case the driver zeroed it again */ >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>> return ret; >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>> break; >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>> bytesperline); >>>>> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); >>>>> + return ops->vidioc_try_fmt_vid_out_mplane ? >>>>> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>> + VIDIOC_TRY_FMT); >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) >>>>> break; >>>>> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct >>>>> v4l2_ioctl_ops *ops, >>>>> return -EINVAL; >>>>> } >>>>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>>> + struct file *file, void *fh, void *arg) >>>>> +{ >>>>> + struct video_device *vfd = video_devdata(file); >>>>> + struct v4l2_ext_pix_format *ef = arg; >>>>> + struct v4l2_format f; >>>>> + int ret = check_fmt(file, ef->type); >>>>> + >>>>> + if (ret) >>>>> + return ret; >>>>> + >>>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>>>> + >>>>> + switch (ef->type) { >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) >>>>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>>>> + ef); >>>>> + break; >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>> + if (ops->vidioc_try_ext_pix_fmt_vid_out) >>>>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>>>> + ef); >>>>> + break; >>>>> + default: >>>>> + return -EINVAL; >>>>> + } >>>>> + >>>>> + v4l2_ext_pix_format_to_format(ef, &f, >>>>> V4L2_IS_CAP_MULTIPLANAR(vfd)); >>>>> + >>>>> + ret = v4l_try_fmt(ops, file, fh, &f); >>>>> + if (ret) >>>>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>>>> + return ret; >>>>> + >>>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>>> + return 0; >>>>> +} >>>>> + >>>>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, >>>>> struct file *file, void *fh, void *arg) >>>>> { >>>>> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info >>>>> v4l2_ioctls[] = { >>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, >>>>> v4l_print_freq_band, 0), >>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, >>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), >>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, >>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | >>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), >>>>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, >>>>> v4l_print_ext_pix_format, 0), >>>>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, >>>>> v4l_print_ext_pix_format, INFO_FL_PRIO), >>>>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, >>>>> v4l_print_ext_pix_format, 0), >>>>> }; >>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) >>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h >>>>> index edb733f21604..c44708dc9355 100644 >>>>> --- a/include/media/v4l2-ioctl.h >>>>> +++ b/include/media/v4l2-ioctl.h >>>>> @@ -48,11 +48,17 @@ struct v4l2_fh; >>>>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>>> * in single plane mode >>>>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that >>>>> implements >>>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>>>> for video >>>>> + * capture >>>>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>>> * @vidioc_g_fmt_vid_out: pointer to the function that implements >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out >>>>> * in single plane mode >>>>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that >>>>> implements >>>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>>>> for video >>>>> + * out >>>>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that >>>>> implements >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video >>>>> overlay output >>>>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements >>>>> @@ -82,11 +88,16 @@ struct v4l2_fh; >>>>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>>> * in single plane mode >>>>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that >>>>> implements >>>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>>>> for video >>>>> + * capture >>>>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>>> * @vidioc_s_fmt_vid_out: pointer to the function that implements >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out >>>>> * in single plane mode >>>>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that >>>>> implements >>>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for >>>>> video out >>>>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that >>>>> implements >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video >>>>> overlay output >>>>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements >>>>> @@ -116,11 +127,16 @@ struct v4l2_fh; >>>>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>>>> capture >>>>> * in single plane mode >>>>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that >>>>> implements >>>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl >>>>> logic for >>>>> + video capture >>>>> * @vidioc_try_fmt_vid_overlay: pointer to the function that >>>>> implements >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>>>> overlay >>>>> * @vidioc_try_fmt_vid_out: pointer to the function that implements >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out >>>>> * in single plane mode >>>>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that >>>>> implements >>>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for >>>>> video out >>>>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that >>>>> implements >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>>>> overlay >>>>> * output >>>>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { >>>>> /* VIDIOC_G_FMT handlers */ >>>>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>>> + struct v4l2_ext_pix_format *ef); >>>>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>>> + struct v4l2_ext_pix_format *ef); >>>>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, >>>>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { >>>>> /* VIDIOC_S_FMT handlers */ >>>>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>>> + struct v4l2_ext_pix_format *ef); >>>>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>>> + struct v4l2_ext_pix_format *ef); >>>>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, >>>>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { >>>>> /* VIDIOC_TRY_FMT handlers */ >>>>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>>> + struct v4l2_ext_pix_format *ef); >>>>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>>> + struct v4l2_ext_pix_format *ef); >>>>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, >>>>> struct v4l2_format *f); >>>>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, >>>>> diff --git a/include/uapi/linux/videodev2.h >>>>> b/include/uapi/linux/videodev2.h >>>>> index d9b7c9177605..a2d850513708 100644 >>>>> --- a/include/uapi/linux/videodev2.h >>>>> +++ b/include/uapi/linux/videodev2.h >>>>> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { >>>>> __u8 reserved[7]; >>>>> } __attribute__ ((packed)); >>>>> +/** >>>>> + * struct v4l2_ext_pix_format - extended single/multiplanar format >>>>> definition >>>>> + * @type: type of the data stream; >>>>> V4L2_BUF_TYPE_VIDEO_CAPTURE or >>>>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT >>>>> + * @width: image width in pixels >>>>> + * @height: image height in pixels >>>>> + * @field: enum v4l2_field; field order (for interlaced video) >>>>> + * @plane_fmt: per-plane information >>>>> + * @pixelformat: little endian four character code (fourcc) >>>>> + * @modifier: modifier applied to the format (used for tiled >>>>> formats >>>>> + * and other kind of HW-specific formats, like compressed >>>>> + * formats) as defined in drm_fourcc.h >>>>> + * @colorspace: enum v4l2_colorspace; supplemental to >>>>> pixelformat >>>>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function >>>>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding >>>>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding >>>>> + * @quantization: enum v4l2_quantization, colorspace quantization >>>>> + * @reserved: extra space reserved for future fields, must be >>>>> set to 0 >>>>> + */ >>>>> +struct v4l2_ext_pix_format { >>>>> + __u32 type; >>>>> + __u32 width; >>>>> + __u32 height; >>>>> + __u32 field; >>>>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; >>>>> + __u32 pixelformat; >>>>> + __u64 modifier; >>>>> + __u32 colorspace; >>>>> + __u32 xfer_func; >>>>> + union { >>>>> + __u32 ycbcr_enc; >>>>> + __u32 hsv_enc; >>>>> + }; >>>>> + __u32 quantization; >>>>> + __u32 reserved[9]; >>>>> +}; >>>>> + >>>>> /** >>>>> * struct v4l2_sdr_format - SDR format definition >>>>> * @pixelformat: little endian four character code (fourcc) >>>>> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { >>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct >>>>> v4l2_query_ext_ctrl) >>>>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct >>>>> v4l2_ext_pix_format) >>>>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct >>>>> v4l2_ext_pix_format) >>>>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct >>>>> v4l2_ext_pix_format) >>>>> + >>>>> /* Reminder: when adding new ioctls please add support for them to >>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ >>>>> > > -- > Regards, > > Laurent Pinchart
On 11/7/22 06:11, Dmitry Osipenko wrote: > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > On 11/5/22 18:19, Hsia-Jun Li wrote: >> Hello Helen >> >> I didn't see any updates from V6 and V7-WIP in your repo. That is what I >> need to for our complex tile formats in our platform. >> >> Any future plane here? >> >> Besides I have some ideas on these patches. > > I was looking into updating this patchset few months ago and the biggest > blocker was the absence of immediate upstream user for this new UAPI. > What your platform is? Synaptics VideoSmart https://www.synaptics.com/products/multimedia-solutions Is the driver stack completely opensource? If you don't include the trusted application(firmware) then yes. I can't post it in a short time, because the firmware is not released yet. While the v4l2 still lacks many features I need here. If I release it in a short time, you would just get a version that outputs non-tile linear pixel formats here, the tile formats were only used for internal buffers. Besides this TEE in our platform is not optee, even I posted it I don't think it could be merged in a short time. > > -- > Best regards, > Dmitry >
Hi Hsia-Jun, On Mon, Nov 07, 2022 at 09:54:11AM +0800, Hsia-Jun Li wrote: > On 11/7/22 03:24, Laurent Pinchart wrote: > > On Sat, Nov 05, 2022 at 11:19:10PM +0800, Hsia-Jun Li wrote: > >> Hello Helen > >> > >> I didn't see any updates from V6 and V7-WIP in your repo. That is what I > >> need to for our complex tile formats in our platform. > >> > >> Any future plane here? > >> > >> Besides I have some ideas on these patches. > >> > >> On 2/24/21 23:12, Helen Koike wrote: > >>> Hi Hans, > >>> > >>> Thank you for your comment, please see my reply below. > >>> > >>> On 2/23/21 9:35 AM, Hans Verkuil wrote: > >>>> Hi Helen, > >>>> > >>>> On 14/01/2021 19:07, Helen Koike wrote: > >>>>> This is part of the multiplanar and singleplanar unification process. > >>>>> v4l2_ext_pix_format is supposed to work for both cases. > >>>>> > >>>>> We also add the concept of modifiers already employed in DRM to expose > >>>>> HW-specific formats (like tiled or compressed formats) and allow > >>>>> exchanging this information with the DRM subsystem in a consistent way. > >>>>> > >>>>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in > >>>>> v4l2_ext_format, other types will be rejected if you use the > >>>>> {G,S,TRY}_EXT_PIX_FMT ioctls. > >>>>> > >>>>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls > >>>>> in drivers, but, in the meantime, the core takes care of converting > >>>>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers > >>>>> can > >>>>> still work if the userspace app/lib uses the new ioctls. > >>>>> > >>>>> The conversion is also done the other around to allow userspace > >>>>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the > >>>>> _ext_ hooks. > >>>> > >>>> I have some small comments below, but also one high level comment: > >>>> > >>>> Regarding M variants of pixelformats: this patch 'normalizes' them to > >>>> regular pixelformats in the extended API. This makes life complicated, > >>>> and I wonder if this is the right approach. > >>>> > >>>> Currently there are two reasons for a driver to support e.g. NV12M: > >>>> either luma and chroma need to be in two memory banks, or the luma > >>>> and chroma planes cannot be contiguous due to alignment requirements. > >>>> > >>>> The first requirement is still valid for drivers that support the > >>>> extended API. > >>>> The second requirement is no longer a reason to support NV12M. But I > >>>> don't think we should just drop NV12M support if it was already supported > >>>> before the conversion to this extended API. Since NV12M allocates two > >>>> buffers > >>>> instead of one, it is still different from a regular NV12. > >>> > >>> I don't see what would be the difference when using NV12 and NV12M in > >>> Ext API. > >>> NV12 could be used for both requirements. It would even allow the second > >>> requirement with a single memory buffer. > >>> > >> Although I don't have problem here to support both single and multiple > >> planes in our hardware. But using the single plane format here is not > >> convience for us. > >> > >> The tile format in our platform is little complex, you can't calculate > >> the stride and sizeimage easily with just the modifier, width and > >> height. Then we don't have a good place to record the offset here. > > > > What else is needed to compute those values ? Can they be computed by > > the kernel driver, or do they have computed by userspace ? > > I could calculate the stride(bytesperline) and sizeimage in the kernel > driver. But it would be better to let the firmware do that. Our driver > would depend on the V4L2_EVENT_SOURCE_CHANGE event. It could indeed be done in the firmware, but that would mean that the device would need to be powered up to implement VIDIOC_S_FMT and VIDIOC_TRY_FMT. The uvcvideo driver does that, which results in very slow startup for applications that try lots of formats. It would be best to avoid it if possible, and calculate the values in software in the kernel. > >> Besides, the luma and chroma planes would always need their own > >> compression meta data planes. If we use NV12 here, that would make a > >> very strange pixel format, one plane for the pixel datas and two planes > >> for the meta data. > >> > >>>> I would prefer that such drivers support both NV12 and NV12M, so no > >>>> automatic conversion. > >>>> > >>>> A related question is how to handle pixelformat enumeration: with the > >>>> extended API an NV12 format might work, but not with the old API (e.g. > >>>> due to memory alignment requirements). I wonder if a > >>>> VIDIOC_ENUM_EXT_PIX_FMT > >>>> isn't needed. > >>> > >>> We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. > >>> If the driver reports NV12M, userspace can use it and the framework > >>> normilizes it. > >>> > >>>> > >>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > >>>> VIDIOC_ENUM_FMT > >>>> would just report NV12M. > >>> > >>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would > >>> report both (unless I'm missing something, which is probably the case). > >>> > >>> The idea was to deprecate the M-variants one day. > >> > >> I was thinking the way in DRM API is better, always assuming it would > >> always in a multiple planes. The only problem is we don't have a way to > >> let the allocator that allocate contiguous memory for planes when we > >> need to do that. > >> > >>>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> > >>>>> Signed-off-by: Helen Koike <helen.koike@collabora.com> > >>>>> --- > >>>>> > >>>>> Changes in v6: > >>>>> The main change here was fixing the conversion, so planes reflects > >>>>> color planes, > >>>>> and to implement this properly I made major refactors compared to > >>>>> the previous > >>>>> version. > >>>>> - struct v4l2_plane_ext_pix_format removed, using struct > >>>>> v4l2_plane_pix_format instead (Tomasz) > >>>>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) > >>>>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) > >>>>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) > >>>>> - refactor conversion functions, so planes are color planes (Tomasz) > >>>>> - Don't explicitly check for e->modifier != 0 in > >>>>> v4l2_ext_pix_format_to_format() (Tomasz) > >>>>> - Use "ef" for extended formats in the framework for consistency > >>>>> (Tomasz) > >>>>> - Handle xfer_func field in conversions (Tomasz) > >>>>> - Zero reserved fields in v4l_s_ext_pix_fmt() and > >>>>> v4l_try_ext_pix_fmt() (Tomasz) > >>>>> - Refactor format functions to use v4l_fmt_ioctl_via_ext() > >>>>> - Several fixes/refactoring/changes > >>>>> - Remove EXT API for touch devices > >>>>> > >>>>> Changes in v5: > >>>>> - change sizes and reorder fields to avoid holes in the struct and make > >>>>> it the same for 32 and 64 bits > >>>>> - removed __attribute__ ((packed)) from uapi structs > >>>>> - Fix doc warning from make htmldocs > >>>>> - Updated commit message with EXT_PIX prefix for the ioctls. > >>>>> > >>>>> Changes in v4: > >>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, > >>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. > >>>>> - Add reserved fields > >>>>> - Removed num_planes from struct v4l2_ext_pix_format > >>>>> - Removed flag field from struct v4l2_ext_pix_format, since the only > >>>>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, > >>>>> where we can use modifiers, or add it back later through the reserved > >>>>> bits. > >>>>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && > >>>>> != MOD_INVALID > >>>>> - Fix type assignment in v4l_g_fmt_ext_pix() > >>>>> - Rebased on top of media/master (post 5.8-rc1) > >>>>> > >>>>> Changes in v3: > >>>>> - Rebased on top of media/master (post 5.4-rc1) > >>>>> > >>>>> Changes in v2: > >>>>> - Move the modifier in v4l2_ext_format (was formerly placed in > >>>>> v4l2_ext_plane) > >>>>> - Fix a few bugs in the converters and add a strict parameter to > >>>>> allow conversion of uninitialized/mis-initialized objects > >>>>> --- > >>>>> drivers/media/v4l2-core/v4l2-dev.c | 27 +- > >>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- > >>>>> include/media/v4l2-ioctl.h | 28 ++ > >>>>> include/uapi/linux/videodev2.h | 41 ++ > >>>>> 4 files changed, 602 insertions(+), 32 deletions(-) > >>>>> > >>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c > >>>>> b/drivers/media/v4l2-core/v4l2-dev.c > >>>>> index f9cff033d0dc..5add58cb6d45 100644 > >>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c > >>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c > >>>>> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct > >>>>> video_device *vdev) > >>>>> ops->vidioc_enum_fmt_vid_overlay)) || > >>>>> (is_tx && ops->vidioc_enum_fmt_vid_out)) > >>>>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); > >>>>> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || > >>>>> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) > >>>>> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > >>>>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || > >>>>> ops->vidioc_g_fmt_vid_cap_mplane || > >>>>> - ops->vidioc_g_fmt_vid_overlay)) || > >>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || > >>>>> (is_tx && (ops->vidioc_g_fmt_vid_out || > >>>>> ops->vidioc_g_fmt_vid_out_mplane || > >>>>> - ops->vidioc_g_fmt_vid_out_overlay))) > >>>>> + ops->vidioc_g_ext_pix_fmt_vid_out))) { > >>>>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); > >>>>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); > >>>>> + } > >>>>> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || > >>>>> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) > >>>>> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > >>>>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || > >>>>> ops->vidioc_s_fmt_vid_cap_mplane || > >>>>> - ops->vidioc_s_fmt_vid_overlay)) || > >>>>> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || > >>>>> (is_tx && (ops->vidioc_s_fmt_vid_out || > >>>>> ops->vidioc_s_fmt_vid_out_mplane || > >>>>> - ops->vidioc_s_fmt_vid_out_overlay))) > >>>>> + ops->vidioc_s_ext_pix_fmt_vid_out))) { > >>>>> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); > >>>>> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); > >>>>> + } > >>>>> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || > >>>>> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) > >>>>> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > >>>>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || > >>>>> ops->vidioc_try_fmt_vid_cap_mplane || > >>>>> - ops->vidioc_try_fmt_vid_overlay)) || > >>>>> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || > >>>>> (is_tx && (ops->vidioc_try_fmt_vid_out || > >>>>> ops->vidioc_try_fmt_vid_out_mplane || > >>>>> - ops->vidioc_try_fmt_vid_out_overlay))) > >>>>> + ops->vidioc_try_ext_pix_fmt_vid_out))) { > >>>>> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); > >>>>> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); > >>>>> + } > >>>>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); > >>>>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); > >>>>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); > >>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c > >>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c > >>>>> index 848286a284f6..a9c07c0a73ec 100644 > >>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c > >>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > >>>>> @@ -18,6 +18,8 @@ > >>>>> #include <linux/videodev2.h> > >>>>> +#include <drm/drm_fourcc.h> > >>>>> + > >>>>> #include <media/v4l2-common.h> > >>>>> #include <media/v4l2-ioctl.h> > >>>>> #include <media/v4l2-ctrls.h> > >>>>> @@ -38,6 +40,11 @@ > >>>>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), > >>>>> (vfd)->valid_ioctls) > >>>>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ > >>>>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ > >>>>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ > >>>>> + V4L2_CAP_VIDEO_M2M_MPLANE)) > >>>>> + > >>>>> struct std_descr { > >>>>> v4l2_std_id std; > >>>>> const char *descr; > >>>>> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, > >>>>> bool write_only) > >>>>> } > >>>>> } > >>>>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) > >>>>> +{ > >>>>> + const struct v4l2_ext_pix_format *ef = arg; > >>>>> + unsigned int i; > >>>>> + > >>>>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier > >>>>> %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, > >>>>> xfer_func=%u\n", > >>>>> + prt_names(ef->type, v4l2_type_names), > >>>>> + ef->width, ef->height, > >>>>> + (ef->pixelformat & 0xff), > >>>>> + (ef->pixelformat >> 8) & 0xff, > >>>>> + (ef->pixelformat >> 16) & 0xff, > >>>>> + (ef->pixelformat >> 24) & 0xff, > >>>>> + ef->modifier, prt_names(ef->field, v4l2_field_names), > >>>>> + ef->colorspace, ef->ycbcr_enc, > >>>>> + ef->quantization, ef->xfer_func); > >>>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; > >>>>> i++) > >>>>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", > >>>>> + i, ef->plane_fmt[i].bytesperline, > >>>>> + ef->plane_fmt[i].sizeimage); > >>>>> +} > >>>>> + > >>>>> static void v4l_print_framebuffer(const void *arg, bool write_only) > >>>>> { > >>>>> const struct v4l2_framebuffer *p = arg; > >>>>> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum > >>>>> v4l2_buf_type type) > >>>>> switch (type) { > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> if ((is_vid || is_tch) && is_rx && > >>>>> - (ops->vidioc_g_fmt_vid_cap || > >>>>> ops->vidioc_g_fmt_vid_cap_mplane)) > >>>>> + (ops->vidioc_g_fmt_vid_cap || > >>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap || > >>>>> + ops->vidioc_g_fmt_vid_cap_mplane)) > >>>>> return 0; > >>>>> break; > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>>>> - if ((is_vid || is_tch) && is_rx && > >>>>> ops->vidioc_g_fmt_vid_cap_mplane) > >>>>> + if ((is_vid || is_tch) && is_rx && > >>>>> + (ops->vidioc_g_fmt_vid_cap_mplane || > >>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) > >>>>> return 0; > >>>>> break; > >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>>>> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum > >>>>> v4l2_buf_type type) > >>>>> break; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> if (is_vid && is_tx && > >>>>> - (ops->vidioc_g_fmt_vid_out || > >>>>> ops->vidioc_g_fmt_vid_out_mplane)) > >>>>> + (ops->vidioc_g_fmt_vid_out || > >>>>> + ops->vidioc_g_ext_pix_fmt_vid_out || > >>>>> + ops->vidioc_g_fmt_vid_out_mplane)) > >>>>> return 0; > >>>>> break; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>>>> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) > >>>>> + if (is_vid && is_tx && > >>>>> + (ops->vidioc_g_ext_pix_fmt_vid_out || > >>>>> + ops->vidioc_g_fmt_vid_out_mplane)) > >>>>> return 0; > >>>>> break; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>>>> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct > >>>>> v4l2_format *fmt) > >>>>> sizeof(fmt->fmt.pix) - offset); > >>>>> } > >>>>> +static void > >>>>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, > >>>>> + struct v4l2_pix_format *pix) > >>>>> +{ > >>>>> + unsigned int i; > >>>>> + > >>>>> + pix->width = ef->width; > >>>>> + pix->height = ef->height; > >>>>> + pix->field = ef->field; > >>>>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > >>>>> + pix->colorspace = ef->colorspace; > >>>>> + pix->ycbcr_enc = ef->ycbcr_enc; > >>>>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> + pix->quantization = ef->quantization; > >>>>> + pix->pixelformat = ef->pixelformat; > >>>>> + pix->bytesperline = ef->plane_fmt[0].bytesperline; > >>>>> + pix->sizeimage = ef->plane_fmt[0].sizeimage; > >>>>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; > >>>>> i++) > >>>>> + pix->sizeimage += ef->plane_fmt[i].sizeimage; > >>>>> +} > >>>>> + > >>>>> +static void > >>>>> +v4l2_ext_pix_format_to_pix_mp_format(const struct > >>>>> v4l2_ext_pix_format *ef, > >>>>> + struct v4l2_pix_format_mplane *pix_mp) > >>>>> +{ > >>>>> + const struct v4l2_format_info *info = > >>>>> + v4l2_format_info(ef->pixelformat); > >>>>> + unsigned int i; > >>>>> + > >>>>> + pix_mp->width = ef->width; > >>>>> + pix_mp->height = ef->height; > >>>>> + pix_mp->field = ef->field; > >>>>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; > >>>>> + pix_mp->colorspace = ef->colorspace; > >>>>> + pix_mp->ycbcr_enc = ef->ycbcr_enc; > >>>>> + pix_mp->quantization = ef->quantization; > >>>>> + pix_mp->pixelformat = ef->pixelformat; > >>>>> + > >>>>> + /* This is true when converting to non-M-variant */ > >>>>> + if (info && info->mem_planes == 1) { > >>>>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; > >>>>> + pix_mp->num_planes = 1; > >>>>> + for (i = 1; i < VIDEO_MAX_PLANES && > >>>>> ef->plane_fmt[i].sizeimage; i++) > >>>>> + pix_mp->plane_fmt[0].sizeimage += > >>>>> ef->plane_fmt[i].sizeimage; > >>>>> + > >>>>> + return; > >>>>> + } > >>>>> + > >>>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; > >>>>> i++) > >>>>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; > >>>>> + pix_mp->num_planes = i; > >>>>> +} > >>>>> + > >>>>> +/* > >>>>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to > >>>>> v4l2_format > >>>>> + * > >>>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. > >>>>> + * @f: A pointer to struct v4l2_format to be filled. > >>>>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. > >>>>> + * > >>>>> + * If pixelformat should be converted to M-variant, change > >>>>> ef->pixelformat > >>>>> + * to the M-variant before calling this function. > >>>>> + */ > >>>>> +static void v4l2_ext_pix_format_to_format(const struct > >>>>> v4l2_ext_pix_format *ef, > >>>>> + struct v4l2_format *f, bool is_mplane) > >>>>> +{ > >>>>> + memset(f, 0, sizeof(*f)); > >>>>> + > >>>>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && > >>>>> + ef->modifier != DRM_FORMAT_MOD_INVALID) > >>>>> + pr_warn("Modifiers are not supported in v4l2_format, > >>>>> ignoring %llx\n", > >>>>> + ef->modifier); > >>>>> + > >>>>> + if (!is_mplane) { > >>>>> + f->type = ef->type; > >>>>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); > >>>>> + return; > >>>>> + } > >>>>> + > >>>>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) > >>>>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; > >>>>> + else > >>>>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; > >>>>> + > >>>>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); > >>>>> +} > >>>>> + > >>>>> +static void > >>>>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, > >>>>> + struct v4l2_ext_pix_format *ef) > >>>>> +{ > >>>>> + const struct v4l2_format_info *info = > >>>>> + v4l2_format_info(pix->pixelformat); > >>>>> + unsigned int i; > >>>>> + > >>>>> + ef->width = pix->width; > >>>>> + ef->height = pix->height; > >>>>> + ef->field = pix->field; > >>>>> + ef->colorspace = pix->colorspace; > >>>>> + ef->ycbcr_enc = pix->ycbcr_enc; > >>>>> + ef->quantization = pix->quantization; > >>>>> + ef->xfer_func = pix->xfer_func; > >>>>> + if (pix->flags) > >>>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); > >>>>> + > >>>>> + /* We assume M-variants won't be used in this function */ > >>>>> + ef->pixelformat = pix->pixelformat; > >>>>> + > >>>>> + ef->plane_fmt[0].bytesperline = pix->bytesperline; > >>>>> + ef->plane_fmt[0].sizeimage = pix->sizeimage; > >>>>> + > >>>>> + if (!info) > >>>>> + return; > >>>>> + > >>>>> + for (i = 1; i < info->comp_planes; i++) { > >>>>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; > >>>>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * > >>>>> + ef->height / info->vdiv; > >>>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > >>>>> + } > >>>>> +} > >>>>> + > >>>>> +static void > >>>>> +v4l2_pix_mp_format_to_ext_pix_format(const struct > >>>>> v4l2_pix_format_mplane *pix_mp, > >>>>> + struct v4l2_ext_pix_format *ef) > >>>>> +{ > >>>>> + const struct v4l2_format_info *info = > >>>>> + v4l2_format_info(pix_mp->pixelformat); > >>>>> + unsigned int i; > >>>>> + > >>>>> + ef->width = pix_mp->width; > >>>>> + ef->height = pix_mp->height; > >>>>> + ef->field = pix_mp->field; > >>>>> + ef->colorspace = pix_mp->colorspace; > >>>>> + ef->ycbcr_enc = pix_mp->ycbcr_enc; > >>>>> + ef->quantization = pix_mp->quantization; > >>>>> + ef->xfer_func = pix_mp->xfer_func; > >>>>> + if (pix_mp->flags) > >>>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); > >>>>> + > >>>>> + if (!info) > >>>>> + return; > >>>>> + > >>>>> + ef->pixelformat = info && info->norm ? > >>>> > >>>> 'info &&' can be dropped, info is always non-NULL here. > >>>> > >>>>> + info->norm : pix_mp->pixelformat; > >>>>> + > >>>>> + if (info->comp_planes == info->mem_planes) { > >>>>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; > >>>>> i++) > >>>>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; > >>>>> + > >>>>> + return; > >>>>> + } > >>>>> + > >>>>> + /* case where mem_planes is 1 and comp_planes > 1 */ > >>>>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; > >>>>> + for (i = 1; i < info->comp_planes; i++) { > >>>>> + ef->plane_fmt[i].bytesperline = > >>>>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; > >>>>> + ef->plane_fmt[i].sizeimage = > >>>>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; > >>>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; > >>>>> + } > >>>>> +} > >>>>> + > >>>>> +/* > >>>>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to > >>>>> v4l2_ext_pix_format > >>>>> + * > >>>>> + * @f: A pointer to struct v4l2_format to be converted. > >>>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. > >>>>> + * > >>>>> + * This method normalize the pixelformat to non-M variant. > >>>>> + */ > >>>>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, > >>>>> + struct v4l2_ext_pix_format *ef) > >>>>> +{ > >>>>> + memset(ef, 0, sizeof(*ef)); > >>>>> + > >>>>> + switch (f->type) { > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> + ef->type = f->type; > >>>>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); > >>>>> + break; > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>>>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > >>>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > >>>>> + break; > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>>>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > >>>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); > >>>>> + break; > >>>>> + default: > >>>>> + WARN("Converting to Ext Pix Format with wrong buffer type > >>>>> %s\n", > >>>>> + prt_names(f->type, v4l2_type_names)); > >>>>> + break; > >>>>> + } > >>>>> +} > >>>>> + > >>>>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, > >>>>> struct file *file, void *fh, void *arg) > >>>>> { > >>>>> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct > >>>>> v4l2_pix_format *p) > >>>>> p->xfer_func = 0; > >>>>> } > >>>>> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, > >>>>> + struct file *file, void *fh, > >>>>> + struct v4l2_format *f, > >>>>> + unsigned int ioctl) > >>>>> +{ > >>>>> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); > >>>>> + struct video_device *vdev = video_devdata(file); > >>>>> + struct v4l2_ext_pix_format ef = {0}; > >>>>> + u32 original_pixfmt = 0; > >>>>> + u32 cap_mask; > >>>>> + int ret; > >>>>> + > >>>>> + if (ioctl != VIDIOC_G_FMT) { > >>>>> + /* > >>>>> + * If CSC attributes are read only, set them to DEFAULT > >>>>> + * to avoid changes by the driver. > >>>>> + */ > >>>>> + if (is_multiplanar) { > >>>>> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > >>>>> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; > >>>>> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > >>>>> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; > >>>>> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; > >>>>> + } > >>>>> + /* Unset the flag to avoid warning in the convertion */ > >>>>> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > >>>>> + > >>>>> + /* Save pixelformat in case M-variant is being used */ > >>>>> + original_pixfmt = f->fmt.pix_mp.pixelformat; > >>>>> + } else { > >>>>> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { > >>>>> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; > >>>>> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; > >>>>> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; > >>>>> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; > >>>>> + } > >>>>> + /* Unset the flag to avoid warning in the convertion */ > >>>>> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; > >>>>> + } > >>>>> + v4l2_format_to_ext_pix_format(f, &ef); > >>>>> + } > >>>>> + > >>>>> + switch (f->type) { > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>>>> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | > >>>>> + V4L2_CAP_VIDEO_M2M_MPLANE; > >>>>> + if (!!(vdev->device_caps & cap_mask) != > >>>>> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) > >>>>> + return -EINVAL; > >>>>> + > >>>>> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; > >>>>> + if (ioctl == VIDIOC_G_FMT) > >>>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); > >>>>> + else if (ioctl == VIDIOC_S_FMT) > >>>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); > >>>>> + else > >>>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > >>>>> + &ef); > >>>>> + break; > >>>>> + > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>>>> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | > >>>>> + V4L2_CAP_VIDEO_M2M_MPLANE; > >>>>> + if (!!(vdev->device_caps & cap_mask) != > >>>>> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) > >>>>> + return -EINVAL; > >>>>> + > >>>>> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; > >>>>> + if (ioctl == VIDIOC_G_FMT) > >>>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); > >>>>> + else if (ioctl == VIDIOC_S_FMT) > >>>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); > >>>>> + else > >>>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > >>>>> + &ef); > >>>>> + break; > >>>>> + > >>>>> + default: > >>>>> + return -EINVAL; > >>>>> + } > >>>>> + > >>>>> + if (ret) > >>>>> + return ret; > >>>>> + > >>>>> + if (original_pixfmt != ef.pixelformat && > >>>>> + v4l2_format_info(original_pixfmt)) > >>>> > >>>> Could this test be simplified to: 'if (original_pixfmt)'? > >>>> > >>>> I.e., if the original pixfmt was saved, then restore it here. > >>>> > >>>>> + ef.pixelformat = original_pixfmt; > >>>>> + > >>>>> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); > >>>>> + return 0; > >>>>> +} > >>>>> + > >>>>> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, > >>>>> struct file *file, void *fh, void *arg) > >>>>> { > >>>>> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> switch (p->type) { > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) > >>>>> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && > >>>>> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); > >>>>> + ret = ops->vidioc_g_fmt_vid_cap ? > >>>>> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> + VIDIOC_G_FMT); > >>>>> /* just in case the driver zeroed it again */ > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) > >>>>> v4l_pix_format_touch(&p->fmt.pix); > >>>>> return ret; > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>>>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > >>>>> + if (ops->vidioc_g_fmt_vid_cap_mplane) > >>>>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); > >>>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) > >>>> > >>>> 'else' can be dropped. > >>>> > >>>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> + VIDIOC_G_FMT); > >>>>> + break; > >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>>>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); > >>>>> case V4L2_BUF_TYPE_VBI_CAPTURE: > >>>>> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: > >>>>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) > >>>>> + if (unlikely(!ops->vidioc_g_fmt_vid_out && > >>>>> + !ops->vidioc_g_ext_pix_fmt_vid_out)) > >>>>> break; > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); > >>>>> + ret = ops->vidioc_g_fmt_vid_out ? > >>>>> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); > >>>>> /* just in case the driver zeroed it again */ > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> return ret; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>>>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > >>>>> + if (ops->vidioc_g_fmt_vid_out_mplane) > >>>>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); > >>>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) > >>>> > >>>> Ditto. > >>>> > >>>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> + VIDIOC_G_FMT); > >>>>> + break; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>>>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); > >>>>> case V4L2_BUF_TYPE_VBI_OUTPUT: > >>>>> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> return -EINVAL; > >>>>> } > >>>>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > >>>>> + struct file *file, void *fh, void *arg) > >>>>> +{ > >>>>> + struct v4l2_ext_pix_format *ef = arg; > >>>>> + struct v4l2_format f = { > >>>>> + .type = ef->type, > >>>>> + }; > >>>>> + int ret = check_fmt(file, ef->type); > >>>>> + > >>>>> + if (ret) > >>>>> + return ret; > >>>>> + > >>>>> + memset(ef, 0, sizeof(*ef)); > >>>>> + ef->type = f.type; > >>>>> + > >>>>> + switch (f.type) { > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) > >>>>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); > >>>>> + break; > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> + if (ops->vidioc_g_ext_pix_fmt_vid_out) > >>>>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); > >>>>> + break; > >>>>> + default: > >>>>> + return -EINVAL; > >>>>> + } > >>>>> + > >>>>> + ret = v4l_g_fmt(ops, file, fh, &f); > >>>>> + if (ret) > >>>>> + return ret; > >>>>> + > >>>>> + v4l2_format_to_ext_pix_format(&f, ef); > >>>>> + return 0; > >>>>> +} > >>>>> + > >>>>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, > >>>>> struct file *file, void *fh, void *arg) > >>>>> { > >>>>> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> switch (p->type) { > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) > >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && > >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>>>> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); > >>>>> + ret = ops->vidioc_s_fmt_vid_cap ? > >>>>> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > >>>>> /* just in case the driver zeroed it again */ > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) > >>>>> v4l_pix_format_touch(&p->fmt.pix); > >>>>> return ret; > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) > >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && > >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>>>> bytesperline); > >>>>> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); > >>>>> + return ops->vidioc_s_fmt_vid_cap_mplane ? > >>>>> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> VIDIOC_S_FMT); > >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>>>> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) > >>>>> break; > >>>>> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > >>>>> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) > >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out && > >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>>>> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); > >>>>> + ret = ops->vidioc_s_fmt_vid_out ? > >>>>> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > >>>>> /* just in case the driver zeroed it again */ > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> return ret; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) > >>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && > >>>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>>>> bytesperline); > >>>>> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); > >>>>> + return ops->vidioc_s_fmt_vid_out_mplane ? > >>>>> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>>>> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) > >>>>> break; > >>>>> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> return -EINVAL; > >>>>> } > >>>>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > >>>>> + struct file *file, void *fh, void *arg) > >>>>> +{ > >>>>> + struct video_device *vfd = video_devdata(file); > >>>>> + struct v4l2_ext_pix_format *ef = arg; > >>>>> + struct v4l2_format f; > >>>>> + int ret = check_fmt(file, ef->type); > >>>>> + > >>>>> + if (ret) > >>>>> + return ret; > >>>>> + > >>>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); > >>>>> + > >>>>> + switch (ef->type) { > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) > >>>>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); > >>>>> + break; > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> + if (ops->vidioc_s_ext_pix_fmt_vid_out) > >>>>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); > >>>>> + break; > >>>>> + default: > >>>>> + return -EINVAL; > >>>>> + } > >>>>> + > >>>>> + v4l2_ext_pix_format_to_format(ef, &f, > >>>>> V4L2_IS_CAP_MULTIPLANAR(vfd)); > >>>>> + > >>>>> + ret = v4l_s_fmt(ops, file, fh, &f); > >>>>> + if (ret) > >>>>> + /* TODO: retry with M-variant of ef->pixelformat? */ > >>>> > >>>> See my comments on this at the top. > >>>> > >>>>> + return ret; > >>>>> + > >>>>> + v4l2_format_to_ext_pix_format(&f, ef); > >>>>> + return 0; > >>>>> +} > >>>>> + > >>>>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, > >>>>> struct file *file, void *fh, void *arg) > >>>>> { > >>>>> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> switch (p->type) { > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) > >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && > >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>>>> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); > >>>>> + ret = ops->vidioc_try_fmt_vid_cap ? > >>>>> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> VIDIOC_TRY_FMT); > >>>>> /* just in case the driver zeroed it again */ > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) > >>>>> v4l_pix_format_touch(&p->fmt.pix); > >>>>> return ret; > >>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: > >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) > >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && > >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>>>> bytesperline); > >>>>> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); > >>>>> + return ops->vidioc_try_fmt_vid_cap_mplane ? > >>>>> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> + VIDIOC_TRY_FMT); > >>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: > >>>>> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) > >>>>> break; > >>>>> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); > >>>>> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) > >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out && > >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix); > >>>>> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); > >>>>> + ret = ops->vidioc_try_fmt_vid_out ? > >>>>> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> VIDIOC_TRY_FMT); > >>>>> /* just in case the driver zeroed it again */ > >>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; > >>>>> return ret; > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: > >>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) > >>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && > >>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) > >>>>> break; > >>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); > >>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) > >>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], > >>>>> bytesperline); > >>>>> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); > >>>>> + return ops->vidioc_try_fmt_vid_out_mplane ? > >>>>> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : > >>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, > >>>>> + VIDIOC_TRY_FMT); > >>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: > >>>>> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) > >>>>> break; > >>>>> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct > >>>>> v4l2_ioctl_ops *ops, > >>>>> return -EINVAL; > >>>>> } > >>>>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, > >>>>> + struct file *file, void *fh, void *arg) > >>>>> +{ > >>>>> + struct video_device *vfd = video_devdata(file); > >>>>> + struct v4l2_ext_pix_format *ef = arg; > >>>>> + struct v4l2_format f; > >>>>> + int ret = check_fmt(file, ef->type); > >>>>> + > >>>>> + if (ret) > >>>>> + return ret; > >>>>> + > >>>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); > >>>>> + > >>>>> + switch (ef->type) { > >>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: > >>>>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) > >>>>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, > >>>>> + ef); > >>>>> + break; > >>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: > >>>>> + if (ops->vidioc_try_ext_pix_fmt_vid_out) > >>>>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, > >>>>> + ef); > >>>>> + break; > >>>>> + default: > >>>>> + return -EINVAL; > >>>>> + } > >>>>> + > >>>>> + v4l2_ext_pix_format_to_format(ef, &f, > >>>>> V4L2_IS_CAP_MULTIPLANAR(vfd)); > >>>>> + > >>>>> + ret = v4l_try_fmt(ops, file, fh, &f); > >>>>> + if (ret) > >>>>> + /* TODO: retry with M-variant of ef->pixelformat? */ > >>>>> + return ret; > >>>>> + > >>>>> + v4l2_format_to_ext_pix_format(&f, ef); > >>>>> + return 0; > >>>>> +} > >>>>> + > >>>>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, > >>>>> struct file *file, void *fh, void *arg) > >>>>> { > >>>>> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info > >>>>> v4l2_ioctls[] = { > >>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, > >>>>> v4l_print_freq_band, 0), > >>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, > >>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), > >>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, > >>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | > >>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), > >>>>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, > >>>>> v4l_print_ext_pix_format, 0), > >>>>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, > >>>>> v4l_print_ext_pix_format, INFO_FL_PRIO), > >>>>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, > >>>>> v4l_print_ext_pix_format, 0), > >>>>> }; > >>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) > >>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h > >>>>> index edb733f21604..c44708dc9355 100644 > >>>>> --- a/include/media/v4l2-ioctl.h > >>>>> +++ b/include/media/v4l2-ioctl.h > >>>>> @@ -48,11 +48,17 @@ struct v4l2_fh; > >>>>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements > >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture > >>>>> * in single plane mode > >>>>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that > >>>>> implements > >>>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic > >>>>> for video > >>>>> + * capture > >>>>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements > >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay > >>>>> * @vidioc_g_fmt_vid_out: pointer to the function that implements > >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out > >>>>> * in single plane mode > >>>>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that > >>>>> implements > >>>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic > >>>>> for video > >>>>> + * out > >>>>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that > >>>>> implements > >>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video > >>>>> overlay output > >>>>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements > >>>>> @@ -82,11 +88,16 @@ struct v4l2_fh; > >>>>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements > >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture > >>>>> * in single plane mode > >>>>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that > >>>>> implements > >>>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic > >>>>> for video > >>>>> + * capture > >>>>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements > >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay > >>>>> * @vidioc_s_fmt_vid_out: pointer to the function that implements > >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out > >>>>> * in single plane mode > >>>>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that > >>>>> implements > >>>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for > >>>>> video out > >>>>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that > >>>>> implements > >>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video > >>>>> overlay output > >>>>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements > >>>>> @@ -116,11 +127,16 @@ struct v4l2_fh; > >>>>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements > >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video > >>>>> capture > >>>>> * in single plane mode > >>>>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that > >>>>> implements > >>>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl > >>>>> logic for > >>>>> + video capture > >>>>> * @vidioc_try_fmt_vid_overlay: pointer to the function that > >>>>> implements > >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video > >>>>> overlay > >>>>> * @vidioc_try_fmt_vid_out: pointer to the function that implements > >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out > >>>>> * in single plane mode > >>>>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that > >>>>> implements > >>>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for > >>>>> video out > >>>>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that > >>>>> implements > >>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video > >>>>> overlay > >>>>> * output > >>>>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { > >>>>> /* VIDIOC_G_FMT handlers */ > >>>>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > >>>>> + struct v4l2_ext_pix_format *ef); > >>>>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, > >>>>> + struct v4l2_ext_pix_format *ef); > >>>>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, > >>>>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { > >>>>> /* VIDIOC_S_FMT handlers */ > >>>>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > >>>>> + struct v4l2_ext_pix_format *ef); > >>>>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, > >>>>> + struct v4l2_ext_pix_format *ef); > >>>>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, > >>>>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { > >>>>> /* VIDIOC_TRY_FMT handlers */ > >>>>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, > >>>>> + struct v4l2_ext_pix_format *ef); > >>>>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, > >>>>> + struct v4l2_ext_pix_format *ef); > >>>>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, > >>>>> struct v4l2_format *f); > >>>>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, > >>>>> diff --git a/include/uapi/linux/videodev2.h > >>>>> b/include/uapi/linux/videodev2.h > >>>>> index d9b7c9177605..a2d850513708 100644 > >>>>> --- a/include/uapi/linux/videodev2.h > >>>>> +++ b/include/uapi/linux/videodev2.h > >>>>> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { > >>>>> __u8 reserved[7]; > >>>>> } __attribute__ ((packed)); > >>>>> +/** > >>>>> + * struct v4l2_ext_pix_format - extended single/multiplanar format > >>>>> definition > >>>>> + * @type: type of the data stream; > >>>>> V4L2_BUF_TYPE_VIDEO_CAPTURE or > >>>>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT > >>>>> + * @width: image width in pixels > >>>>> + * @height: image height in pixels > >>>>> + * @field: enum v4l2_field; field order (for interlaced video) > >>>>> + * @plane_fmt: per-plane information > >>>>> + * @pixelformat: little endian four character code (fourcc) > >>>>> + * @modifier: modifier applied to the format (used for tiled > >>>>> formats > >>>>> + * and other kind of HW-specific formats, like compressed > >>>>> + * formats) as defined in drm_fourcc.h > >>>>> + * @colorspace: enum v4l2_colorspace; supplemental to > >>>>> pixelformat > >>>>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function > >>>>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding > >>>>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding > >>>>> + * @quantization: enum v4l2_quantization, colorspace quantization > >>>>> + * @reserved: extra space reserved for future fields, must be > >>>>> set to 0 > >>>>> + */ > >>>>> +struct v4l2_ext_pix_format { > >>>>> + __u32 type; > >>>>> + __u32 width; > >>>>> + __u32 height; > >>>>> + __u32 field; > >>>>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; > >>>>> + __u32 pixelformat; > >>>>> + __u64 modifier; > >>>>> + __u32 colorspace; > >>>>> + __u32 xfer_func; > >>>>> + union { > >>>>> + __u32 ycbcr_enc; > >>>>> + __u32 hsv_enc; > >>>>> + }; > >>>>> + __u32 quantization; > >>>>> + __u32 reserved[9]; > >>>>> +}; > >>>>> + > >>>>> /** > >>>>> * struct v4l2_sdr_format - SDR format definition > >>>>> * @pixelformat: little endian four character code (fourcc) > >>>>> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { > >>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct > >>>>> v4l2_query_ext_ctrl) > >>>>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct > >>>>> v4l2_ext_pix_format) > >>>>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct > >>>>> v4l2_ext_pix_format) > >>>>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct > >>>>> v4l2_ext_pix_format) > >>>>> + > >>>>> /* Reminder: when adding new ioctls please add support for them to > >>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ > >>>>> > > > > -- > > Regards, > > > > Laurent Pinchart > > -- > Hsia-Jun(Randy) Li
On Mon, Nov 07, 2022 at 01:11:32AM +0300, Dmitry Osipenko wrote: > On 11/5/22 18:19, Hsia-Jun Li wrote: > > Hello Helen > > > > I didn't see any updates from V6 and V7-WIP in your repo. That is what I > > need to for our complex tile formats in our platform. > > > > Any future plane here? > > > > Besides I have some ideas on these patches. > > I was looking into updating this patchset few months ago and the biggest > blocker was the absence of immediate upstream user for this new UAPI. > What your platform is? Is the driver stack completely opensource? libcamera could be a good place to test (part of) this API in userspace. We could really do with the data offset feature for instance.
Hi Hsia-Jun, Helen no longer works for Collabora and left the community. I've added her latest email. Helen, are you still interested in hearing about this? Regards, Hans On 05/11/2022 16:19, Hsia-Jun Li wrote: > Hello Helen > > I didn't see any updates from V6 and V7-WIP in your repo. That is what I need to for our complex tile formats in our platform. > > Any future plane here? > > Besides I have some ideas on these patches. > > On 2/24/21 23:12, Helen Koike wrote: >> Hi Hans, >> >> Thank you for your comment, please see my reply below. >> >> On 2/23/21 9:35 AM, Hans Verkuil wrote: >>> Hi Helen, >>> >>> On 14/01/2021 19:07, Helen Koike wrote: >>>> This is part of the multiplanar and singleplanar unification process. >>>> v4l2_ext_pix_format is supposed to work for both cases. >>>> >>>> We also add the concept of modifiers already employed in DRM to expose >>>> HW-specific formats (like tiled or compressed formats) and allow >>>> exchanging this information with the DRM subsystem in a consistent way. >>>> >>>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in >>>> v4l2_ext_format, other types will be rejected if you use the >>>> {G,S,TRY}_EXT_PIX_FMT ioctls. >>>> >>>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls >>>> in drivers, but, in the meantime, the core takes care of converting >>>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can >>>> still work if the userspace app/lib uses the new ioctls. >>>> >>>> The conversion is also done the other around to allow userspace >>>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the >>>> _ext_ hooks. >>> >>> I have some small comments below, but also one high level comment: >>> >>> Regarding M variants of pixelformats: this patch 'normalizes' them to >>> regular pixelformats in the extended API. This makes life complicated, >>> and I wonder if this is the right approach. >>> >>> Currently there are two reasons for a driver to support e.g. NV12M: >>> either luma and chroma need to be in two memory banks, or the luma >>> and chroma planes cannot be contiguous due to alignment requirements. >>> >>> The first requirement is still valid for drivers that support the extended API. >>> The second requirement is no longer a reason to support NV12M. But I >>> don't think we should just drop NV12M support if it was already supported >>> before the conversion to this extended API. Since NV12M allocates two buffers >>> instead of one, it is still different from a regular NV12. >> >> I don't see what would be the difference when using NV12 and NV12M in Ext API. >> NV12 could be used for both requirements. It would even allow the second >> requirement with a single memory buffer. >> > Although I don't have problem here to support both single and multiple planes in our hardware. But using the single plane format here is not convience for us. > > The tile format in our platform is little complex, you can't calculate the stride and sizeimage easily with just the modifier, width and height. Then we don't have a good place to record the offset here. > > Besides, the luma and chroma planes would always need their own compression meta data planes. If we use NV12 here, that would make a very strange pixel format, one plane for the pixel datas and two > planes for the meta data. > >>> >>> I would prefer that such drivers support both NV12 and NV12M, so no >>> automatic conversion. >>> >>> A related question is how to handle pixelformat enumeration: with the >>> extended API an NV12 format might work, but not with the old API (e.g. >>> due to memory alignment requirements). I wonder if a VIDIOC_ENUM_EXT_PIX_FMT >>> isn't needed. >> >> We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. >> If the driver reports NV12M, userspace can use it and the framework >> normilizes it. >> >>> >>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while VIDIOC_ENUM_FMT >>> would just report NV12M. >> >> If NV12 and NV12M are equivalent in Ext API, I don't see why we would >> report both (unless I'm missing something, which is probably the case). >> >> The idea was to deprecate the M-variants one day. > I was thinking the way in DRM API is better, always assuming it would always in a multiple planes. The only problem is we don't have a way to let the allocator that allocate contiguous memory for > planes when we need to do that. >> >> Regards, >> Helen >> >>> >>>> >>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> >>>> Signed-off-by: Helen Koike <helen.koike@collabora.com> >>>> --- >>>> >>>> Changes in v6: >>>> The main change here was fixing the conversion, so planes reflects color planes, >>>> and to implement this properly I made major refactors compared to the previous >>>> version. >>>> - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz) >>>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) >>>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) >>>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) >>>> - refactor conversion functions, so planes are color planes (Tomasz) >>>> - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz) >>>> - Use "ef" for extended formats in the framework for consistency (Tomasz) >>>> - Handle xfer_func field in conversions (Tomasz) >>>> - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz) >>>> - Refactor format functions to use v4l_fmt_ioctl_via_ext() >>>> - Several fixes/refactoring/changes >>>> - Remove EXT API for touch devices >>>> >>>> Changes in v5: >>>> - change sizes and reorder fields to avoid holes in the struct and make >>>> it the same for 32 and 64 bits >>>> - removed __attribute__ ((packed)) from uapi structs >>>> - Fix doc warning from make htmldocs >>>> - Updated commit message with EXT_PIX prefix for the ioctls. >>>> >>>> Changes in v4: >>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, >>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >>>> - Add reserved fields >>>> - Removed num_planes from struct v4l2_ext_pix_format >>>> - Removed flag field from struct v4l2_ext_pix_format, since the only >>>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, >>>> where we can use modifiers, or add it back later through the reserved >>>> bits. >>>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && >>>> != MOD_INVALID >>>> - Fix type assignment in v4l_g_fmt_ext_pix() >>>> - Rebased on top of media/master (post 5.8-rc1) >>>> >>>> Changes in v3: >>>> - Rebased on top of media/master (post 5.4-rc1) >>>> >>>> Changes in v2: >>>> - Move the modifier in v4l2_ext_format (was formerly placed in >>>> v4l2_ext_plane) >>>> - Fix a few bugs in the converters and add a strict parameter to >>>> allow conversion of uninitialized/mis-initialized objects >>>> --- >>>> drivers/media/v4l2-core/v4l2-dev.c | 27 +- >>>> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- >>>> include/media/v4l2-ioctl.h | 28 ++ >>>> include/uapi/linux/videodev2.h | 41 ++ >>>> 4 files changed, 602 insertions(+), 32 deletions(-) >>>> >>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c >>>> index f9cff033d0dc..5add58cb6d45 100644 >>>> --- a/drivers/media/v4l2-core/v4l2-dev.c >>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c >>>> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct video_device *vdev) >>>> ops->vidioc_enum_fmt_vid_overlay)) || >>>> (is_tx && ops->vidioc_enum_fmt_vid_out)) >>>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); >>>> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || >>>> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) >>>> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || >>>> ops->vidioc_g_fmt_vid_cap_mplane || >>>> - ops->vidioc_g_fmt_vid_overlay)) || >>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || >>>> (is_tx && (ops->vidioc_g_fmt_vid_out || >>>> ops->vidioc_g_fmt_vid_out_mplane || >>>> - ops->vidioc_g_fmt_vid_out_overlay))) >>>> + ops->vidioc_g_ext_pix_fmt_vid_out))) { >>>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); >>>> + } >>>> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || >>>> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) >>>> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || >>>> ops->vidioc_s_fmt_vid_cap_mplane || >>>> - ops->vidioc_s_fmt_vid_overlay)) || >>>> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || >>>> (is_tx && (ops->vidioc_s_fmt_vid_out || >>>> ops->vidioc_s_fmt_vid_out_mplane || >>>> - ops->vidioc_s_fmt_vid_out_overlay))) >>>> + ops->vidioc_s_ext_pix_fmt_vid_out))) { >>>> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>>> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); >>>> + } >>>> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || >>>> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) >>>> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || >>>> ops->vidioc_try_fmt_vid_cap_mplane || >>>> - ops->vidioc_try_fmt_vid_overlay)) || >>>> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || >>>> (is_tx && (ops->vidioc_try_fmt_vid_out || >>>> ops->vidioc_try_fmt_vid_out_mplane || >>>> - ops->vidioc_try_fmt_vid_out_overlay))) >>>> + ops->vidioc_try_ext_pix_fmt_vid_out))) { >>>> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>>> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); >>>> + } >>>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); >>>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); >>>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); >>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c >>>> index 848286a284f6..a9c07c0a73ec 100644 >>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >>>> @@ -18,6 +18,8 @@ >>>> #include <linux/videodev2.h> >>>> +#include <drm/drm_fourcc.h> >>>> + >>>> #include <media/v4l2-common.h> >>>> #include <media/v4l2-ioctl.h> >>>> #include <media/v4l2-ctrls.h> >>>> @@ -38,6 +40,11 @@ >>>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls) >>>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ >>>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ >>>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ >>>> + V4L2_CAP_VIDEO_M2M_MPLANE)) >>>> + >>>> struct std_descr { >>>> v4l2_std_id std; >>>> const char *descr; >>>> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, bool write_only) >>>> } >>>> } >>>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) >>>> +{ >>>> + const struct v4l2_ext_pix_format *ef = arg; >>>> + unsigned int i; >>>> + >>>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n", >>>> + prt_names(ef->type, v4l2_type_names), >>>> + ef->width, ef->height, >>>> + (ef->pixelformat & 0xff), >>>> + (ef->pixelformat >> 8) & 0xff, >>>> + (ef->pixelformat >> 16) & 0xff, >>>> + (ef->pixelformat >> 24) & 0xff, >>>> + ef->modifier, prt_names(ef->field, v4l2_field_names), >>>> + ef->colorspace, ef->ycbcr_enc, >>>> + ef->quantization, ef->xfer_func); >>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >>>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", >>>> + i, ef->plane_fmt[i].bytesperline, >>>> + ef->plane_fmt[i].sizeimage); >>>> +} >>>> + >>>> static void v4l_print_framebuffer(const void *arg, bool write_only) >>>> { >>>> const struct v4l2_framebuffer *p = arg; >>>> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) >>>> switch (type) { >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> if ((is_vid || is_tch) && is_rx && >>>> - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane)) >>>> + (ops->vidioc_g_fmt_vid_cap || >>>> + ops->vidioc_g_ext_pix_fmt_vid_cap || >>>> + ops->vidioc_g_fmt_vid_cap_mplane)) >>>> return 0; >>>> break; >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>> - if ((is_vid || is_tch) && is_rx && ops->vidioc_g_fmt_vid_cap_mplane) >>>> + if ((is_vid || is_tch) && is_rx && >>>> + (ops->vidioc_g_fmt_vid_cap_mplane || >>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) >>>> return 0; >>>> break; >>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) >>>> break; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> if (is_vid && is_tx && >>>> - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane)) >>>> + (ops->vidioc_g_fmt_vid_out || >>>> + ops->vidioc_g_ext_pix_fmt_vid_out || >>>> + ops->vidioc_g_fmt_vid_out_mplane)) >>>> return 0; >>>> break; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) >>>> + if (is_vid && is_tx && >>>> + (ops->vidioc_g_ext_pix_fmt_vid_out || >>>> + ops->vidioc_g_fmt_vid_out_mplane)) >>>> return 0; >>>> break; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct v4l2_format *fmt) >>>> sizeof(fmt->fmt.pix) - offset); >>>> } >>>> +static void >>>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, >>>> + struct v4l2_pix_format *pix) >>>> +{ >>>> + unsigned int i; >>>> + >>>> + pix->width = ef->width; >>>> + pix->height = ef->height; >>>> + pix->field = ef->field; >>>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>>> + pix->colorspace = ef->colorspace; >>>> + pix->ycbcr_enc = ef->ycbcr_enc; >>>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> + pix->quantization = ef->quantization; >>>> + pix->pixelformat = ef->pixelformat; >>>> + pix->bytesperline = ef->plane_fmt[0].bytesperline; >>>> + pix->sizeimage = ef->plane_fmt[0].sizeimage; >>>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >>>> + pix->sizeimage += ef->plane_fmt[i].sizeimage; >>>> +} >>>> + >>>> +static void >>>> +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef, >>>> + struct v4l2_pix_format_mplane *pix_mp) >>>> +{ >>>> + const struct v4l2_format_info *info = >>>> + v4l2_format_info(ef->pixelformat); >>>> + unsigned int i; >>>> + >>>> + pix_mp->width = ef->width; >>>> + pix_mp->height = ef->height; >>>> + pix_mp->field = ef->field; >>>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>>> + pix_mp->colorspace = ef->colorspace; >>>> + pix_mp->ycbcr_enc = ef->ycbcr_enc; >>>> + pix_mp->quantization = ef->quantization; >>>> + pix_mp->pixelformat = ef->pixelformat; >>>> + >>>> + /* This is true when converting to non-M-variant */ >>>> + if (info && info->mem_planes == 1) { >>>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; >>>> + pix_mp->num_planes = 1; >>>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >>>> + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage; >>>> + >>>> + return; >>>> + } >>>> + >>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) >>>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; >>>> + pix_mp->num_planes = i; >>>> +} >>>> + >>>> +/* >>>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format >>>> + * >>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. >>>> + * @f: A pointer to struct v4l2_format to be filled. >>>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. >>>> + * >>>> + * If pixelformat should be converted to M-variant, change ef->pixelformat >>>> + * to the M-variant before calling this function. >>>> + */ >>>> +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef, >>>> + struct v4l2_format *f, bool is_mplane) >>>> +{ >>>> + memset(f, 0, sizeof(*f)); >>>> + >>>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && >>>> + ef->modifier != DRM_FORMAT_MOD_INVALID) >>>> + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n", >>>> + ef->modifier); >>>> + >>>> + if (!is_mplane) { >>>> + f->type = ef->type; >>>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); >>>> + return; >>>> + } >>>> + >>>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) >>>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; >>>> + else >>>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; >>>> + >>>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); >>>> +} >>>> + >>>> +static void >>>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, >>>> + struct v4l2_ext_pix_format *ef) >>>> +{ >>>> + const struct v4l2_format_info *info = >>>> + v4l2_format_info(pix->pixelformat); >>>> + unsigned int i; >>>> + >>>> + ef->width = pix->width; >>>> + ef->height = pix->height; >>>> + ef->field = pix->field; >>>> + ef->colorspace = pix->colorspace; >>>> + ef->ycbcr_enc = pix->ycbcr_enc; >>>> + ef->quantization = pix->quantization; >>>> + ef->xfer_func = pix->xfer_func; >>>> + if (pix->flags) >>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); >>>> + >>>> + /* We assume M-variants won't be used in this function */ >>>> + ef->pixelformat = pix->pixelformat; >>>> + >>>> + ef->plane_fmt[0].bytesperline = pix->bytesperline; >>>> + ef->plane_fmt[0].sizeimage = pix->sizeimage; >>>> + >>>> + if (!info) >>>> + return; >>>> + >>>> + for (i = 1; i < info->comp_planes; i++) { >>>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; >>>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * >>>> + ef->height / info->vdiv; >>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>>> + } >>>> +} >>>> + >>>> +static void >>>> +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp, >>>> + struct v4l2_ext_pix_format *ef) >>>> +{ >>>> + const struct v4l2_format_info *info = >>>> + v4l2_format_info(pix_mp->pixelformat); >>>> + unsigned int i; >>>> + >>>> + ef->width = pix_mp->width; >>>> + ef->height = pix_mp->height; >>>> + ef->field = pix_mp->field; >>>> + ef->colorspace = pix_mp->colorspace; >>>> + ef->ycbcr_enc = pix_mp->ycbcr_enc; >>>> + ef->quantization = pix_mp->quantization; >>>> + ef->xfer_func = pix_mp->xfer_func; >>>> + if (pix_mp->flags) >>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); >>>> + >>>> + if (!info) >>>> + return; >>>> + >>>> + ef->pixelformat = info && info->norm ? >>> >>> 'info &&' can be dropped, info is always non-NULL here. >>> >>>> + info->norm : pix_mp->pixelformat; >>>> + >>>> + if (info->comp_planes == info->mem_planes) { >>>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++) >>>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; >>>> + >>>> + return; >>>> + } >>>> + >>>> + /* case where mem_planes is 1 and comp_planes > 1 */ >>>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; >>>> + for (i = 1; i < info->comp_planes; i++) { >>>> + ef->plane_fmt[i].bytesperline = >>>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; >>>> + ef->plane_fmt[i].sizeimage = >>>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; >>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>>> + } >>>> +} >>>> + >>>> +/* >>>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format >>>> + * >>>> + * @f: A pointer to struct v4l2_format to be converted. >>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. >>>> + * >>>> + * This method normalize the pixelformat to non-M variant. >>>> + */ >>>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, >>>> + struct v4l2_ext_pix_format *ef) >>>> +{ >>>> + memset(ef, 0, sizeof(*ef)); >>>> + >>>> + switch (f->type) { >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> + ef->type = f->type; >>>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); >>>> + break; >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>>> + break; >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>>> + break; >>>> + default: >>>> + WARN("Converting to Ext Pix Format with wrong buffer type %s\n", >>>> + prt_names(f->type, v4l2_type_names)); >>>> + break; >>>> + } >>>> +} >>>> + >>>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, >>>> struct file *file, void *fh, void *arg) >>>> { >>>> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct v4l2_pix_format *p) >>>> p->xfer_func = 0; >>>> } >>>> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, >>>> + struct file *file, void *fh, >>>> + struct v4l2_format *f, >>>> + unsigned int ioctl) >>>> +{ >>>> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); >>>> + struct video_device *vdev = video_devdata(file); >>>> + struct v4l2_ext_pix_format ef = {0}; >>>> + u32 original_pixfmt = 0; >>>> + u32 cap_mask; >>>> + int ret; >>>> + >>>> + if (ioctl != VIDIOC_G_FMT) { >>>> + /* >>>> + * If CSC attributes are read only, set them to DEFAULT >>>> + * to avoid changes by the driver. >>>> + */ >>>> + if (is_multiplanar) { >>>> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>>> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; >>>> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>>> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; >>>> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>>> + } >>>> + /* Unset the flag to avoid warning in the convertion */ >>>> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>>> + >>>> + /* Save pixelformat in case M-variant is being used */ >>>> + original_pixfmt = f->fmt.pix_mp.pixelformat; >>>> + } else { >>>> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>>> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; >>>> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>>> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; >>>> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>>> + } >>>> + /* Unset the flag to avoid warning in the convertion */ >>>> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>>> + } >>>> + v4l2_format_to_ext_pix_format(f, &ef); >>>> + } >>>> + >>>> + switch (f->type) { >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | >>>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>>> + if (!!(vdev->device_caps & cap_mask) != >>>> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) >>>> + return -EINVAL; >>>> + >>>> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>>> + if (ioctl == VIDIOC_G_FMT) >>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); >>>> + else if (ioctl == VIDIOC_S_FMT) >>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); >>>> + else >>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>>> + &ef); >>>> + break; >>>> + >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | >>>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>>> + if (!!(vdev->device_caps & cap_mask) != >>>> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) >>>> + return -EINVAL; >>>> + >>>> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>>> + if (ioctl == VIDIOC_G_FMT) >>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); >>>> + else if (ioctl == VIDIOC_S_FMT) >>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); >>>> + else >>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>>> + &ef); >>>> + break; >>>> + >>>> + default: >>>> + return -EINVAL; >>>> + } >>>> + >>>> + if (ret) >>>> + return ret; >>>> + >>>> + if (original_pixfmt != ef.pixelformat && >>>> + v4l2_format_info(original_pixfmt)) >>> >>> Could this test be simplified to: 'if (original_pixfmt)'? >>> >>> I.e., if the original pixfmt was saved, then restore it here. >>> >>>> + ef.pixelformat = original_pixfmt; >>>> + >>>> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); >>>> + return 0; >>>> +} >>>> + >>>> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>>> struct file *file, void *fh, void *arg) >>>> { >>>> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>>> switch (p->type) { >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) >>>> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && >>>> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) >>>> break; >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); >>>> + ret = ops->vidioc_g_fmt_vid_cap ? >>>> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>> + VIDIOC_G_FMT); >>>> /* just in case the driver zeroed it again */ >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>> v4l_pix_format_touch(&p->fmt.pix); >>>> return ret; >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>>> + if (ops->vidioc_g_fmt_vid_cap_mplane) >>>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>> >>> 'else' can be dropped. >>> >>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>> + VIDIOC_G_FMT); >>>> + break; >>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); >>>> case V4L2_BUF_TYPE_VBI_CAPTURE: >>>> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>>> �� case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: >>>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) >>>> + if (unlikely(!ops->vidioc_g_fmt_vid_out && >>>> + !ops->vidioc_g_ext_pix_fmt_vid_out)) >>>> break; >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); >>>> + ret = ops->vidioc_g_fmt_vid_out ? >>>> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); >>>> /* just in case the driver zeroed it again */ >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> return ret; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>>> + if (ops->vidioc_g_fmt_vid_out_mplane) >>>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) >>> >>> Ditto. >>> >>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>> + VIDIOC_G_FMT); >>>> + break; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); >>>> case V4L2_BUF_TYPE_VBI_OUTPUT: >>>> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>>> return -EINVAL; >>>> } >>>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>> + struct file *file, void *fh, void *arg) >>>> +{ >>>> + struct v4l2_ext_pix_format *ef = arg; >>>> + struct v4l2_format f = { >>>> + .type = ef->type, >>>> + }; >>>> + int ret = check_fmt(file, ef->type); >>>> + >>>> + if (ret) >>>> + return ret; >>>> + >>>> + memset(ef, 0, sizeof(*ef)); >>>> + ef->type = f.type; >>>> + >>>> + switch (f.type) { >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); >>>> + break; >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> + if (ops->vidioc_g_ext_pix_fmt_vid_out) >>>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); >>>> + break; >>>> + default: >>>> + return -EINVAL; >>>> + } >>>> + >>>> + ret = v4l_g_fmt(ops, file, fh, &f); >>>> + if (ret) >>>> + return ret; >>>> + >>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>> + return 0; >>>> +} >>>> + >>>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>>> struct file *file, void *fh, void *arg) >>>> { >>>> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>>> switch (p->type) { >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) >>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && >>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); >>>> + ret = ops->vidioc_s_fmt_vid_cap ? >>>> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>> /* just in case the driver zeroed it again */ >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>> v4l_pix_format_touch(&p->fmt.pix); >>>> return ret; >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) >>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && >>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>> bytesperline); >>>> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); >>>> + return ops->vidioc_s_fmt_vid_cap_mplane ? >>>> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) >>>> break; >>>> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>>> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) >>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out && >>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); >>>> + ret = ops->vidioc_s_fmt_vid_out ? >>>> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>> /* just in case the driver zeroed it again */ >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> return ret; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) >>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && >>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>> bytesperline); >>>> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); >>>> + return ops->vidioc_s_fmt_vid_out_mplane ? >>>> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) >>>> break; >>>> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>>> return -EINVAL; >>>> } >>>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>> + struct file *file, void *fh, void *arg) >>>> +{ >>>> + struct video_device *vfd = video_devdata(file); >>>> + struct v4l2_ext_pix_format *ef = arg; >>>> + struct v4l2_format f; >>>> + int ret = check_fmt(file, ef->type); >>>> + >>>> + if (ret) >>>> + return ret; >>>> + >>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>>> + >>>> + switch (ef->type) { >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) >>>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); >>>> + break; >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> + if (ops->vidioc_s_ext_pix_fmt_vid_out) >>>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); >>>> + break; >>>> + default: >>>> + return -EINVAL; >>>> + } >>>> + >>>> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); >>>> + >>>> + ret = v4l_s_fmt(ops, file, fh, &f); >>>> + if (ret) >>>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>> >>> See my comments on this at the top. >>> >>>> + return ret; >>>> + >>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>> + return 0; >>>> +} >>>> + >>>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>>> struct file *file, void *fh, void *arg) >>>> { >>>> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>>> switch (p->type) { >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) >>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && >>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); >>>> + ret = ops->vidioc_try_fmt_vid_cap ? >>>> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); >>>> /* just in case the driver zeroed it again */ >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>> v4l_pix_format_touch(&p->fmt.pix); >>>> return ret; >>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) >>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && >>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>> bytesperline); >>>> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); >>>> + return ops->vidioc_try_fmt_vid_cap_mplane ? >>>> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>> + VIDIOC_TRY_FMT); >>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) >>>> break; >>>> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>>> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) >>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out && >>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); >>>> + ret = ops->vidioc_try_fmt_vid_out ? >>>> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); >>>> /* just in case the driver zeroed it again */ >>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>> return ret; >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) >>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && >>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>> break; >>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>> bytesperline); >>>> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); >>>> + return ops->vidioc_try_fmt_vid_out_mplane ? >>>> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : >>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>> + VIDIOC_TRY_FMT); >>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) >>>> break; >>>> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>>> return -EINVAL; >>>> } >>>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>> + struct file *file, void *fh, void *arg) >>>> +{ >>>> + struct video_device *vfd = video_devdata(file); >>>> + struct v4l2_ext_pix_format *ef = arg; >>>> + struct v4l2_format f; >>>> + int ret = check_fmt(file, ef->type); >>>> + >>>> + if (ret) >>>> + return ret; >>>> + >>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>>> + >>>> + switch (ef->type) { >>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) >>>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>>> + ef); >>>> + break; >>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>> + if (ops->vidioc_try_ext_pix_fmt_vid_out) >>>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>>> + ef); >>>> + break; >>>> + default: >>>> + return -EINVAL; >>>> + } >>>> + >>>> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); >>>> + >>>> + ret = v4l_try_fmt(ops, file, fh, &f); >>>> + if (ret) >>>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>>> + return ret; >>>> + >>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>> + return 0; >>>> +} >>>> + >>>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, >>>> struct file *file, void *fh, void *arg) >>>> { >>>> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { >>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0), >>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), >>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), >>>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0), >>>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO), >>>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0), >>>> }; >>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) >>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h >>>> index edb733f21604..c44708dc9355 100644 >>>> --- a/include/media/v4l2-ioctl.h >>>> +++ b/include/media/v4l2-ioctl.h >>>> @@ -48,11 +48,17 @@ struct v4l2_fh; >>>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements >>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>> * in single plane mode >>>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements >>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video >>>> + * capture >>>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements >>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>> * @vidioc_g_fmt_vid_out: pointer to the function that implements >>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out >>>> * in single plane mode >>>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements >>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video >>>> + * out >>>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements >>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output >>>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements >>>> @@ -82,11 +88,16 @@ struct v4l2_fh; >>>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements >>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>> * in single plane mode >>>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements >>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video >>>> + * capture >>>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements >>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>> * @vidioc_s_fmt_vid_out: pointer to the function that implements >>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out >>>> * in single plane mode >>>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements >>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out >>>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements >>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output >>>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements >>>> @@ -116,11 +127,16 @@ struct v4l2_fh; >>>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements >>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>> * in single plane mode >>>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements >>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for >>>> + video capture >>>> * @vidioc_try_fmt_vid_overlay: pointer to the function that implements >>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>> * @vidioc_try_fmt_vid_out: pointer to the function that implements >>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out >>>> * in single plane mode >>>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements >>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out >>>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements >>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>> * output >>>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { >>>> /* VIDIOC_G_FMT handlers */ >>>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>> + struct v4l2_ext_pix_format *ef); >>>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>> + struct v4l2_ext_pix_format *ef); >>>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, >>>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { >>>> /* VIDIOC_S_FMT handlers */ >>>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>> + struct v4l2_ext_pix_format *ef); >>>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>> + struct v4l2_ext_pix_format *ef); >>>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, >>>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { >>>> /* VIDIOC_TRY_FMT handlers */ >>>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>> + struct v4l2_ext_pix_format *ef); >>>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>> + struct v4l2_ext_pix_format *ef); >>>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, >>>> struct v4l2_format *f); >>>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, >>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >>>> index d9b7c9177605..a2d850513708 100644 >>>> --- a/include/uapi/linux/videodev2.h >>>> +++ b/include/uapi/linux/videodev2.h >>>> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { >>>> __u8 reserved[7]; >>>> } __attribute__ ((packed)); >>>> +/** >>>> + * struct v4l2_ext_pix_format - extended single/multiplanar format definition >>>> + * @type: type of the data stream; V4L2_BUF_TYPE_VIDEO_CAPTURE or >>>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT >>>> + * @width: image width in pixels >>>> + * @height: image height in pixels >>>> + * @field: enum v4l2_field; field order (for interlaced video) >>>> + * @plane_fmt: per-plane information >>>> + * @pixelformat: little endian four character code (fourcc) >>>> + * @modifier: modifier applied to the format (used for tiled formats >>>> + * and other kind of HW-specific formats, like compressed >>>> + * formats) as defined in drm_fourcc.h >>>> + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat >>>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function >>>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding >>>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding >>>> + * @quantization: enum v4l2_quantization, colorspace quantization >>>> + * @reserved: extra space reserved for future fields, must be set to 0 >>>> + */ >>>> +struct v4l2_ext_pix_format { >>>> + __u32 type; >>>> + __u32 width; >>>> + __u32 height; >>>> + __u32 field; >>>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; >>>> + __u32 pixelformat; >>>> + __u64 modifier; >>>> + __u32 colorspace; >>>> + __u32 xfer_func; >>>> + union { >>>> + __u32 ycbcr_enc; >>>> + __u32 hsv_enc; >>>> + }; >>>> + __u32 quantization; >>>> + __u32 reserved[9]; >>>> +}; >>>> + >>>> /** >>>> * struct v4l2_sdr_format - SDR format definition >>>> * @pixelformat: little endian four character code (fourcc) >>>> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { >>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) >>>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format) >>>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format) >>>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format) >>>> + >>>> /* Reminder: when adding new ioctls please add support for them to >>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ >>>> >>> >>> Regards, >>> >>> Hans >>> >> >
On 11/7/22 16:28, Laurent Pinchart wrote: > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > Hi Hsia-Jun, > > On Mon, Nov 07, 2022 at 09:54:11AM +0800, Hsia-Jun Li wrote: >> On 11/7/22 03:24, Laurent Pinchart wrote: >>> On Sat, Nov 05, 2022 at 11:19:10PM +0800, Hsia-Jun Li wrote: >>>> Hello Helen >>>> >>>> I didn't see any updates from V6 and V7-WIP in your repo. That is what I >>>> need to for our complex tile formats in our platform. >>>> >>>> Any future plane here? >>>> >>>> Besides I have some ideas on these patches. >>>> >>>> On 2/24/21 23:12, Helen Koike wrote: >>>>> Hi Hans, >>>>> >>>>> Thank you for your comment, please see my reply below. >>>>> >>>>> On 2/23/21 9:35 AM, Hans Verkuil wrote: >>>>>> Hi Helen, >>>>>> >>>>>> On 14/01/2021 19:07, Helen Koike wrote: >>>>>>> This is part of the multiplanar and singleplanar unification process. >>>>>>> v4l2_ext_pix_format is supposed to work for both cases. >>>>>>> >>>>>>> We also add the concept of modifiers already employed in DRM to expose >>>>>>> HW-specific formats (like tiled or compressed formats) and allow >>>>>>> exchanging this information with the DRM subsystem in a consistent way. >>>>>>> >>>>>>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in >>>>>>> v4l2_ext_format, other types will be rejected if you use the >>>>>>> {G,S,TRY}_EXT_PIX_FMT ioctls. >>>>>>> >>>>>>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls >>>>>>> in drivers, but, in the meantime, the core takes care of converting >>>>>>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers >>>>>>> can >>>>>>> still work if the userspace app/lib uses the new ioctls. >>>>>>> >>>>>>> The conversion is also done the other around to allow userspace >>>>>>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the >>>>>>> _ext_ hooks. >>>>>> >>>>>> I have some small comments below, but also one high level comment: >>>>>> >>>>>> Regarding M variants of pixelformats: this patch 'normalizes' them to >>>>>> regular pixelformats in the extended API. This makes life complicated, >>>>>> and I wonder if this is the right approach. >>>>>> >>>>>> Currently there are two reasons for a driver to support e.g. NV12M: >>>>>> either luma and chroma need to be in two memory banks, or the luma >>>>>> and chroma planes cannot be contiguous due to alignment requirements. >>>>>> >>>>>> The first requirement is still valid for drivers that support the >>>>>> extended API. >>>>>> The second requirement is no longer a reason to support NV12M. But I >>>>>> don't think we should just drop NV12M support if it was already supported >>>>>> before the conversion to this extended API. Since NV12M allocates two >>>>>> buffers >>>>>> instead of one, it is still different from a regular NV12. >>>>> >>>>> I don't see what would be the difference when using NV12 and NV12M in >>>>> Ext API. >>>>> NV12 could be used for both requirements. It would even allow the second >>>>> requirement with a single memory buffer. >>>>> >>>> Although I don't have problem here to support both single and multiple >>>> planes in our hardware. But using the single plane format here is not >>>> convience for us. >>>> >>>> The tile format in our platform is little complex, you can't calculate >>>> the stride and sizeimage easily with just the modifier, width and >>>> height. Then we don't have a good place to record the offset here. >>> >>> What else is needed to compute those values ? Can they be computed by >>> the kernel driver, or do they have computed by userspace ? >> >> I could calculate the stride(bytesperline) and sizeimage in the kernel >> driver. But it would be better to let the firmware do that. Our driver >> would depend on the V4L2_EVENT_SOURCE_CHANGE event. > > It could indeed be done in the firmware, but that would mean that the > device would need to be powered up to implement VIDIOC_S_FMT and > VIDIOC_TRY_FMT. Although I would power on the device here but that is not necessary. The firmware is running in TEE(CPU) and a remote processor. Parsing the bitstream could be done in TEE part of firmware. The only reason that I need to power on the device is that I can't do it in the TEE. Besides, there would be encrypted video streams(DRM video), the only possible to know their resolutions and other buffer informations should only be done in TEE. The uvcvideo driver does that, which results in very > slow startup for applications that try lots of formats. It would be best > to avoid it if possible, and calculate the values in software in the > kernel. We have had this problem here. If I allocate the motion vector buffer by driver internally. I have to do either in buf_init() or start_streaming(). When I did it in the STREAMON of the capture queue, it would spend lots of time on allocating. I think I need a stage that the user won't allocate more CAPTURE buffer. > >>>> Besides, the luma and chroma planes would always need their own >>>> compression meta data planes. If we use NV12 here, that would make a >>>> very strange pixel format, one plane for the pixel datas and two planes >>>> for the meta data. >>>> >>>>>> I would prefer that such drivers support both NV12 and NV12M, so no >>>>>> automatic conversion. >>>>>> >>>>>> A related question is how to handle pixelformat enumeration: with the >>>>>> extended API an NV12 format might work, but not with the old API (e.g. >>>>>> due to memory alignment requirements). I wonder if a >>>>>> VIDIOC_ENUM_EXT_PIX_FMT >>>>>> isn't needed. >>>>> >>>>> We need VIDIOC_ENUM_EXT_PIX_FMT for modifiers, but we can add it later. >>>>> If the driver reports NV12M, userspace can use it and the framework >>>>> normilizes it. >>>>> >>>>>> >>>>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while >>>>>> VIDIOC_ENUM_FMT >>>>>> would just report NV12M. >>>>> >>>>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would >>>>> report both (unless I'm missing something, which is probably the case). >>>>> >>>>> The idea was to deprecate the M-variants one day. >>>> >>>> I was thinking the way in DRM API is better, always assuming it would >>>> always in a multiple planes. The only problem is we don't have a way to >>>> let the allocator that allocate contiguous memory for planes when we >>>> need to do that. >>>> >>>>>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> >>>>>>> Signed-off-by: Helen Koike <helen.koike@collabora.com> >>>>>>> --- >>>>>>> >>>>>>> Changes in v6: >>>>>>> The main change here was fixing the conversion, so planes reflects >>>>>>> color planes, >>>>>>> and to implement this properly I made major refactors compared to >>>>>>> the previous >>>>>>> version. >>>>>>> - struct v4l2_plane_ext_pix_format removed, using struct >>>>>>> v4l2_plane_pix_format instead (Tomasz) >>>>>>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) >>>>>>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) >>>>>>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) >>>>>>> - refactor conversion functions, so planes are color planes (Tomasz) >>>>>>> - Don't explicitly check for e->modifier != 0 in >>>>>>> v4l2_ext_pix_format_to_format() (Tomasz) >>>>>>> - Use "ef" for extended formats in the framework for consistency >>>>>>> (Tomasz) >>>>>>> - Handle xfer_func field in conversions (Tomasz) >>>>>>> - Zero reserved fields in v4l_s_ext_pix_fmt() and >>>>>>> v4l_try_ext_pix_fmt() (Tomasz) >>>>>>> - Refactor format functions to use v4l_fmt_ioctl_via_ext() >>>>>>> - Several fixes/refactoring/changes >>>>>>> - Remove EXT API for touch devices >>>>>>> >>>>>>> Changes in v5: >>>>>>> - change sizes and reorder fields to avoid holes in the struct and make >>>>>>> it the same for 32 and 64 bits >>>>>>> - removed __attribute__ ((packed)) from uapi structs >>>>>>> - Fix doc warning from make htmldocs >>>>>>> - Updated commit message with EXT_PIX prefix for the ioctls. >>>>>>> >>>>>>> Changes in v4: >>>>>>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, >>>>>>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. >>>>>>> - Add reserved fields >>>>>>> - Removed num_planes from struct v4l2_ext_pix_format >>>>>>> - Removed flag field from struct v4l2_ext_pix_format, since the only >>>>>>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, >>>>>>> where we can use modifiers, or add it back later through the reserved >>>>>>> bits. >>>>>>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && >>>>>>> != MOD_INVALID >>>>>>> - Fix type assignment in v4l_g_fmt_ext_pix() >>>>>>> - Rebased on top of media/master (post 5.8-rc1) >>>>>>> >>>>>>> Changes in v3: >>>>>>> - Rebased on top of media/master (post 5.4-rc1) >>>>>>> >>>>>>> Changes in v2: >>>>>>> - Move the modifier in v4l2_ext_format (was formerly placed in >>>>>>> v4l2_ext_plane) >>>>>>> - Fix a few bugs in the converters and add a strict parameter to >>>>>>> allow conversion of uninitialized/mis-initialized objects >>>>>>> --- >>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 27 +- >>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- >>>>>>> include/media/v4l2-ioctl.h | 28 ++ >>>>>>> include/uapi/linux/videodev2.h | 41 ++ >>>>>>> 4 files changed, 602 insertions(+), 32 deletions(-) >>>>>>> >>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c >>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c >>>>>>> index f9cff033d0dc..5add58cb6d45 100644 >>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c >>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c >>>>>>> @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct >>>>>>> video_device *vdev) >>>>>>> ops->vidioc_enum_fmt_vid_overlay)) || >>>>>>> (is_tx && ops->vidioc_enum_fmt_vid_out)) >>>>>>> set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); >>>>>>> + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || >>>>>>> + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) >>>>>>> + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>>>>>> if ((is_rx && (ops->vidioc_g_fmt_vid_cap || >>>>>>> ops->vidioc_g_fmt_vid_cap_mplane || >>>>>>> - ops->vidioc_g_fmt_vid_overlay)) || >>>>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) || >>>>>>> (is_tx && (ops->vidioc_g_fmt_vid_out || >>>>>>> ops->vidioc_g_fmt_vid_out_mplane || >>>>>>> - ops->vidioc_g_fmt_vid_out_overlay))) >>>>>>> + ops->vidioc_g_ext_pix_fmt_vid_out))) { >>>>>>> set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); >>>>>>> + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); >>>>>>> + } >>>>>>> + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || >>>>>>> + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) >>>>>>> + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>>>>>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap || >>>>>>> ops->vidioc_s_fmt_vid_cap_mplane || >>>>>>> - ops->vidioc_s_fmt_vid_overlay)) || >>>>>>> + ops->vidioc_s_ext_pix_fmt_vid_cap)) || >>>>>>> (is_tx && (ops->vidioc_s_fmt_vid_out || >>>>>>> ops->vidioc_s_fmt_vid_out_mplane || >>>>>>> - ops->vidioc_s_fmt_vid_out_overlay))) >>>>>>> + ops->vidioc_s_ext_pix_fmt_vid_out))) { >>>>>>> set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); >>>>>>> + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); >>>>>>> + } >>>>>>> + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || >>>>>>> + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) >>>>>>> + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>>>>>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap || >>>>>>> ops->vidioc_try_fmt_vid_cap_mplane || >>>>>>> - ops->vidioc_try_fmt_vid_overlay)) || >>>>>>> + ops->vidioc_try_ext_pix_fmt_vid_cap)) || >>>>>>> (is_tx && (ops->vidioc_try_fmt_vid_out || >>>>>>> ops->vidioc_try_fmt_vid_out_mplane || >>>>>>> - ops->vidioc_try_fmt_vid_out_overlay))) >>>>>>> + ops->vidioc_try_ext_pix_fmt_vid_out))) { >>>>>>> set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); >>>>>>> + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); >>>>>>> + } >>>>>>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); >>>>>>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); >>>>>>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); >>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c >>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c >>>>>>> index 848286a284f6..a9c07c0a73ec 100644 >>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >>>>>>> @@ -18,6 +18,8 @@ >>>>>>> #include <linux/videodev2.h> >>>>>>> +#include <drm/drm_fourcc.h> >>>>>>> + >>>>>>> #include <media/v4l2-common.h> >>>>>>> #include <media/v4l2-ioctl.h> >>>>>>> #include <media/v4l2-ctrls.h> >>>>>>> @@ -38,6 +40,11 @@ >>>>>>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), >>>>>>> (vfd)->valid_ioctls) >>>>>>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ >>>>>>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ >>>>>>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ >>>>>>> + V4L2_CAP_VIDEO_M2M_MPLANE)) >>>>>>> + >>>>>>> struct std_descr { >>>>>>> v4l2_std_id std; >>>>>>> const char *descr; >>>>>>> @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, >>>>>>> bool write_only) >>>>>>> } >>>>>>> } >>>>>>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only) >>>>>>> +{ >>>>>>> + const struct v4l2_ext_pix_format *ef = arg; >>>>>>> + unsigned int i; >>>>>>> + >>>>>>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier >>>>>>> %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, >>>>>>> xfer_func=%u\n", >>>>>>> + prt_names(ef->type, v4l2_type_names), >>>>>>> + ef->width, ef->height, >>>>>>> + (ef->pixelformat & 0xff), >>>>>>> + (ef->pixelformat >> 8) & 0xff, >>>>>>> + (ef->pixelformat >> 16) & 0xff, >>>>>>> + (ef->pixelformat >> 24) & 0xff, >>>>>>> + ef->modifier, prt_names(ef->field, v4l2_field_names), >>>>>>> + ef->colorspace, ef->ycbcr_enc, >>>>>>> + ef->quantization, ef->xfer_func); >>>>>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>>>>>> i++) >>>>>>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", >>>>>>> + i, ef->plane_fmt[i].bytesperline, >>>>>>> + ef->plane_fmt[i].sizeimage); >>>>>>> +} >>>>>>> + >>>>>>> static void v4l_print_framebuffer(const void *arg, bool write_only) >>>>>>> { >>>>>>> const struct v4l2_framebuffer *p = arg; >>>>>>> @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum >>>>>>> v4l2_buf_type type) >>>>>>> switch (type) { >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> if ((is_vid || is_tch) && is_rx && >>>>>>> - (ops->vidioc_g_fmt_vid_cap || >>>>>>> ops->vidioc_g_fmt_vid_cap_mplane)) >>>>>>> + (ops->vidioc_g_fmt_vid_cap || >>>>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap || >>>>>>> + ops->vidioc_g_fmt_vid_cap_mplane)) >>>>>>> return 0; >>>>>>> break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>>>> - if ((is_vid || is_tch) && is_rx && >>>>>>> ops->vidioc_g_fmt_vid_cap_mplane) >>>>>>> + if ((is_vid || is_tch) && is_rx && >>>>>>> + (ops->vidioc_g_fmt_vid_cap_mplane || >>>>>>> + ops->vidioc_g_ext_pix_fmt_vid_cap)) >>>>>>> return 0; >>>>>>> break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>>>> @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum >>>>>>> v4l2_buf_type type) >>>>>>> break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> if (is_vid && is_tx && >>>>>>> - (ops->vidioc_g_fmt_vid_out || >>>>>>> ops->vidioc_g_fmt_vid_out_mplane)) >>>>>>> + (ops->vidioc_g_fmt_vid_out || >>>>>>> + ops->vidioc_g_ext_pix_fmt_vid_out || >>>>>>> + ops->vidioc_g_fmt_vid_out_mplane)) >>>>>>> return 0; >>>>>>> break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>>>> - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) >>>>>>> + if (is_vid && is_tx && >>>>>>> + (ops->vidioc_g_ext_pix_fmt_vid_out || >>>>>>> + ops->vidioc_g_fmt_vid_out_mplane)) >>>>>>> return 0; >>>>>>> break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>>>> @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct >>>>>>> v4l2_format *fmt) >>>>>>> sizeof(fmt->fmt.pix) - offset); >>>>>>> } >>>>>>> +static void >>>>>>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, >>>>>>> + struct v4l2_pix_format *pix) >>>>>>> +{ >>>>>>> + unsigned int i; >>>>>>> + >>>>>>> + pix->width = ef->width; >>>>>>> + pix->height = ef->height; >>>>>>> + pix->field = ef->field; >>>>>>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>>>>>> + pix->colorspace = ef->colorspace; >>>>>>> + pix->ycbcr_enc = ef->ycbcr_enc; >>>>>>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> + pix->quantization = ef->quantization; >>>>>>> + pix->pixelformat = ef->pixelformat; >>>>>>> + pix->bytesperline = ef->plane_fmt[0].bytesperline; >>>>>>> + pix->sizeimage = ef->plane_fmt[0].sizeimage; >>>>>>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>>>>>> i++) >>>>>>> + pix->sizeimage += ef->plane_fmt[i].sizeimage; >>>>>>> +} >>>>>>> + >>>>>>> +static void >>>>>>> +v4l2_ext_pix_format_to_pix_mp_format(const struct >>>>>>> v4l2_ext_pix_format *ef, >>>>>>> + struct v4l2_pix_format_mplane *pix_mp) >>>>>>> +{ >>>>>>> + const struct v4l2_format_info *info = >>>>>>> + v4l2_format_info(ef->pixelformat); >>>>>>> + unsigned int i; >>>>>>> + >>>>>>> + pix_mp->width = ef->width; >>>>>>> + pix_mp->height = ef->height; >>>>>>> + pix_mp->field = ef->field; >>>>>>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; >>>>>>> + pix_mp->colorspace = ef->colorspace; >>>>>>> + pix_mp->ycbcr_enc = ef->ycbcr_enc; >>>>>>> + pix_mp->quantization = ef->quantization; >>>>>>> + pix_mp->pixelformat = ef->pixelformat; >>>>>>> + >>>>>>> + /* This is true when converting to non-M-variant */ >>>>>>> + if (info && info->mem_planes == 1) { >>>>>>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; >>>>>>> + pix_mp->num_planes = 1; >>>>>>> + for (i = 1; i < VIDEO_MAX_PLANES && >>>>>>> ef->plane_fmt[i].sizeimage; i++) >>>>>>> + pix_mp->plane_fmt[0].sizeimage += >>>>>>> ef->plane_fmt[i].sizeimage; >>>>>>> + >>>>>>> + return; >>>>>>> + } >>>>>>> + >>>>>>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; >>>>>>> i++) >>>>>>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; >>>>>>> + pix_mp->num_planes = i; >>>>>>> +} >>>>>>> + >>>>>>> +/* >>>>>>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to >>>>>>> v4l2_format >>>>>>> + * >>>>>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. >>>>>>> + * @f: A pointer to struct v4l2_format to be filled. >>>>>>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f. >>>>>>> + * >>>>>>> + * If pixelformat should be converted to M-variant, change >>>>>>> ef->pixelformat >>>>>>> + * to the M-variant before calling this function. >>>>>>> + */ >>>>>>> +static void v4l2_ext_pix_format_to_format(const struct >>>>>>> v4l2_ext_pix_format *ef, >>>>>>> + struct v4l2_format *f, bool is_mplane) >>>>>>> +{ >>>>>>> + memset(f, 0, sizeof(*f)); >>>>>>> + >>>>>>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && >>>>>>> + ef->modifier != DRM_FORMAT_MOD_INVALID) >>>>>>> + pr_warn("Modifiers are not supported in v4l2_format, >>>>>>> ignoring %llx\n", >>>>>>> + ef->modifier); >>>>>>> + >>>>>>> + if (!is_mplane) { >>>>>>> + f->type = ef->type; >>>>>>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); >>>>>>> + return; >>>>>>> + } >>>>>>> + >>>>>>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) >>>>>>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; >>>>>>> + else >>>>>>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; >>>>>>> + >>>>>>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); >>>>>>> +} >>>>>>> + >>>>>>> +static void >>>>>>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, >>>>>>> + struct v4l2_ext_pix_format *ef) >>>>>>> +{ >>>>>>> + const struct v4l2_format_info *info = >>>>>>> + v4l2_format_info(pix->pixelformat); >>>>>>> + unsigned int i; >>>>>>> + >>>>>>> + ef->width = pix->width; >>>>>>> + ef->height = pix->height; >>>>>>> + ef->field = pix->field; >>>>>>> + ef->colorspace = pix->colorspace; >>>>>>> + ef->ycbcr_enc = pix->ycbcr_enc; >>>>>>> + ef->quantization = pix->quantization; >>>>>>> + ef->xfer_func = pix->xfer_func; >>>>>>> + if (pix->flags) >>>>>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); >>>>>>> + >>>>>>> + /* We assume M-variants won't be used in this function */ >>>>>>> + ef->pixelformat = pix->pixelformat; >>>>>>> + >>>>>>> + ef->plane_fmt[0].bytesperline = pix->bytesperline; >>>>>>> + ef->plane_fmt[0].sizeimage = pix->sizeimage; >>>>>>> + >>>>>>> + if (!info) >>>>>>> + return; >>>>>>> + >>>>>>> + for (i = 1; i < info->comp_planes; i++) { >>>>>>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; >>>>>>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * >>>>>>> + ef->height / info->vdiv; >>>>>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>>>>>> + } >>>>>>> +} >>>>>>> + >>>>>>> +static void >>>>>>> +v4l2_pix_mp_format_to_ext_pix_format(const struct >>>>>>> v4l2_pix_format_mplane *pix_mp, >>>>>>> + struct v4l2_ext_pix_format *ef) >>>>>>> +{ >>>>>>> + const struct v4l2_format_info *info = >>>>>>> + v4l2_format_info(pix_mp->pixelformat); >>>>>>> + unsigned int i; >>>>>>> + >>>>>>> + ef->width = pix_mp->width; >>>>>>> + ef->height = pix_mp->height; >>>>>>> + ef->field = pix_mp->field; >>>>>>> + ef->colorspace = pix_mp->colorspace; >>>>>>> + ef->ycbcr_enc = pix_mp->ycbcr_enc; >>>>>>> + ef->quantization = pix_mp->quantization; >>>>>>> + ef->xfer_func = pix_mp->xfer_func; >>>>>>> + if (pix_mp->flags) >>>>>>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); >>>>>>> + >>>>>>> + if (!info) >>>>>>> + return; >>>>>>> + >>>>>>> + ef->pixelformat = info && info->norm ? >>>>>> >>>>>> 'info &&' can be dropped, info is always non-NULL here. >>>>>> >>>>>>> + info->norm : pix_mp->pixelformat; >>>>>>> + >>>>>>> + if (info->comp_planes == info->mem_planes) { >>>>>>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; >>>>>>> i++) >>>>>>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; >>>>>>> + >>>>>>> + return; >>>>>>> + } >>>>>>> + >>>>>>> + /* case where mem_planes is 1 and comp_planes > 1 */ >>>>>>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; >>>>>>> + for (i = 1; i < info->comp_planes; i++) { >>>>>>> + ef->plane_fmt[i].bytesperline = >>>>>>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv; >>>>>>> + ef->plane_fmt[i].sizeimage = >>>>>>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; >>>>>>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; >>>>>>> + } >>>>>>> +} >>>>>>> + >>>>>>> +/* >>>>>>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to >>>>>>> v4l2_ext_pix_format >>>>>>> + * >>>>>>> + * @f: A pointer to struct v4l2_format to be converted. >>>>>>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. >>>>>>> + * >>>>>>> + * This method normalize the pixelformat to non-M variant. >>>>>>> + */ >>>>>>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, >>>>>>> + struct v4l2_ext_pix_format *ef) >>>>>>> +{ >>>>>>> + memset(ef, 0, sizeof(*ef)); >>>>>>> + >>>>>>> + switch (f->type) { >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> + ef->type = f->type; >>>>>>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); >>>>>>> + break; >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>>>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>>>>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>>>>>> + break; >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>>>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>>>>>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); >>>>>>> + break; >>>>>>> + default: >>>>>>> + WARN("Converting to Ext Pix Format with wrong buffer type >>>>>>> %s\n", >>>>>>> + prt_names(f->type, v4l2_type_names)); >>>>>>> + break; >>>>>>> + } >>>>>>> +} >>>>>>> + >>>>>>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops, >>>>>>> struct file *file, void *fh, void *arg) >>>>>>> { >>>>>>> @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct >>>>>>> v4l2_pix_format *p) >>>>>>> p->xfer_func = 0; >>>>>>> } >>>>>>> +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, >>>>>>> + struct file *file, void *fh, >>>>>>> + struct v4l2_format *f, >>>>>>> + unsigned int ioctl) >>>>>>> +{ >>>>>>> + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); >>>>>>> + struct video_device *vdev = video_devdata(file); >>>>>>> + struct v4l2_ext_pix_format ef = {0}; >>>>>>> + u32 original_pixfmt = 0; >>>>>>> + u32 cap_mask; >>>>>>> + int ret; >>>>>>> + >>>>>>> + if (ioctl != VIDIOC_G_FMT) { >>>>>>> + /* >>>>>>> + * If CSC attributes are read only, set them to DEFAULT >>>>>>> + * to avoid changes by the driver. >>>>>>> + */ >>>>>>> + if (is_multiplanar) { >>>>>>> + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>>>>>> + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; >>>>>>> + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>>>>>> + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; >>>>>>> + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>>>>>> + } >>>>>>> + /* Unset the flag to avoid warning in the convertion */ >>>>>>> + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>>>>>> + >>>>>>> + /* Save pixelformat in case M-variant is being used */ >>>>>>> + original_pixfmt = f->fmt.pix_mp.pixelformat; >>>>>>> + } else { >>>>>>> + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { >>>>>>> + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; >>>>>>> + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; >>>>>>> + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; >>>>>>> + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; >>>>>>> + } >>>>>>> + /* Unset the flag to avoid warning in the convertion */ >>>>>>> + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; >>>>>>> + } >>>>>>> + v4l2_format_to_ext_pix_format(f, &ef); >>>>>>> + } >>>>>>> + >>>>>>> + switch (f->type) { >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>>>> + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | >>>>>>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>>>>>> + if (!!(vdev->device_caps & cap_mask) != >>>>>>> + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) >>>>>>> + return -EINVAL; >>>>>>> + >>>>>>> + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; >>>>>>> + if (ioctl == VIDIOC_G_FMT) >>>>>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); >>>>>>> + else if (ioctl == VIDIOC_S_FMT) >>>>>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); >>>>>>> + else >>>>>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>>>>>> + &ef); >>>>>>> + break; >>>>>>> + >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>>>> + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | >>>>>>> + V4L2_CAP_VIDEO_M2M_MPLANE; >>>>>>> + if (!!(vdev->device_caps & cap_mask) != >>>>>>> + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) >>>>>>> + return -EINVAL; >>>>>>> + >>>>>>> + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; >>>>>>> + if (ioctl == VIDIOC_G_FMT) >>>>>>> + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); >>>>>>> + else if (ioctl == VIDIOC_S_FMT) >>>>>>> + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); >>>>>>> + else >>>>>>> + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>>>>>> + &ef); >>>>>>> + break; >>>>>>> + >>>>>>> + default: >>>>>>> + return -EINVAL; >>>>>>> + } >>>>>>> + >>>>>>> + if (ret) >>>>>>> + return ret; >>>>>>> + >>>>>>> + if (original_pixfmt != ef.pixelformat && >>>>>>> + v4l2_format_info(original_pixfmt)) >>>>>> >>>>>> Could this test be simplified to: 'if (original_pixfmt)'? >>>>>> >>>>>> I.e., if the original pixfmt was saved, then restore it here. >>>>>> >>>>>>> + ef.pixelformat = original_pixfmt; >>>>>>> + >>>>>>> + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); >>>>>>> + return 0; >>>>>>> +} >>>>>>> + >>>>>>> static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, >>>>>>> struct file *file, void *fh, void *arg) >>>>>>> { >>>>>>> @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> switch (p->type) { >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) >>>>>>> + if (unlikely(!ops->vidioc_g_fmt_vid_cap && >>>>>>> + !ops->vidioc_g_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); >>>>>>> + ret = ops->vidioc_g_fmt_vid_cap ? >>>>>>> + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> + VIDIOC_G_FMT); >>>>>>> /* just in case the driver zeroed it again */ >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>>>>> v4l_pix_format_touch(&p->fmt.pix); >>>>>>> return ret; >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>>>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>>>>>> + if (ops->vidioc_g_fmt_vid_cap_mplane) >>>>>>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); >>>>>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>>>>> >>>>>> 'else' can be dropped. >>>>>> >>>>>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> + VIDIOC_G_FMT); >>>>>>> + break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>>>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); >>>>>>> case V4L2_BUF_TYPE_VBI_CAPTURE: >>>>>>> @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: >>>>>>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> - if (unlikely(!ops->vidioc_g_fmt_vid_out)) >>>>>>> + if (unlikely(!ops->vidioc_g_fmt_vid_out && >>>>>>> + !ops->vidioc_g_ext_pix_fmt_vid_out)) >>>>>>> break; >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); >>>>>>> + ret = ops->vidioc_g_fmt_vid_out ? >>>>>>> + ops->vidioc_g_fmt_vid_out(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); >>>>>>> /* just in case the driver zeroed it again */ >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> return ret; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>>>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>>>>>> + if (ops->vidioc_g_fmt_vid_out_mplane) >>>>>>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); >>>>>>> + else if (ops->vidioc_g_ext_pix_fmt_vid_out) >>>>>> >>>>>> Ditto. >>>>>> >>>>>>> + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> + VIDIOC_G_FMT); >>>>>>> + break; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>>>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); >>>>>>> case V4L2_BUF_TYPE_VBI_OUTPUT: >>>>>>> @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> return -EINVAL; >>>>>>> } >>>>>>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>>>>> + struct file *file, void *fh, void *arg) >>>>>>> +{ >>>>>>> + struct v4l2_ext_pix_format *ef = arg; >>>>>>> + struct v4l2_format f = { >>>>>>> + .type = ef->type, >>>>>>> + }; >>>>>>> + int ret = check_fmt(file, ef->type); >>>>>>> + >>>>>>> + if (ret) >>>>>>> + return ret; >>>>>>> + >>>>>>> + memset(ef, 0, sizeof(*ef)); >>>>>>> + ef->type = f.type; >>>>>>> + >>>>>>> + switch (f.type) { >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap) >>>>>>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); >>>>>>> + break; >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> + if (ops->vidioc_g_ext_pix_fmt_vid_out) >>>>>>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); >>>>>>> + break; >>>>>>> + default: >>>>>>> + return -EINVAL; >>>>>>> + } >>>>>>> + >>>>>>> + ret = v4l_g_fmt(ops, file, fh, &f); >>>>>>> + if (ret) >>>>>>> + return ret; >>>>>>> + >>>>>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>>>>> + return 0; >>>>>>> +} >>>>>>> + >>>>>>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, >>>>>>> struct file *file, void *fh, void *arg) >>>>>>> { >>>>>>> @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> switch (p->type) { >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) >>>>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap && >>>>>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>>>> - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); >>>>>>> + ret = ops->vidioc_s_fmt_vid_cap ? >>>>>>> + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>>>>> /* just in case the driver zeroed it again */ >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>>>>> v4l_pix_format_touch(&p->fmt.pix); >>>>>>> return ret; >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) >>>>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && >>>>>>> + !ops->vidioc_s_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>>>> bytesperline); >>>>>>> - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); >>>>>>> + return ops->vidioc_s_fmt_vid_cap_mplane ? >>>>>>> + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> VIDIOC_S_FMT); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>>>> if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) >>>>>>> break; >>>>>>> @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>>>>>> return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out)) >>>>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out && >>>>>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>>>> - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); >>>>>>> + ret = ops->vidioc_s_fmt_vid_out ? >>>>>>> + ops->vidioc_s_fmt_vid_out(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>>>>> /* just in case the driver zeroed it again */ >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> return ret; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>>>> - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) >>>>>>> + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && >>>>>>> + !ops->vidioc_s_ext_pix_fmt_vid_out)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>>>> bytesperline); >>>>>>> - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); >>>>>>> + return ops->vidioc_s_fmt_vid_out_mplane ? >>>>>>> + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>>>> if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) >>>>>>> break; >>>>>>> @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> return -EINVAL; >>>>>>> } >>>>>>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>>>>> + struct file *file, void *fh, void *arg) >>>>>>> +{ >>>>>>> + struct video_device *vfd = video_devdata(file); >>>>>>> + struct v4l2_ext_pix_format *ef = arg; >>>>>>> + struct v4l2_format f; >>>>>>> + int ret = check_fmt(file, ef->type); >>>>>>> + >>>>>>> + if (ret) >>>>>>> + return ret; >>>>>>> + >>>>>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>>>>>> + >>>>>>> + switch (ef->type) { >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap) >>>>>>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); >>>>>>> + break; >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> + if (ops->vidioc_s_ext_pix_fmt_vid_out) >>>>>>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); >>>>>>> + break; >>>>>>> + default: >>>>>>> + return -EINVAL; >>>>>>> + } >>>>>>> + >>>>>>> + v4l2_ext_pix_format_to_format(ef, &f, >>>>>>> V4L2_IS_CAP_MULTIPLANAR(vfd)); >>>>>>> + >>>>>>> + ret = v4l_s_fmt(ops, file, fh, &f); >>>>>>> + if (ret) >>>>>>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>>>>> >>>>>> See my comments on this at the top. >>>>>> >>>>>>> + return ret; >>>>>>> + >>>>>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>>>>> + return 0; >>>>>>> +} >>>>>>> + >>>>>>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, >>>>>>> struct file *file, void *fh, void *arg) >>>>>>> { >>>>>>> @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> switch (p->type) { >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) >>>>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap && >>>>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>>>> - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); >>>>>>> + ret = ops->vidioc_try_fmt_vid_cap ? >>>>>>> + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> VIDIOC_TRY_FMT); >>>>>>> /* just in case the driver zeroed it again */ >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> if (vfd->vfl_type == VFL_TYPE_TOUCH) >>>>>>> v4l_pix_format_touch(&p->fmt.pix); >>>>>>> return ret; >>>>>>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: >>>>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) >>>>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && >>>>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>>>> bytesperline); >>>>>>> - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); >>>>>>> + return ops->vidioc_try_fmt_vid_cap_mplane ? >>>>>>> + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> + VIDIOC_TRY_FMT); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OVERLAY: >>>>>>> if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) >>>>>>> break; >>>>>>> @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); >>>>>>> return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out)) >>>>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out && >>>>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix); >>>>>>> - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); >>>>>>> + ret = ops->vidioc_try_fmt_vid_out ? >>>>>>> + ops->vidioc_try_fmt_vid_out(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> VIDIOC_TRY_FMT); >>>>>>> /* just in case the driver zeroed it again */ >>>>>>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; >>>>>>> return ret; >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: >>>>>>> - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) >>>>>>> + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && >>>>>>> + !ops->vidioc_try_ext_pix_fmt_vid_cap)) >>>>>>> break; >>>>>>> CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); >>>>>>> for (i = 0; i < p->fmt.pix_mp.num_planes; i++) >>>>>>> CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], >>>>>>> bytesperline); >>>>>>> - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); >>>>>>> + return ops->vidioc_try_fmt_vid_out_mplane ? >>>>>>> + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : >>>>>>> + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, >>>>>>> + VIDIOC_TRY_FMT); >>>>>>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: >>>>>>> if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) >>>>>>> break; >>>>>>> @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct >>>>>>> v4l2_ioctl_ops *ops, >>>>>>> return -EINVAL; >>>>>>> } >>>>>>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, >>>>>>> + struct file *file, void *fh, void *arg) >>>>>>> +{ >>>>>>> + struct video_device *vfd = video_devdata(file); >>>>>>> + struct v4l2_ext_pix_format *ef = arg; >>>>>>> + struct v4l2_format f; >>>>>>> + int ret = check_fmt(file, ef->type); >>>>>>> + >>>>>>> + if (ret) >>>>>>> + return ret; >>>>>>> + >>>>>>> + memset(ef->reserved, 0, sizeof(ef->reserved)); >>>>>>> + >>>>>>> + switch (ef->type) { >>>>>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE: >>>>>>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap) >>>>>>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, >>>>>>> + ef); >>>>>>> + break; >>>>>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT: >>>>>>> + if (ops->vidioc_try_ext_pix_fmt_vid_out) >>>>>>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, >>>>>>> + ef); >>>>>>> + break; >>>>>>> + default: >>>>>>> + return -EINVAL; >>>>>>> + } >>>>>>> + >>>>>>> + v4l2_ext_pix_format_to_format(ef, &f, >>>>>>> V4L2_IS_CAP_MULTIPLANAR(vfd)); >>>>>>> + >>>>>>> + ret = v4l_try_fmt(ops, file, fh, &f); >>>>>>> + if (ret) >>>>>>> + /* TODO: retry with M-variant of ef->pixelformat? */ >>>>>>> + return ret; >>>>>>> + >>>>>>> + v4l2_format_to_ext_pix_format(&f, ef); >>>>>>> + return 0; >>>>>>> +} >>>>>>> + >>>>>>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops, >>>>>>> struct file *file, void *fh, void *arg) >>>>>>> { >>>>>>> @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info >>>>>>> v4l2_ioctls[] = { >>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, >>>>>>> v4l_print_freq_band, 0), >>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, >>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), >>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, >>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | >>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), >>>>>>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, >>>>>>> v4l_print_ext_pix_format, 0), >>>>>>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, >>>>>>> v4l_print_ext_pix_format, INFO_FL_PRIO), >>>>>>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, >>>>>>> v4l_print_ext_pix_format, 0), >>>>>>> }; >>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) >>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h >>>>>>> index edb733f21604..c44708dc9355 100644 >>>>>>> --- a/include/media/v4l2-ioctl.h >>>>>>> +++ b/include/media/v4l2-ioctl.h >>>>>>> @@ -48,11 +48,17 @@ struct v4l2_fh; >>>>>>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>>>>> * in single plane mode >>>>>>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that >>>>>>> implements >>>>>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>>>>>> for video >>>>>>> + * capture >>>>>>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>>>>> * @vidioc_g_fmt_vid_out: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out >>>>>>> * in single plane mode >>>>>>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that >>>>>>> implements >>>>>>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>>>>>> for video >>>>>>> + * out >>>>>>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that >>>>>>> implements >>>>>>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video >>>>>>> overlay output >>>>>>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements >>>>>>> @@ -82,11 +88,16 @@ struct v4l2_fh; >>>>>>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture >>>>>>> * in single plane mode >>>>>>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that >>>>>>> implements >>>>>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic >>>>>>> for video >>>>>>> + * capture >>>>>>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay >>>>>>> * @vidioc_s_fmt_vid_out: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out >>>>>>> * in single plane mode >>>>>>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that >>>>>>> implements >>>>>>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for >>>>>>> video out >>>>>>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that >>>>>>> implements >>>>>>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video >>>>>>> overlay output >>>>>>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements >>>>>>> @@ -116,11 +127,16 @@ struct v4l2_fh; >>>>>>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>>>>>> capture >>>>>>> * in single plane mode >>>>>>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that >>>>>>> implements >>>>>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl >>>>>>> logic for >>>>>>> + video capture >>>>>>> * @vidioc_try_fmt_vid_overlay: pointer to the function that >>>>>>> implements >>>>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>>>>>> overlay >>>>>>> * @vidioc_try_fmt_vid_out: pointer to the function that implements >>>>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out >>>>>>> * in single plane mode >>>>>>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that >>>>>>> implements >>>>>>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for >>>>>>> video out >>>>>>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that >>>>>>> implements >>>>>>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video >>>>>>> overlay >>>>>>> * output >>>>>>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { >>>>>>> /* VIDIOC_G_FMT handlers */ >>>>>>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>>>>> + struct v4l2_ext_pix_format *ef); >>>>>>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>>>>> + struct v4l2_ext_pix_format *ef); >>>>>>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, >>>>>>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { >>>>>>> /* VIDIOC_S_FMT handlers */ >>>>>>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>>>>> + struct v4l2_ext_pix_format *ef); >>>>>>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>>>>> + struct v4l2_ext_pix_format *ef); >>>>>>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, >>>>>>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { >>>>>>> /* VIDIOC_TRY_FMT handlers */ >>>>>>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, >>>>>>> + struct v4l2_ext_pix_format *ef); >>>>>>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, >>>>>>> + struct v4l2_ext_pix_format *ef); >>>>>>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, >>>>>>> struct v4l2_format *f); >>>>>>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, >>>>>>> diff --git a/include/uapi/linux/videodev2.h >>>>>>> b/include/uapi/linux/videodev2.h >>>>>>> index d9b7c9177605..a2d850513708 100644 >>>>>>> --- a/include/uapi/linux/videodev2.h >>>>>>> +++ b/include/uapi/linux/videodev2.h >>>>>>> @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { >>>>>>> __u8 reserved[7]; >>>>>>> } __attribute__ ((packed)); >>>>>>> +/** >>>>>>> + * struct v4l2_ext_pix_format - extended single/multiplanar format >>>>>>> definition >>>>>>> + * @type: type of the data stream; >>>>>>> V4L2_BUF_TYPE_VIDEO_CAPTURE or >>>>>>> + * V4L2_BUF_TYPE_VIDEO_OUTPUT >>>>>>> + * @width: image width in pixels >>>>>>> + * @height: image height in pixels >>>>>>> + * @field: enum v4l2_field; field order (for interlaced video) >>>>>>> + * @plane_fmt: per-plane information >>>>>>> + * @pixelformat: little endian four character code (fourcc) >>>>>>> + * @modifier: modifier applied to the format (used for tiled >>>>>>> formats >>>>>>> + * and other kind of HW-specific formats, like compressed >>>>>>> + * formats) as defined in drm_fourcc.h >>>>>>> + * @colorspace: enum v4l2_colorspace; supplemental to >>>>>>> pixelformat >>>>>>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function >>>>>>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding >>>>>>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding >>>>>>> + * @quantization: enum v4l2_quantization, colorspace quantization >>>>>>> + * @reserved: extra space reserved for future fields, must be >>>>>>> set to 0 >>>>>>> + */ >>>>>>> +struct v4l2_ext_pix_format { >>>>>>> + __u32 type; >>>>>>> + __u32 width; >>>>>>> + __u32 height; >>>>>>> + __u32 field; >>>>>>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; >>>>>>> + __u32 pixelformat; >>>>>>> + __u64 modifier; >>>>>>> + __u32 colorspace; >>>>>>> + __u32 xfer_func; >>>>>>> + union { >>>>>>> + __u32 ycbcr_enc; >>>>>>> + __u32 hsv_enc; >>>>>>> + }; >>>>>>> + __u32 quantization; >>>>>>> + __u32 reserved[9]; >>>>>>> +}; >>>>>>> + >>>>>>> /** >>>>>>> * struct v4l2_sdr_format - SDR format definition >>>>>>> * @pixelformat: little endian four character code (fourcc) >>>>>>> @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { >>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct >>>>>>> v4l2_query_ext_ctrl) >>>>>>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct >>>>>>> v4l2_ext_pix_format) >>>>>>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct >>>>>>> v4l2_ext_pix_format) >>>>>>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct >>>>>>> v4l2_ext_pix_format) >>>>>>> + >>>>>>> /* Reminder: when adding new ioctls please add support for them to >>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ >>>>>>> >>> >>> -- >>> Regards, >>> >>> Laurent Pinchart >> >> -- >> Hsia-Jun(Randy) Li > > -- > Regards, > > Laurent Pinchart
On Sun, Nov 6, 2022 at 2:11 PM Dmitry Osipenko <dmitry.osipenko@collabora.com> wrote: > > On 11/5/22 18:19, Hsia-Jun Li wrote: > > Hello Helen > > > > I didn't see any updates from V6 and V7-WIP in your repo. That is what I > > need to for our complex tile formats in our platform. > > > > Any future plane here? > > > > Besides I have some ideas on these patches. > > I was looking into updating this patchset few months ago and the biggest > blocker was the absence of immediate upstream user for this new UAPI. > What your platform is? Is the driver stack completely opensource? > ChromeOS had interest in this for enabling UBWC for the venus driver. We have a workaround at the moment, but would be interested. So not immediate need, but would hopefully be a user at some point. -Fritz > -- > Best regards, > Dmitry >
On 11/7/22 11:30, Laurent Pinchart wrote: > On Mon, Nov 07, 2022 at 01:11:32AM +0300, Dmitry Osipenko wrote: >> On 11/5/22 18:19, Hsia-Jun Li wrote: >>> Hello Helen >>> >>> I didn't see any updates from V6 and V7-WIP in your repo. That is what I >>> need to for our complex tile formats in our platform. >>> >>> Any future plane here? >>> >>> Besides I have some ideas on these patches. >> >> I was looking into updating this patchset few months ago and the biggest >> blocker was the absence of immediate upstream user for this new UAPI. >> What your platform is? Is the driver stack completely opensource? > > libcamera could be a good place to test (part of) this API in userspace. > We could really do with the data offset feature for instance. Since we don't have a user for the new UAPI right now, perhaps we can put the new UAPI behind the staging Kconfig until it will get a real/production user. On the other hand, I had unpleasant experience with having UAPI gated by staging Kconfig for other kernel subsystem in a sense that it permanently stays a staging UAPI because nobody has time for unstaging it. I see a common demand in this new UAPI which only becomes stronger over time. Won't hurt to refresh this patchset. I had a rebased version locally, but likely it needs to be rebased again. Will try to allocate time for this.
Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : > > > VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > > > VIDIOC_ENUM_FMT > > > would just report NV12M. > > > > If NV12 and NV12M are equivalent in Ext API, I don't see why we would > > report both (unless I'm missing something, which is probably the case). > > > > The idea was to deprecate the M-variants one day. > I was thinking the way in DRM API is better, always assuming it would > always in a multiple planes. The only problem is we don't have a way to > let the allocator that allocate contiguous memory for planes when we > need to do that. Its not too late to allow this to be negotiated, but I would move this out of the pixel format definition to stop the explosion of duplicate pixel formats, which is a nightmare to deal with. If I simplify the discussion, we want to negotiate contiguity with the driver. The new FMT structure should have a CONTIGUOUS flag. So if userpace sets: S_FMT(NV12, CONTIGUOUS) The driver can accepts, and return the unmodified structure, or may drop the CONTIGUOUS flag, which would mean its not supported. Could be the other way around too. As for allocation, if you have CONTIGUOUS flag set, userspace does not have to export or map memory for each planes, as they are the same. We simply need to define the offset as relative to their allocation, which I think is the most sensible thing. Nicolas
On 11/11/22 01:06, Nicolas Dufresne wrote: > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : >>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while >>>> VIDIOC_ENUM_FMT >>>> would just report NV12M. >>> >>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would >>> report both (unless I'm missing something, which is probably the case). >>> >>> The idea was to deprecate the M-variants one day. >> I was thinking the way in DRM API is better, always assuming it would >> always in a multiple planes. The only problem is we don't have a way to >> let the allocator that allocate contiguous memory for planes when we >> need to do that. > > Its not too late to allow this to be negotiated, but I would move this out of > the pixel format definition to stop the explosion of duplicate pixel formats, > which is a nightmare to deal with. I wonder whether we need to keep the pixel formats in videodev2.h anymore. If we would like to use the modifiers from drm_fourcc.h, why don't we use their pixel formats, they should be the same values of non-M variant pixel formats of v4l2. Let videodev2.h only maintain the those codecs or motion based compressed (pixel) formats. If I simplify the discussion, we want to > negotiate contiguity with the driver. The new FMT structure should have a > CONTIGUOUS flag. So if userpace sets: > > S_FMT(NV12, CONTIGUOUS) I wonder whether we would allow some planes being contiguous while some would not. For example, the graphics planes could be in a contiguous memory address while its compression metadata are not. Although that is not the case of our platform. I believe it sounds like reasonable case for improving the performance, two meta planes could resident in a different memory bank. That lead to another question which I forgot whether I mention it before. There are four modifiers in DRM while we would only one in these patches. From the EGL https://registry.khronos.org/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import_modifiers.txt The modifier for echo plane could be different. I wish it would be better to create a framebuffer being aware of which planes are graphics or metadata. I wonder whether it would be better that convincing the DRM maintainer adding a non vendor flag for contiguous memory allocation here(DRM itself don't need it). While whether the memory could be contiguous for these vendor pixel formats, it is complex vendor defined. > > The driver can accepts, and return the unmodified structure, or may drop the > CONTIGUOUS flag, which would mean its not supported. Could be the other way > around too. As for allocation, if you have CONTIGUOUS flag set, userspace does > not have to export or map memory for each planes, as they are the same. We > simply need to define the offset as relative to their allocation, which I think > is the most sensible thing. > > Nicolas >
On Fri, Nov 11, 2022 at 12:04 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote: > > > > On 11/11/22 01:06, Nicolas Dufresne wrote: > > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > > > > Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : > >>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > >>>> VIDIOC_ENUM_FMT > >>>> would just report NV12M. > >>> > >>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would > >>> report both (unless I'm missing something, which is probably the case). > >>> > >>> The idea was to deprecate the M-variants one day. > >> I was thinking the way in DRM API is better, always assuming it would > >> always in a multiple planes. The only problem is we don't have a way to > >> let the allocator that allocate contiguous memory for planes when we > >> need to do that. > > > > Its not too late to allow this to be negotiated, but I would move this out of > > the pixel format definition to stop the explosion of duplicate pixel formats, > > which is a nightmare to deal with. > I wonder whether we need to keep the pixel formats in videodev2.h > anymore. If we would like to use the modifiers from drm_fourcc.h, why > don't we use their pixel formats, they should be the same values of > non-M variant pixel formats of v4l2. > > Let videodev2.h only maintain the those codecs or motion based > compressed (pixel) formats. > > If I simplify the discussion, we want to > > negotiate contiguity with the driver. The new FMT structure should have a > > CONTIGUOUS flag. So if userpace sets: > > > > S_FMT(NV12, CONTIGUOUS) > I wonder whether we would allow some planes being contiguous while some > would not. For example, the graphics planes could be in a contiguous > memory address while its compression metadata are not. > Although that is not the case of our platform. I believe it sounds like > reasonable case for improving the performance, two meta planes could > resident in a different memory bank. I feel like this would be only useful in the MMAP mode. Looking at how the other UAPIs are evolving, things are going towards userspace-managed allocations, using, for example, DMA-buf heaps. I think we should follow the trend and keep the MMAP mode just at the same level of functionality as is today and focus on improvements and new functionality for the DMABUF mode. > > That lead to another question which I forgot whether I mention it before. > > There are four modifiers in DRM while we would only one in these patches. > From the EGL > https://registry.khronos.org/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import_modifiers.txt > > The modifier for echo plane could be different. I wish it would be > better to create a framebuffer being aware of which planes are graphics > or metadata. What's an echo plane? That said, it indeed looks like we may want to be consistent with DRM here and allow per-plane modifiers. > > I wonder whether it would be better that convincing the DRM maintainer > adding a non vendor flag for contiguous memory allocation here(DRM > itself don't need it). > While whether the memory could be contiguous for these vendor pixel > formats, it is complex vendor defined. Memory allocation doesn't sound to me like it is related to formats or modifiers in any way. I agree with Nicolas that if we want to allow the userspace to specify if the memory should be contiguous or not, that should be a separate flag and actually I'd probably see it in REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. > > > > > The driver can accepts, and return the unmodified structure, or may drop the > > CONTIGUOUS flag, which would mean its not supported. Could be the other way > > around too. As for allocation, if you have CONTIGUOUS flag set, userspace does > > not have to export or map memory for each planes, as they are the same. We > > simply need to define the offset as relative to their allocation, which I think > > is the most sensible thing. > > > > Nicolas > > > > -- > Hsia-Jun(Randy) Li
On 11/11/22 13:48, Tomasz Figa wrote: > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > On Fri, Nov 11, 2022 at 12:04 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote: >> >> >> >> On 11/11/22 01:06, Nicolas Dufresne wrote: >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. >>> >>> >>> Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : >>>>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while >>>>>> VIDIOC_ENUM_FMT >>>>>> would just report NV12M. >>>>> >>>>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would >>>>> report both (unless I'm missing something, which is probably the case). >>>>> >>>>> The idea was to deprecate the M-variants one day. >>>> I was thinking the way in DRM API is better, always assuming it would >>>> always in a multiple planes. The only problem is we don't have a way to >>>> let the allocator that allocate contiguous memory for planes when we >>>> need to do that. >>> >>> Its not too late to allow this to be negotiated, but I would move this out of >>> the pixel format definition to stop the explosion of duplicate pixel formats, >>> which is a nightmare to deal with. >> I wonder whether we need to keep the pixel formats in videodev2.h >> anymore. If we would like to use the modifiers from drm_fourcc.h, why >> don't we use their pixel formats, they should be the same values of >> non-M variant pixel formats of v4l2. >> >> Let videodev2.h only maintain the those codecs or motion based >> compressed (pixel) formats. >> >> If I simplify the discussion, we want to >>> negotiate contiguity with the driver. The new FMT structure should have a >>> CONTIGUOUS flag. So if userpace sets: >>> >>> S_FMT(NV12, CONTIGUOUS) >> I wonder whether we would allow some planes being contiguous while some >> would not. For example, the graphics planes could be in a contiguous >> memory address while its compression metadata are not. >> Although that is not the case of our platform. I believe it sounds like >> reasonable case for improving the performance, two meta planes could >> resident in a different memory bank. > > I feel like this would be only useful in the MMAP mode. Looking at how > the other UAPIs are evolving, things are going towards > userspace-managed allocations, using, for example, DMA-buf heaps. I > think we should follow the trend and keep the MMAP mode just at the > same level of functionality as is today and focus on improvements and > new functionality for the DMABUF mode. > I know there are still some devices(encoder) which only have one register for storing the address of a graphics buffer. >> >> That lead to another question which I forgot whether I mention it before. >> >> There are four modifiers in DRM while we would only one in these patches. >> From the EGL >> https://urldefense.proofpoint.com/v2/url?u=https-3A__registry.khronos.org_EGL_extensions_EXT_EGL-5FEXT-5Fimage-5Fdma-5Fbuf-5Fimport-5Fmodifiers.txt&d=DwIFaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=mCebYOAiZK6pbpH1MrZGq-ZkDW-OqORCSwsCEX9ScgdXk_yfWZFJPC5aC93CUg5F&s=rtmW_t2LYoJ6g3Y5wgyICmABu-2Npw3JCOlvUVIYH2o&e= >> >> The modifier for echo plane could be different. I wish it would be >> better to create a framebuffer being aware of which planes are graphics >> or metadata. > > What's an echo plane? > They could be DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED DRM_FORMAT_MOD_SYNA_MTR DRM_FORMAT_MOD_SYNA_MTR Or DRM_FORMAT_MOD_SYNA_V4H3P8_64L4 DRM_FORMAT_MOD_SYNA_V4H3P8_64L4 in our platform. It could give a better idea on what is stored in a plane. > That said, it indeed looks like we may want to be consistent with DRM > here and allow per-plane modifiers. > >> >> I wonder whether it would be better that convincing the DRM maintainer >> adding a non vendor flag for contiguous memory allocation here(DRM >> itself don't need it). >> While whether the memory could be contiguous for these vendor pixel >> formats, it is complex vendor defined. > > Memory allocation doesn't sound to me like it is related to formats or > modifiers in any way. I agree with Nicolas that if we want to allow > the userspace to specify if the memory should be contiguous or not, > that should be a separate flag and actually I'd probably see it in > REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. > I agree with that. But here is a problem, if there was a display device(DRM) that only supports contiguous planes in a frame buffer. How do we be aware of that? >> >>> >>> The driver can accepts, and return the unmodified structure, or may drop the >>> CONTIGUOUS flag, which would mean its not supported. Could be the other way >>> around too. As for allocation, if you have CONTIGUOUS flag set, userspace does >>> not have to export or map memory for each planes, as they are the same. We >>> simply need to define the offset as relative to their allocation, which I think >>> is the most sensible thing. >>> >>> Nicolas >>> >> >> -- >> Hsia-Jun(Randy) Li
Hi Tomasz, On Fri, Nov 11, 2022 at 02:48:48PM +0900, Tomasz Figa wrote: > On Fri, Nov 11, 2022 at 12:04 PM Hsia-Jun Li wrote: > > On 11/11/22 01:06, Nicolas Dufresne wrote: > > > Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : > > >>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > > >>>> VIDIOC_ENUM_FMT > > >>>> would just report NV12M. > > >>> > > >>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would > > >>> report both (unless I'm missing something, which is probably the case). > > >>> > > >>> The idea was to deprecate the M-variants one day. > > >> I was thinking the way in DRM API is better, always assuming it would > > >> always in a multiple planes. The only problem is we don't have a way to > > >> let the allocator that allocate contiguous memory for planes when we > > >> need to do that. > > > > > > Its not too late to allow this to be negotiated, but I would move this out of > > > the pixel format definition to stop the explosion of duplicate pixel formats, > > > which is a nightmare to deal with. > > > > I wonder whether we need to keep the pixel formats in videodev2.h > > anymore. If we would like to use the modifiers from drm_fourcc.h, why > > don't we use their pixel formats, they should be the same values of > > non-M variant pixel formats of v4l2. > > > > Let videodev2.h only maintain the those codecs or motion based > > compressed (pixel) formats. > > > > If I simplify the discussion, we want to > > > > > negotiate contiguity with the driver. The new FMT structure should have a > > > CONTIGUOUS flag. So if userpace sets: > > > > > > S_FMT(NV12, CONTIGUOUS) > > > > I wonder whether we would allow some planes being contiguous while some > > would not. For example, the graphics planes could be in a contiguous > > memory address while its compression metadata are not. > > Although that is not the case of our platform. I believe it sounds like > > reasonable case for improving the performance, two meta planes could > > resident in a different memory bank. > > I feel like this would be only useful in the MMAP mode. Looking at how > the other UAPIs are evolving, things are going towards > userspace-managed allocations, using, for example, DMA-buf heaps. I > think we should follow the trend and keep the MMAP mode just at the > same level of functionality as is today and focus on improvements and > new functionality for the DMABUF mode. I agree, but we will need an API to expose the memory constraints of the device, or userspace won't be able to allocate memory compatible with the hardware or driver requirements. > > That lead to another question which I forgot whether I mention it before. > > > > There are four modifiers in DRM while we would only one in these patches. > > From the EGL > > https://registry.khronos.org/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import_modifiers.txt > > > > The modifier for echo plane could be different. I wish it would be > > better to create a framebuffer being aware of which planes are graphics > > or metadata. > > What's an echo plane? > > That said, it indeed looks like we may want to be consistent with DRM > here and allow per-plane modifiers. > > > I wonder whether it would be better that convincing the DRM maintainer > > adding a non vendor flag for contiguous memory allocation here(DRM > > itself don't need it). > > While whether the memory could be contiguous for these vendor pixel > > formats, it is complex vendor defined. > > Memory allocation doesn't sound to me like it is related to formats or > modifiers in any way. I agree with Nicolas that if we want to allow > the userspace to specify if the memory should be contiguous or not, > that should be a separate flag and actually I'd probably see it in > REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. I like how DRM decouples allocation of buffer objects and creation of frame buffers. > > > The driver can accepts, and return the unmodified structure, or may drop the > > > CONTIGUOUS flag, which would mean its not supported. Could be the other way > > > around too. As for allocation, if you have CONTIGUOUS flag set, userspace does > > > not have to export or map memory for each planes, as they are the same. We > > > simply need to define the offset as relative to their allocation, which I think > > > is the most sensible thing.
On Fri, Nov 11, 2022 at 3:31 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote: > > > > On 11/11/22 13:48, Tomasz Figa wrote: > > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > > > > On Fri, Nov 11, 2022 at 12:04 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote: > >> > >> > >> > >> On 11/11/22 01:06, Nicolas Dufresne wrote: > >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > >>> > >>> > >>> Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : > >>>>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > >>>>>> VIDIOC_ENUM_FMT > >>>>>> would just report NV12M. > >>>>> > >>>>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would > >>>>> report both (unless I'm missing something, which is probably the case). > >>>>> > >>>>> The idea was to deprecate the M-variants one day. > >>>> I was thinking the way in DRM API is better, always assuming it would > >>>> always in a multiple planes. The only problem is we don't have a way to > >>>> let the allocator that allocate contiguous memory for planes when we > >>>> need to do that. > >>> > >>> Its not too late to allow this to be negotiated, but I would move this out of > >>> the pixel format definition to stop the explosion of duplicate pixel formats, > >>> which is a nightmare to deal with. > >> I wonder whether we need to keep the pixel formats in videodev2.h > >> anymore. If we would like to use the modifiers from drm_fourcc.h, why > >> don't we use their pixel formats, they should be the same values of > >> non-M variant pixel formats of v4l2. > >> > >> Let videodev2.h only maintain the those codecs or motion based > >> compressed (pixel) formats. > >> > >> If I simplify the discussion, we want to > >>> negotiate contiguity with the driver. The new FMT structure should have a > >>> CONTIGUOUS flag. So if userpace sets: > >>> > >>> S_FMT(NV12, CONTIGUOUS) > >> I wonder whether we would allow some planes being contiguous while some > >> would not. For example, the graphics planes could be in a contiguous > >> memory address while its compression metadata are not. > >> Although that is not the case of our platform. I believe it sounds like > >> reasonable case for improving the performance, two meta planes could > >> resident in a different memory bank. > > > > I feel like this would be only useful in the MMAP mode. Looking at how > > the other UAPIs are evolving, things are going towards > > userspace-managed allocations, using, for example, DMA-buf heaps. I > > think we should follow the trend and keep the MMAP mode just at the > > same level of functionality as is today and focus on improvements and > > new functionality for the DMABUF mode. > > > I know there are still some devices(encoder) which only have one > register for storing the address of a graphics buffer. For those, the legacy MMAP mode (with existing functionality) can be successfully used, we wouldn't be removing it any time soon. Just don't want to design new functionality specifically for the legacy mode. > >> > >> That lead to another question which I forgot whether I mention it before. > >> > >> There are four modifiers in DRM while we would only one in these patches. > >> From the EGL > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__registry.khronos.org_EGL_extensions_EXT_EGL-5FEXT-5Fimage-5Fdma-5Fbuf-5Fimport-5Fmodifiers.txt&d=DwIFaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=mCebYOAiZK6pbpH1MrZGq-ZkDW-OqORCSwsCEX9ScgdXk_yfWZFJPC5aC93CUg5F&s=rtmW_t2LYoJ6g3Y5wgyICmABu-2Npw3JCOlvUVIYH2o&e= > >> > >> The modifier for echo plane could be different. I wish it would be > >> better to create a framebuffer being aware of which planes are graphics > >> or metadata. > > > > What's an echo plane? > > > They could be > DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED > DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED > DRM_FORMAT_MOD_SYNA_MTR > DRM_FORMAT_MOD_SYNA_MTR > Or > DRM_FORMAT_MOD_SYNA_V4H3P8_64L4 > DRM_FORMAT_MOD_SYNA_V4H3P8_64L4 > > in our platform. It could give a better idea on what is stored in a plane. Yes, that's what I was thinking, but my question is more about what those planes hold. Are you sure that they should be planes of the same buffer rather than separate buffers? > > That said, it indeed looks like we may want to be consistent with DRM > > here and allow per-plane modifiers. > > > >> > >> I wonder whether it would be better that convincing the DRM maintainer > >> adding a non vendor flag for contiguous memory allocation here(DRM > >> itself don't need it). > >> While whether the memory could be contiguous for these vendor pixel > >> formats, it is complex vendor defined. > > > > Memory allocation doesn't sound to me like it is related to formats or > > modifiers in any way. I agree with Nicolas that if we want to allow > > the userspace to specify if the memory should be contiguous or not, > > that should be a separate flag and actually I'd probably see it in > > REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. > > > I agree with that. But here is a problem, if there was a display > device(DRM) that only supports contiguous planes in a frame buffer. > How do we be aware of that? That's why I think the MMAP mode is not scalable and shouldn't be expanded anymore. Both V4L2 and DRM devices should describe their constraints to the userspace and then the userspace should allocate accordingly from the right DMA-buf heap. (Or as Android and ChromeOS do, just have a central allocator library that understands the constraints, so there is no need to query the drivers.) > >> > >>> > >>> The driver can accepts, and return the unmodified structure, or may drop the > >>> CONTIGUOUS flag, which would mean its not supported. Could be the other way > >>> around too. As for allocation, if you have CONTIGUOUS flag set, userspace does > >>> not have to export or map memory for each planes, as they are the same. We > >>> simply need to define the offset as relative to their allocation, which I think > >>> is the most sensible thing. > >>> > >>> Nicolas > >>> > >> > >> -- > >> Hsia-Jun(Randy) Li > > -- > Hsia-Jun(Randy) Li
On Fri, Nov 11, 2022 at 5:43 PM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hi Tomasz, > > On Fri, Nov 11, 2022 at 02:48:48PM +0900, Tomasz Figa wrote: > > On Fri, Nov 11, 2022 at 12:04 PM Hsia-Jun Li wrote: > > > On 11/11/22 01:06, Nicolas Dufresne wrote: > > > > Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : > > > >>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while > > > >>>> VIDIOC_ENUM_FMT > > > >>>> would just report NV12M. > > > >>> > > > >>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would > > > >>> report both (unless I'm missing something, which is probably the case). > > > >>> > > > >>> The idea was to deprecate the M-variants one day. > > > >> I was thinking the way in DRM API is better, always assuming it would > > > >> always in a multiple planes. The only problem is we don't have a way to > > > >> let the allocator that allocate contiguous memory for planes when we > > > >> need to do that. > > > > > > > > Its not too late to allow this to be negotiated, but I would move this out of > > > > the pixel format definition to stop the explosion of duplicate pixel formats, > > > > which is a nightmare to deal with. > > > > > > I wonder whether we need to keep the pixel formats in videodev2.h > > > anymore. If we would like to use the modifiers from drm_fourcc.h, why > > > don't we use their pixel formats, they should be the same values of > > > non-M variant pixel formats of v4l2. > > > > > > Let videodev2.h only maintain the those codecs or motion based > > > compressed (pixel) formats. > > > > > > If I simplify the discussion, we want to > > > > > > > negotiate contiguity with the driver. The new FMT structure should have a > > > > CONTIGUOUS flag. So if userpace sets: > > > > > > > > S_FMT(NV12, CONTIGUOUS) > > > > > > I wonder whether we would allow some planes being contiguous while some > > > would not. For example, the graphics planes could be in a contiguous > > > memory address while its compression metadata are not. > > > Although that is not the case of our platform. I believe it sounds like > > > reasonable case for improving the performance, two meta planes could > > > resident in a different memory bank. > > > > I feel like this would be only useful in the MMAP mode. Looking at how > > the other UAPIs are evolving, things are going towards > > userspace-managed allocations, using, for example, DMA-buf heaps. I > > think we should follow the trend and keep the MMAP mode just at the > > same level of functionality as is today and focus on improvements and > > new functionality for the DMABUF mode. > > I agree, but we will need an API to expose the memory constraints of the > device, or userspace won't be able to allocate memory compatible with > the hardware or driver requirements. Yes, I fully agree and that's why I think we should rather focus our efforts in that direction rather than expanding the existing MMAP capabilities. > > > > That lead to another question which I forgot whether I mention it before. > > > > > > There are four modifiers in DRM while we would only one in these patches. > > > From the EGL > > > https://registry.khronos.org/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import_modifiers.txt > > > > > > The modifier for echo plane could be different. I wish it would be > > > better to create a framebuffer being aware of which planes are graphics > > > or metadata. > > > > What's an echo plane? > > > > That said, it indeed looks like we may want to be consistent with DRM > > here and allow per-plane modifiers. > > > > > I wonder whether it would be better that convincing the DRM maintainer > > > adding a non vendor flag for contiguous memory allocation here(DRM > > > itself don't need it). > > > While whether the memory could be contiguous for these vendor pixel > > > formats, it is complex vendor defined. > > > > Memory allocation doesn't sound to me like it is related to formats or > > modifiers in any way. I agree with Nicolas that if we want to allow > > the userspace to specify if the memory should be contiguous or not, > > that should be a separate flag and actually I'd probably see it in > > REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. > > I like how DRM decouples allocation of buffer objects and creation of > frame buffers. Exactly why I proposed so rather than coupling it with S_FMT. (But then it's moot if we decide to focus on DMABUF mode.) > > > > > The driver can accepts, and return the unmodified structure, or may drop the > > > > CONTIGUOUS flag, which would mean its not supported. Could be the other way > > > > around too. As for allocation, if you have CONTIGUOUS flag set, userspace does > > > > not have to export or map memory for each planes, as they are the same. We > > > > simply need to define the offset as relative to their allocation, which I think > > > > is the most sensible thing. > > -- > Regards, > > Laurent Pinchart
On 11/11/22 16:52, Tomasz Figa wrote: > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > On Fri, Nov 11, 2022 at 3:31 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote: >> >> >> >> On 11/11/22 13:48, Tomasz Figa wrote: >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. >>> >>> >>> On Fri, Nov 11, 2022 at 12:04 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote: >>>> >>>> >>>> >>>> On 11/11/22 01:06, Nicolas Dufresne wrote: >>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. >>>>> >>>>> >>>>> Le samedi 05 novembre 2022 à 23:19 +0800, Hsia-Jun Li a écrit : >>>>>>>> VIDIOC_ENUM_EXT_PIX_FMT would report NV12 and NV12M, while >>>>>>>> VIDIOC_ENUM_FMT >>>>>>>> would just report NV12M. >>>>>>> >>>>>>> If NV12 and NV12M are equivalent in Ext API, I don't see why we would >>>>>>> report both (unless I'm missing something, which is probably the case). >>>>>>> >>>>>>> The idea was to deprecate the M-variants one day. >>>>>> I was thinking the way in DRM API is better, always assuming it would >>>>>> always in a multiple planes. The only problem is we don't have a way to >>>>>> let the allocator that allocate contiguous memory for planes when we >>>>>> need to do that. >>>>> >>>>> Its not too late to allow this to be negotiated, but I would move this out of >>>>> the pixel format definition to stop the explosion of duplicate pixel formats, >>>>> which is a nightmare to deal with. >>>> I wonder whether we need to keep the pixel formats in videodev2.h >>>> anymore. If we would like to use the modifiers from drm_fourcc.h, why >>>> don't we use their pixel formats, they should be the same values of >>>> non-M variant pixel formats of v4l2. >>>> >>>> Let videodev2.h only maintain the those codecs or motion based >>>> compressed (pixel) formats. >>>> >>>> If I simplify the discussion, we want to >>>>> negotiate contiguity with the driver. The new FMT structure should have a >>>>> CONTIGUOUS flag. So if userpace sets: >>>>> >>>>> S_FMT(NV12, CONTIGUOUS) >>>> I wonder whether we would allow some planes being contiguous while some >>>> would not. For example, the graphics planes could be in a contiguous >>>> memory address while its compression metadata are not. >>>> Although that is not the case of our platform. I believe it sounds like >>>> reasonable case for improving the performance, two meta planes could >>>> resident in a different memory bank. >>> >>> I feel like this would be only useful in the MMAP mode. Looking at how >>> the other UAPIs are evolving, things are going towards >>> userspace-managed allocations, using, for example, DMA-buf heaps. I >>> think we should follow the trend and keep the MMAP mode just at the >>> same level of functionality as is today and focus on improvements and >>> new functionality for the DMABUF mode. >>> >> I know there are still some devices(encoder) which only have one >> register for storing the address of a graphics buffer. > > For those, the legacy MMAP mode (with existing functionality) can be > successfully used, we wouldn't be removing it any time soon. Just > don't want to design new functionality specifically for the legacy > mode. > But it prevents the encoder using the buffer from the outside. For example, there was an PCI-e interface camera which would write to the system memory where is configured to its register, then we would like to encode those buffers. >>>> >>>> That lead to another question which I forgot whether I mention it before. >>>> >>>> There are four modifiers in DRM while we would only one in these patches. >>>> From the EGL >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__registry.khronos.org_EGL_extensions_EXT_EGL-5FEXT-5Fimage-5Fdma-5Fbuf-5Fimport-5Fmodifiers.txt&d=DwIFaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=mCebYOAiZK6pbpH1MrZGq-ZkDW-OqORCSwsCEX9ScgdXk_yfWZFJPC5aC93CUg5F&s=rtmW_t2LYoJ6g3Y5wgyICmABu-2Npw3JCOlvUVIYH2o&e= >>>> >>>> The modifier for echo plane could be different. I wish it would be >>>> better to create a framebuffer being aware of which planes are graphics >>>> or metadata. >>> >>> What's an echo plane? >>> >> They could be >> DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED >> DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED >> DRM_FORMAT_MOD_SYNA_MTR >> DRM_FORMAT_MOD_SYNA_MTR >> Or >> DRM_FORMAT_MOD_SYNA_V4H3P8_64L4 >> DRM_FORMAT_MOD_SYNA_V4H3P8_64L4 >> >> in our platform. It could give a better idea on what is stored in a plane. > > Yes, that's what I was thinking, but my question is more about what > those planes hold. DRM_FORMAT_MOD_SYNA_V4H1* or DRM_FORMAT_MOD_SYNA_V4H3P8* would be the luma and chroma (un)compressed data here. They are modifiers to NV12 and NV15. Are you sure that they should be planes of the same > buffer rather than separate buffers? I am not sure about your question here. I prefer they are in a different memory plane. But not all Android APIs support that. If I just think about our platform and GNU Linux, I won't care about those limitations. > >>> That said, it indeed looks like we may want to be consistent with DRM >>> here and allow per-plane modifiers. >>> >>>> >>>> I wonder whether it would be better that convincing the DRM maintainer >>>> adding a non vendor flag for contiguous memory allocation here(DRM >>>> itself don't need it). >>>> While whether the memory could be contiguous for these vendor pixel >>>> formats, it is complex vendor defined. >>> >>> Memory allocation doesn't sound to me like it is related to formats or >>> modifiers in any way. I agree with Nicolas that if we want to allow >>> the userspace to specify if the memory should be contiguous or not, >>> that should be a separate flag and actually I'd probably see it in >>> REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. >>> >> I agree with that. But here is a problem, if there was a display >> device(DRM) that only supports contiguous planes in a frame buffer. >> How do we be aware of that? > > That's why I think the MMAP mode is not scalable and shouldn't be > expanded anymore. Both V4L2 and DRM devices should describe their > constraints to the userspace and then the userspace should allocate > accordingly from the right DMA-buf heap. (Or as Android and ChromeOS > do, just have a central allocator library that understands the > constraints, so there is no need to query the drivers.) > Because we are working for embedded platforms which don't have memory beyond the system memory. I believe those GPU vendors would hate idea of DMAheap only. >>>> >>>>> >>>>> The driver can accepts, and return the unmodified structure, or may drop the >>>>> CONTIGUOUS flag, which would mean its not supported. Could be the other way >>>>> around too. As for allocation, if you have CONTIGUOUS flag set, userspace does >>>>> not have to export or map memory for each planes, as they are the same. We >>>>> simply need to define the offset as relative to their allocation, which I think >>>>> is the most sensible thing. >>>>> >>>>> Nicolas >>>>> >>>> >>>> -- >>>> Hsia-Jun(Randy) Li >> >> -- >> Hsia-Jun(Randy) Li
Le vendredi 11 novembre 2022 à 11:03 +0800, Hsia-Jun Li a écrit : > There are four modifiers in DRM while we would only one in these patches. > From the EGL > https://registry.khronos.org/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import_modifiers.txt > > The modifier for echo plane could be different. I wish it would be > better to create a framebuffer being aware of which planes are graphics > or metadata. I was told that having different modifiers per plane is not supported and just an historical artifact of an over-engineered API design. Nicolas
Le vendredi 11 novembre 2022 à 17:52 +0900, Tomasz Figa a écrit : > > > Memory allocation doesn't sound to me like it is related to formats or > > > modifiers in any way. I agree with Nicolas that if we want to allow > > > the userspace to specify if the memory should be contiguous or not, > > > that should be a separate flag and actually I'd probably see it in > > > REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. > > > > > I agree with that. But here is a problem, if there was a display > > device(DRM) that only supports contiguous planes in a frame buffer. > > How do we be aware of that? > > That's why I think the MMAP mode is not scalable and shouldn't be > expanded anymore. Both V4L2 and DRM devices should describe their > constraints to the userspace and then the userspace should allocate > accordingly from the right DMA-buf heap. (Or as Android and ChromeOS > do, just have a central allocator library that understands the > constraints, so there is no need to query the drivers.) Just pointing out, they "hardcode" the constraints, they don't truly understand them. Also, the Android/ChromeOS implementation is not a great playground, as it completely ignores the constrait already exposed by V4L2 API (the sizeimage and bytesperline found in the FMT structure). You would not have to implement Hantro and Rockchip motion vector size calculation there if you'd simply use the sizeimage. Nicolas
Le vendredi 11 novembre 2022 à 17:54 +0900, Tomasz Figa a écrit : > > > I feel like this would be only useful in the MMAP mode. Looking at how > > > the other UAPIs are evolving, things are going towards > > > userspace-managed allocations, using, for example, DMA-buf heaps. I > > > think we should follow the trend and keep the MMAP mode just at the > > > same level of functionality as is today and focus on improvements and > > > new functionality for the DMABUF mode. > > > > I agree, but we will need an API to expose the memory constraints of the > > device, or userspace won't be able to allocate memory compatible with > > the hardware or driver requirements. > > Yes, I fully agree and that's why I think we should rather focus our > efforts in that direction rather than expanding the existing MMAP > capabilities. I was once told that MMAP was mandatory to support in v4l2 drivers. I'd like to get some clarification on the subject for sure. We can't break compat, unless we spin v4l3 here. One thing that come to mind, is that its not true that a V4L2 driver can always be importer only. For cards like Xplorer X1600P PCIe Accelerator [1] from Blaize (they are using a modified, but still generic, Hantro Driver), the CODECs memory should be allocated on the card for best performance. Only the driver is aware that there is memory on that card, and so it must export the buffers. In a DMABuf import only future, that basically means the driver must implement a DMABuf HEAP driver, and to make this usable by generic software, the constraints API need to support telling userspace that this Heap is to be used. This gap easily extend to DRM, which have per driver API to allocate memory, and in some cases these API must be used (they don't have heaps for on-card memory allocation). When this is happening inside the GFX stack, it works very well, but when you need to integrate this with some V4L2 driver, its not really practical and requires something like minigbm/gralloc, which is non-generic. [1] https://www.blaize.com/products/ai-edge-computing-platforms/
> On Nov 16, 2022, at 12:03 AM, Nicolas Dufresne <nicolas@ndufresne.ca> wrote: > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. > > > Le vendredi 11 novembre 2022 à 17:52 +0900, Tomasz Figa a écrit : >>>> Memory allocation doesn't sound to me like it is related to formats or >>>> modifiers in any way. I agree with Nicolas that if we want to allow >>>> the userspace to specify if the memory should be contiguous or not, >>>> that should be a separate flag and actually I'd probably see it in >>>> REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format. >>> I agree with that. But here is a problem, if there was a display >>> device(DRM) that only supports contiguous planes in a frame buffer. >>> How do we be aware of that? >> >> That's why I think the MMAP mode is not scalable and shouldn't be >> expanded anymore. Both V4L2 and DRM devices should describe their >> constraints to the userspace and then the userspace should allocate >> accordingly from the right DMA-buf heap. (Or as Android and ChromeOS >> do, just have a central allocator library that understands the >> constraints, so there is no need to query the drivers.) > > Just pointing out, they "hardcode" the constraints, they don't truly understand > them. Also, the Android/ChromeOS implementation is not a great playground, as it > completely ignores the constrait already exposed by V4L2 API (the sizeimage and > bytesperline found in the FMT structure). You would not have to implement Hantro > and Rockchip motion vector size calculation there if you'd simply use the > sizeimage. > About the allocation, I have my own plan here. Beyond the draft of DMAHeap allocator I threw last time. I would like to modify the v4l2_queue API: 1. allowing different allocators for each memory plane(that matches the demand of synaptics video smart platform, which graphic planes support IOMMU while the compression metadata planes don’t). 2. New API that let driver allocate all memory planes for a buffer in a call And DMAHeap V4l2 allocator would become a template or currying. > Nicolas
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index f9cff033d0dc..5add58cb6d45 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -608,27 +608,42 @@ static void determine_valid_ioctls(struct video_device *vdev) ops->vidioc_enum_fmt_vid_overlay)) || (is_tx && ops->vidioc_enum_fmt_vid_out)) set_bit(_IOC_NR(VIDIOC_ENUM_FMT), valid_ioctls); + if ((is_rx && ops->vidioc_g_fmt_vid_overlay) || + (is_tx && ops->vidioc_g_fmt_vid_out_overlay)) + set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); if ((is_rx && (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane || - ops->vidioc_g_fmt_vid_overlay)) || + ops->vidioc_g_ext_pix_fmt_vid_cap)) || (is_tx && (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane || - ops->vidioc_g_fmt_vid_out_overlay))) + ops->vidioc_g_ext_pix_fmt_vid_out))) { set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls); + set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls); + } + if ((is_rx && ops->vidioc_s_fmt_vid_overlay) || + (is_tx && ops->vidioc_s_fmt_vid_out_overlay)) + set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); if ((is_rx && (ops->vidioc_s_fmt_vid_cap || ops->vidioc_s_fmt_vid_cap_mplane || - ops->vidioc_s_fmt_vid_overlay)) || + ops->vidioc_s_ext_pix_fmt_vid_cap)) || (is_tx && (ops->vidioc_s_fmt_vid_out || ops->vidioc_s_fmt_vid_out_mplane || - ops->vidioc_s_fmt_vid_out_overlay))) + ops->vidioc_s_ext_pix_fmt_vid_out))) { set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls); + set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls); + } + if ((is_rx && ops->vidioc_try_fmt_vid_overlay) || + (is_tx && ops->vidioc_try_fmt_vid_out_overlay)) + set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); if ((is_rx && (ops->vidioc_try_fmt_vid_cap || ops->vidioc_try_fmt_vid_cap_mplane || - ops->vidioc_try_fmt_vid_overlay)) || + ops->vidioc_try_ext_pix_fmt_vid_cap)) || (is_tx && (ops->vidioc_try_fmt_vid_out || ops->vidioc_try_fmt_vid_out_mplane || - ops->vidioc_try_fmt_vid_out_overlay))) + ops->vidioc_try_ext_pix_fmt_vid_out))) { set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls); + set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls); + } SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay); SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf); SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 848286a284f6..a9c07c0a73ec 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -18,6 +18,8 @@ #include <linux/videodev2.h> +#include <drm/drm_fourcc.h> + #include <media/v4l2-common.h> #include <media/v4l2-ioctl.h> #include <media/v4l2-ctrls.h> @@ -38,6 +40,11 @@ #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls) +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \ + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \ + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \ + V4L2_CAP_VIDEO_M2M_MPLANE)) + struct std_descr { v4l2_std_id std; const char *descr; @@ -379,6 +386,27 @@ static void v4l_print_format(const void *arg, bool write_only) } } +static void v4l_print_ext_pix_format(const void *arg, bool write_only) +{ + const struct v4l2_ext_pix_format *ef = arg; + unsigned int i; + + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n", + prt_names(ef->type, v4l2_type_names), + ef->width, ef->height, + (ef->pixelformat & 0xff), + (ef->pixelformat >> 8) & 0xff, + (ef->pixelformat >> 16) & 0xff, + (ef->pixelformat >> 24) & 0xff, + ef->modifier, prt_names(ef->field, v4l2_field_names), + ef->colorspace, ef->ycbcr_enc, + ef->quantization, ef->xfer_func); + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n", + i, ef->plane_fmt[i].bytesperline, + ef->plane_fmt[i].sizeimage); +} + static void v4l_print_framebuffer(const void *arg, bool write_only) { const struct v4l2_framebuffer *p = arg; @@ -963,11 +991,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) switch (type) { case V4L2_BUF_TYPE_VIDEO_CAPTURE: if ((is_vid || is_tch) && is_rx && - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane)) + (ops->vidioc_g_fmt_vid_cap || + ops->vidioc_g_ext_pix_fmt_vid_cap || + ops->vidioc_g_fmt_vid_cap_mplane)) return 0; break; case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: - if ((is_vid || is_tch) && is_rx && ops->vidioc_g_fmt_vid_cap_mplane) + if ((is_vid || is_tch) && is_rx && + (ops->vidioc_g_fmt_vid_cap_mplane || + ops->vidioc_g_ext_pix_fmt_vid_cap)) return 0; break; case V4L2_BUF_TYPE_VIDEO_OVERLAY: @@ -976,11 +1008,15 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) break; case V4L2_BUF_TYPE_VIDEO_OUTPUT: if (is_vid && is_tx && - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane)) + (ops->vidioc_g_fmt_vid_out || + ops->vidioc_g_ext_pix_fmt_vid_out || + ops->vidioc_g_fmt_vid_out_mplane)) return 0; break; case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: - if (is_vid && is_tx && ops->vidioc_g_fmt_vid_out_mplane) + if (is_vid && is_tx && + (ops->vidioc_g_ext_pix_fmt_vid_out || + ops->vidioc_g_fmt_vid_out_mplane)) return 0; break; case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: @@ -1060,6 +1096,204 @@ static void v4l_sanitize_format(struct v4l2_format *fmt) sizeof(fmt->fmt.pix) - offset); } +static void +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef, + struct v4l2_pix_format *pix) +{ + unsigned int i; + + pix->width = ef->width; + pix->height = ef->height; + pix->field = ef->field; + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC; + pix->colorspace = ef->colorspace; + pix->ycbcr_enc = ef->ycbcr_enc; + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC; + pix->quantization = ef->quantization; + pix->pixelformat = ef->pixelformat; + pix->bytesperline = ef->plane_fmt[0].bytesperline; + pix->sizeimage = ef->plane_fmt[0].sizeimage; + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) + pix->sizeimage += ef->plane_fmt[i].sizeimage; +} + +static void +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef, + struct v4l2_pix_format_mplane *pix_mp) +{ + const struct v4l2_format_info *info = + v4l2_format_info(ef->pixelformat); + unsigned int i; + + pix_mp->width = ef->width; + pix_mp->height = ef->height; + pix_mp->field = ef->field; + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC; + pix_mp->colorspace = ef->colorspace; + pix_mp->ycbcr_enc = ef->ycbcr_enc; + pix_mp->quantization = ef->quantization; + pix_mp->pixelformat = ef->pixelformat; + + /* This is true when converting to non-M-variant */ + if (info && info->mem_planes == 1) { + pix_mp->plane_fmt[0] = ef->plane_fmt[0]; + pix_mp->num_planes = 1; + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage; + + return; + } + + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++) + pix_mp->plane_fmt[i] = ef->plane_fmt[i]; + pix_mp->num_planes = i; +} + +/* + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format + * + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted. + * @f: A pointer to struct v4l2_format to be filled. + * @is_mplane: Bool indicating if multiplanar API should be used in @f. + * + * If pixelformat should be converted to M-variant, change ef->pixelformat + * to the M-variant before calling this function. + */ +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef, + struct v4l2_format *f, bool is_mplane) +{ + memset(f, 0, sizeof(*f)); + + if (ef->modifier != DRM_FORMAT_MOD_LINEAR && + ef->modifier != DRM_FORMAT_MOD_INVALID) + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n", + ef->modifier); + + if (!is_mplane) { + f->type = ef->type; + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix); + return; + } + + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + else + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp); +} + +static void +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix, + struct v4l2_ext_pix_format *ef) +{ + const struct v4l2_format_info *info = + v4l2_format_info(pix->pixelformat); + unsigned int i; + + ef->width = pix->width; + ef->height = pix->height; + ef->field = pix->field; + ef->colorspace = pix->colorspace; + ef->ycbcr_enc = pix->ycbcr_enc; + ef->quantization = pix->quantization; + ef->xfer_func = pix->xfer_func; + if (pix->flags) + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags); + + /* We assume M-variants won't be used in this function */ + ef->pixelformat = pix->pixelformat; + + ef->plane_fmt[0].bytesperline = pix->bytesperline; + ef->plane_fmt[0].sizeimage = pix->sizeimage; + + if (!info) + return; + + for (i = 1; i < info->comp_planes; i++) { + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv; + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline * + ef->height / info->vdiv; + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; + } +} + +static void +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp, + struct v4l2_ext_pix_format *ef) +{ + const struct v4l2_format_info *info = + v4l2_format_info(pix_mp->pixelformat); + unsigned int i; + + ef->width = pix_mp->width; + ef->height = pix_mp->height; + ef->field = pix_mp->field; + ef->colorspace = pix_mp->colorspace; + ef->ycbcr_enc = pix_mp->ycbcr_enc; + ef->quantization = pix_mp->quantization; + ef->xfer_func = pix_mp->xfer_func; + if (pix_mp->flags) + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags); + + if (!info) + return; + + ef->pixelformat = info && info->norm ? + info->norm : pix_mp->pixelformat; + + if (info->comp_planes == info->mem_planes) { + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++) + ef->plane_fmt[i] = pix_mp->plane_fmt[i]; + + return; + } + + /* case where mem_planes is 1 and comp_planes > 1 */ + ef->plane_fmt[0] = pix_mp->plane_fmt[0]; + for (i = 1; i < info->comp_planes; i++) { + ef->plane_fmt[i].bytesperline = + pix_mp->plane_fmt[0].bytesperline / info->hdiv; + ef->plane_fmt[i].sizeimage = + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv; + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage; + } +} + +/* + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format + * + * @f: A pointer to struct v4l2_format to be converted. + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled. + * + * This method normalize the pixelformat to non-M variant. + */ +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f, + struct v4l2_ext_pix_format *ef) +{ + memset(ef, 0, sizeof(*ef)); + + switch (f->type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + ef->type = f->type; + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef); + break; + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); + break; + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT; + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef); + break; + default: + WARN("Converting to Ext Pix Format with wrong buffer type %s\n", + prt_names(f->type, v4l2_type_names)); + break; + } +} + static int v4l_querycap(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -1565,6 +1799,100 @@ static void v4l_pix_format_touch(struct v4l2_pix_format *p) p->xfer_func = 0; } +static int v4l_fmt_ioctl_via_ext(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, + struct v4l2_format *f, + unsigned int ioctl) +{ + bool is_multiplanar = V4L2_TYPE_IS_MULTIPLANAR(f->type); + struct video_device *vdev = video_devdata(file); + struct v4l2_ext_pix_format ef = {0}; + u32 original_pixfmt = 0; + u32 cap_mask; + int ret; + + if (ioctl != VIDIOC_G_FMT) { + /* + * If CSC attributes are read only, set them to DEFAULT + * to avoid changes by the driver. + */ + if (is_multiplanar) { + if (!(f->fmt.pix_mp.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; + } + /* Unset the flag to avoid warning in the convertion */ + f->fmt.pix_mp.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; + + /* Save pixelformat in case M-variant is being used */ + original_pixfmt = f->fmt.pix_mp.pixelformat; + } else { + if (!(f->fmt.pix.flags & V4L2_PIX_FMT_FLAG_SET_CSC)) { + f->fmt.pix.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix.quantization = V4L2_QUANTIZATION_DEFAULT; + f->fmt.pix.xfer_func = V4L2_XFER_FUNC_DEFAULT; + } + /* Unset the flag to avoid warning in the convertion */ + f->fmt.pix.flags &= ~V4L2_PIX_FMT_FLAG_SET_CSC; + } + v4l2_format_to_ext_pix_format(f, &ef); + } + + switch (f->type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + cap_mask = V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_M2M_MPLANE; + if (!!(vdev->device_caps & cap_mask) != + (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) + return -EINVAL; + + ef.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + if (ioctl == VIDIOC_G_FMT) + ret = ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, &ef); + else if (ioctl == VIDIOC_S_FMT) + ret = ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, &ef); + else + ret = ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, + &ef); + break; + + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + cap_mask = V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_M2M_MPLANE; + if (!!(vdev->device_caps & cap_mask) != + (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)) + return -EINVAL; + + ef.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; + if (ioctl == VIDIOC_G_FMT) + ret = ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, &ef); + else if (ioctl == VIDIOC_S_FMT) + ret = ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, &ef); + else + ret = ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, + &ef); + break; + + default: + return -EINVAL; + } + + if (ret) + return ret; + + if (original_pixfmt != ef.pixelformat && + v4l2_format_info(original_pixfmt)) + ef.pixelformat = original_pixfmt; + + v4l2_ext_pix_format_to_format(&ef, f, is_multiplanar); + return 0; +} + static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -1601,17 +1929,26 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, switch (p->type) { case V4L2_BUF_TYPE_VIDEO_CAPTURE: - if (unlikely(!ops->vidioc_g_fmt_vid_cap)) + if (unlikely(!ops->vidioc_g_fmt_vid_cap && + !ops->vidioc_g_ext_pix_fmt_vid_cap)) break; p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; - ret = ops->vidioc_g_fmt_vid_cap(file, fh, arg); + ret = ops->vidioc_g_fmt_vid_cap ? + ops->vidioc_g_fmt_vid_cap(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, + VIDIOC_G_FMT); /* just in case the driver zeroed it again */ p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; if (vfd->vfl_type == VFL_TYPE_TOUCH) v4l_pix_format_touch(&p->fmt.pix); return ret; case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); + if (ops->vidioc_g_fmt_vid_cap_mplane) + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg); + else if (ops->vidioc_g_ext_pix_fmt_vid_cap) + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, + VIDIOC_G_FMT); + break; case V4L2_BUF_TYPE_VIDEO_OVERLAY: return ops->vidioc_g_fmt_vid_overlay(file, fh, arg); case V4L2_BUF_TYPE_VBI_CAPTURE: @@ -1619,15 +1956,23 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE: return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg); case V4L2_BUF_TYPE_VIDEO_OUTPUT: - if (unlikely(!ops->vidioc_g_fmt_vid_out)) + if (unlikely(!ops->vidioc_g_fmt_vid_out && + !ops->vidioc_g_ext_pix_fmt_vid_out)) break; p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; - ret = ops->vidioc_g_fmt_vid_out(file, fh, arg); + ret = ops->vidioc_g_fmt_vid_out ? + ops->vidioc_g_fmt_vid_out(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_G_FMT); /* just in case the driver zeroed it again */ p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; return ret; case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); + if (ops->vidioc_g_fmt_vid_out_mplane) + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg); + else if (ops->vidioc_g_ext_pix_fmt_vid_out) + return v4l_fmt_ioctl_via_ext(ops, file, fh, arg, + VIDIOC_G_FMT); + break; case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg); case V4L2_BUF_TYPE_VBI_OUTPUT: @@ -1646,6 +1991,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, return -EINVAL; } +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct v4l2_ext_pix_format *ef = arg; + struct v4l2_format f = { + .type = ef->type, + }; + int ret = check_fmt(file, ef->type); + + if (ret) + return ret; + + memset(ef, 0, sizeof(*ef)); + ef->type = f.type; + + switch (f.type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + if (ops->vidioc_g_ext_pix_fmt_vid_cap) + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef); + break; + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + if (ops->vidioc_g_ext_pix_fmt_vid_out) + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef); + break; + default: + return -EINVAL; + } + + ret = v4l_g_fmt(ops, file, fh, &f); + if (ret) + return ret; + + v4l2_format_to_ext_pix_format(&f, ef); + return 0; +} + static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -1664,23 +2045,29 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, switch (p->type) { case V4L2_BUF_TYPE_VIDEO_CAPTURE: - if (unlikely(!ops->vidioc_s_fmt_vid_cap)) + if (unlikely(!ops->vidioc_s_fmt_vid_cap && + !ops->vidioc_s_ext_pix_fmt_vid_cap)) break; CLEAR_AFTER_FIELD(p, fmt.pix); - ret = ops->vidioc_s_fmt_vid_cap(file, fh, arg); + ret = ops->vidioc_s_fmt_vid_cap ? + ops->vidioc_s_fmt_vid_cap(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); /* just in case the driver zeroed it again */ p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; if (vfd->vfl_type == VFL_TYPE_TOUCH) v4l_pix_format_touch(&p->fmt.pix); return ret; case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: - if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane)) + if (unlikely(!ops->vidioc_s_fmt_vid_cap_mplane && + !ops->vidioc_s_ext_pix_fmt_vid_cap)) break; CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); for (i = 0; i < p->fmt.pix_mp.num_planes; i++) CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], bytesperline); - return ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg); + return ops->vidioc_s_fmt_vid_cap_mplane ? + ops->vidioc_s_fmt_vid_cap_mplane(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); case V4L2_BUF_TYPE_VIDEO_OVERLAY: if (unlikely(!ops->vidioc_s_fmt_vid_overlay)) break; @@ -1697,21 +2084,27 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); return ops->vidioc_s_fmt_sliced_vbi_cap(file, fh, arg); case V4L2_BUF_TYPE_VIDEO_OUTPUT: - if (unlikely(!ops->vidioc_s_fmt_vid_out)) + if (unlikely(!ops->vidioc_s_fmt_vid_out && + !ops->vidioc_s_ext_pix_fmt_vid_out)) break; CLEAR_AFTER_FIELD(p, fmt.pix); - ret = ops->vidioc_s_fmt_vid_out(file, fh, arg); + ret = ops->vidioc_s_fmt_vid_out ? + ops->vidioc_s_fmt_vid_out(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); /* just in case the driver zeroed it again */ p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; return ret; case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: - if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane)) + if (unlikely(!ops->vidioc_s_fmt_vid_out_mplane && + !ops->vidioc_s_ext_pix_fmt_vid_out)) break; CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); for (i = 0; i < p->fmt.pix_mp.num_planes; i++) CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], bytesperline); - return ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg); + return ops->vidioc_s_fmt_vid_out_mplane ? + ops->vidioc_s_fmt_vid_out_mplane(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_S_FMT); case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: if (unlikely(!ops->vidioc_s_fmt_vid_out_overlay)) break; @@ -1751,6 +2144,43 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, return -EINVAL; } +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct video_device *vfd = video_devdata(file); + struct v4l2_ext_pix_format *ef = arg; + struct v4l2_format f; + int ret = check_fmt(file, ef->type); + + if (ret) + return ret; + + memset(ef->reserved, 0, sizeof(ef->reserved)); + + switch (ef->type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + if (ops->vidioc_s_ext_pix_fmt_vid_cap) + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef); + break; + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + if (ops->vidioc_s_ext_pix_fmt_vid_out) + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef); + break; + default: + return -EINVAL; + } + + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); + + ret = v4l_s_fmt(ops, file, fh, &f); + if (ret) + /* TODO: retry with M-variant of ef->pixelformat? */ + return ret; + + v4l2_format_to_ext_pix_format(&f, ef); + return 0; +} + static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -1766,23 +2196,30 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, switch (p->type) { case V4L2_BUF_TYPE_VIDEO_CAPTURE: - if (unlikely(!ops->vidioc_try_fmt_vid_cap)) + if (unlikely(!ops->vidioc_try_fmt_vid_cap && + !ops->vidioc_try_ext_pix_fmt_vid_cap)) break; CLEAR_AFTER_FIELD(p, fmt.pix); - ret = ops->vidioc_try_fmt_vid_cap(file, fh, arg); + ret = ops->vidioc_try_fmt_vid_cap ? + ops->vidioc_try_fmt_vid_cap(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); /* just in case the driver zeroed it again */ p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; if (vfd->vfl_type == VFL_TYPE_TOUCH) v4l_pix_format_touch(&p->fmt.pix); return ret; case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: - if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane)) + if (unlikely(!ops->vidioc_try_fmt_vid_cap_mplane && + !ops->vidioc_try_ext_pix_fmt_vid_cap)) break; CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); for (i = 0; i < p->fmt.pix_mp.num_planes; i++) CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], bytesperline); - return ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg); + return ops->vidioc_try_fmt_vid_cap_mplane ? + ops->vidioc_try_fmt_vid_cap_mplane(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, + VIDIOC_TRY_FMT); case V4L2_BUF_TYPE_VIDEO_OVERLAY: if (unlikely(!ops->vidioc_try_fmt_vid_overlay)) break; @@ -1799,21 +2236,28 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, CLEAR_AFTER_FIELD(p, fmt.sliced.io_size); return ops->vidioc_try_fmt_sliced_vbi_cap(file, fh, arg); case V4L2_BUF_TYPE_VIDEO_OUTPUT: - if (unlikely(!ops->vidioc_try_fmt_vid_out)) + if (unlikely(!ops->vidioc_try_fmt_vid_out && + !ops->vidioc_try_ext_pix_fmt_vid_cap)) break; CLEAR_AFTER_FIELD(p, fmt.pix); - ret = ops->vidioc_try_fmt_vid_out(file, fh, arg); + ret = ops->vidioc_try_fmt_vid_out ? + ops->vidioc_try_fmt_vid_out(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, VIDIOC_TRY_FMT); /* just in case the driver zeroed it again */ p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; return ret; case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: - if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane)) + if (unlikely(!ops->vidioc_try_fmt_vid_out_mplane && + !ops->vidioc_try_ext_pix_fmt_vid_cap)) break; CLEAR_AFTER_FIELD(p, fmt.pix_mp.xfer_func); for (i = 0; i < p->fmt.pix_mp.num_planes; i++) CLEAR_AFTER_FIELD(&p->fmt.pix_mp.plane_fmt[i], bytesperline); - return ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg); + return ops->vidioc_try_fmt_vid_out_mplane ? + ops->vidioc_try_fmt_vid_out_mplane(file, fh, arg) : + v4l_fmt_ioctl_via_ext(ops, file, fh, arg, + VIDIOC_TRY_FMT); case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY: if (unlikely(!ops->vidioc_try_fmt_vid_out_overlay)) break; @@ -1853,6 +2297,45 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, return -EINVAL; } +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct video_device *vfd = video_devdata(file); + struct v4l2_ext_pix_format *ef = arg; + struct v4l2_format f; + int ret = check_fmt(file, ef->type); + + if (ret) + return ret; + + memset(ef->reserved, 0, sizeof(ef->reserved)); + + switch (ef->type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + if (ops->vidioc_try_ext_pix_fmt_vid_cap) + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh, + ef); + break; + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + if (ops->vidioc_try_ext_pix_fmt_vid_out) + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh, + ef); + break; + default: + return -EINVAL; + } + + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd)); + + ret = v4l_try_fmt(ops, file, fh, &f); + if (ret) + /* TODO: retry with M-variant of ef->pixelformat? */ + return ret; + + v4l2_format_to_ext_pix_format(&f, ef); + return 0; +} + static int v4l_streamon(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -2854,6 +3337,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0), IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)), IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)), + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0), + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO), + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0), }; #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls) diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index edb733f21604..c44708dc9355 100644 --- a/include/media/v4l2-ioctl.h +++ b/include/media/v4l2-ioctl.h @@ -48,11 +48,17 @@ struct v4l2_fh; * @vidioc_g_fmt_vid_cap: pointer to the function that implements * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture * in single plane mode + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video + * capture * @vidioc_g_fmt_vid_overlay: pointer to the function that implements * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay * @vidioc_g_fmt_vid_out: pointer to the function that implements * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out * in single plane mode + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video + * out * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output * @vidioc_g_fmt_vbi_cap: pointer to the function that implements @@ -82,11 +88,16 @@ struct v4l2_fh; * @vidioc_s_fmt_vid_cap: pointer to the function that implements * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture * in single plane mode + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video + * capture * @vidioc_s_fmt_vid_overlay: pointer to the function that implements * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay * @vidioc_s_fmt_vid_out: pointer to the function that implements * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out * in single plane mode + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output * @vidioc_s_fmt_vbi_cap: pointer to the function that implements @@ -116,11 +127,16 @@ struct v4l2_fh; * @vidioc_try_fmt_vid_cap: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture * in single plane mode + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for + video capture * @vidioc_try_fmt_vid_overlay: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay * @vidioc_try_fmt_vid_out: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out * in single plane mode + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay * output @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops { /* VIDIOC_G_FMT handlers */ int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh, + struct v4l2_ext_pix_format *ef); int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh, struct v4l2_format *f); int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh, + struct v4l2_ext_pix_format *ef); int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh, struct v4l2_format *f); int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh, @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops { /* VIDIOC_S_FMT handlers */ int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh, + struct v4l2_ext_pix_format *ef); int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh, struct v4l2_format *f); int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh, + struct v4l2_ext_pix_format *ef); int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh, struct v4l2_format *f); int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh, @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops { /* VIDIOC_TRY_FMT handlers */ int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh, + struct v4l2_ext_pix_format *ef); int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh, struct v4l2_format *f); int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh, + struct v4l2_ext_pix_format *ef); int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh, struct v4l2_format *f); int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh, diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index d9b7c9177605..a2d850513708 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -2270,6 +2270,43 @@ struct v4l2_pix_format_mplane { __u8 reserved[7]; } __attribute__ ((packed)); +/** + * struct v4l2_ext_pix_format - extended single/multiplanar format definition + * @type: type of the data stream; V4L2_BUF_TYPE_VIDEO_CAPTURE or + * V4L2_BUF_TYPE_VIDEO_OUTPUT + * @width: image width in pixels + * @height: image height in pixels + * @field: enum v4l2_field; field order (for interlaced video) + * @plane_fmt: per-plane information + * @pixelformat: little endian four character code (fourcc) + * @modifier: modifier applied to the format (used for tiled formats + * and other kind of HW-specific formats, like compressed + * formats) as defined in drm_fourcc.h + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding + * @quantization: enum v4l2_quantization, colorspace quantization + * @reserved: extra space reserved for future fields, must be set to 0 + */ +struct v4l2_ext_pix_format { + __u32 type; + __u32 width; + __u32 height; + __u32 field; + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES]; + __u32 pixelformat; + __u64 modifier; + __u32 colorspace; + __u32 xfer_func; + union { + __u32 ycbcr_enc; + __u32 hsv_enc; + }; + __u32 quantization; + __u32 reserved[9]; +}; + /** * struct v4l2_sdr_format - SDR format definition * @pixelformat: little endian four character code (fourcc) @@ -2583,6 +2620,10 @@ struct v4l2_create_buffers { #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format) +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format) +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format) + /* Reminder: when adding new ioctls please add support for them to drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
This is part of the multiplanar and singleplanar unification process. v4l2_ext_pix_format is supposed to work for both cases. We also add the concept of modifiers already employed in DRM to expose HW-specific formats (like tiled or compressed formats) and allow exchanging this information with the DRM subsystem in a consistent way. Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in v4l2_ext_format, other types will be rejected if you use the {G,S,TRY}_EXT_PIX_FMT ioctls. New hooks have been added to v4l2_ioctl_ops to support those new ioctls in drivers, but, in the meantime, the core takes care of converting {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can still work if the userspace app/lib uses the new ioctls. The conversion is also done the other around to allow userspace apps/libs using {S,G,TRY}_FMT to work with drivers implementing the _ext_ hooks. Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> Signed-off-by: Helen Koike <helen.koike@collabora.com> --- Changes in v6: The main change here was fixing the conversion, so planes reflects color planes, and to implement this properly I made major refactors compared to the previous version. - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz) - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans) - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans) - do not set Ext ioctls as valid for vid_out_overlay (Tomasz) - refactor conversion functions, so planes are color planes (Tomasz) - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz) - Use "ef" for extended formats in the framework for consistency (Tomasz) - Handle xfer_func field in conversions (Tomasz) - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz) - Refactor format functions to use v4l_fmt_ioctl_via_ext() - Several fixes/refactoring/changes - Remove EXT API for touch devices Changes in v5: - change sizes and reorder fields to avoid holes in the struct and make it the same for 32 and 64 bits - removed __attribute__ ((packed)) from uapi structs - Fix doc warning from make htmldocs - Updated commit message with EXT_PIX prefix for the ioctls. Changes in v4: - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format, making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types. - Add reserved fields - Removed num_planes from struct v4l2_ext_pix_format - Removed flag field from struct v4l2_ext_pix_format, since the only defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1, where we can use modifiers, or add it back later through the reserved bits. - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR && != MOD_INVALID - Fix type assignment in v4l_g_fmt_ext_pix() - Rebased on top of media/master (post 5.8-rc1) Changes in v3: - Rebased on top of media/master (post 5.4-rc1) Changes in v2: - Move the modifier in v4l2_ext_format (was formerly placed in v4l2_ext_plane) - Fix a few bugs in the converters and add a strict parameter to allow conversion of uninitialized/mis-initialized objects --- drivers/media/v4l2-core/v4l2-dev.c | 27 +- drivers/media/v4l2-core/v4l2-ioctl.c | 538 +++++++++++++++++++++++++-- include/media/v4l2-ioctl.h | 28 ++ include/uapi/linux/videodev2.h | 41 ++ 4 files changed, 602 insertions(+), 32 deletions(-)