[v3,10/11] media: v4l2: Add DELETE_BUF ioctl

Commit Message

Benjamin Gaignard June 22, 2023, 1:13 p.m. UTC

VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.

Signed-off-by: Benjamin Gaignard <[e-mail]>
---
 .../userspace-api/media/v4l/user-func.rst     |  1 +
 .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
 .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
 .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
 drivers/media/v4l2-core/v4l2-dev.c            |  1 +
 drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
 include/media/v4l2-ioctl.h                    |  4 ++
 include/media/videobuf2-core.h                |  9 ++++
 include/media/videobuf2-v4l2.h                | 11 ++++
 include/uapi/linux/videodev2.h                |  2 +
 10 files changed, 128 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst

Comments

kernel test robot June 22, 2023, 11:12 p.m. UTC | #1

Hi Benjamin,

kernel test robot noticed the following build errors:

[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base:   git://linuxtv.org/media_tree.git master
patch link:    https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-buildonly-randconfig-r004-20230622 (https://download.01.org/0day-ci/archive/20230623/[e-mail]/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230623/[e-mail]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[e-mail]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[e-mail]/

All errors (new ones prefixed by >>, old ones prefixed by <<):

>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!

kernel test robot June 23, 2023, 12:25 a.m. UTC | #2

Hi Benjamin,

kernel test robot noticed the following build errors:

[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base:   git://linuxtv.org/media_tree.git master
patch link:    https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-randconfig-i012-20230622 (https://download.01.org/0day-ci/archive/20230623/[e-mail]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/[e-mail]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[e-mail]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[e-mail]/

All errors (new ones prefixed by >>, old ones prefixed by <<):

>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!

Ming Qian June 26, 2023, 7:08 a.m. UTC | #3

Hi Benjamin,

>-----Original Message-----
>From: Benjamin Gaignard <[e-mail]>
>Sent: 2023年6月22日 21:14
>To: [e-mail]; [e-mail]; [e-mail];
>Ming Qian <[e-mail]>; [e-mail];
>[e-mail]; [e-mail]; hverkuil-
>[e-mail]; [e-mail]
>Cc: [e-mail]; [e-mail]; linux-arm-
>[e-mail]; [e-mail]; linux-arm-
>[e-mail]; [e-mail]; linux-
>[e-mail]; [e-mail]; Benjamin Gaignard
><[e-mail]>
>Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
>Signed-off-by: Benjamin Gaignard <[e-mail]>
>---
> .../userspace-api/media/v4l/user-func.rst     |  1 +
> .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
> drivers/media/v4l2-core/v4l2-dev.c            |  1 +
> drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
> include/media/v4l2-ioctl.h                    |  4 ++
> include/media/videobuf2-core.h                |  9 ++++
> include/media/videobuf2-v4l2.h                | 11 ++++
> include/uapi/linux/videodev2.h                |  2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>buf.rst
>
>diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>b/Documentation/userspace-api/media/v4l/user-func.rst
>index 15ff0bf7bbe6..8c74016e12fd 100644
>--- a/Documentation/userspace-api/media/v4l/user-func.rst
>+++ b/Documentation/userspace-api/media/v4l/user-func.rst
>@@ -17,6 +17,7 @@ Function Reference
>     vidioc-dbg-g-chip-info
>     vidioc-dbg-g-register
>     vidioc-decoder-cmd
>+    vidioc-delete-buf
>     vidioc-dqevent
>     vidioc-dv-timings-cap
>     vidioc-encoder-cmd
>diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>new file mode 100644
>index 000000000000..0e7ce58f91bc
>--- /dev/null
>+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>@@ -0,0 +1,51 @@
>+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>+c:namespace:: V4L
>+
>+.. _VIDIOC_DELETE_BUF:
>+
>+************************
>+ioctl VIDIOC_DELETE_BUF
>+************************
>+
>+Name
>+====
>+
>+VIDIOC_DELETE_BUF - Delete a buffer from a queue
>+
>+Synopsis
>+========
>+
>+.. c:macro:: VIDIOC_DELETE_BUF
>+
>+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>+
>+Arguments
>+=========
>+
>+``fd``
>+    File descriptor returned by :c:func:`open()`.
>+
>+``argp``
>+    Pointer to struct :c:type:`v4l2_buffer`.
>+
>+Description
>+===========
>+
>+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>+delete a buffer from a queue.
>+
>+The struct :c:type:`v4l2_buffer` structure is specified in
>+:ref:`buffer`.
>+
>+Return Value
>+============
>+
>+On success 0 is returned, on error -1 and the ``errno`` variable is set
>+appropriately. The generic error codes are described at the
>+:ref:`Generic Error Codes <gen-errors>` chapter.
>+
>+EBUSY
>+    File I/O is in progress.
>+
>+EINVAL
>+    The buffer ``index`` doesn't exist in the queue.
>diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>b/drivers/media/common/videobuf2/videobuf2-core.c
>index 899783f67580..aa546c972c3d 100644
>--- a/drivers/media/common/videobuf2/videobuf2-core.c
>+++ b/drivers/media/common/videobuf2/videobuf2-core.c
>@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>unsigned int index, void *pb)  }
>EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>+       struct vb2_buffer *vb;
>+
>+       vb = vb2_get_buffer(q, index);
>+       if (!vb) {
>+               dprintk(q, 1, "invalid buffer index %d\n", index);
>+               return -EINVAL;
>+       }
>+
>+       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>+               dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>+               return -EINVAL;
>+       }
>+
>+       if (vb->planes[0].mem_priv)
>+               call_void_vb_qop(vb, buf_cleanup, vb);
>+
>+       /* Free MMAP buffers or release USERPTR buffers */
>+       if (q->memory == VB2_MEMORY_MMAP)
>+               __vb2_buf_mem_free(vb);
>+       else if (q->memory == VB2_MEMORY_DMABUF)
>+               __vb2_buf_dmabuf_put(vb);
>+       else
>+               __vb2_buf_userptr_put(vb);
>+
>+       vb2_queue_remove_buffer(q, vb);
>+       kfree(vb);

The num_buffers is not changed, Is that on purpose?
num_buffers doesn't mean the number of vb2 buffers,  but only decides the max index of allocated vb2 buffer?
Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?

And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.

>+
>+       dprintk(q, 2, "buffer %d deleted\n", index);
>+       return 0;
>+}
>+
> /*
>  * vb2_start_streaming() - Attempt to start streaming.
>  * @q:         videobuf2 queue
>diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>index 724135d41f7f..cea666c17b41 100644
>--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>media_device *mdev,  }  EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>+       return vb2_core_delete_buf(q, b->index); }
>+EXPORT_SYMBOL_GPL(vb2_delete_buf);
>+
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>{
>        unsigned requested_planes = 1;
>diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>core/v4l2-dev.c
>index f81279492682..80ace2e1e932 100644
>--- a/drivers/media/v4l2-core/v4l2-dev.c
>+++ b/drivers/media/v4l2-core/v4l2-dev.c
>@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>*vdev)
>                SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
>                SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>                SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>+               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>+ vidioc_delete_buf);
>        }
>
>        if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>index a858acea6547..1c737279d3ef 100644
>--- a/drivers/media/v4l2-core/v4l2-ioctl.c
>+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>v4l2_ioctl_ops *ops,
>        return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);  }
>
>+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>+                         struct file *file, void *fh, void *arg) {
>+       struct v4l2_buffer *b = arg;
>+       int ret = check_fmt(file, b->type);
>+
>+       return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>+
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>                                struct file *file, void *fh, void *arg)  { @@ -2905,6 +2914,7
>@@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>        IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>v4l_print_freq_band, 0),
>        IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>        IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>id)),
>+       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>+ INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
>diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>edb733f21604..2f232ed884c7 100644
>--- a/include/media/v4l2-ioctl.h
>+++ b/include/media/v4l2-ioctl.h
>@@ -163,6 +163,8 @@ struct v4l2_fh;
>  *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>  * @vidioc_prepare_buf: pointer to the function that implements
>  *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>+ * @vidioc_delete_buf: pointer to the function that implements
>+ *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>  * @vidioc_overlay: pointer to the function that implements
>  *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>  * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>@@ struct v4l2_ioctl_ops {
>                                  struct v4l2_create_buffers *b);
>        int (*vidioc_prepare_buf)(struct file *file, void *fh,
>                                  struct v4l2_buffer *b);
>+       int (*vidioc_delete_buf)(struct file *file, void *fh,
>+                                struct v4l2_buffer *b);
>
>        int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>        int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>080b783d608d..0f9e68f76b77 100644
>--- a/include/media/videobuf2-core.h
>+++ b/include/media/videobuf2-core.h
>@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>enum vb2_memory memory,
>  */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
>+/**
>+ * vb2_core_delete_buf() -
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @index:     id number of the buffer.
>+ *
>+ *  Return: returns zero on success; an error code otherwise.
>+ */
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>+
> /**
>  * vb2_core_qbuf() - Queue a buffer from userspace
>  *
>diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>index 88a7a565170e..3beeb4c735f0 100644
>--- a/include/media/videobuf2-v4l2.h
>+++ b/include/media/videobuf2-v4l2.h
>@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>v4l2_create_buffers *create);
>  */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>                    struct v4l2_buffer *b);
>+/**
>+ * vb2_delete_buf() - Delete the buffer from the queue
>+ *
>+ * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>+ * @b:         buffer structure passed from userspace to
>+ *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>+ *
>+ * The return values from this function are intended to be directly
>+returned
>+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>+ */
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
>  * vb2_qbuf() - Queue a buffer from userspace diff --git
>a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>aee75eb9e686..31bba1915642 100644
>--- a/include/uapi/linux/videodev2.h
>+++ b/include/uapi/linux/videodev2.h
>@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {  #define
>VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>+#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>+
>
> /* Reminder: when adding new ioctls please add support for them to
>    drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>--
>2.39.2

Benjamin Gaignard June 26, 2023, 7:48 a.m. UTC | #4

Le 26/06/2023 à 09:08, Ming Qian a écrit :
> Hi Benjamin,
>
>> -----Original Message-----
>> From: Benjamin Gaignard <[e-mail]>
>> Sent: 2023年6月22日 21:14
>> To: [e-mail]; [e-mail]; [e-mail];
>> Ming Qian <[e-mail]>; [e-mail];
>> [e-mail]; [e-mail]; hverkuil-
>> [e-mail]; [e-mail]
>> Cc: [e-mail]; [e-mail]; linux-arm-
>> [e-mail]; [e-mail]; linux-arm-
>> [e-mail]; [e-mail]; linux-
>> [e-mail]; [e-mail]; Benjamin Gaignard
>> <[e-mail]>
>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>
>> Caution: This is an external email. Please take care when clicking links or
>> opening attachments. When in doubt, report the message using the 'Report
>> this email' button
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>> ---
>> .../userspace-api/media/v4l/user-func.rst     |  1 +
>> .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>> drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>> include/media/v4l2-ioctl.h                    |  4 ++
>> include/media/videobuf2-core.h                |  9 ++++
>> include/media/videobuf2-v4l2.h                | 11 ++++
>> include/uapi/linux/videodev2.h                |  2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>> buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>>      vidioc-dbg-g-chip-info
>>      vidioc-dbg-g-register
>>      vidioc-decoder-cmd
>> +    vidioc-delete-buf
>>      vidioc-dqevent
>>      vidioc-dv-timings-cap
>>      vidioc-encoder-cmd
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>> +c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> +    File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> +    Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> +    File I/O is in progress.
>> +
>> +EINVAL
>> +    The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb)  }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>> +       struct vb2_buffer *vb;
>> +
>> +       vb = vb2_get_buffer(q, index);
>> +       if (!vb) {
>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>> +               return -EINVAL;
>> +       }
>> +
>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> +               dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>> +               return -EINVAL;
>> +       }
>> +
>> +       if (vb->planes[0].mem_priv)
>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> +       /* Free MMAP buffers or release USERPTR buffers */
>> +       if (q->memory == VB2_MEMORY_MMAP)
>> +               __vb2_buf_mem_free(vb);
>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>> +               __vb2_buf_dmabuf_put(vb);
>> +       else
>> +               __vb2_buf_userptr_put(vb);
>> +
>> +       vb2_queue_remove_buffer(q, vb);
>> +       kfree(vb);
> The num_buffers is not changed, Is that on purpose?
> num_buffers doesn't mean the number of vb2 buffers,  but only decides the max index of allocated vb2 buffer?
> Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?

Yes num_buffers isn't change and still represent the highest index of allocated buffer.
DELETE_BUF remove the buffer from the list and free the memory.
That could a create a hole in vb2 queue, that why the first patch of this series change
all places where vb2 queue array was used by calls to helpers function and also check the return value.
num_buffers is used as the lowest possible value when finding the free indexes for buffers (see vb2_queue_add_buffer())
this way I can guaranty that the indexes a continuous which is a requirement for create_bufs.
I choose this solution because XArray API doesn't offer way to find continuous free range.
It doesn't seem impossible to add it but this series is already big enough from my point of view.

Regards,
Benjamin

>
> And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>
>> +
>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>> +       return 0;
>> +}
>> +
>> /*
>>   * vb2_start_streaming() - Attempt to start streaming.
>>   * @q:         videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev,  }  EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>> +       return vb2_core_delete_buf(q, b->index); }
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>> {
>>         unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>> core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>>                 SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> + vidioc_delete_buf);
>>         }
>>
>>         if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>>         return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);  }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> +                         struct file *file, void *fh, void *arg) {
>> +       struct v4l2_buffer *b = arg;
>> +       int ret = check_fmt(file, b->type);
>> +
>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>                                 struct file *file, void *fh, void *arg)  { @@ -2905,6 +2914,7
>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>         IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>>         IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>         IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>> id)),
>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>> + INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>> edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>   *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>   * @vidioc_prepare_buf: pointer to the function that implements
>>   *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>   * @vidioc_overlay: pointer to the function that implements
>>   *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>   * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>> @@ struct v4l2_ioctl_ops {
>>                                   struct v4l2_create_buffers *b);
>>         int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>                                   struct v4l2_buffer *b);
>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>> +                                struct v4l2_buffer *b);
>>
>>         int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>>         int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>> 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>>   */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index:     id number of the buffer.
>> + *
>> + *  Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>>   * vb2_core_qbuf() - Queue a buffer from userspace
>>   *
>> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>>   */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>                     struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b:         buffer structure passed from userspace to
>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> +returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>>   * vb2_qbuf() - Queue a buffer from userspace diff --git
>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>> aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {  #define
>> VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>>     drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2

Benjamin Gaignard June 26, 2023, 7:50 a.m. UTC | #5

Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>
> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <[e-mail]>
>>> Sent: 2023年6月22日 21:14
>>> To: [e-mail]; [e-mail]; [e-mail];
>>> Ming Qian <[e-mail]>; [e-mail];
>>> [e-mail]; [e-mail]; hverkuil-
>>> [e-mail]; [e-mail]
>>> Cc: [e-mail]; [e-mail]; 
>>> linux-arm-
>>> [e-mail]; [e-mail]; 
>>> linux-arm-
>>> [e-mail]; [e-mail]; linux-
>>> [e-mail]; [e-mail]; Benjamin Gaignard
>>> <[e-mail]>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking 
>>> links or
>>> opening attachments. When in doubt, report the message using the 
>>> 'Report
>>> this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst     |  1 +
>>> .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>> include/media/v4l2-ioctl.h                    |  4 ++
>>> include/media/videobuf2-core.h                |  9 ++++
>>> include/media/videobuf2-v4l2.h                | 11 ++++
>>> include/uapi/linux/videodev2.h                |  2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>>      vidioc-dbg-g-chip-info
>>>      vidioc-dbg-g-register
>>>      vidioc-decoder-cmd
>>> +    vidioc-delete-buf
>>>      vidioc-dqevent
>>>      vidioc-dv-timings-cap
>>>      vidioc-encoder-cmd
>>> diff --git 
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> +    File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is 
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> +    File I/O is in progress.
>>> +
>>> +EINVAL
>>> +    The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb)  }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> +       struct vb2_buffer *vb;
>>> +
>>> +       vb = vb2_get_buffer(q, index);
>>> +       if (!vb) {
>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>> +               return -EINVAL;
>>> +       }
>>> +
>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> +               dprintk(q, 1, "can't delete non dequeued buffer 
>>> index %d\n", index);
>>> +               return -EINVAL;
>>> +       }
>>> +
>>> +       if (vb->planes[0].mem_priv)
>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>> +               __vb2_buf_mem_free(vb);
>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>> +               __vb2_buf_dmabuf_put(vb);
>>> +       else
>>> +               __vb2_buf_userptr_put(vb);
>>> +
>>> +       vb2_queue_remove_buffer(q, vb);
>>> +       kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers,  but only decides 
>> the max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free 
>> the vb2 queue? is it a hole in vb2 queue?
>
> Yes num_buffers isn't change and still represent the highest index of 
> allocated buffer.
> DELETE_BUF remove the buffer from the list and free the memory.
> That could a create a hole in vb2 queue, that why the first patch of 
> this series change
> all places where vb2 queue array was used by calls to helpers function 
> and also check the return value.
> num_buffers is used as the lowest possible value when finding the free 
> indexes for buffers (see vb2_queue_add_buffer())
> this way I can guaranty that the indexes a continuous which is a 
> requirement for create_bufs.
> I choose this solution because XArray API doesn't offer way to find 
> continuous free range.
> It doesn't seem impossible to add it but this series is already big 
> enough from my point of view.
>
> Regards,
> Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over 
>> the present buffers in vb2 queue.

Sorry I forgot this comment.
Where do you think I can improve the code ?

Thanks,
Benjamin

>>
>>> +
>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>> +       return 0;
>>> +}
>>> +
>>> /*
>>>   * vb2_start_streaming() - Attempt to start streaming.
>>>   * @q:         videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev,  }  EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> +       return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers 
>>> *create)
>>> {
>>>         unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct 
>>> video_device
>>> *vdev)
>>>                 SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, 
>>> vidioc_prepare_buf);
>>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, 
>>> vidioc_streamoff);
>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>>         }
>>>
>>>         if (is_vid || is_vbi || is_meta) { diff --git 
>>> a/drivers/media/v4l2-core/v4l2-
>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>>         return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);  }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> +                         struct file *file, void *fh, void *arg) {
>>> +       struct v4l2_buffer *b = arg;
>>> +       int ret = check_fmt(file, b->type);
>>> +
>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>                                 struct file *file, void *fh, void 
>>> *arg)  { @@ -2905,6 +2914,7
>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>>         IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>>         IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>         IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | 
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>>> + INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h 
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>   *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>   * @vidioc_prepare_buf: pointer to the function that implements
>>>   *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>   * @vidioc_overlay: pointer to the function that implements
>>>   *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>   * @vidioc_g_fbuf: pointer to the function that implements @@ 
>>> -422,6 +424,8
>>> @@ struct v4l2_ioctl_ops {
>>>                                   struct v4l2_create_buffers *b);
>>>         int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>                                   struct v4l2_buffer *b);
>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> +                                struct v4l2_buffer *b);
>>>
>>>         int (*vidioc_overlay)(struct file *file, void *fh, unsigned 
>>> int i);
>>>         int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>>   */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, 
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index:     id number of the buffer.
>>> + *
>>> + *  Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>>   * vb2_core_qbuf() - Queue a buffer from userspace
>>>   *
>>> diff --git a/include/media/videobuf2-v4l2.h 
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>>   */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>                     struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b:         buffer structure passed from userspace to
>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>>   * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {  #define
>>> VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct 
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>>     drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> -- 
>>> 2.39.2

Ming Qian June 26, 2023, 8:04 a.m. UTC | #6

>-----Original Message-----
>From: Benjamin Gaignard <[e-mail]>
>Sent: 2023年6月26日 15:49
>To: Ming Qian <[e-mail]>; [e-mail];
>[e-mail]; [e-mail];
>[e-mail]; [e-mail];
>[e-mail]; [e-mail];
>[e-mail]
>Cc: [e-mail]; [e-mail]; linux-arm-
>[e-mail]; [e-mail]; linux-arm-
>[e-mail]; [e-mail]; linux-
>[e-mail]; [e-mail]
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <[e-mail]>
>>> Sent: 2023年6月22日 21:14
>>> To: [e-mail]; [e-mail];
>[e-mail];
>>> Ming Qian <[e-mail]>; [e-mail];
>>> [e-mail]; [e-mail]; hverkuil-
>>> [e-mail]; [e-mail]
>>> Cc: [e-mail]; [e-mail];
>>> linux-arm- [e-mail];
>>> [e-mail]; linux-arm- [e-mail];
>>> [e-mail]; linux- [e-mail];
>>> [e-mail]; Benjamin Gaignard
>>> <[e-mail]>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or opening attachments. When in doubt, report the message using
>>> the 'Report this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst     |  1 +
>>> .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>> include/media/v4l2-ioctl.h                    |  4 ++
>>> include/media/videobuf2-core.h                |  9 ++++
>>> include/media/videobuf2-v4l2.h                | 11 ++++
>>> include/uapi/linux/videodev2.h                |  2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>>      vidioc-dbg-g-chip-info
>>>      vidioc-dbg-g-register
>>>      vidioc-decoder-cmd
>>> +    vidioc-delete-buf
>>>      vidioc-dqevent
>>>      vidioc-dv-timings-cap
>>>      vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> +    File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>> +to delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> +set appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> +    File I/O is in progress.
>>> +
>>> +EINVAL
>>> +    The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb)  }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> +       struct vb2_buffer *vb;
>>> +
>>> +       vb = vb2_get_buffer(q, index);
>>> +       if (!vb) {
>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>> +               return -EINVAL;
>>> +       }
>>> +
>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> +               dprintk(q, 1, "can't delete non dequeued buffer index %d\n",
>index);
>>> +               return -EINVAL;
>>> +       }
>>> +
>>> +       if (vb->planes[0].mem_priv)
>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>> +               __vb2_buf_mem_free(vb);
>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>> +               __vb2_buf_dmabuf_put(vb);
>>> +       else
>>> +               __vb2_buf_userptr_put(vb);
>>> +
>>> +       vb2_queue_remove_buffer(q, vb);
>>> +       kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers,  but only decides the
>max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free the
>vb2 queue? is it a hole in vb2 queue?
>
>Yes num_buffers isn't change and still represent the highest index of allocated
>buffer.
>DELETE_BUF remove the buffer from the list and free the memory.
>That could a create a hole in vb2 queue, that why the first patch of this series
>change all places where vb2 queue array was used by calls to helpers function
>and also check the return value.
>num_buffers is used as the lowest possible value when finding the free
>indexes for buffers (see vb2_queue_add_buffer()) this way I can guaranty that
>the indexes a continuous which is a requirement for create_bufs.
>I choose this solution because XArray API doesn't offer way to find continuous
>free range.
>It doesn't seem impossible to add it but this series is already big enough from
>my point of view.
>

Thank you for your detailed explanation.

>Regards,
>Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over the
>present buffers in vb2 queue.
>>
>>> +
>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>> +       return 0;
>>> +}
>>> +
>>> /*
>>>   * vb2_start_streaming() - Attempt to start streaming.
>>>   * @q:         videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev,  }  EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> +       return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create) {
>>>         unsigned requested_planes = 1; diff --git
>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>>                 SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>vidioc_prepare_buf);
>>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>>         }
>>>
>>>         if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2- ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>>         return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);  }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> +                         struct file *file, void *fh, void *arg) {
>>> +       struct v4l2_buffer *b = arg;
>>> +       int ret = check_fmt(file, b->type);
>>> +
>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>                                 struct file *file, void *fh, void
>>> *arg)  { @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>v4l2_ioctls[] = {
>>>         IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>>         IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>         IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>   *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>   * @vidioc_prepare_buf: pointer to the function that implements
>>>   *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>   * @vidioc_overlay: pointer to the function that implements
>>>   *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>   * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6
>>> +424,8 @@ struct v4l2_ioctl_ops {
>>>                                   struct v4l2_create_buffers *b);
>>>         int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>                                   struct v4l2_buffer *b);
>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> +                                struct v4l2_buffer *b);
>>>
>>>         int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>>>         int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>> index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>>   */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index:     id number of the buffer.
>>> + *
>>> + *  Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>>   * vb2_core_qbuf() - Queue a buffer from userspace
>>>   *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>> 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>>   */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>                     struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b:         buffer structure passed from userspace to
>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>>   * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {  #define
>>> VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>>     drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2

Ming Qian June 26, 2023, 8:13 a.m. UTC | #7

>From: Benjamin Gaignard <[e-mail]>
>Sent: 2023年6月26日 15:50
>To: Ming Qian <[e-mail]>; [e-mail];
>[e-mail]; [e-mail];
>[e-mail]; [e-mail];
>[e-mail]; [e-mail];
>[e-mail]
>Cc: [e-mail]; [e-mail]; linux-arm-
>[e-mail]; [e-mail]; linux-arm-
>[e-mail]; [e-mail]; linux-
>[e-mail]; [e-mail]
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>>
>> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>>> Hi Benjamin,
>>>
>>>> -----Original Message-----
>>>> From: Benjamin Gaignard <[e-mail]>
>>>> Sent: 2023年6月22日 21:14
>>>> To: [e-mail]; [e-mail];
>>>> [e-mail]; Ming Qian <[e-mail]>;
>>>> [e-mail]; [e-mail];
>>>> [e-mail]; hverkuil- [e-mail];
>>>> [e-mail]
>>>> Cc: [e-mail]; [e-mail];
>>>> linux-arm-
>>>> [e-mail]; [e-mail];
>>>> linux-arm-
>>>> [e-mail]; [e-mail]; linux-
>>>> [e-mail]; [e-mail]; Benjamin Gaignard
>>>> <[e-mail]>
>>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>>
>>>> Caution: This is an external email. Please take care when clicking
>>>> links or opening attachments. When in doubt, report the message
>>>> using the 'Report this email' button
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>> .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>> include/media/v4l2-ioctl.h                    |  4 ++
>>>> include/media/videobuf2-core.h                |  9 ++++
>>>> include/media/videobuf2-v4l2.h                | 11 ++++
>>>> include/uapi/linux/videodev2.h                |  2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>>> buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>      vidioc-dbg-g-chip-info
>>>>      vidioc-dbg-g-register
>>>>      vidioc-decoder-cmd
>>>> +    vidioc-delete-buf
>>>>      vidioc-dqevent
>>>>      vidioc-dv-timings-cap
>>>>      vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>>> +c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> +    File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>>> +to delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> +    File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>*q,
>>>> unsigned int index, void *pb)  }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>>> +       struct vb2_buffer *vb;
>>>> +
>>>> +       vb = vb2_get_buffer(q, index);
>>>> +       if (!vb) {
>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> +               return -EINVAL;
>>>> +       }
>>>> +
>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> +               dprintk(q, 1, "can't delete non dequeued buffer
>>>> index %d\n", index);
>>>> +               return -EINVAL;
>>>> +       }
>>>> +
>>>> +       if (vb->planes[0].mem_priv)
>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>> +               __vb2_buf_mem_free(vb);
>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>> +               __vb2_buf_dmabuf_put(vb);
>>>> +       else
>>>> +               __vb2_buf_userptr_put(vb);
>>>> +
>>>> +       vb2_queue_remove_buffer(q, vb);
>>>> +       kfree(vb);
>>> The num_buffers is not changed, Is that on purpose?
>>> num_buffers doesn't mean the number of vb2 buffers,  but only decides
>>> the max index of allocated vb2 buffer?
>>> Once the index is deleted, then it won't be used anymore, unless free
>>> the vb2 queue? is it a hole in vb2 queue?
>>
>> Yes num_buffers isn't change and still represent the highest index of
>> allocated buffer.
>> DELETE_BUF remove the buffer from the list and free the memory.
>> That could a create a hole in vb2 queue, that why the first patch of
>> this series change all places where vb2 queue array was used by calls
>> to helpers function and also check the return value.
>> num_buffers is used as the lowest possible value when finding the free
>> indexes for buffers (see vb2_queue_add_buffer()) this way I can
>> guaranty that the indexes a continuous which is a requirement for
>> create_bufs.
>> I choose this solution because XArray API doesn't offer way to find
>> continuous free range.
>> It doesn't seem impossible to add it but this series is already big
>> enough from my point of view.
>>
>> Regards,
>> Benjamin
>>
>>>
>>> And you can use xa_for_each() instead the for loop to iterate over
>>> the present buffers in vb2 queue.
>
>Sorry I forgot this comment.
>Where do you think I can improve the code ?

That's what I meant is the following code can use xa_for_each() directly.
Indeed the for loop also work well, so please ignore it.

for (buffer = 0; buffer < q->num_buffers; ++buffer) {
        struct vb2_buffer *vb = vb2_get_buffer(q, buffer);

>
>Thanks,
>Benjamin
>
>>>
>>>> +
>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>> +       return 0;
>>>> +}
>>>> +
>>>> /*
>>>>   * vb2_start_streaming() - Attempt to start streaming.
>>>>   * @q:         videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev,  }  EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>>> +       return vb2_core_delete_buf(q, b->index); }
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>>         unsigned requested_planes = 1; diff --git
>>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device
>>>> *vdev)
>>>>                 SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>>>                 SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> + vidioc_delete_buf);
>>>>         }
>>>>
>>>>         if (is_vid || is_vbi || is_meta) { diff --git
>>>> a/drivers/media/v4l2-core/v4l2-
>>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>>         return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);  }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> +                         struct file *file, void *fh, void *arg) {
>>>> +       struct v4l2_buffer *b = arg;
>>>> +       int ret = check_fmt(file, b->type);
>>>> +
>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>                                 struct file *file, void *fh, void
>>>> *arg)  { @@ -2905,6 +2914,7
>>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>>>         IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>>         IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>         IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>>> id)),
>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index
>>>> edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>   *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>   * @vidioc_prepare_buf: pointer to the function that implements
>>>>   *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>   * @vidioc_overlay: pointer to the function that implements
>>>>   *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>   * @vidioc_g_fbuf: pointer to the function that implements @@
>>>> -422,6 +424,8
>>>> @@ struct v4l2_ioctl_ops {
>>>>                                   struct v4l2_create_buffers *b);
>>>>         int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>                                   struct v4l2_buffer *b);
>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> +                                struct v4l2_buffer *b);
>>>>
>>>>         int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>>         int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>>> index
>>>> 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>>   */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index:     id number of the buffer.
>>>> + *
>>>> + *  Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>>   * vb2_core_qbuf() - Queue a buffer from userspace
>>>>   *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>>> 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>>   */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>                     struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b:         buffer structure passed from userspace to
>>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> +returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>>   * vb2_qbuf() - Queue a buffer from userspace diff --git
>>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index
>>>> aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {  #define
>>>> VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>     drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2

Hsia-Jun Li June 27, 2023, 7:30 a.m. UTC | #8

On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
> Signed-off-by: Benjamin Gaignard <[e-mail]>
> ---
>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>   .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>   include/media/v4l2-ioctl.h                    |  4 ++
>   include/media/videobuf2-core.h                |  9 ++++
>   include/media/videobuf2-v4l2.h                | 11 ++++
>   include/uapi/linux/videodev2.h                |  2 +
>   10 files changed, 128 insertions(+)
>   create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
> index 15ff0bf7bbe6..8c74016e12fd 100644
> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> @@ -17,6 +17,7 @@ Function Reference
>       vidioc-dbg-g-chip-info
>       vidioc-dbg-g-register
>       vidioc-decoder-cmd
> +    vidioc-delete-buf
>       vidioc-dqevent
>       vidioc-dv-timings-cap
>       vidioc-encoder-cmd
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> new file mode 100644
> index 000000000000..0e7ce58f91bc
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> @@ -0,0 +1,51 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +.. c:namespace:: V4L
> +
> +.. _VIDIOC_DELETE_BUF:
> +
> +************************
> +ioctl VIDIOC_DELETE_BUF
> +************************
> +
> +Name
> +====
> +
> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> +
> +Synopsis
> +========
> +
> +.. c:macro:: VIDIOC_DELETE_BUF
> +
> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> +
> +Arguments
> +=========
> +
> +``fd``
> +    File descriptor returned by :c:func:`open()`.
> +
> +``argp``
> +    Pointer to struct :c:type:`v4l2_buffer`.
> +
> +Description
> +===========
> +
> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
> +delete a buffer from a queue.
> +
> +The struct :c:type:`v4l2_buffer` structure is specified in
> +:ref:`buffer`.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +EBUSY
> +    File I/O is in progress.
> +
> +EINVAL
> +    The buffer ``index`` doesn't exist in the queue.
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 899783f67580..aa546c972c3d 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
>   }
>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> +{
> +       struct vb2_buffer *vb;
> +
> +       vb = vb2_get_buffer(q, index);
> +       if (!vb) {
> +               dprintk(q, 1, "invalid buffer index %d\n", index);
> +               return -EINVAL;
> +       }
> +
> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> +               dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
> +               return -EINVAL;
> +       }
> +
I know the driver could implement its own 
v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is used 
by the hardware as a future reference frame.
But I think we need a flag to let the user know which buffer is still 
used by the hardware.
Alternative ref case is safe, we only know it's existing when it is 
dequeued in current V4L2 buffer mechanism.
While the Golden reference frame, such long term reference frame could 
last much longer.
> +       if (vb->planes[0].mem_priv)
> +               call_void_vb_qop(vb, buf_cleanup, vb);
> +
> +       /* Free MMAP buffers or release USERPTR buffers */
> +       if (q->memory == VB2_MEMORY_MMAP)
> +               __vb2_buf_mem_free(vb);
> +       else if (q->memory == VB2_MEMORY_DMABUF)
> +               __vb2_buf_dmabuf_put(vb);
> +       else
> +               __vb2_buf_userptr_put(vb);
> +
> +       vb2_queue_remove_buffer(q, vb);
> +       kfree(vb);
> +
> +       dprintk(q, 2, "buffer %d deleted\n", index);
> +       return 0;
> +}
> +
>   /*
>    * vb2_start_streaming() - Attempt to start streaming.
>    * @q:         videobuf2 queue
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 724135d41f7f..cea666c17b41 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>   }
>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> +{
> +       return vb2_core_delete_buf(q, b->index);
> +}
> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> +
>   int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>   {
>          unsigned requested_planes = 1;
> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
> index f81279492682..80ace2e1e932 100644
> --- a/drivers/media/v4l2-core/v4l2-dev.c
> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
>          }
>
>          if (is_vid || is_vbi || is_meta) {
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index a858acea6547..1c737279d3ef 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>   }
>
> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> +                         struct file *file, void *fh, void *arg)
> +{
> +       struct v4l2_buffer *b = arg;
> +       int ret = check_fmt(file, b->type);
> +
> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> +}
> +
>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>                                  struct file *file, void *fh, void *arg)
>   {
> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
>   };
>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> index edb733f21604..2f232ed884c7 100644
> --- a/include/media/v4l2-ioctl.h
> +++ b/include/media/v4l2-ioctl.h
> @@ -163,6 +163,8 @@ struct v4l2_fh;
>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>    * @vidioc_prepare_buf: pointer to the function that implements
>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> + * @vidioc_delete_buf: pointer to the function that implements
> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>    * @vidioc_overlay: pointer to the function that implements
>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>    * @vidioc_g_fbuf: pointer to the function that implements
> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>                                    struct v4l2_create_buffers *b);
>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>                                    struct v4l2_buffer *b);
> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
> +                                struct v4l2_buffer *b);
>
>          int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 080b783d608d..0f9e68f76b77 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
>    */
>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
> +/**
> + * vb2_core_delete_buf() -
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @index:     id number of the buffer.
> + *
> + *  Return: returns zero on success; an error code otherwise.
> + */
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> +
>   /**
>    * vb2_core_qbuf() - Queue a buffer from userspace
>    *
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 88a7a565170e..3beeb4c735f0 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
>    */
>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>                      struct v4l2_buffer *b);
> +/**
> + * vb2_delete_buf() - Delete the buffer from the queue
> + *
> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
> + * @b:         buffer structure passed from userspace to
> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> + *
> + * The return values from this function are intended to be directly returned
> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> + */
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
>   /**
>    * vb2_qbuf() - Queue a buffer from userspace
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index aee75eb9e686..31bba1915642 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct v4l2_query_ext_ctrl)
> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
> +
>
>   /* Reminder: when adding new ioctls please add support for them to
>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> --
> 2.39.2
>

Benjamin Gaignard June 27, 2023, 8:43 a.m. UTC | #9

Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open 
>> attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>> ---
>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>   .../media/v4l/vidioc-delete-buf.rst           | 51 +++++++++++++++++++
>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>   include/media/v4l2-ioctl.h                    |  4 ++
>>   include/media/videobuf2-core.h                |  9 ++++
>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>   include/uapi/linux/videodev2.h                |  2 +
>>   10 files changed, 128 insertions(+)
>>   create mode 100644 
>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst 
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>>       vidioc-dbg-g-chip-info
>>       vidioc-dbg-g-register
>>       vidioc-decoder-cmd
>> +    vidioc-delete-buf
>>       vidioc-dqevent
>>       vidioc-dv-timings-cap
>>       vidioc-encoder-cmd
>> diff --git 
>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst 
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>> +.. c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> +    File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> +    Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> +    File I/O is in progress.
>> +
>> +EINVAL
>> +    The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, 
>> unsigned int index, void *pb)
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>> +{
>> +       struct vb2_buffer *vb;
>> +
>> +       vb = vb2_get_buffer(q, index);
>> +       if (!vb) {
>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>> +               return -EINVAL;
>> +       }
>> +
>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> +               dprintk(q, 1, "can't delete non dequeued buffer index 
>> %d\n", index);
>> +               return -EINVAL;
>> +       }
>> +
> I know the driver could implement its own 
> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is 
> used by the hardware as a future reference frame.
> But I think we need a flag to let the user know which buffer is still 
> used by the hardware.
> Alternative ref case is safe, we only know it's existing when it is 
> dequeued in current V4L2 buffer mechanism.
> While the Golden reference frame, such long term reference frame could 
> last much longer.

It is up to userland stack to know frames life time, it got the information for that.

>> +       if (vb->planes[0].mem_priv)
>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> +       /* Free MMAP buffers or release USERPTR buffers */
>> +       if (q->memory == VB2_MEMORY_MMAP)
>> +               __vb2_buf_mem_free(vb);
>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>> +               __vb2_buf_dmabuf_put(vb);
>> +       else
>> +               __vb2_buf_userptr_put(vb);
>> +
>> +       vb2_queue_remove_buffer(q, vb);
>> +       kfree(vb);
>> +
>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>> +       return 0;
>> +}
>> +
>>   /*
>>    * vb2_start_streaming() - Attempt to start streaming.
>>    * @q:         videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c 
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct 
>> media_device *mdev,
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>> +{
>> +       return vb2_core_delete_buf(q, b->index);
>> +}
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>>   int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers 
>> *create)
>>   {
>>          unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c 
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct 
>> video_device *vdev)
>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, 
>> vidioc_prepare_buf);
>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, 
>> vidioc_streamoff);
>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, 
>> vidioc_delete_buf);
>>          }
>>
>>          if (is_vid || is_vbi || is_meta) {
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct 
>> v4l2_ioctl_ops *ops,
>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>   }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> +                         struct file *file, void *fh, void *arg)
>> +{
>> +       struct v4l2_buffer *b = arg;
>> +       int ret = check_fmt(file, b->type);
>> +
>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>> +}
>> +
>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>                                  struct file *file, void *fh, void *arg)
>>   {
>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info 
>> v4l2_ioctls[] = {
>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, 
>> v4l_print_freq_band, 0),
>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, 
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, 
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | 
>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, 
>> v4l_print_buffer, INFO_FL_QUEUE),
>>   };
>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>> index edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>    * @vidioc_prepare_buf: pointer to the function that implements
>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>    * @vidioc_overlay: pointer to the function that implements
>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>    * @vidioc_g_fbuf: pointer to the function that implements
>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>                                    struct v4l2_create_buffers *b);
>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>                                    struct v4l2_buffer *b);
>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>> +                                struct v4l2_buffer *b);
>>
>>          int (*vidioc_overlay)(struct file *file, void *fh, unsigned 
>> int i);
>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>> diff --git a/include/media/videobuf2-core.h 
>> b/include/media/videobuf2-core.h
>> index 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, 
>> enum vb2_memory memory,
>>    */
>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, 
>> void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index:     id number of the buffer.
>> + *
>> + *  Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>>   /**
>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>    *
>> diff --git a/include/media/videobuf2-v4l2.h 
>> b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct 
>> v4l2_create_buffers *create);
>>    */
>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>                      struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b:         buffer structure passed from userspace to
>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly 
>> returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>>   /**
>>    * vb2_qbuf() - Queue a buffer from userspace
>> diff --git a/include/uapi/linux/videodev2.h 
>> b/include/uapi/linux/videodev2.h
>> index aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct 
>> v4l2_dbg_chip_info)
>>
>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct 
>> v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>>   /* Reminder: when adding new ioctls please add support for them to
>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> -- 
>> 2.39.2
>>

Hsia-Jun Li June 27, 2023, 8:47 a.m. UTC | #10

On 6/27/23 16:43, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open 
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>> ---
>>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>   .../media/v4l/vidioc-delete-buf.rst           | 51 
>>> +++++++++++++++++++
>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>   include/media/v4l2-ioctl.h                    |  4 ++
>>>   include/media/videobuf2-core.h                |  9 ++++
>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>   include/uapi/linux/videodev2.h                |  2 +
>>>   10 files changed, 128 insertions(+)
>>>   create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>>       vidioc-dbg-g-chip-info
>>>       vidioc-dbg-g-register
>>>       vidioc-decoder-cmd
>>> +    vidioc-delete-buf
>>>       vidioc-dqevent
>>>       vidioc-dv-timings-cap
>>>       vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>> +.. c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> +    File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is 
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> +    File I/O is in progress.
>>> +
>>> +EINVAL
>>> +    The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb)
>>>   }
>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>> +{
>>> +       struct vb2_buffer *vb;
>>> +
>>> +       vb = vb2_get_buffer(q, index);
>>> +       if (!vb) {
>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>> +               return -EINVAL;
>>> +       }
>>> +
>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> +               dprintk(q, 1, "can't delete non dequeued buffer index
>>> %d\n", index);
>>> +               return -EINVAL;
>>> +       }
>>> +
>> I know the driver could implement its own
>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>> used by the hardware as a future reference frame.
>> But I think we need a flag to let the user know which buffer is still
>> used by the hardware.
>> Alternative ref case is safe, we only know it's existing when it is
>> dequeued in current V4L2 buffer mechanism.
>> While the Golden reference frame, such long term reference frame could
>> last much longer.
>
> It is up to userland stack to know frames life time, it got the 
> information for that.

That is true for the stateless codec driver.

While application for stateful decoder could never do that. It also 
breaks what the document said:

"The backing memory of |CAPTURE| buffers that are used as reference 
frames by the stream may be read by the hardware even after they are 
dequeued. Consequently, the client should avoid writing into this memory 
while the |CAPTURE| queue is streaming. Failure to observe this may 
result in corruption of decoded frames."

>
>>> +       if (vb->planes[0].mem_priv)
>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>> +               __vb2_buf_mem_free(vb);
>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>> +               __vb2_buf_dmabuf_put(vb);
>>> +       else
>>> +               __vb2_buf_userptr_put(vb);
>>> +
>>> +       vb2_queue_remove_buffer(q, vb);
>>> +       kfree(vb);
>>> +
>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>> +       return 0;
>>> +}
>>> +
>>>   /*
>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>    * @q:         videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev,
>>>   }
>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>> +{
>>> +       return vb2_core_delete_buf(q, b->index);
>>> +}
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>>   int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>>   {
>>>          unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device *vdev)
>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON, 
>>> vidioc_streamon);
>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> vidioc_delete_buf);
>>>          }
>>>
>>>          if (is_vid || is_vbi || is_meta) {
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>   }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> +                         struct file *file, void *fh, void *arg)
>>> +{
>>> +       struct v4l2_buffer *b = arg;
>>> +       int ret = check_fmt(file, b->type);
>>> +
>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>> +}
>>> +
>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>                                  struct file *file, void *fh, void 
>>> *arg)
>>>   {
>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>> v4l2_ioctls[] = {
>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>   };
>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>    * @vidioc_overlay: pointer to the function that implements
>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>                                    struct v4l2_create_buffers *b);
>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>                                    struct v4l2_buffer *b);
>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> +                                struct v4l2_buffer *b);
>>>
>>>          int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>>    */
>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index:     id number of the buffer.
>>> + *
>>> + *  Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>>   /**
>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>    *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>>    */
>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>                      struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b:         buffer structure passed from userspace to
>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>>   /**
>>>    * vb2_qbuf() - Queue a buffer from userspace
>>> diff --git a/include/uapi/linux/videodev2.h
>>> b/include/uapi/linux/videodev2.h
>>> index aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>> v4l2_dbg_chip_info)
>>>
>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>>   /* Reminder: when adding new ioctls please add support for them to
>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> -- 
>>> 2.39.2
>>>

Hsia-Jun Li June 30, 2023, 9:43 a.m. UTC | #11

On 6/27/23 16:47, Hsia-Jun Li wrote:
> CAUTION: Email originated externally, do not click links or open 
> attachments unless you recognize the sender and know the content is safe.
>
>
> On 6/27/23 16:43, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>> ---
>>>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>   .../media/v4l/vidioc-delete-buf.rst           | 51
>>>> +++++++++++++++++++
>>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>   include/media/v4l2-ioctl.h                    |  4 ++
>>>>   include/media/videobuf2-core.h                |  9 ++++
>>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>>   include/uapi/linux/videodev2.h                |  2 +
>>>>   10 files changed, 128 insertions(+)
>>>>   create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>       vidioc-dbg-g-chip-info
>>>>       vidioc-dbg-g-register
>>>>       vidioc-decoder-cmd
>>>> +    vidioc-delete-buf
>>>>       vidioc-dqevent
>>>>       vidioc-dv-timings-cap
>>>>       vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>> +.. c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> +    File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` 
>>>> ioctl to
>>>> +delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> +    File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>> unsigned int index, void *pb)
>>>>   }
>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>> +{
>>>> +       struct vb2_buffer *vb;
>>>> +
>>>> +       vb = vb2_get_buffer(q, index);
>>>> +       if (!vb) {
>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> +               return -EINVAL;
>>>> +       }
>>>> +
>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> +               dprintk(q, 1, "can't delete non dequeued buffer index
>>>> %d\n", index);
>>>> +               return -EINVAL;
>>>> +       }
>>>> +
>>> I know the driver could implement its own
>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>> used by the hardware as a future reference frame.
>>> But I think we need a flag to let the user know which buffer is still
>>> used by the hardware.
>>> Alternative ref case is safe, we only know it's existing when it is
>>> dequeued in current V4L2 buffer mechanism.
>>> While the Golden reference frame, such long term reference frame could
>>> last much longer.
>>
>> It is up to userland stack to know frames life time, it got the
>> information for that.
>
> That is true for the stateless codec driver.
>
> While application for stateful decoder could never do that. It also
> breaks what the document said:
>
> "The backing memory of |CAPTURE| buffers that are used as reference
> frames by the stream may be read by the hardware even after they are
> dequeued. Consequently, the client should avoid writing into this memory
> while the |CAPTURE| queue is streaming. Failure to observe this may
> result in corruption of decoded frames."
>
>>
>>>> +       if (vb->planes[0].mem_priv)
>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>> +               __vb2_buf_mem_free(vb);

Here is another problem for the existing application, the mmap() from 
the mmap offset or exportbuffer fd would not create a reference to 
buffer in this step(while the exportbuffer would create one itself).

When you delete a buffer, you may not release it from its virtual memory 
space, leaving a corrupted virtual memory space. Also this behavior is 
right, because mmap(2) says:

"After  the  mmap()  call has returned, the file descriptor, fd, can be 
closed immediately without invalidating the map‐ping."

>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>> +               __vb2_buf_dmabuf_put(vb);
>>>> +       else
>>>> +               __vb2_buf_userptr_put(vb);
>>>> +
>>>> +       vb2_queue_remove_buffer(q, vb);
>>>> +       kfree(vb);
>>>> +
>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>> +       return 0;
>>>> +}
>>>> +
>>>>   /*
>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>    * @q:         videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev,
>>>>   }
>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>> +{
>>>> +       return vb2_core_delete_buf(q, b->index);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>>   int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>>   {
>>>>          unsigned requested_planes = 1;
>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>> index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device *vdev)
>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>> vidioc_streamon);
>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> vidioc_delete_buf);
>>>>          }
>>>>
>>>>          if (is_vid || is_vbi || is_meta) {
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>   }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> +                         struct file *file, void *fh, void *arg)
>>>> +{
>>>> +       struct v4l2_buffer *b = arg;
>>>> +       int ret = check_fmt(file, b->type);
>>>> +
>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>> +}
>>>> +
>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>                                  struct file *file, void *fh, void
>>>> *arg)
>>>>   {
>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>> v4l2_ioctls[] = {
>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>   };
>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>                                    struct v4l2_create_buffers *b);
>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>                                    struct v4l2_buffer *b);
>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> +                                struct v4l2_buffer *b);
>>>>
>>>>          int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>>    */
>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index:     id number of the buffer.
>>>> + *
>>>> + *  Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>>   /**
>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>    *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>>    */
>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>                      struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b:         buffer structure passed from userspace to
>>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>>   /**
>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>> diff --git a/include/uapi/linux/videodev2.h
>>>> b/include/uapi/linux/videodev2.h
>>>> index aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>> v4l2_dbg_chip_info)
>>>>
>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>>   /* Reminder: when adding new ioctls please add support for them to
>>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> -- 
>>>> 2.39.2
>>>>
> -- 
> Hsia-Jun(Randy) Li
>

Benjamin Gaignard July 3, 2023, 8:12 a.m. UTC | #12

Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
>> CAUTION: Email originated externally, do not click links or open 
>> attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is 
>>> safe.
>>>
>>>
>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>>> ---
>>>>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>>   .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>> +++++++++++++++++++
>>>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>   include/media/v4l2-ioctl.h                    |  4 ++
>>>>>   include/media/videobuf2-core.h                |  9 ++++
>>>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>   include/uapi/linux/videodev2.h                |  2 +
>>>>>   10 files changed, 128 insertions(+)
>>>>>   create mode 100644
>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>       vidioc-dbg-g-chip-info
>>>>>       vidioc-dbg-g-register
>>>>>       vidioc-decoder-cmd
>>>>> +    vidioc-delete-buf
>>>>>       vidioc-dqevent
>>>>>       vidioc-dv-timings-cap
>>>>>       vidioc-encoder-cmd
>>>>> diff --git
>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> new file mode 100644
>>>>> index 000000000000..0e7ce58f91bc
>>>>> --- /dev/null
>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> @@ -0,0 +1,51 @@
>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>> +.. c:namespace:: V4L
>>>>> +
>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>> +
>>>>> +************************
>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>> +************************
>>>>> +
>>>>> +Name
>>>>> +====
>>>>> +
>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>> +
>>>>> +Synopsis
>>>>> +========
>>>>> +
>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>> +
>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>> +
>>>>> +Arguments
>>>>> +=========
>>>>> +
>>>>> +``fd``
>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>> +
>>>>> +``argp``
>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>> +
>>>>> +Description
>>>>> +===========
>>>>> +
>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` 
>>>>> ioctl to
>>>>> +delete a buffer from a queue.
>>>>> +
>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>> +:ref:`buffer`.
>>>>> +
>>>>> +Return Value
>>>>> +============
>>>>> +
>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>> set
>>>>> +appropriately. The generic error codes are described at the
>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>> +
>>>>> +EBUSY
>>>>> +    File I/O is in progress.
>>>>> +
>>>>> +EINVAL
>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 899783f67580..aa546c972c3d 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>> unsigned int index, void *pb)
>>>>>   }
>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>> +{
>>>>> +       struct vb2_buffer *vb;
>>>>> +
>>>>> +       vb = vb2_get_buffer(q, index);
>>>>> +       if (!vb) {
>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>> +               return -EINVAL;
>>>>> +       }
>>>>> +
>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer index
>>>>> %d\n", index);
>>>>> +               return -EINVAL;
>>>>> +       }
>>>>> +
>>>> I know the driver could implement its own
>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>> used by the hardware as a future reference frame.
>>>> But I think we need a flag to let the user know which buffer is still
>>>> used by the hardware.
>>>> Alternative ref case is safe, we only know it's existing when it is
>>>> dequeued in current V4L2 buffer mechanism.
>>>> While the Golden reference frame, such long term reference frame could
>>>> last much longer.
>>>
>>> It is up to userland stack to know frames life time, it got the
>>> information for that.
>>
>> That is true for the stateless codec driver.
>>
>> While application for stateful decoder could never do that. It also
>> breaks what the document said:
>>
>> "The backing memory of |CAPTURE| buffers that are used as reference
>> frames by the stream may be read by the hardware even after they are
>> dequeued. Consequently, the client should avoid writing into this memory
>> while the |CAPTURE| queue is streaming. Failure to observe this may
>> result in corruption of decoded frames."
>>
>>>
>>>>> +       if (vb->planes[0].mem_priv)
>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>> +
>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>> +               __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from 
> the mmap offset or exportbuffer fd would not create a reference to 
> buffer in this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual 
> memory space, leaving a corrupted virtual memory space. Also this 
> behavior is right, because mmap(2) says:
>
> "After  the  mmap()  call has returned, the file descriptor, fd, can 
> be closed immediately without invalidating the map‐ping."

Existing applications do not call DELETE_BUF ioctl and when call it they will be aware that the buffer is removed.
I have done it in GStreamer:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/commit/fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9

Regards,
Benjamin

>
>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>> +       else
>>>>> +               __vb2_buf_userptr_put(vb);
>>>>> +
>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>> +       kfree(vb);
>>>>> +
>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>> +       return 0;
>>>>> +}
>>>>> +
>>>>>   /*
>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>    * @q:         videobuf2 queue
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>> media_device *mdev,
>>>>>   }
>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>> +{
>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>> +
>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>> *create)
>>>>>   {
>>>>>          unsigned requested_planes = 1;
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> index f81279492682..80ace2e1e932 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>> video_device *vdev)
>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>> vidioc_prepare_buf);
>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>> vidioc_streamon);
>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>> vidioc_streamoff);
>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>> vidioc_delete_buf);
>>>>>          }
>>>>>
>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index a858acea6547..1c737279d3ef 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>> v4l2_ioctl_ops *ops,
>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>   }
>>>>>
>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>> +                         struct file *file, void *fh, void *arg)
>>>>> +{
>>>>> +       struct v4l2_buffer *b = arg;
>>>>> +       int ret = check_fmt(file, b->type);
>>>>> +
>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>> +}
>>>>> +
>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>                                  struct file *file, void *fh, void
>>>>> *arg)
>>>>>   {
>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>> v4l2_ioctls[] = {
>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>> v4l_print_freq_band, 0),
>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>   };
>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>
>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>> index edb733f21604..2f232ed884c7 100644
>>>>> --- a/include/media/v4l2-ioctl.h
>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>                                    struct v4l2_create_buffers *b);
>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>                                    struct v4l2_buffer *b);
>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>> +                                struct v4l2_buffer *b);
>>>>>
>>>>>          int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>> int i);
>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>> enum vb2_memory memory,
>>>>>    */
>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>> void *pb);
>>>>>
>>>>> +/**
>>>>> + * vb2_core_delete_buf() -
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @index:     id number of the buffer.
>>>>> + *
>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>> + */
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>> +
>>>>>   /**
>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>    *
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>> v4l2_create_buffers *create);
>>>>>    */
>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>                      struct v4l2_buffer *b);
>>>>> +/**
>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>> + *
>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @b:         buffer structure passed from userspace to
>>>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>> + *
>>>>> + * The return values from this function are intended to be directly
>>>>> returned
>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>> + */
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>
>>>>>   /**
>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>> b/include/uapi/linux/videodev2.h
>>>>> index aee75eb9e686..31bba1915642 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>> v4l2_dbg_chip_info)
>>>>>
>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>> v4l2_query_ext_ctrl)
>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>>> +
>>>>>
>>>>>   /* Reminder: when adding new ioctls please add support for them to
>>>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>> -- 
>>>>> 2.39.2
>>>>>
>> -- 
>> Hsia-Jun(Randy) Li
>>

Hsia-Jun Li July 3, 2023, 8:19 a.m. UTC | #13

On 7/3/23 16:12, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open 
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>>>> ---
>>>>>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>>>   .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>>> +++++++++++++++++++
>>>>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>>   include/media/v4l2-ioctl.h                    |  4 ++
>>>>>>   include/media/videobuf2-core.h                |  9 ++++
>>>>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>>   include/uapi/linux/videodev2.h                |  2 +
>>>>>>   10 files changed, 128 insertions(+)
>>>>>>   create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>       vidioc-dbg-g-chip-info
>>>>>>       vidioc-dbg-g-register
>>>>>>       vidioc-decoder-cmd
>>>>>> +    vidioc-delete-buf
>>>>>>       vidioc-dqevent
>>>>>>       vidioc-dv-timings-cap
>>>>>>       vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>> ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> +    File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>>   }
>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> +       struct vb2_buffer *vb;
>>>>>> +
>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>> +       if (!vb) {
>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> +               return -EINVAL;
>>>>>> +       }
>>>>>> +
>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer 
>>>>>> index
>>>>>> %d\n", index);
>>>>>> +               return -EINVAL;
>>>>>> +       }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame 
>>>>> could
>>>>> last much longer.
>>>>
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>>
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this 
>>> memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>
>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>> +               __vb2_buf_mem_free(vb);
>>
>> Here is another problem for the existing application, the mmap() from
>> the mmap offset or exportbuffer fd would not create a reference to
>> buffer in this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual
>> memory space, leaving a corrupted virtual memory space. Also this
>> behavior is right, because mmap(2) says:
>>
>> "After  the  mmap()  call has returned, the file descriptor, fd, can
>> be closed immediately without invalidating the map‐ping."
>
> Existing applications do not call DELETE_BUF ioctl and when call it 
> they will be aware that the buffer is removed.
> I have done it in GStreamer:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>

I have read that.

There is not a VP8 parser in Gstreamer, while a parser would not work 
when deal with the secure video(userspace can't access the data context 
at all).

Besides, this adds extra work for the application for a stateful codec 
driver. The application need to parser the bitstream and track the dpb.

I don't mind if you could fix the nonfiction mechanism for those 
non-display frame and internal reference state.

That could be requirement for codec firmware that its driver could 
support this DELETE_BUF ioctl() feature.

>
> Regards,
> Benjamin
>
>>
>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>> +       else
>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>> +       kfree(vb);
>>>>>> +
>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> +       return 0;
>>>>>> +}
>>>>>> +
>>>>>>   /*
>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>    * @q:         videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>>   }
>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct 
>>>>>> v4l2_create_buffers
>>>>>> *create)
>>>>>>   {
>>>>>>          unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>>          }
>>>>>>
>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>   }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> +                         struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>                                  struct file *file, void *fh, void
>>>>>> *arg)
>>>>>>   {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>   };
>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>                                    struct v4l2_create_buffers *b);
>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>                                    struct v4l2_buffer *b);
>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> +                                struct v4l2_buffer *b);
>>>>>>
>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>>    */
>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index:     id number of the buffer.
>>>>>> + *
>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>>   /**
>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>    *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>>    */
>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device 
>>>>>> *mdev,
>>>>>>                      struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>>   /**
>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>>   /* Reminder: when adding new ioctls please add support for them to
>>>>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> -- 
>>>>>> 2.39.2
>>>>>>
>>> -- 
>>> Hsia-Jun(Randy) Li
>>>

Benjamin Gaignard July 3, 2023, 8:52 a.m. UTC | #14

Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:12, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open 
>> attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>
>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>>>>> ---
>>>>>>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>>>>   .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>>>> +++++++++++++++++++
>>>>>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>>>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>>>   include/media/v4l2-ioctl.h                    |  4 ++
>>>>>>>   include/media/videobuf2-core.h                |  9 ++++
>>>>>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>>>   include/uapi/linux/videodev2.h                |  2 +
>>>>>>>   10 files changed, 128 insertions(+)
>>>>>>>   create mode 100644
>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>
>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>       vidioc-dbg-g-chip-info
>>>>>>>       vidioc-dbg-g-register
>>>>>>>       vidioc-decoder-cmd
>>>>>>> +    vidioc-delete-buf
>>>>>>>       vidioc-dqevent
>>>>>>>       vidioc-dv-timings-cap
>>>>>>>       vidioc-encoder-cmd
>>>>>>> diff --git
>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> new file mode 100644
>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>> --- /dev/null
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> @@ -0,0 +1,51 @@
>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>> +.. c:namespace:: V4L
>>>>>>> +
>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>> +
>>>>>>> +************************
>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>> +************************
>>>>>>> +
>>>>>>> +Name
>>>>>>> +====
>>>>>>> +
>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>> +
>>>>>>> +Synopsis
>>>>>>> +========
>>>>>>> +
>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>> +
>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>> +
>>>>>>> +Arguments
>>>>>>> +=========
>>>>>>> +
>>>>>>> +``fd``
>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>> +
>>>>>>> +``argp``
>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>> +
>>>>>>> +Description
>>>>>>> +===========
>>>>>>> +
>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>> ioctl to
>>>>>>> +delete a buffer from a queue.
>>>>>>> +
>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>> +:ref:`buffer`.
>>>>>>> +
>>>>>>> +Return Value
>>>>>>> +============
>>>>>>> +
>>>>>>> +On success 0 is returned, on error -1 and the ``errno`` 
>>>>>>> variable is
>>>>>>> set
>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>> +
>>>>>>> +EBUSY
>>>>>>> +    File I/O is in progress.
>>>>>>> +
>>>>>>> +EINVAL
>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue 
>>>>>>> *q,
>>>>>>> unsigned int index, void *pb)
>>>>>>>   }
>>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>> +{
>>>>>>> +       struct vb2_buffer *vb;
>>>>>>> +
>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>> +       if (!vb) {
>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>> +               return -EINVAL;
>>>>>>> +       }
>>>>>>> +
>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer 
>>>>>>> index
>>>>>>> %d\n", index);
>>>>>>> +               return -EINVAL;
>>>>>>> +       }
>>>>>>> +
>>>>>> I know the driver could implement its own
>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>> used by the hardware as a future reference frame.
>>>>>> But I think we need a flag to let the user know which buffer is 
>>>>>> still
>>>>>> used by the hardware.
>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>> While the Golden reference frame, such long term reference frame 
>>>>>> could
>>>>>> last much longer.
>>>>>
>>>>> It is up to userland stack to know frames life time, it got the
>>>>> information for that.
>>>>
>>>> That is true for the stateless codec driver.
>>>>
>>>> While application for stateful decoder could never do that. It also
>>>> breaks what the document said:
>>>>
>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>> frames by the stream may be read by the hardware even after they are
>>>> dequeued. Consequently, the client should avoid writing into this 
>>>> memory
>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>> result in corruption of decoded frames."
>>>>
>>>>>
>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>> +
>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>> +               __vb2_buf_mem_free(vb);
>>>
>>> Here is another problem for the existing application, the mmap() from
>>> the mmap offset or exportbuffer fd would not create a reference to
>>> buffer in this step(while the exportbuffer would create one itself).
>>>
>>> When you delete a buffer, you may not release it from its virtual
>>> memory space, leaving a corrupted virtual memory space. Also this
>>> behavior is right, because mmap(2) says:
>>>
>>> "After  the  mmap()  call has returned, the file descriptor, fd, can
>>> be closed immediately without invalidating the map‐ping."
>>
>> Existing applications do not call DELETE_BUF ioctl and when call it 
>> they will be aware that the buffer is removed.
>> I have done it in GStreamer:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>>
>
> I have read that.
>
> There is not a VP8 parser in Gstreamer, while a parser would not work 
> when deal with the secure video(userspace can't access the data 
> context at all).
>
> Besides, this adds extra work for the application for a stateful codec 
> driver. The application need to parser the bitstream and track the dpb.
>
> I don't mind if you could fix the nonfiction mechanism for those 
> non-display frame and internal reference state.
>
> That could be requirement for codec firmware that its driver could 
> support this DELETE_BUF ioctl() feature.

Sorry I don't see the link with my patches here...
I have work on non-secure VP9 on stateless codec.
DELETE_BUF ioctl is optional and the main goal is to offer a way to applications
to save memory if they know when they could delete buffers without risk.

>
>>
>> Regards,
>> Benjamin
>>
>>>
>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>> +       else
>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>> +
>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>> +       kfree(vb);
>>>>>>> +
>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>> +       return 0;
>>>>>>> +}
>>>>>>> +
>>>>>>>   /*
>>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>    * @q:         videobuf2 queue
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, 
>>>>>>> struct
>>>>>>> media_device *mdev,
>>>>>>>   }
>>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>> +{
>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>> +}
>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>> +
>>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct 
>>>>>>> v4l2_create_buffers
>>>>>>> *create)
>>>>>>>   {
>>>>>>>          unsigned requested_planes = 1;
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>> video_device *vdev)
>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>> vidioc_prepare_buf);
>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>> vidioc_streamon);
>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>> vidioc_streamoff);
>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>> vidioc_delete_buf);
>>>>>>>          }
>>>>>>>
>>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>   }
>>>>>>>
>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>> +                         struct file *file, void *fh, void *arg)
>>>>>>> +{
>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>> +
>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>> +}
>>>>>>> +
>>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>                                  struct file *file, void *fh, void
>>>>>>> *arg)
>>>>>>>   {
>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>> v4l2_ioctls[] = {
>>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>> v4l_print_freq_band, 0),
>>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>   };
>>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>
>>>>>>> diff --git a/include/media/v4l2-ioctl.h 
>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>                                    struct v4l2_create_buffers *b);
>>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>                                    struct v4l2_buffer *b);
>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>
>>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh, 
>>>>>>> unsigned
>>>>>>> int i);
>>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>> enum vb2_memory memory,
>>>>>>>    */
>>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>> void *pb);
>>>>>>>
>>>>>>> +/**
>>>>>>> + * vb2_core_delete_buf() -
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @index:     id number of the buffer.
>>>>>>> + *
>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>> + */
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>> +
>>>>>>>   /**
>>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>    *
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, 
>>>>>>> struct
>>>>>>> v4l2_create_buffers *create);
>>>>>>>    */
>>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device 
>>>>>>> *mdev,
>>>>>>>                      struct v4l2_buffer *b);
>>>>>>> +/**
>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>> + *
>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>> + *
>>>>>>> + * The return values from this function are intended to be 
>>>>>>> directly
>>>>>>> returned
>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>> + */
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>
>>>>>>>   /**
>>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>> v4l2_dbg_chip_info)
>>>>>>>
>>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>> v4l2_query_ext_ctrl)
>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>>>>> +
>>>>>>>
>>>>>>>   /* Reminder: when adding new ioctls please add support for 
>>>>>>> them to
>>>>>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>> -- 
>>>>>>> 2.39.2
>>>>>>>
>>>> -- 
>>>> Hsia-Jun(Randy) Li
>>>>

Hsia-Jun Li July 3, 2023, 9:20 a.m. UTC | #15

On 7/3/23 16:52, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open 
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>>>>>> ---
>>>>>>>>   .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>>>>>   .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>>>>   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>>>>>   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>>>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>>>>   include/media/v4l2-ioctl.h                    |  4 ++
>>>>>>>>   include/media/videobuf2-core.h                |  9 ++++
>>>>>>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>>>>   include/uapi/linux/videodev2.h                |  2 +
>>>>>>>>   10 files changed, 128 insertions(+)
>>>>>>>>   create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>       vidioc-dbg-g-chip-info
>>>>>>>>       vidioc-dbg-g-register
>>>>>>>>       vidioc-decoder-cmd
>>>>>>>> +    vidioc-delete-buf
>>>>>>>>       vidioc-dqevent
>>>>>>>>       vidioc-dv-timings-cap
>>>>>>>>       vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer 
>>>>>>>> *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>> ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>> variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> +    File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>> *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>>   }
>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> +       struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>>> +       if (!vb) {
>>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> +               return -EINVAL;
>>>>>>>> +       }
>>>>>>>> +
>>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>> index
>>>>>>>> %d\n", index);
>>>>>>>> +               return -EINVAL;
>>>>>>>> +       }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>> still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>> could
>>>>>>> last much longer.
>>>>>>
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>>
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>> memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>
>>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> +               __vb2_buf_mem_free(vb);
>>>>
>>>> Here is another problem for the existing application, the mmap() from
>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual
>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>> behavior is right, because mmap(2) says:
>>>>
>>>> "After  the  mmap()  call has returned, the file descriptor, fd, can
>>>> be closed immediately without invalidating the map‐ping."
>>>
>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>> they will be aware that the buffer is removed.
>>> I have done it in GStreamer:
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>>>
>>>
>>
>> I have read that.
>>
>> There is not a VP8 parser in Gstreamer, while a parser would not work
>> when deal with the secure video(userspace can't access the data
>> context at all).
>>
>> Besides, this adds extra work for the application for a stateful codec
>> driver. The application need to parser the bitstream and track the dpb.
>>
>> I don't mind if you could fix the nonfiction mechanism for those
>> non-display frame and internal reference state.
>>
>> That could be requirement for codec firmware that its driver could
>> support this DELETE_BUF ioctl() feature.
>
> Sorry I don't see the link with my patches here...
> I have work on non-secure VP9 on stateless codec.
> DELETE_BUF ioctl is optional and the main goal is to offer a way to 
> applications
> to save memory if they know when they could delete buffers without risk.

I try to explain why I think this design in not "complete". One problem 
resolved, more problems would occur.


For non-secure video, those applications have worked:

- It would break what stateful means here, application need to 
acquire(parse) the information that driver should offer.

   Or it would break the decoding model.

- Your Gstreamer sample code or this design won't work for AV1.

For all the future possible secure video:

- This feature could never be used from the current design.


>
>>
>>>
>>> Regards,
>>> Benjamin
>>>
>>>>
>>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>>> +       else
>>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>>> +       kfree(vb);
>>>>>>>> +
>>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> +       return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>   /*
>>>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>    * @q:         videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> media_device *mdev,
>>>>>>>>   }
>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>>   {
>>>>>>>>          unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>>          }
>>>>>>>>
>>>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>>   }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> +                         struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>                                  struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>>   {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, 
>>>>>>>> match)),
>>>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>   };
>>>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>                                    struct v4l2_create_buffers *b);
>>>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>                                    struct v4l2_buffer *b);
>>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>> unsigned
>>>>>>>> int i);
>>>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>>    */
>>>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int 
>>>>>>>> index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index:     id number of the buffer.
>>>>>>>> + *
>>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>>   /**
>>>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>    *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>    */
>>>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>> *mdev,
>>>>>>>>                      struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be
>>>>>>>> directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>>   /**
>>>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct 
>>>>>>>> v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>>   /* Reminder: when adding new ioctls please add support for
>>>>>>>> them to
>>>>>>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> -- 
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> -- 
>>>>> Hsia-Jun(Randy) Li
>>>>>

Benjamin Gaignard July 3, 2023, 10:35 a.m. UTC | #16

Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:52, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open 
>> attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard 
>>>>>>>>> <[e-mail]>
>>>>>>>>> ---
>>>>>>>>>   .../userspace-api/media/v4l/user-func.rst     | 1 +
>>>>>>>>>   .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>>>>>> +++++++++++++++++++
>>>>>>>>>   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>>>>>   .../media/common/videobuf2/videobuf2-v4l2.c   | 6 +++
>>>>>>>>>   drivers/media/v4l2-core/v4l2-dev.c            | 1 +
>>>>>>>>>   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>>>>>   include/media/v4l2-ioctl.h                    | 4 ++
>>>>>>>>>   include/media/videobuf2-core.h                | 9 ++++
>>>>>>>>>   include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>>>>>   include/uapi/linux/videodev2.h                | 2 +
>>>>>>>>>   10 files changed, 128 insertions(+)
>>>>>>>>>   create mode 100644
>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>
>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>       vidioc-dbg-g-chip-info
>>>>>>>>>       vidioc-dbg-g-register
>>>>>>>>>       vidioc-decoder-cmd
>>>>>>>>> +    vidioc-delete-buf
>>>>>>>>>       vidioc-dqevent
>>>>>>>>>       vidioc-dv-timings-cap
>>>>>>>>>       vidioc-encoder-cmd
>>>>>>>>> diff --git
>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> new file mode 100644
>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>> --- /dev/null
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>> +
>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>> +
>>>>>>>>> +************************
>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>> +************************
>>>>>>>>> +
>>>>>>>>> +Name
>>>>>>>>> +====
>>>>>>>>> +
>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>> +
>>>>>>>>> +Synopsis
>>>>>>>>> +========
>>>>>>>>> +
>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>> +
>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer 
>>>>>>>>> *argp)``
>>>>>>>>> +
>>>>>>>>> +Arguments
>>>>>>>>> +=========
>>>>>>>>> +
>>>>>>>>> +``fd``
>>>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>>>> +
>>>>>>>>> +``argp``
>>>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>> +
>>>>>>>>> +Description
>>>>>>>>> +===========
>>>>>>>>> +
>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>> ioctl to
>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>> +
>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>> +:ref:`buffer`.
>>>>>>>>> +
>>>>>>>>> +Return Value
>>>>>>>>> +============
>>>>>>>>> +
>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>> variable is
>>>>>>>>> set
>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>> +
>>>>>>>>> +EBUSY
>>>>>>>>> +    File I/O is in progress.
>>>>>>>>> +
>>>>>>>>> +EINVAL
>>>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>   }
>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>>> +{
>>>>>>>>> +       struct vb2_buffer *vb;
>>>>>>>>> +
>>>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>>>> +       if (!vb) {
>>>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", 
>>>>>>>>> index);
>>>>>>>>> +               return -EINVAL;
>>>>>>>>> +       }
>>>>>>>>> +
>>>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>> index
>>>>>>>>> %d\n", index);
>>>>>>>>> +               return -EINVAL;
>>>>>>>>> +       }
>>>>>>>>> +
>>>>>>>> I know the driver could implement its own
>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>> still
>>>>>>>> used by the hardware.
>>>>>>>> Alternative ref case is safe, we only know it's existing when 
>>>>>>>> it is
>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>> could
>>>>>>>> last much longer.
>>>>>>>
>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>> information for that.
>>>>>>
>>>>>> That is true for the stateless codec driver.
>>>>>>
>>>>>> While application for stateful decoder could never do that. It also
>>>>>> breaks what the document said:
>>>>>>
>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>> frames by the stream may be read by the hardware even after they are
>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>> memory
>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>> result in corruption of decoded frames."
>>>>>>
>>>>>>>
>>>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>> +
>>>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>> +               __vb2_buf_mem_free(vb);
>>>>>
>>>>> Here is another problem for the existing application, the mmap() from
>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>
>>>>> When you delete a buffer, you may not release it from its virtual
>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>> behavior is right, because mmap(2) says:
>>>>>
>>>>> "After  the  mmap()  call has returned, the file descriptor, fd, can
>>>>> be closed immediately without invalidating the map‐ping."
>>>>
>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>> they will be aware that the buffer is removed.
>>>> I have done it in GStreamer:
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>>>>
>>>>
>>>
>>> I have read that.
>>>
>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>> when deal with the secure video(userspace can't access the data
>>> context at all).
>>>
>>> Besides, this adds extra work for the application for a stateful codec
>>> driver. The application need to parser the bitstream and track the dpb.
>>>
>>> I don't mind if you could fix the nonfiction mechanism for those
>>> non-display frame and internal reference state.
>>>
>>> That could be requirement for codec firmware that its driver could
>>> support this DELETE_BUF ioctl() feature.
>>
>> Sorry I don't see the link with my patches here...
>> I have work on non-secure VP9 on stateless codec.
>> DELETE_BUF ioctl is optional and the main goal is to offer a way to 
>> applications
>> to save memory if they know when they could delete buffers without risk.
>
> I try to explain why I think this design in not "complete". One 
> problem resolved, more problems would occur.
>
>
> For non-secure video, those applications have worked:
>
> - It would break what stateful means here, application need to 
> acquire(parse) the information that driver should offer.
>
>   Or it would break the decoding model.
>
> - Your Gstreamer sample code or this design won't work for AV1.
why ?
>
> For all the future possible secure video:
>
> - This feature could never be used from the current design.
>
>
>>
>>>
>>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>>
>>>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>>>> +       else
>>>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>>>> +
>>>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>>>> +       kfree(vb);
>>>>>>>>> +
>>>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>> +       return 0;
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>>   /*
>>>>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>    * @q:         videobuf2 queue
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> media_device *mdev,
>>>>>>>>>   }
>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>> +{
>>>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>>>> +}
>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>> +
>>>>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>> v4l2_create_buffers
>>>>>>>>> *create)
>>>>>>>>>   {
>>>>>>>>>          unsigned requested_planes = 1;
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>> video_device *vdev)
>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>> vidioc_streamon);
>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>> vidioc_streamoff);
>>>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>          }
>>>>>>>>>
>>>>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>>>   }
>>>>>>>>>
>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> +                         struct file *file, void *fh, void *arg)
>>>>>>>>> +{
>>>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>>>> +
>>>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>                                  struct file *file, void *fh, 
>>>>>>>>> void
>>>>>>>>> *arg)
>>>>>>>>>   {
>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, 
>>>>>>>>> match)),
>>>>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>   };
>>>>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>
>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>    * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>                                    struct v4l2_create_buffers 
>>>>>>>>> *b);
>>>>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>                                    struct v4l2_buffer *b);
>>>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>> unsigned
>>>>>>>>> int i);
>>>>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue 
>>>>>>>>> *q,
>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>    */
>>>>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int 
>>>>>>>>> index,
>>>>>>>>> void *pb);
>>>>>>>>>
>>>>>>>>> +/**
>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>> + * @index:     id number of the buffer.
>>>>>>>>> + *
>>>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>>>> + */
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int 
>>>>>>>>> index);
>>>>>>>>> +
>>>>>>>>>   /**
>>>>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>    *
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>    */
>>>>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>> *mdev,
>>>>>>>>>                      struct v4l2_buffer *b);
>>>>>>>>> +/**
>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>> + *
>>>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 
>>>>>>>>> queue.
>>>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>> + *
>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>> directly
>>>>>>>>> returned
>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>> + */
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>>   /**
>>>>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>
>>>>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct 
>>>>>>>>> v4l2_buffer)
>>>>>>>>> +
>>>>>>>>>
>>>>>>>>>   /* Reminder: when adding new ioctls please add support for
>>>>>>>>> them to
>>>>>>>>>      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>> -- 
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> -- 
>>>>>> Hsia-Jun(Randy) Li
>>>>>>

Hsia-Jun Li July 3, 2023, 11:06 a.m. UTC | #17

Sorry for html version, again.

On 7/3/23 18:35, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open 
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>
>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>> <[e-mail]>
>>>>>>>>>> ---
>>>>>>>>>>   .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>   .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>   .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>   .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>   drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>   drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>   include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>   include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>   include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>   include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>   10 files changed, 128 insertions(+)
>>>>>>>>>>   create mode 100644
>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>
>>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>       vidioc-dbg-g-chip-info
>>>>>>>>>>       vidioc-dbg-g-register
>>>>>>>>>>       vidioc-decoder-cmd
>>>>>>>>>> +    vidioc-delete-buf
>>>>>>>>>>       vidioc-dqevent
>>>>>>>>>>       vidioc-dv-timings-cap
>>>>>>>>>>       vidioc-encoder-cmd
>>>>>>>>>> diff --git
>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> new file mode 100644
>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>> --- /dev/null
>>>>>>>>>> +++ 
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>> +
>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>> +
>>>>>>>>>> +************************
>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>> +************************
>>>>>>>>>> +
>>>>>>>>>> +Name
>>>>>>>>>> +====
>>>>>>>>>> +
>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>> +
>>>>>>>>>> +Synopsis
>>>>>>>>>> +========
>>>>>>>>>> +
>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>> +
>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>> *argp)``
>>>>>>>>>> +
>>>>>>>>>> +Arguments
>>>>>>>>>> +=========
>>>>>>>>>> +
>>>>>>>>>> +``fd``
>>>>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>>>>> +
>>>>>>>>>> +``argp``
>>>>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Description
>>>>>>>>>> +===========
>>>>>>>>>> +
>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>> ioctl to
>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>> +
>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Return Value
>>>>>>>>>> +============
>>>>>>>>>> +
>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>> variable is
>>>>>>>>>> set
>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>> +
>>>>>>>>>> +EBUSY
>>>>>>>>>> +    File I/O is in progress.
>>>>>>>>>> +
>>>>>>>>>> +EINVAL
>>>>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>   }
>>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int 
>>>>>>>>>> index)
>>>>>>>>>> +{
>>>>>>>>>> +       struct vb2_buffer *vb;
>>>>>>>>>> +
>>>>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>>>>> +       if (!vb) {
>>>>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>> index);
>>>>>>>>>> +               return -EINVAL;
>>>>>>>>>> +       }
>>>>>>>>>> +
>>>>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>> index
>>>>>>>>>> %d\n", index);
>>>>>>>>>> +               return -EINVAL;
>>>>>>>>>> +       }
>>>>>>>>>> +
>>>>>>>>> I know the driver could implement its own
>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a 
>>>>>>>>> buffer is
>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>> still
>>>>>>>>> used by the hardware.
>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>> it is
>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>> could
>>>>>>>>> last much longer.
>>>>>>>>
>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>> information for that.
>>>>>>>
>>>>>>> That is true for the stateless codec driver.
>>>>>>>
>>>>>>> While application for stateful decoder could never do that. It also
>>>>>>> breaks what the document said:
>>>>>>>
>>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>>> frames by the stream may be read by the hardware even after they 
>>>>>>> are
>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>> memory
>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>>> result in corruption of decoded frames."
>>>>>>>
>>>>>>>>
>>>>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>> +
>>>>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>> +               __vb2_buf_mem_free(vb);
>>>>>>
>>>>>> Here is another problem for the existing application, the mmap() 
>>>>>> from
>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>>
>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>> behavior is right, because mmap(2) says:
>>>>>>
>>>>>> "After  the  mmap()  call has returned, the file descriptor, fd, can
>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>
>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>> they will be aware that the buffer is removed.
>>>>> I have done it in GStreamer:
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>>>>>
>>>>>
>>>>>
>>>>
>>>> I have read that.
>>>>
>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>> when deal with the secure video(userspace can't access the data
>>>> context at all).
>>>>
>>>> Besides, this adds extra work for the application for a stateful codec
>>>> driver. The application need to parser the bitstream and track the 
>>>> dpb.
>>>>
>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>> non-display frame and internal reference state.
>>>>
>>>> That could be requirement for codec firmware that its driver could
>>>> support this DELETE_BUF ioctl() feature.
>>>
>>> Sorry I don't see the link with my patches here...
>>> I have work on non-secure VP9 on stateless codec.
>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>> applications
>>> to save memory if they know when they could delete buffers without 
>>> risk.
>>
>> I try to explain why I think this design in not "complete". One
>> problem resolved, more problems would occur.
>>
>>
>> For non-secure video, those applications have worked:
>>
>> - It would break what stateful means here, application need to
>> acquire(parse) the information that driver should offer.
>>
>>   Or it would break the decoding model.
>>
>> - Your Gstreamer sample code or this design won't work for AV1.
> why ?

A frame in AV1 could be put to display order with a future frame. That 
is quite different to those ITU codecs.

Supposing a frame (we call it frame 0) without show_display frame, you 
could delete it (just want to do that) once it is decoded.

Actually, unless 7 future frames have came and parser has acknowledged 
them, it is not safe to delete that frame.

Or a show_exist that make frame 0 present.

In my personal opinion, this feature could break many uAPI's, I have 
said my worrying in the other emails.
>>
>> For all the future possible secure video:
>>
>> - This feature could never be used from the current design.
>>
>>
>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Benjamin
>>>>>
>>>>>>
>>>>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>>>>> +       else
>>>>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>>>>> +
>>>>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>>>>> +       kfree(vb);
>>>>>>>>>> +
>>>>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>> +       return 0;
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>>   /*
>>>>>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>    * @q:         videobuf2 queue
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> media_device *mdev,
>>>>>>>>>>   }
>>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>> +{
>>>>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>>>>> +}
>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>> +
>>>>>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>> *create)
>>>>>>>>>>   {
>>>>>>>>>>          unsigned requested_planes = 1;
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>> video_device *vdev)
>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>          }
>>>>>>>>>>
>>>>>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, fh, 
>>>>>>>>>> b);
>>>>>>>>>>   }
>>>>>>>>>>
>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> +                         struct file *file, void *fh, void 
>>>>>>>>>> *arg)
>>>>>>>>>> +{
>>>>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>>>>> +
>>>>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>                                  struct file *file, void *fh,
>>>>>>>>>> void
>>>>>>>>>> *arg)
>>>>>>>>>>   {
>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>> match)),
>>>>>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>   };
>>>>>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>
>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>    * @vidioc_prepare_buf: pointer to the function that 
>>>>>>>>>> implements
>>>>>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>                                    struct v4l2_create_buffers
>>>>>>>>>> *b);
>>>>>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>>                                    struct v4l2_buffer *b);
>>>>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>> unsigned
>>>>>>>>>> int i);
>>>>>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>    */
>>>>>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index,
>>>>>>>>>> void *pb);
>>>>>>>>>>
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>> + * @index:     id number of the buffer.
>>>>>>>>>> + *
>>>>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index);
>>>>>>>>>> +
>>>>>>>>>>   /**
>>>>>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>    *
>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>    */
>>>>>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>> *mdev,
>>>>>>>>>>                      struct v4l2_buffer *b);
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>> + *
>>>>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2
>>>>>>>>>> queue.
>>>>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>> + *
>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>> directly
>>>>>>>>>> returned
>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>>   /**
>>>>>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>
>>>>>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct
>>>>>>>>>> v4l2_buffer)
>>>>>>>>>> +
>>>>>>>>>>
>>>>>>>>>>   /* Reminder: when adding new ioctls please add support for
>>>>>>>>>> them to
>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>> -- 
>>>>>>>>>> 2.39.2
>>>>>>>>>>
>>>>>>> -- 
>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>

Benjamin Gaignard July 3, 2023, 11:17 a.m. UTC | #18

Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>
>
> On 7/3/23 18:35, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open 
>> attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or 
>>>>>>>>>>> open
>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>> content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>> <[e-mail]>
>>>>>>>>>>> ---
>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>>   10 files changed, 128 insertions(+)
>>>>>>>>>>>   create mode 100644
>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>
>>>>>>>>>>> diff --git 
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>>       vidioc-dbg-g-chip-info
>>>>>>>>>>>       vidioc-dbg-g-register
>>>>>>>>>>>       vidioc-decoder-cmd
>>>>>>>>>>> +    vidioc-delete-buf
>>>>>>>>>>>       vidioc-dqevent
>>>>>>>>>>>       vidioc-dv-timings-cap
>>>>>>>>>>>       vidioc-encoder-cmd
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> new file mode 100644
>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>> --- /dev/null
>>>>>>>>>>> +++ 
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>> +
>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>> +
>>>>>>>>>>> +************************
>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>> +************************
>>>>>>>>>>> +
>>>>>>>>>>> +Name
>>>>>>>>>>> +====
>>>>>>>>>>> +
>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>> +
>>>>>>>>>>> +Synopsis
>>>>>>>>>>> +========
>>>>>>>>>>> +
>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>> +
>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>> *argp)``
>>>>>>>>>>> +
>>>>>>>>>>> +Arguments
>>>>>>>>>>> +=========
>>>>>>>>>>> +
>>>>>>>>>>> +``fd``
>>>>>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>>>>>> +
>>>>>>>>>>> +``argp``
>>>>>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Description
>>>>>>>>>>> +===========
>>>>>>>>>>> +
>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>> ioctl to
>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>> +
>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Return Value
>>>>>>>>>>> +============
>>>>>>>>>>> +
>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>> variable is
>>>>>>>>>>> set
>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>> +
>>>>>>>>>>> +EBUSY
>>>>>>>>>>> +    File I/O is in progress.
>>>>>>>>>>> +
>>>>>>>>>>> +EINVAL
>>>>>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct 
>>>>>>>>>>> vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>>   }
>>>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int 
>>>>>>>>>>> index)
>>>>>>>>>>> +{
>>>>>>>>>>> +       struct vb2_buffer *vb;
>>>>>>>>>>> +
>>>>>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>>>>>> +       if (!vb) {
>>>>>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>> index);
>>>>>>>>>>> +               return -EINVAL;
>>>>>>>>>>> +       }
>>>>>>>>>>> +
>>>>>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>>> index
>>>>>>>>>>> %d\n", index);
>>>>>>>>>>> +               return -EINVAL;
>>>>>>>>>>> +       }
>>>>>>>>>>> +
>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a 
>>>>>>>>>> buffer is
>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>> still
>>>>>>>>>> used by the hardware.
>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>> it is
>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>>> could
>>>>>>>>>> last much longer.
>>>>>>>>>
>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>> information for that.
>>>>>>>>
>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>
>>>>>>>> While application for stateful decoder could never do that. It 
>>>>>>>> also
>>>>>>>> breaks what the document said:
>>>>>>>>
>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as 
>>>>>>>> reference
>>>>>>>> frames by the stream may be read by the hardware even after 
>>>>>>>> they are
>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>> memory
>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this 
>>>>>>>> may
>>>>>>>> result in corruption of decoded frames."
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>> +
>>>>>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>> +               __vb2_buf_mem_free(vb);
>>>>>>>
>>>>>>> Here is another problem for the existing application, the mmap() 
>>>>>>> from
>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>> buffer in this step(while the exportbuffer would create one 
>>>>>>> itself).
>>>>>>>
>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>
>>>>>>> "After  the  mmap()  call has returned, the file descriptor, fd, 
>>>>>>> can
>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>
>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>> they will be aware that the buffer is removed.
>>>>>> I have done it in GStreamer:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I have read that.
>>>>>
>>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>>> when deal with the secure video(userspace can't access the data
>>>>> context at all).
>>>>>
>>>>> Besides, this adds extra work for the application for a stateful 
>>>>> codec
>>>>> driver. The application need to parser the bitstream and track the 
>>>>> dpb.
>>>>>
>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>> non-display frame and internal reference state.
>>>>>
>>>>> That could be requirement for codec firmware that its driver could
>>>>> support this DELETE_BUF ioctl() feature.
>>>>
>>>> Sorry I don't see the link with my patches here...
>>>> I have work on non-secure VP9 on stateless codec.
>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>> applications
>>>> to save memory if they know when they could delete buffers without 
>>>> risk.
>>>
>>> I try to explain why I think this design in not "complete". One
>>> problem resolved, more problems would occur.
>>>
>>>
>>> For non-secure video, those applications have worked:
>>>
>>> - It would break what stateful means here, application need to
>>> acquire(parse) the information that driver should offer.
>>>
>>>   Or it would break the decoding model.
>>>
>>> - Your Gstreamer sample code or this design won't work for AV1.
>> why ?
>
> A frame in AV1 could be put to display order with a future frame. That 
> is quite different to those ITU codecs.
>
> Supposing a frame (we call it frame 0) without show_display frame, you 
> could delete it (just want to do that) once it is decoded.
>
> Actually, unless 7 future frames have came and parser has acknowledged 
> them, it is not safe to delete that frame.
>
> Or a show_exist that make frame 0 present.
>
> In my personal opinion, this feature could break many uAPI's, I have 
> said my worrying in the other emails.
>
Of course if the application delete a useful buffer it is a problem but an application issue.
I have test this code using GStreamer with VP9, VP8 and HEVC on Verisilicon driver without issues.
I will be happy if someone could test it on another driver and/or other userlands (like ffmpeg).

>>>
>>> For all the future possible secure video:
>>>
>>> - This feature could never be used from the current design.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Benjamin
>>>>>>
>>>>>>>
>>>>>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>> +       else
>>>>>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>>>>>> +
>>>>>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>> +       kfree(vb);
>>>>>>>>>>> +
>>>>>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>> +       return 0;
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>>   /*
>>>>>>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>>    * @q:         videobuf2 queue
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>>   }
>>>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>>> +{
>>>>>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>> +}
>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>> +
>>>>>>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>> *create)
>>>>>>>>>>>   {
>>>>>>>>>>>          unsigned requested_planes = 1;
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>>          }
>>>>>>>>>>>
>>>>>>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, 
>>>>>>>>>>> fh, b);
>>>>>>>>>>>   }
>>>>>>>>>>>
>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> +                         struct file *file, void *fh, void 
>>>>>>>>>>> *arg)
>>>>>>>>>>> +{
>>>>>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>>>>>> +
>>>>>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>                                  struct file *file, void *fh,
>>>>>>>>>>> void
>>>>>>>>>>> *arg)
>>>>>>>>>>>   {
>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, 
>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, 
>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>> match)),
>>>>>>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>>   };
>>>>>>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>>    * @vidioc_prepare_buf: pointer to the function that 
>>>>>>>>>>> implements
>>>>>>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>>                                    struct v4l2_create_buffers
>>>>>>>>>>> *b);
>>>>>>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>>>                                    struct v4l2_buffer *b);
>>>>>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>>>>>
>>>>>>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>> unsigned
>>>>>>>>>>> int i);
>>>>>>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>>    */
>>>>>>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index,
>>>>>>>>>>> void *pb);
>>>>>>>>>>>
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>> + * @index:     id number of the buffer.
>>>>>>>>>>> + *
>>>>>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index);
>>>>>>>>>>> +
>>>>>>>>>>>   /**
>>>>>>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>    *
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>>    */
>>>>>>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>> *mdev,
>>>>>>>>>>>                      struct v4l2_buffer *b);
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>> + *
>>>>>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>> queue.
>>>>>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>> + *
>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>> directly
>>>>>>>>>>> returned
>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer 
>>>>>>>>>>> *b);
>>>>>>>>>>>
>>>>>>>>>>>   /**
>>>>>>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>
>>>>>>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct
>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>> +
>>>>>>>>>>>
>>>>>>>>>>>   /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>> them to
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>> -- 
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> -- 
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
> -- 
> Hsia-Jun(Randy) Li

ayaka July 3, 2023, 3:42 p.m. UTC | #19

On 2023/7/3 19:17, Benjamin Gaignard wrote:
>
> Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>>
>>
>> On 7/3/23 18:35, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open 
>>> attachments unless you recognize the sender and know the content is 
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the 
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>>
>>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>>> CAUTION: Email originated externally, do not click links or 
>>>>>>>>>>>> open
>>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>>> content is
>>>>>>>>>>>> safe.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a 
>>>>>>>>>>>> queue.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>>> <[e-mail]>
>>>>>>>>>>>> ---
>>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>>>   10 files changed, 128 insertions(+)
>>>>>>>>>>>>   create mode 100644
>>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git 
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>>>       vidioc-dbg-g-chip-info
>>>>>>>>>>>>       vidioc-dbg-g-register
>>>>>>>>>>>>       vidioc-decoder-cmd
>>>>>>>>>>>> +    vidioc-delete-buf
>>>>>>>>>>>>       vidioc-dqevent
>>>>>>>>>>>>       vidioc-dv-timings-cap
>>>>>>>>>>>>       vidioc-encoder-cmd
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> new file mode 100644
>>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>>> --- /dev/null
>>>>>>>>>>>> +++ 
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>>> +
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +
>>>>>>>>>>>> +Name
>>>>>>>>>>>> +====
>>>>>>>>>>>> +
>>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>>> +
>>>>>>>>>>>> +Synopsis
>>>>>>>>>>>> +========
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>>> +
>>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>>> *argp)``
>>>>>>>>>>>> +
>>>>>>>>>>>> +Arguments
>>>>>>>>>>>> +=========
>>>>>>>>>>>> +
>>>>>>>>>>>> +``fd``
>>>>>>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +``argp``
>>>>>>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Description
>>>>>>>>>>>> +===========
>>>>>>>>>>>> +
>>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>>> ioctl to
>>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>>> +
>>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Return Value
>>>>>>>>>>>> +============
>>>>>>>>>>>> +
>>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>>> variable is
>>>>>>>>>>>> set
>>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EBUSY
>>>>>>>>>>>> +    File I/O is in progress.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EINVAL
>>>>>>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct 
>>>>>>>>>>>> vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>>>   }
>>>>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int 
>>>>>>>>>>>> index)
>>>>>>>>>>>> +{
>>>>>>>>>>>> +       struct vb2_buffer *vb;
>>>>>>>>>>>> +
>>>>>>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>>>>>>> +       if (!vb) {
>>>>>>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>>> index);
>>>>>>>>>>>> +               return -EINVAL;
>>>>>>>>>>>> +       }
>>>>>>>>>>>> +
>>>>>>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>>> +               dprintk(q, 1, "can't delete non dequeued 
>>>>>>>>>>>> buffer
>>>>>>>>>>>> index
>>>>>>>>>>>> %d\n", index);
>>>>>>>>>>>> +               return -EINVAL;
>>>>>>>>>>>> +       }
>>>>>>>>>>>> +
>>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a 
>>>>>>>>>>> buffer is
>>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>>> still
>>>>>>>>>>> used by the hardware.
>>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>>> it is
>>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>>> While the Golden reference frame, such long term reference 
>>>>>>>>>>> frame
>>>>>>>>>>> could
>>>>>>>>>>> last much longer.
>>>>>>>>>>
>>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>>> information for that.
>>>>>>>>>
>>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>>
>>>>>>>>> While application for stateful decoder could never do that. It 
>>>>>>>>> also
>>>>>>>>> breaks what the document said:
>>>>>>>>>
>>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as 
>>>>>>>>> reference
>>>>>>>>> frames by the stream may be read by the hardware even after 
>>>>>>>>> they are
>>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>>> memory
>>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe 
>>>>>>>>> this may
>>>>>>>>> result in corruption of decoded frames."
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>>> +               __vb2_buf_mem_free(vb);
>>>>>>>>
>>>>>>>> Here is another problem for the existing application, the 
>>>>>>>> mmap() from
>>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>>> buffer in this step(while the exportbuffer would create one 
>>>>>>>> itself).
>>>>>>>>
>>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>>
>>>>>>>> "After  the  mmap()  call has returned, the file descriptor, 
>>>>>>>> fd, can
>>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>>
>>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>>> they will be aware that the buffer is removed.
>>>>>>> I have done it in GStreamer:
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e= 
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I have read that.
>>>>>>
>>>>>> There is not a VP8 parser in Gstreamer, while a parser would not 
>>>>>> work
>>>>>> when deal with the secure video(userspace can't access the data
>>>>>> context at all).
>>>>>>
>>>>>> Besides, this adds extra work for the application for a stateful 
>>>>>> codec
>>>>>> driver. The application need to parser the bitstream and track 
>>>>>> the dpb.
>>>>>>
>>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>>> non-display frame and internal reference state.
>>>>>>
>>>>>> That could be requirement for codec firmware that its driver could
>>>>>> support this DELETE_BUF ioctl() feature.
>>>>>
>>>>> Sorry I don't see the link with my patches here...
>>>>> I have work on non-secure VP9 on stateless codec.
>>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>>> applications
>>>>> to save memory if they know when they could delete buffers without 
>>>>> risk.
>>>>
>>>> I try to explain why I think this design in not "complete". One
>>>> problem resolved, more problems would occur.
>>>>
>>>>
>>>> For non-secure video, those applications have worked:
>>>>
>>>> - It would break what stateful means here, application need to
>>>> acquire(parse) the information that driver should offer.
>>>>
>>>>   Or it would break the decoding model.
>>>>
>>>> - Your Gstreamer sample code or this design won't work for AV1.
>>> why ?
>>
>> A frame in AV1 could be put to display order with a future frame. 
>> That is quite different to those ITU codecs.
>>
>> Supposing a frame (we call it frame 0) without show_display frame, 
>> you could delete it (just want to do that) once it is decoded.
>>
>> Actually, unless 7 future frames have came and parser has 
>> acknowledged them, it is not safe to delete that frame.
>>
>> Or a show_exist that make frame 0 present.
>>
>> In my personal opinion, this feature could break many uAPI's, I have 
>> said my worrying in the other emails.
>>
> Of course if the application delete a useful buffer it is a problem 
> but an application issue.

I don't think this feature is only applied to stateless codec. Stateful 
codec could reclaim the previous memory as well.

The implementation now just leave all those problems to the conscience 
of the application, no protection from the kernel which could crash the 
hardware easily.

I am not against  on this feature, I just wish it would be more safe and 
more complete.

> I have test this code using GStreamer with VP9, VP8 and HEVC on 
> Verisilicon driver without issues.

Even with the secure decoding, at least someone should offer the DPB 
info or we just don't need this.

I am sure stateless decoder could work well with that despite the 
performance impact on buffer looking up.

>
> I will be happy if someone could test it on another driver and/or 
> other userlands (like ffmpeg).
>
>>>>
>>>> For all the future possible secure video:
>>>>
>>>> - This feature could never be used from the current design.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Benjamin
>>>>>>>
>>>>>>>>
>>>>>>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>>> +       else
>>>>>>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>>> +       kfree(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>>> +       return 0;
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>>   /*
>>>>>>>>>>>>    * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>>>    * @q:         videobuf2 queue
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>>>   }
>>>>>>>>>>>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer 
>>>>>>>>>>>> *b)
>>>>>>>>>>>> +{
>>>>>>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>>> +
>>>>>>>>>>>>   int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>>> *create)
>>>>>>>>>>>>   {
>>>>>>>>>>>>          unsigned requested_planes = 1;
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>>>                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>>>          }
>>>>>>>>>>>>
>>>>>>>>>>>>          if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>>>          return ret ? ret : ops->vidioc_prepare_buf(file, 
>>>>>>>>>>>> fh, b);
>>>>>>>>>>>>   }
>>>>>>>>>>>>
>>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> +                         struct file *file, void *fh, void 
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> +{
>>>>>>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>>>>>>> +
>>>>>>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, 
>>>>>>>>>>>> b);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>>   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>>                                  struct file *file, void *fh,
>>>>>>>>>>>> void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>>   {
>>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>>>          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, 
>>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>>>          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, 
>>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>>> match)),
>>>>>>>>>>>>          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>>>   };
>>>>>>>>>>>>   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>>>    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>>>    * @vidioc_prepare_buf: pointer to the function that 
>>>>>>>>>>>> implements
>>>>>>>>>>>>    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that 
>>>>>>>>>>>> implements
>>>>>>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>>>    * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>>>    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>>>    * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>>>                                    struct v4l2_create_buffers
>>>>>>>>>>>> *b);
>>>>>>>>>>>>          int (*vidioc_prepare_buf)(struct file *file, void 
>>>>>>>>>>>> *fh,
>>>>>>>>>>>>                                    struct v4l2_buffer *b);
>>>>>>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>>>>>>
>>>>>>>>>>>>          int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>>> unsigned
>>>>>>>>>>>> int i);
>>>>>>>>>>>>          int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>>>    */
>>>>>>>>>>>>   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index,
>>>>>>>>>>>> void *pb);
>>>>>>>>>>>>
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>>> + * @index:     id number of the buffer.
>>>>>>>>>>>> + *
>>>>>>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index);
>>>>>>>>>>>> +
>>>>>>>>>>>>   /**
>>>>>>>>>>>>    * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>>    *
>>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>>>    */
>>>>>>>>>>>>   int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>>> *mdev,
>>>>>>>>>>>>                      struct v4l2_buffer *b);
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>>> queue.
>>>>>>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>>> directly
>>>>>>>>>>>> returned
>>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer 
>>>>>>>>>>>> *b);
>>>>>>>>>>>>
>>>>>>>>>>>>   /**
>>>>>>>>>>>>    * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>>>   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>>
>>>>>>>>>>>>   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct
>>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>>> +
>>>>>>>>>>>>
>>>>>>>>>>>>   /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>>> them to
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>>> -- 
>>>>>>>>>>>> 2.39.2
>>>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>>
>> -- 
>> Hsia-Jun(Randy) Li

Tomasz Figa July 13, 2023, 9:09 a.m. UTC | #20

On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
> 
> On 6/27/23 16:47, Hsia-Jun Li wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> > 
> > 
> > On 6/27/23 16:43, Benjamin Gaignard wrote:
> > > CAUTION: Email originated externally, do not click links or open
> > > attachments unless you recognize the sender and know the content is
> > > safe.
> > > 
> > > 
> > > Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> > > > 
> > > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > > CAUTION: Email originated externally, do not click links or open
> > > > > attachments unless you recognize the sender and know the content is
> > > > > safe.
> > > > > 
> > > > > 
> > > > > VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> > > > > 
> > > > > Signed-off-by: Benjamin Gaignard <[e-mail]>
> > > > > ---
> > > > >   .../userspace-api/media/v4l/user-func.rst     |  1 +
> > > > >   .../media/v4l/vidioc-delete-buf.rst           | 51
> > > > > +++++++++++++++++++
> > > > >   .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
> > > > >   .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
> > > > >   drivers/media/v4l2-core/v4l2-dev.c            |  1 +
> > > > >   drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
> > > > >   include/media/v4l2-ioctl.h                    |  4 ++
> > > > >   include/media/videobuf2-core.h                |  9 ++++
> > > > >   include/media/videobuf2-v4l2.h                | 11 ++++
> > > > >   include/uapi/linux/videodev2.h                |  2 +
> > > > >   10 files changed, 128 insertions(+)
> > > > >   create mode 100644
> > > > > Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > 
> > > > > diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > index 15ff0bf7bbe6..8c74016e12fd 100644
> > > > > --- a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > @@ -17,6 +17,7 @@ Function Reference
> > > > >       vidioc-dbg-g-chip-info
> > > > >       vidioc-dbg-g-register
> > > > >       vidioc-decoder-cmd
> > > > > +    vidioc-delete-buf
> > > > >       vidioc-dqevent
> > > > >       vidioc-dv-timings-cap
> > > > >       vidioc-encoder-cmd
> > > > > diff --git
> > > > > a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > new file mode 100644
> > > > > index 000000000000..0e7ce58f91bc
> > > > > --- /dev/null
> > > > > +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > @@ -0,0 +1,51 @@
> > > > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > > > > +.. c:namespace:: V4L
> > > > > +
> > > > > +.. _VIDIOC_DELETE_BUF:
> > > > > +
> > > > > +************************
> > > > > +ioctl VIDIOC_DELETE_BUF
> > > > > +************************
> > > > > +
> > > > > +Name
> > > > > +====
> > > > > +
> > > > > +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> > > > > +
> > > > > +Synopsis
> > > > > +========
> > > > > +
> > > > > +.. c:macro:: VIDIOC_DELETE_BUF
> > > > > +
> > > > > +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> > > > > +
> > > > > +Arguments
> > > > > +=========
> > > > > +
> > > > > +``fd``
> > > > > +    File descriptor returned by :c:func:`open()`.
> > > > > +
> > > > > +``argp``
> > > > > +    Pointer to struct :c:type:`v4l2_buffer`.
> > > > > +
> > > > > +Description
> > > > > +===========
> > > > > +
> > > > > +Applications can optionally call the
> > > > > :ref:`VIDIOC_DELETE_BUF` ioctl to
> > > > > +delete a buffer from a queue.
> > > > > +
> > > > > +The struct :c:type:`v4l2_buffer` structure is specified in
> > > > > +:ref:`buffer`.
> > > > > +
> > > > > +Return Value
> > > > > +============
> > > > > +
> > > > > +On success 0 is returned, on error -1 and the ``errno`` variable is
> > > > > set
> > > > > +appropriately. The generic error codes are described at the
> > > > > +:ref:`Generic Error Codes <gen-errors>` chapter.
> > > > > +
> > > > > +EBUSY
> > > > > +    File I/O is in progress.
> > > > > +
> > > > > +EINVAL
> > > > > +    The buffer ``index`` doesn't exist in the queue.
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > index 899783f67580..aa546c972c3d 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> > > > > unsigned int index, void *pb)
> > > > >   }
> > > > >   EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> > > > > 
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> > > > > +{
> > > > > +       struct vb2_buffer *vb;
> > > > > +
> > > > > +       vb = vb2_get_buffer(q, index);
> > > > > +       if (!vb) {
> > > > > +               dprintk(q, 1, "invalid buffer index %d\n", index);
> > > > > +               return -EINVAL;
> > > > > +       }
> > > > > +
> > > > > +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> > > > > +               dprintk(q, 1, "can't delete non dequeued buffer index
> > > > > %d\n", index);
> > > > > +               return -EINVAL;
> > > > > +       }
> > > > > +
> > > > I know the driver could implement its own
> > > > v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> > > > used by the hardware as a future reference frame.
> > > > But I think we need a flag to let the user know which buffer is still
> > > > used by the hardware.
> > > > Alternative ref case is safe, we only know it's existing when it is
> > > > dequeued in current V4L2 buffer mechanism.
> > > > While the Golden reference frame, such long term reference frame could
> > > > last much longer.
> > > 
> > > It is up to userland stack to know frames life time, it got the
> > > information for that.
> > 
> > That is true for the stateless codec driver.
> > 
> > While application for stateful decoder could never do that. It also
> > breaks what the document said:
> > 
> > "The backing memory of |CAPTURE| buffers that are used as reference
> > frames by the stream may be read by the hardware even after they are
> > dequeued. Consequently, the client should avoid writing into this memory
> > while the |CAPTURE| queue is streaming. Failure to observe this may
> > result in corruption of decoded frames."
> > 
> > > 
> > > > > +       if (vb->planes[0].mem_priv)
> > > > > +               call_void_vb_qop(vb, buf_cleanup, vb);
> > > > > +
> > > > > +       /* Free MMAP buffers or release USERPTR buffers */
> > > > > +       if (q->memory == VB2_MEMORY_MMAP)
> > > > > +               __vb2_buf_mem_free(vb);
> 
> Here is another problem for the existing application, the mmap() from the
> mmap offset or exportbuffer fd would not create a reference to buffer in
> this step(while the exportbuffer would create one itself).
> 
> When you delete a buffer, you may not release it from its virtual memory
> space, leaving a corrupted virtual memory space.

What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
memory, it just decrements a reference counter. The VMA holds its own,
so the buffer is only fully released when the application calls
munmap().

Best regards,
Tomasz

> Also this behavior is
> right, because mmap(2) says:
> 
> "After  the  mmap()  call has returned, the file descriptor, fd, can be
> closed immediately without invalidating the map‐ping."
> 
> > > > > +       else if (q->memory == VB2_MEMORY_DMABUF)
> > > > > +               __vb2_buf_dmabuf_put(vb);
> > > > > +       else
> > > > > +               __vb2_buf_userptr_put(vb);
> > > > > +
> > > > > +       vb2_queue_remove_buffer(q, vb);
> > > > > +       kfree(vb);
> > > > > +
> > > > > +       dprintk(q, 2, "buffer %d deleted\n", index);
> > > > > +       return 0;
> > > > > +}
> > > > > +
> > > > >   /*
> > > > >    * vb2_start_streaming() - Attempt to start streaming.
> > > > >    * @q:         videobuf2 queue
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > index 724135d41f7f..cea666c17b41 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> > > > > media_device *mdev,
> > > > >   }
> > > > >   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> > > > > 
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> > > > > +{
> > > > > +       return vb2_core_delete_buf(q, b->index);
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> > > > > +
> > > > >   int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> > > > > *create)
> > > > >   {
> > > > >          unsigned requested_planes = 1;
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > index f81279492682..80ace2e1e932 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> > > > > video_device *vdev)
> > > > >                  SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> > > > > vidioc_prepare_buf);
> > > > >                  SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> > > > > vidioc_streamon);
> > > > >                  SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> > > > > vidioc_streamoff);
> > > > > +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> > > > > vidioc_delete_buf);
> > > > >          }
> > > > > 
> > > > >          if (is_vid || is_vbi || is_meta) {
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > index a858acea6547..1c737279d3ef 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> > > > > v4l2_ioctl_ops *ops,
> > > > >          return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> > > > >   }
> > > > > 
> > > > > +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> > > > > +                         struct file *file, void *fh, void *arg)
> > > > > +{
> > > > > +       struct v4l2_buffer *b = arg;
> > > > > +       int ret = check_fmt(file, b->type);
> > > > > +
> > > > > +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> > > > > +}
> > > > > +
> > > > >   static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> > > > >                                  struct file *file, void *fh, void
> > > > > *arg)
> > > > >   {
> > > > > @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> > > > > v4l2_ioctls[] = {
> > > > >          IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> > > > > v4l_print_freq_band, 0),
> > > > >          IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> > > > > v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> > > > >          IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> > > > > v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> > > > > INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> > > > > +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> > > > > v4l_print_buffer, INFO_FL_QUEUE),
> > > > >   };
> > > > >   #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> > > > > 
> > > > > diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> > > > > index edb733f21604..2f232ed884c7 100644
> > > > > --- a/include/media/v4l2-ioctl.h
> > > > > +++ b/include/media/v4l2-ioctl.h
> > > > > @@ -163,6 +163,8 @@ struct v4l2_fh;
> > > > >    *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> > > > >    * @vidioc_prepare_buf: pointer to the function that implements
> > > > >    *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> > > > > + * @vidioc_delete_buf: pointer to the function that implements
> > > > > + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> > > > >    * @vidioc_overlay: pointer to the function that implements
> > > > >    *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> > > > >    * @vidioc_g_fbuf: pointer to the function that implements
> > > > > @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> > > > >                                    struct v4l2_create_buffers *b);
> > > > >          int (*vidioc_prepare_buf)(struct file *file, void *fh,
> > > > >                                    struct v4l2_buffer *b);
> > > > > +       int (*vidioc_delete_buf)(struct file *file, void *fh,
> > > > > +                                struct v4l2_buffer *b);
> > > > > 
> > > > >          int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> > > > > int i);
> > > > >          int (*vidioc_g_fbuf)(struct file *file, void *fh,
> > > > > diff --git a/include/media/videobuf2-core.h
> > > > > b/include/media/videobuf2-core.h
> > > > > index 080b783d608d..0f9e68f76b77 100644
> > > > > --- a/include/media/videobuf2-core.h
> > > > > +++ b/include/media/videobuf2-core.h
> > > > > @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> > > > > enum vb2_memory memory,
> > > > >    */
> > > > >   int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> > > > > void *pb);
> > > > > 
> > > > > +/**
> > > > > + * vb2_core_delete_buf() -
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @index:     id number of the buffer.
> > > > > + *
> > > > > + *  Return: returns zero on success; an error code otherwise.
> > > > > + */
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> > > > > +
> > > > >   /**
> > > > >    * vb2_core_qbuf() - Queue a buffer from userspace
> > > > >    *
> > > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > > b/include/media/videobuf2-v4l2.h
> > > > > index 88a7a565170e..3beeb4c735f0 100644
> > > > > --- a/include/media/videobuf2-v4l2.h
> > > > > +++ b/include/media/videobuf2-v4l2.h
> > > > > @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> > > > > v4l2_create_buffers *create);
> > > > >    */
> > > > >   int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> > > > >                      struct v4l2_buffer *b);
> > > > > +/**
> > > > > + * vb2_delete_buf() - Delete the buffer from the queue
> > > > > + *
> > > > > + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @b:         buffer structure passed from userspace to
> > > > > + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> > > > > + *
> > > > > + * The return values from this function are intended to be directly
> > > > > returned
> > > > > + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> > > > > + */
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> > > > > 
> > > > >   /**
> > > > >    * vb2_qbuf() - Queue a buffer from userspace
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index aee75eb9e686..31bba1915642 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> > > > >   #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
> > > > > v4l2_dbg_chip_info)
> > > > > 
> > > > >   #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
> > > > > v4l2_query_ext_ctrl)
> > > > > +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
> > > > > +
> > > > > 
> > > > >   /* Reminder: when adding new ioctls please add support for them to
> > > > >      drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> > > > > -- 
> > > > > 2.39.2
> > > > > 
> > -- 
> > Hsia-Jun(Randy) Li
> > 
> -- 
> Hsia-Jun(Randy) Li
>

Hsia-Jun Li July 17, 2023, 2:16 a.m. UTC | #21

On 7/13/23 17:09, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>>>> ---
>>>>>>    .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>>>    .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>>> +++++++++++++++++++
>>>>>>    .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>>    .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>>>    drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>>>    drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>>    include/media/v4l2-ioctl.h                    |  4 ++
>>>>>>    include/media/videobuf2-core.h                |  9 ++++
>>>>>>    include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>>    include/uapi/linux/videodev2.h                |  2 +
>>>>>>    10 files changed, 128 insertions(+)
>>>>>>    create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>        vidioc-dbg-g-chip-info
>>>>>>        vidioc-dbg-g-register
>>>>>>        vidioc-decoder-cmd
>>>>>> +    vidioc-delete-buf
>>>>>>        vidioc-dqevent
>>>>>>        vidioc-dv-timings-cap
>>>>>>        vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the
>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> +    File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>>    }
>>>>>>    EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> +       struct vb2_buffer *vb;
>>>>>> +
>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>> +       if (!vb) {
>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> +               return -EINVAL;
>>>>>> +       }
>>>>>> +
>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>> %d\n", index);
>>>>>> +               return -EINVAL;
>>>>>> +       }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame could
>>>>> last much longer.
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>> +               __vb2_buf_mem_free(vb);
>> Here is another problem for the existing application, the mmap() from the
>> mmap offset or exportbuffer fd would not create a reference to buffer in
>> this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual memory
>> space, leaving a corrupted virtual memory space.
> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> memory, it just decrements a reference counter.

struct dma_buf_ops->mmap() may not increase a reference to its buffer.

While struct vb2_mem_ops->get_dmabuf() would.

> The VMA holds its own,
> so the buffer is only fully released when the application calls
> munmap().

DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF 
ioctl() make the no other user could access to this.

>
> Best regards,
> Tomasz
>
>> Also this behavior is
>> right, because mmap(2) says:
>>
>> "After  the  mmap()  call has returned, the file descriptor, fd, can be
>> closed immediately without invalidating the map‐ping."
>>
>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>> +       else
>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>> +       kfree(vb);
>>>>>> +
>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> +       return 0;
>>>>>> +}
>>>>>> +
>>>>>>    /*
>>>>>>     * vb2_start_streaming() - Attempt to start streaming.
>>>>>>     * @q:         videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>>    }
>>>>>>    EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>>    int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>> *create)
>>>>>>    {
>>>>>>           unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>>                   SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>>                   SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>>                   SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>>           }
>>>>>>
>>>>>>           if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>>           return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>    }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> +                         struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>>    static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>                                   struct file *file, void *fh, void
>>>>>> *arg)
>>>>>>    {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>>           IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>>           IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>           IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>    };
>>>>>>    #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>     *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>     * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>     *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>     * @vidioc_overlay: pointer to the function that implements
>>>>>>     *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>     * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>                                     struct v4l2_create_buffers *b);
>>>>>>           int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>                                     struct v4l2_buffer *b);
>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> +                                struct v4l2_buffer *b);
>>>>>>
>>>>>>           int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>>           int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>>     */
>>>>>>    int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index:     id number of the buffer.
>>>>>> + *
>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>>    /**
>>>>>>     * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>     *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>>     */
>>>>>>    int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>>                       struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>>    /**
>>>>>>     * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>    #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>>    #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>>    /* Reminder: when adding new ioctls please add support for them to
>>>>>>       drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
>> --
>> Hsia-Jun(Randy) Li
>>

Tomasz Figa July 28, 2023, 6:57 a.m. UTC | #22

On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <[e-mail]> wrote:
>
>
> On 7/13/23 17:09, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
> >> On 6/27/23 16:47, Hsia-Jun Li wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> On 6/27/23 16:43, Benjamin Gaignard wrote:
> >>>> CAUTION: Email originated externally, do not click links or open
> >>>> attachments unless you recognize the sender and know the content is
> >>>> safe.
> >>>>
> >>>>
> >>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> >>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>> attachments unless you recognize the sender and know the content is
> >>>>>> safe.
> >>>>>>
> >>>>>>
> >>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> >>>>>>
> >>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
> >>>>>> ---
> >>>>>>    .../userspace-api/media/v4l/user-func.rst     |  1 +
> >>>>>>    .../media/v4l/vidioc-delete-buf.rst           | 51
> >>>>>> +++++++++++++++++++
> >>>>>>    .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
> >>>>>>    .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
> >>>>>>    drivers/media/v4l2-core/v4l2-dev.c            |  1 +
> >>>>>>    drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
> >>>>>>    include/media/v4l2-ioctl.h                    |  4 ++
> >>>>>>    include/media/videobuf2-core.h                |  9 ++++
> >>>>>>    include/media/videobuf2-v4l2.h                | 11 ++++
> >>>>>>    include/uapi/linux/videodev2.h                |  2 +
> >>>>>>    10 files changed, 128 insertions(+)
> >>>>>>    create mode 100644
> >>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>>
> >>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
> >>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> @@ -17,6 +17,7 @@ Function Reference
> >>>>>>        vidioc-dbg-g-chip-info
> >>>>>>        vidioc-dbg-g-register
> >>>>>>        vidioc-decoder-cmd
> >>>>>> +    vidioc-delete-buf
> >>>>>>        vidioc-dqevent
> >>>>>>        vidioc-dv-timings-cap
> >>>>>>        vidioc-encoder-cmd
> >>>>>> diff --git
> >>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> new file mode 100644
> >>>>>> index 000000000000..0e7ce58f91bc
> >>>>>> --- /dev/null
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> @@ -0,0 +1,51 @@
> >>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>>>> +.. c:namespace:: V4L
> >>>>>> +
> >>>>>> +.. _VIDIOC_DELETE_BUF:
> >>>>>> +
> >>>>>> +************************
> >>>>>> +ioctl VIDIOC_DELETE_BUF
> >>>>>> +************************
> >>>>>> +
> >>>>>> +Name
> >>>>>> +====
> >>>>>> +
> >>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> >>>>>> +
> >>>>>> +Synopsis
> >>>>>> +========
> >>>>>> +
> >>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
> >>>>>> +
> >>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> >>>>>> +
> >>>>>> +Arguments
> >>>>>> +=========
> >>>>>> +
> >>>>>> +``fd``
> >>>>>> +    File descriptor returned by :c:func:`open()`.
> >>>>>> +
> >>>>>> +``argp``
> >>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
> >>>>>> +
> >>>>>> +Description
> >>>>>> +===========
> >>>>>> +
> >>>>>> +Applications can optionally call the
> >>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
> >>>>>> +delete a buffer from a queue.
> >>>>>> +
> >>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
> >>>>>> +:ref:`buffer`.
> >>>>>> +
> >>>>>> +Return Value
> >>>>>> +============
> >>>>>> +
> >>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
> >>>>>> set
> >>>>>> +appropriately. The generic error codes are described at the
> >>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
> >>>>>> +
> >>>>>> +EBUSY
> >>>>>> +    File I/O is in progress.
> >>>>>> +
> >>>>>> +EINVAL
> >>>>>> +    The buffer ``index`` doesn't exist in the queue.
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> index 899783f67580..aa546c972c3d 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> >>>>>> unsigned int index, void *pb)
> >>>>>>    }
> >>>>>>    EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> >>>>>> +{
> >>>>>> +       struct vb2_buffer *vb;
> >>>>>> +
> >>>>>> +       vb = vb2_get_buffer(q, index);
> >>>>>> +       if (!vb) {
> >>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
> >>>>>> +               return -EINVAL;
> >>>>>> +       }
> >>>>>> +
> >>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> >>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer index
> >>>>>> %d\n", index);
> >>>>>> +               return -EINVAL;
> >>>>>> +       }
> >>>>>> +
> >>>>> I know the driver could implement its own
> >>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> >>>>> used by the hardware as a future reference frame.
> >>>>> But I think we need a flag to let the user know which buffer is still
> >>>>> used by the hardware.
> >>>>> Alternative ref case is safe, we only know it's existing when it is
> >>>>> dequeued in current V4L2 buffer mechanism.
> >>>>> While the Golden reference frame, such long term reference frame could
> >>>>> last much longer.
> >>>> It is up to userland stack to know frames life time, it got the
> >>>> information for that.
> >>> That is true for the stateless codec driver.
> >>>
> >>> While application for stateful decoder could never do that. It also
> >>> breaks what the document said:
> >>>
> >>> "The backing memory of |CAPTURE| buffers that are used as reference
> >>> frames by the stream may be read by the hardware even after they are
> >>> dequeued. Consequently, the client should avoid writing into this memory
> >>> while the |CAPTURE| queue is streaming. Failure to observe this may
> >>> result in corruption of decoded frames."
> >>>
> >>>>>> +       if (vb->planes[0].mem_priv)
> >>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
> >>>>>> +
> >>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
> >>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
> >>>>>> +               __vb2_buf_mem_free(vb);
> >> Here is another problem for the existing application, the mmap() from the
> >> mmap offset or exportbuffer fd would not create a reference to buffer in
> >> this step(while the exportbuffer would create one itself).
> >>
> >> When you delete a buffer, you may not release it from its virtual memory
> >> space, leaving a corrupted virtual memory space.
> > What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> > memory, it just decrements a reference counter.
>
> struct dma_buf_ops->mmap() may not increase a reference to its buffer.

Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
would increase a reference to the buffer. They both lead to
vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
vb2_common_vm_open() that increases the buffer refcount.

Best regards,
Tomasz

>
> While struct vb2_mem_ops->get_dmabuf() would.
>
> > The VMA holds its own,
> > so the buffer is only fully released when the application calls
> > munmap().
>
> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
> ioctl() make the no other user could access to this.
>
> >
> > Best regards,
> > Tomasz
> >
> >> Also this behavior is
> >> right, because mmap(2) says:
> >>
> >> "After  the  mmap()  call has returned, the file descriptor, fd, can be
> >> closed immediately without invalidating the map‐ping."
> >>
> >>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
> >>>>>> +               __vb2_buf_dmabuf_put(vb);
> >>>>>> +       else
> >>>>>> +               __vb2_buf_userptr_put(vb);
> >>>>>> +
> >>>>>> +       vb2_queue_remove_buffer(q, vb);
> >>>>>> +       kfree(vb);
> >>>>>> +
> >>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
> >>>>>> +       return 0;
> >>>>>> +}
> >>>>>> +
> >>>>>>    /*
> >>>>>>     * vb2_start_streaming() - Attempt to start streaming.
> >>>>>>     * @q:         videobuf2 queue
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> index 724135d41f7f..cea666c17b41 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> >>>>>> media_device *mdev,
> >>>>>>    }
> >>>>>>    EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> >>>>>> +{
> >>>>>> +       return vb2_core_delete_buf(q, b->index);
> >>>>>> +}
> >>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> >>>>>> +
> >>>>>>    int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> >>>>>> *create)
> >>>>>>    {
> >>>>>>           unsigned requested_planes = 1;
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> index f81279492682..80ace2e1e932 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> >>>>>> video_device *vdev)
> >>>>>>                   SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> >>>>>> vidioc_prepare_buf);
> >>>>>>                   SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> >>>>>> vidioc_streamon);
> >>>>>>                   SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> >>>>>> vidioc_streamoff);
> >>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> >>>>>> vidioc_delete_buf);
> >>>>>>           }
> >>>>>>
> >>>>>>           if (is_vid || is_vbi || is_meta) {
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> index a858acea6547..1c737279d3ef 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> >>>>>> v4l2_ioctl_ops *ops,
> >>>>>>           return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> >>>>>>    }
> >>>>>>
> >>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> >>>>>> +                         struct file *file, void *fh, void *arg)
> >>>>>> +{
> >>>>>> +       struct v4l2_buffer *b = arg;
> >>>>>> +       int ret = check_fmt(file, b->type);
> >>>>>> +
> >>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> >>>>>> +}
> >>>>>> +
> >>>>>>    static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> >>>>>>                                   struct file *file, void *fh, void
> >>>>>> *arg)
> >>>>>>    {
> >>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> >>>>>> v4l2_ioctls[] = {
> >>>>>>           IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> >>>>>> v4l_print_freq_band, 0),
> >>>>>>           IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> >>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> >>>>>>           IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> >>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> >>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> >>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> >>>>>> v4l_print_buffer, INFO_FL_QUEUE),
> >>>>>>    };
> >>>>>>    #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> >>>>>>
> >>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> >>>>>> index edb733f21604..2f232ed884c7 100644
> >>>>>> --- a/include/media/v4l2-ioctl.h
> >>>>>> +++ b/include/media/v4l2-ioctl.h
> >>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
> >>>>>>     *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> >>>>>>     * @vidioc_prepare_buf: pointer to the function that implements
> >>>>>>     *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> >>>>>> + * @vidioc_delete_buf: pointer to the function that implements
> >>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> >>>>>>     * @vidioc_overlay: pointer to the function that implements
> >>>>>>     *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> >>>>>>     * @vidioc_g_fbuf: pointer to the function that implements
> >>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> >>>>>>                                     struct v4l2_create_buffers *b);
> >>>>>>           int (*vidioc_prepare_buf)(struct file *file, void *fh,
> >>>>>>                                     struct v4l2_buffer *b);
> >>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
> >>>>>> +                                struct v4l2_buffer *b);
> >>>>>>
> >>>>>>           int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> >>>>>> int i);
> >>>>>>           int (*vidioc_g_fbuf)(struct file *file, void *fh,
> >>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>> b/include/media/videobuf2-core.h
> >>>>>> index 080b783d608d..0f9e68f76b77 100644
> >>>>>> --- a/include/media/videobuf2-core.h
> >>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> >>>>>> enum vb2_memory memory,
> >>>>>>     */
> >>>>>>    int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> >>>>>> void *pb);
> >>>>>>
> >>>>>> +/**
> >>>>>> + * vb2_core_delete_buf() -
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @index:     id number of the buffer.
> >>>>>> + *
> >>>>>> + *  Return: returns zero on success; an error code otherwise.
> >>>>>> + */
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> >>>>>> +
> >>>>>>    /**
> >>>>>>     * vb2_core_qbuf() - Queue a buffer from userspace
> >>>>>>     *
> >>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>> index 88a7a565170e..3beeb4c735f0 100644
> >>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> >>>>>> v4l2_create_buffers *create);
> >>>>>>     */
> >>>>>>    int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> >>>>>>                       struct v4l2_buffer *b);
> >>>>>> +/**
> >>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
> >>>>>> + *
> >>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @b:         buffer structure passed from userspace to
> >>>>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> >>>>>> + *
> >>>>>> + * The return values from this function are intended to be directly
> >>>>>> returned
> >>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> >>>>>> + */
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> >>>>>>
> >>>>>>    /**
> >>>>>>     * vb2_qbuf() - Queue a buffer from userspace
> >>>>>> diff --git a/include/uapi/linux/videodev2.h
> >>>>>> b/include/uapi/linux/videodev2.h
> >>>>>> index aee75eb9e686..31bba1915642 100644
> >>>>>> --- a/include/uapi/linux/videodev2.h
> >>>>>> +++ b/include/uapi/linux/videodev2.h
> >>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> >>>>>>    #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
> >>>>>> v4l2_dbg_chip_info)
> >>>>>>
> >>>>>>    #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
> >>>>>> v4l2_query_ext_ctrl)
> >>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
> >>>>>> +
> >>>>>>
> >>>>>>    /* Reminder: when adding new ioctls please add support for them to
> >>>>>>       drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> >>>>>> --
> >>>>>> 2.39.2
> >>>>>>
> >>> --
> >>> Hsia-Jun(Randy) Li
> >>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>

Hsia-Jun Li July 28, 2023, 7:26 a.m. UTC | #23

On 7/28/23 14:57, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <[e-mail]> wrote:
>>
>>
>> On 7/13/23 17:09, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <[e-mail]>
>>>>>>>> ---
>>>>>>>>     .../userspace-api/media/v4l/user-func.rst     |  1 +
>>>>>>>>     .../media/v4l/vidioc-delete-buf.rst           | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>>     .../media/common/videobuf2/videobuf2-core.c   | 33 ++++++++++++
>>>>>>>>     .../media/common/videobuf2/videobuf2-v4l2.c   |  6 +++
>>>>>>>>     drivers/media/v4l2-core/v4l2-dev.c            |  1 +
>>>>>>>>     drivers/media/v4l2-core/v4l2-ioctl.c          | 10 ++++
>>>>>>>>     include/media/v4l2-ioctl.h                    |  4 ++
>>>>>>>>     include/media/videobuf2-core.h                |  9 ++++
>>>>>>>>     include/media/videobuf2-v4l2.h                | 11 ++++
>>>>>>>>     include/uapi/linux/videodev2.h                |  2 +
>>>>>>>>     10 files changed, 128 insertions(+)
>>>>>>>>     create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>         vidioc-dbg-g-chip-info
>>>>>>>>         vidioc-dbg-g-register
>>>>>>>>         vidioc-decoder-cmd
>>>>>>>> +    vidioc-delete-buf
>>>>>>>>         vidioc-dqevent
>>>>>>>>         vidioc-dv-timings-cap
>>>>>>>>         vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> +    File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> +    Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the
>>>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> +    File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> +    The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>>     }
>>>>>>>>     EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> +       struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> +       vb = vb2_get_buffer(q, index);
>>>>>>>> +       if (!vb) {
>>>>>>>> +               dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> +               return -EINVAL;
>>>>>>>> +       }
>>>>>>>> +
>>>>>>>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> +               dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>>>> %d\n", index);
>>>>>>>> +               return -EINVAL;
>>>>>>>> +       }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame could
>>>>>>> last much longer.
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>>> +       if (vb->planes[0].mem_priv)
>>>>>>>> +               call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> +       /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> +       if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> +               __vb2_buf_mem_free(vb);
>>>> Here is another problem for the existing application, the mmap() from the
>>>> mmap offset or exportbuffer fd would not create a reference to buffer in
>>>> this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual memory
>>>> space, leaving a corrupted virtual memory space.
>>> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
>>> memory, it just decrements a reference counter.
>>
>> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
I think we are talking the same refcount.
That is vb2_vmarea_handler->refcount.
While, I am thinking about refcount from vb2_dc_buf.
> 
> Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
> would increase a reference to the buffer. They both lead to
> vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
> vb2_common_vm_open() that increases the buffer refcount.
> 
> Best regards,
> Tomasz
> 
>>
>> While struct vb2_mem_ops->get_dmabuf() would.
>>
>>> The VMA holds its own,
>>> so the buffer is only fully released when the application calls
>>> munmap().
>>
>> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
>> ioctl() make the no other user could access to this.
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>> Also this behavior is
>>>> right, because mmap(2) says:
>>>>
>>>> "After  the  mmap()  call has returned, the file descriptor, fd, can be
>>>> closed immediately without invalidating the map‐ping."
>>>>
>>>>>>>> +       else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> +               __vb2_buf_dmabuf_put(vb);
>>>>>>>> +       else
>>>>>>>> +               __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> +       vb2_queue_remove_buffer(q, vb);
>>>>>>>> +       kfree(vb);
>>>>>>>> +
>>>>>>>> +       dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> +       return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>     /*
>>>>>>>>      * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>      * @q:         videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>>>> media_device *mdev,
>>>>>>>>     }
>>>>>>>>     EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> +       return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>>     int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>>     {
>>>>>>>>            unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>>                    SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>>                    SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>>                    SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> +               SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>>            }
>>>>>>>>
>>>>>>>>            if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>            return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>>     }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> +                         struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> +       struct v4l2_buffer *b = arg;
>>>>>>>> +       int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> +       return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>     static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>                                    struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>>     {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>            IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>            IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>>>            IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> +       IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>     };
>>>>>>>>     #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>      *     :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>      * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>>      *     :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + *     :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>      * @vidioc_overlay: pointer to the function that implements
>>>>>>>>      *     :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>      * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>                                      struct v4l2_create_buffers *b);
>>>>>>>>            int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>                                      struct v4l2_buffer *b);
>>>>>>>> +       int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> +                                struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>>            int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>>>> int i);
>>>>>>>>            int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>>      */
>>>>>>>>     int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index:     id number of the buffer.
>>>>>>>> + *
>>>>>>>> + *  Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>>     /**
>>>>>>>>      * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>      *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>      */
>>>>>>>>     int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>>>>                        struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b:         buffer structure passed from userspace to
>>>>>>>> + *             &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>>     /**
>>>>>>>>      * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>     #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>>     #define VIDIOC_QUERY_EXT_CTRL  _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF      _IOWR('V', 104, struct v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>>     /* Reminder: when adding new ioctls please add support for them to
>>>>>>>>        drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>

Patch

diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 15ff0bf7bbe6..8c74016e12fd 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -17,6 +17,7 @@  Function Reference
     vidioc-dbg-g-chip-info
     vidioc-dbg-g-register
     vidioc-decoder-cmd
+    vidioc-delete-buf
     vidioc-dqevent
     vidioc-dv-timings-cap
     vidioc-encoder-cmd
diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
new file mode 100644
index 000000000000..0e7ce58f91bc
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
@@ -0,0 +1,51 @@ 
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. c:namespace:: V4L
+
+.. _VIDIOC_DELETE_BUF:
+
+************************
+ioctl VIDIOC_DELETE_BUF
+************************
+
+Name
+====
+
+VIDIOC_DELETE_BUF - Delete a buffer from a queue
+
+Synopsis
+========
+
+.. c:macro:: VIDIOC_DELETE_BUF
+
+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
+
+Arguments
+=========
+
+``fd``
+    File descriptor returned by :c:func:`open()`.
+
+``argp``
+    Pointer to struct :c:type:`v4l2_buffer`.
+
+Description
+===========
+
+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
+delete a buffer from a queue.
+
+The struct :c:type:`v4l2_buffer` structure is specified in
+:ref:`buffer`.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+EBUSY
+    File I/O is in progress.
+
+EINVAL
+    The buffer ``index`` doesn't exist in the queue.
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 899783f67580..aa546c972c3d 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1637,6 +1637,39 @@  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
+{
+	struct vb2_buffer *vb;
+
+	vb = vb2_get_buffer(q, index);
+	if (!vb) {
+		dprintk(q, 1, "invalid buffer index %d\n", index);
+		return -EINVAL;
+	}
+
+	if (vb->state != VB2_BUF_STATE_DEQUEUED) {
+		dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
+		return -EINVAL;
+	}
+
+	if (vb->planes[0].mem_priv)
+		call_void_vb_qop(vb, buf_cleanup, vb);
+
+	/* Free MMAP buffers or release USERPTR buffers */
+	if (q->memory == VB2_MEMORY_MMAP)
+		__vb2_buf_mem_free(vb);
+	else if (q->memory == VB2_MEMORY_DMABUF)
+		__vb2_buf_dmabuf_put(vb);
+	else
+		__vb2_buf_userptr_put(vb);
+
+	vb2_queue_remove_buffer(q, vb);
+	kfree(vb);
+
+	dprintk(q, 2, "buffer %d deleted\n", index);
+	return 0;
+}
+
 /*
  * vb2_start_streaming() - Attempt to start streaming.
  * @q:		videobuf2 queue
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 724135d41f7f..cea666c17b41 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -751,6 +751,12 @@  int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
 }
 EXPORT_SYMBOL_GPL(vb2_prepare_buf);
 
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
+{
+	return vb2_core_delete_buf(q, b->index);
+}
+EXPORT_SYMBOL_GPL(vb2_delete_buf);
+
 int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
 {
 	unsigned requested_planes = 1;
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index f81279492682..80ace2e1e932 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -720,6 +720,7 @@  static void determine_valid_ioctls(struct video_device *vdev)
 		SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
 		SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
 		SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
+		SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
 	}
 
 	if (is_vid || is_vbi || is_meta) {
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index a858acea6547..1c737279d3ef 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2156,6 +2156,15 @@  static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
 	return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
 }
 
+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
+			  struct file *file, void *fh, void *arg)
+{
+	struct v4l2_buffer *b = arg;
+	int ret = check_fmt(file, b->type);
+
+	return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
+}
+
 static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
 				struct file *file, void *fh, void *arg)
 {
@@ -2905,6 +2914,7 @@  static const struct v4l2_ioctl_info v4l2_ioctls[] = {
 	IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
 	IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
 	IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
+	IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
 };
 #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
 
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index edb733f21604..2f232ed884c7 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -163,6 +163,8 @@  struct v4l2_fh;
  *	:ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
  * @vidioc_prepare_buf: pointer to the function that implements
  *	:ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
+ * @vidioc_delete_buf: pointer to the function that implements
+ *	:ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
  * @vidioc_overlay: pointer to the function that implements
  *	:ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
  * @vidioc_g_fbuf: pointer to the function that implements
@@ -422,6 +424,8 @@  struct v4l2_ioctl_ops {
 				  struct v4l2_create_buffers *b);
 	int (*vidioc_prepare_buf)(struct file *file, void *fh,
 				  struct v4l2_buffer *b);
+	int (*vidioc_delete_buf)(struct file *file, void *fh,
+				 struct v4l2_buffer *b);
 
 	int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
 	int (*vidioc_g_fbuf)(struct file *file, void *fh,
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 080b783d608d..0f9e68f76b77 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -840,6 +840,15 @@  int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
  */
 int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
 
+/**
+ * vb2_core_delete_buf() -
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @index:	id number of the buffer.
+ *
+ *  Return: returns zero on success; an error code otherwise.
+ */
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
+
 /**
  * vb2_core_qbuf() - Queue a buffer from userspace
  *
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 88a7a565170e..3beeb4c735f0 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -114,6 +114,17 @@  int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
  */
 int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
 		    struct v4l2_buffer *b);
+/**
+ * vb2_delete_buf() - Delete the buffer from the queue
+ *
+ * @q:		pointer to &struct vb2_queue with videobuf2 queue.
+ * @b:		buffer structure passed from userspace to
+ *		&v4l2_ioctl_ops->vidioc_delete_buf handler in driver
+ *
+ * The return values from this function are intended to be directly returned
+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
+ */
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
 
 /**
  * vb2_qbuf() - Queue a buffer from userspace
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index aee75eb9e686..31bba1915642 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2702,6 +2702,8 @@  struct v4l2_create_buffers {
 #define VIDIOC_DBG_G_CHIP_INFO  _IOWR('V', 102, struct v4l2_dbg_chip_info)
 
 #define VIDIOC_QUERY_EXT_CTRL	_IOWR('V', 103, struct v4l2_query_ext_ctrl)
+#define VIDIOC_DELETE_BUF	_IOWR('V', 104, struct v4l2_buffer)
+
 
 /* Reminder: when adding new ioctls please add support for them to
    drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */