| Message ID | 20181030105328.0667ec68@coco.lan (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL,for,v4.20-rc1] new experimental media request API | expand |
On Tue, Oct 30, 2018 at 6:53 AM Mauro Carvalho Chehab <mchehab+samsung@kernel.org> wrote: > > For a new media API: the request API Ugh. I don't know how much being in staging matters - if people start using it, they start using it. "Staging" does not mean "regressions don't matter". But pulled, Linus
Hi Linus, Em Wed, 31 Oct 2018 11:05:09 -0700 Linus Torvalds <torvalds@linux-foundation.org> escreveu: > On Tue, Oct 30, 2018 at 6:53 AM Mauro Carvalho Chehab > <mchehab+samsung@kernel.org> wrote: > > > > For a new media API: the request API > > Ugh. I don't know how much being in staging matters - if people start > using it, they start using it. > > "Staging" does not mean "regressions don't matter". Yes, I know. This shouldn't affect normal cameras and generic V4L2 apps, as this is for very advanced use cases. So, we hope that people won't start using it for a while. The main interested party on this is Google CromeOS. We're working together with them in order to do upstream first. They are well aware that the API may change. So, I don't expect any complaints from their side if the API would require further changes. The point is that this API is complex and ensuring that it will work properly is not easy. We've been thinking about a solution for the Camera HAL 2 for a long time (I guess the first discussions were done back in 2008). The big problem is that V4L2 API was designed to work with a stream, while Google HAL API expects to have control for each individual frame. The Google API allows, for example that, inside a stream, the first frame would have a VGA resolution, the next one a 4K resolution (for example, when the user clicks on a camera button) and then returning back to VGA (it actually allows full control for every single frame). This is something that it is not possible to do with the "standard" V4L2 API without stopping and restarting a stream (with increases a lot the latency). Solving it is so complex that we decided to start with a completely new type of Linux media drivers (stateless decoders). In long term, the same API should be used by not only by decoders, but also for encoders and complex cameras (those with an image signal processor inside a SoC chipset). In order to be sure that it is possible to implement the way we did, We need to be able to add it to the Kernel somehow and to have enough drivers that will let us test all possible scenarios. That will allow to adapt a version of the camera HAL for testing and see if it behaves as expected. > But pulled, Thanks! Anyway, we'll try to rush the tests for this API in order to try sending any fixes that might be disruptive before the final release. Regards, Mauro
On Wed, Oct 31, 2018 at 11:05 AM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > But pulled, I have no idea how I missed this during the actual test compile after the pull (and yes, I'm sure I did one), but after doing a couple of more pulls I finally did notice. After the media pull I get this warning: ./usr/include/linux/v4l2-controls.h:1105: found __[us]{8,16,32,64} type without #include <linux/types.h> and sure enough, the recent changes to include/uapi/linux/v4l2-controls.h add those new structures use the "__uXY" types without including the header to define them. It's harmless in the short term and the kernel build itself obviously doesn't care apart from the warning, but please fix it. Linus
Em Wed, 31 Oct 2018 15:32:03 -0700 Linus Torvalds <torvalds@linux-foundation.org> escreveu: > On Wed, Oct 31, 2018 at 11:05 AM Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > > > But pulled, > > I have no idea how I missed this during the actual test compile after > the pull (and yes, I'm sure I did one), but after doing a couple of > more pulls I finally did notice. > > After the media pull I get this warning: > > ./usr/include/linux/v4l2-controls.h:1105: found __[us]{8,16,32,64} > type without #include <linux/types.h> > > and sure enough, the recent changes to > > include/uapi/linux/v4l2-controls.h > > add those new structures use the "__uXY" types without including the > header to define them. > > It's harmless in the short term and the kernel build itself obviously > doesn't care apart from the warning, but please fix it. I also missed this one. Perhaps it depends on gcc version, or is it a new warning after some changes? I remember there was some patchsets floating around related to change some warnings. Anyway, I'll send you a fix for it soon. Thanks, Mauro
Hi! > Em Wed, 31 Oct 2018 11:05:09 -0700 > Linus Torvalds <torvalds@linux-foundation.org> escreveu: > > > On Tue, Oct 30, 2018 at 6:53 AM Mauro Carvalho Chehab > > <mchehab+samsung@kernel.org> wrote: > > > > > > For a new media API: the request API > > > > Ugh. I don't know how much being in staging matters - if people start > > using it, they start using it. > > > > "Staging" does not mean "regressions don't matter". > > Yes, I know. > > This shouldn't affect normal cameras and generic V4L2 apps, as this > is for very advanced use cases. So, we hope that people won't start > using it for a while. > > The main interested party on this is Google CromeOS. We're working > together with them in order to do upstream first. They are well aware > that the API may change. So, I don't expect any complaints from their > side if the API would require further changes. You may want to simply disable it in Kconfig... ChromeOS people can enable it easily, and if it never worked in the mainline, you get some wiggle room :-). Pavel
On 11/11/2018 09:28 PM, Pavel Machek wrote: > Hi! > >> Em Wed, 31 Oct 2018 11:05:09 -0700 >> Linus Torvalds <torvalds@linux-foundation.org> escreveu: >> >>> On Tue, Oct 30, 2018 at 6:53 AM Mauro Carvalho Chehab >>> <mchehab+samsung@kernel.org> wrote: >>>> >>>> For a new media API: the request API >>> >>> Ugh. I don't know how much being in staging matters - if people start >>> using it, they start using it. >>> >>> "Staging" does not mean "regressions don't matter". >> >> Yes, I know. >> >> This shouldn't affect normal cameras and generic V4L2 apps, as this >> is for very advanced use cases. So, we hope that people won't start >> using it for a while. >> >> The main interested party on this is Google CromeOS. We're working >> together with them in order to do upstream first. They are well aware >> that the API may change. So, I don't expect any complaints from their >> side if the API would require further changes. > > You may want to simply disable it in Kconfig... ChromeOS people can > enable it easily, and if it never worked in the mainline, you get some > wiggle room :-). > Pavel > For the record: there are no known issues with the Request API. The only outstanding issue is with a control introduced in the staging cedrus driver which will have to change since the method used to refer to MPEG reference frames is wrong. We're working on that and should have this fixed soon (patches have been posted and I'm waiting for test feedback). This control has nothing to do with the Request API, so I see no reason to put the Request API under a config option. Drivers needs to explicit enable support for the Request API anyway, so all existing drivers will return an error if userspace attempts to use this API. Regards, Hans
Hi Linus, Please pull from: git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media tags/media/v4.20-2 For a new media API: the request API This API is needed to support device drivers that can dynamically change their parameters for each new frame. The latest versions of Google camera and codec HAL depends on such feature. At this stage, it supports only stateless codecs. It has been discussed for a long time (at least over the last 3-4 years), and we finally reached to something that seem to work. This series contain both the API and core changes required to support it and a new m2m decoder driver (cedrus). As the current API is still experimental, the only real driver using it (cedrus) was added at staging[1]. We intend to keep it there for a while, in order to test the API. Only when we're sure that this API works for other cases (like encoders), we'll move this driver out of staging and set the API into a stone. [1] We added support for the vivid virtual driver (used only for testing) to it too, as it makes easier to test the API for the ones that don't have the cedrus hardware. Thanks! Mauro The following changes since commit d842a7cf938b6e0f8a1aa9f1aec0476c9a599310: media: adv7842: enable reduced fps detection (2018-08-31 10:03:51 -0400) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media tags/media/v4.20-2 for you to fetch changes up to e4183d3256e3cd668e899d06af66da5aac3a51af: media: dt-bindings: Document the Rockchip VPU bindings (2018-10-05 07:00:43 -0400) ---------------------------------------------------------------- media updates for v4.20-rc1 ---------------------------------------------------------------- Alexandre Courbot (2): media: Documentation: v4l: document request API media: videodev2.h: add request_fd field to v4l2_ext_controls Ezequiel Garcia (1): media: dt-bindings: Document the Rockchip VPU bindings Hans Verkuil (44): media: uapi/linux/media.h: add request API media: media-request: implement media requests media: media-request: add media_request_get_by_fd media: media-request: add media_request_object_find media: v4l2-device.h: add v4l2_device_supports_requests() helper media: v4l2-dev: lock req_queue_mutex media: v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev media: v4l2-ctrls: prepare internal structs for request API media: v4l2-ctrls: alloc memory for p_req media: v4l2-ctrls: use ref in helper instead of ctrl media: v4l2-ctrls: add core request support media: v4l2-ctrls: support g/s_ext_ctrls for requests media: v4l2-ctrls: add v4l2_ctrl_request_hdl_find/put/ctrl_find functions media: videobuf2-v4l2: move __fill_v4l2_buffer() function media: videobuf2-v4l2: replace if by switch in __fill_vb2_buffer() media: vb2: store userspace data in vb2_v4l2_buffer media: davinci_vpfe: remove bogus vb2->state check media: vb2: drop VB2_BUF_STATE_PREPARED, use bool prepared/synced instead media: videodev2.h: Add request_fd field to v4l2_buffer media: vb2: add init_buffer buffer op media: videobuf2-core: embed media_request_object media: videobuf2-core: integrate with media requests media: videobuf2-v4l2: integrate with media requests media: videobuf2-core: add request helper functions media: videobuf2-v4l2: add vb2_request_queue/validate helpers media: videobuf2-core: add uses_requests/qbuf flags media: videobuf2-v4l2: refuse qbuf if queue uses requests or vv. media: v4l2-mem2mem: add vb2_m2m_request_queue media: vim2m: use workqueue media: vim2m: support requests media: vivid: add mc media: vivid: add request support media: media-request: return -EINVAL for invalid request_fds media: v4l2-ctrls: return -EACCES if request wasn't completed media: buffer.rst: only set V4L2_BUF_FLAG_REQUEST_FD for QBUF media: videodev2.h: add new capabilities for buffer types media: vb2: set reqbufs/create_bufs capabilities media: media-request: add media_request_(un)lock_for_access media: v4l2-ctrls: use media_request_(un)lock_for_access media: v4l2-ctrls: improve media_request_(un)lock_for_update media: media-request: EPERM -> EACCES/EBUSY media: media-request: update documentation media: v4l2-compat-ioctl32.c: add missing documentation for a field media: v4l2-ctrls.c: initialize an error return code with zero Paul Kocialkowski (5): media: videobuf2-core: Rework and rename helper for request buffer count media: v4l: Add definitions for MPEG-2 slice format and metadata media: v4l: Add definition for the Sunxi tiled NV12 format media: dt-bindings: media: Document bindings for the Cedrus VPU driver media: platform: Add Cedrus VPU decoder driver Sakari Ailus (1): media: doc: Add media-request.h header to documentation build Documentation/devicetree/bindings/media/cedrus.txt | 54 ++ .../devicetree/bindings/media/rockchip-vpu.txt | 29 + Documentation/media/kapi/mc-core.rst | 2 + .../media/uapi/mediactl/media-controller.rst | 1 + Documentation/media/uapi/mediactl/media-funcs.rst | 6 + .../uapi/mediactl/media-ioc-request-alloc.rst | 66 +++ .../uapi/mediactl/media-request-ioc-queue.rst | 78 +++ .../uapi/mediactl/media-request-ioc-reinit.rst | 51 ++ Documentation/media/uapi/mediactl/request-api.rst | 252 +++++++++ .../media/uapi/mediactl/request-func-close.rst | 49 ++ .../media/uapi/mediactl/request-func-ioctl.rst | 67 +++ .../media/uapi/mediactl/request-func-poll.rst | 77 +++ Documentation/media/uapi/v4l/buffer.rst | 29 +- Documentation/media/uapi/v4l/extended-controls.rst | 176 ++++++ Documentation/media/uapi/v4l/pixfmt-compressed.rst | 16 + Documentation/media/uapi/v4l/pixfmt-reserved.rst | 15 +- .../media/uapi/v4l/vidioc-create-bufs.rst | 14 +- .../media/uapi/v4l/vidioc-g-ext-ctrls.rst | 59 +- Documentation/media/uapi/v4l/vidioc-qbuf.rst | 37 +- Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 14 +- Documentation/media/uapi/v4l/vidioc-reqbufs.rst | 42 +- Documentation/media/videodev2.h.rst.exceptions | 3 + MAINTAINERS | 7 + drivers/media/Makefile | 3 +- drivers/media/common/videobuf2/videobuf2-core.c | 260 +++++++-- drivers/media/common/videobuf2/videobuf2-v4l2.c | 528 ++++++++++++------ drivers/media/dvb-core/dvb_vb2.c | 5 +- drivers/media/dvb-frontends/rtl2832_sdr.c | 5 +- drivers/media/media-device.c | 24 +- drivers/media/media-request.c | 501 +++++++++++++++++ drivers/media/pci/bt8xx/bttv-driver.c | 2 +- drivers/media/pci/cx23885/cx23885-417.c | 2 +- drivers/media/pci/cx88/cx88-blackbird.c | 2 +- drivers/media/pci/cx88/cx88-video.c | 2 +- drivers/media/pci/saa7134/saa7134-empress.c | 4 +- drivers/media/pci/saa7134/saa7134-video.c | 2 +- drivers/media/platform/exynos4-is/fimc-capture.c | 2 +- drivers/media/platform/omap3isp/ispvideo.c | 4 +- drivers/media/platform/rcar-vin/rcar-core.c | 2 +- drivers/media/platform/rcar_drif.c | 2 +- drivers/media/platform/s3c-camif/camif-capture.c | 4 +- drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 4 +- drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 4 +- drivers/media/platform/soc_camera/soc_camera.c | 7 +- drivers/media/platform/vim2m.c | 50 +- drivers/media/platform/vivid/vivid-core.c | 74 +++ drivers/media/platform/vivid/vivid-core.h | 8 + drivers/media/platform/vivid/vivid-ctrls.c | 46 +- drivers/media/platform/vivid/vivid-kthread-cap.c | 12 + drivers/media/platform/vivid/vivid-kthread-out.c | 12 + drivers/media/platform/vivid/vivid-sdr-cap.c | 16 + drivers/media/platform/vivid/vivid-vbi-cap.c | 10 + drivers/media/platform/vivid/vivid-vbi-out.c | 10 + drivers/media/platform/vivid/vivid-vid-cap.c | 10 + drivers/media/platform/vivid/vivid-vid-out.c | 10 + drivers/media/usb/cpia2/cpia2_v4l.c | 2 +- drivers/media/usb/cx231xx/cx231xx-417.c | 2 +- drivers/media/usb/cx231xx/cx231xx-video.c | 4 +- drivers/media/usb/msi2500/msi2500.c | 2 +- drivers/media/usb/tm6000/tm6000-video.c | 2 +- drivers/media/usb/uvc/uvc_queue.c | 5 +- drivers/media/usb/uvc/uvc_v4l2.c | 3 +- drivers/media/usb/uvc/uvcvideo.h | 1 + drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 19 +- drivers/media/v4l2-core/v4l2-ctrls.c | 612 ++++++++++++++++++++- drivers/media/v4l2-core/v4l2-dev.c | 18 +- drivers/media/v4l2-core/v4l2-device.c | 3 +- drivers/media/v4l2-core/v4l2-ioctl.c | 50 +- drivers/media/v4l2-core/v4l2-mem2mem.c | 67 ++- drivers/media/v4l2-core/v4l2-subdev.c | 9 +- drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/davinci_vpfe/vpfe_video.c | 7 +- drivers/staging/media/imx/imx-media-dev.c | 2 +- drivers/staging/media/imx/imx-media-fim.c | 2 +- drivers/staging/media/omap4iss/iss_video.c | 3 +- drivers/staging/media/sunxi/Kconfig | 15 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/cedrus/Kconfig | 14 + drivers/staging/media/sunxi/cedrus/Makefile | 3 + drivers/staging/media/sunxi/cedrus/TODO | 7 + drivers/staging/media/sunxi/cedrus/cedrus.c | 431 +++++++++++++++ drivers/staging/media/sunxi/cedrus/cedrus.h | 167 ++++++ drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 70 +++ drivers/staging/media/sunxi/cedrus/cedrus_dec.h | 27 + drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 327 +++++++++++ drivers/staging/media/sunxi/cedrus/cedrus_hw.h | 30 + drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c | 246 +++++++++ drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 235 ++++++++ drivers/staging/media/sunxi/cedrus/cedrus_video.c | 542 ++++++++++++++++++ drivers/staging/media/sunxi/cedrus/cedrus_video.h | 30 + drivers/usb/gadget/function/uvc_queue.c | 2 +- include/media/media-device.h | 29 + include/media/media-request.h | 442 +++++++++++++++ include/media/v4l2-ctrls.h | 141 ++++- include/media/v4l2-device.h | 11 + include/media/v4l2-mem2mem.h | 4 + include/media/videobuf2-core.h | 64 ++- include/media/videobuf2-v4l2.h | 20 +- include/uapi/linux/media.h | 8 + include/uapi/linux/v4l2-controls.h | 65 +++ include/uapi/linux/videodev2.h | 33 +- 102 files changed, 6166 insertions(+), 385 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/cedrus.txt create mode 100644 Documentation/devicetree/bindings/media/rockchip-vpu.txt create mode 100644 Documentation/media/uapi/mediactl/media-ioc-request-alloc.rst create mode 100644 Documentation/media/uapi/mediactl/media-request-ioc-queue.rst create mode 100644 Documentation/media/uapi/mediactl/media-request-ioc-reinit.rst create mode 100644 Documentation/media/uapi/mediactl/request-api.rst create mode 100644 Documentation/media/uapi/mediactl/request-func-close.rst create mode 100644 Documentation/media/uapi/mediactl/request-func-ioctl.rst create mode 100644 Documentation/media/uapi/mediactl/request-func-poll.rst create mode 100644 drivers/media/media-request.c create mode 100644 drivers/staging/media/sunxi/Kconfig create mode 100644 drivers/staging/media/sunxi/Makefile create mode 100644 drivers/staging/media/sunxi/cedrus/Kconfig create mode 100644 drivers/staging/media/sunxi/cedrus/Makefile create mode 100644 drivers/staging/media/sunxi/cedrus/TODO create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_regs.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.h create mode 100644 include/media/media-request.h