| Message ID | 1540851790-1777-1-git-send-email-yong.zhi@intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Intel IPU3 ImgU patchset | expand |
Hi Yong, Thanks for the update! On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > Hi, > > This series adds support for the Intel IPU3 (Image Processing Unit) > ImgU which is essentially a modern memory-to-memory ISP. It implements > raw Bayer to YUV image format conversion as well as a large number of > other pixel processing algorithms for improving the image quality. > > Meta data formats are defined for image statistics (3A, i.e. automatic > white balance, exposure and focus, histogram and local area contrast > enhancement) as well as for the pixel processing algorithm parameters. > The documentation for these formats is currently not included in the > patchset but will be added in a future version of this set. > > The algorithm parameters need to be considered specific to a given frame > and typically a large number of these parameters change on frame to frame > basis. Additionally, the parameters are highly structured (and not a flat > space of independent configuration primitives). They also reflect the > data structures used by the firmware and the hardware. On top of that, > the algorithms require highly specialized user space to make meaningful > use of them. For these reasons it has been chosen video buffers to pass > the parameters to the device. > > On individual patches: > > The heart of ImgU is the CSS, or Camera Subsystem, which contains the > image processors and HW accelerators. > > The 3A statistics and other firmware parameter computation related > functions are implemented in patch 11. > > All IPU3 pipeline default settings can be found in patch 10. > > To access DDR via ImgU's own memory space, IPU3 is also equipped with > its own MMU unit, the driver is implemented in patch 6. > > Patch 7 uses above driver for DMA mapping operation. > > The communication between IPU3 firmware and driver is implemented with circular > queues in patch 8. > > Patch 9 provide some utility functions and manage IPU3 fw download and > install. > > The firmware which is called ipu3-fw.bin can be downloaded from: > > git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > > Firmware ABI is defined in patches 4 and 5. > > Patches 12 and 13 are of the same file, the former contains all h/w programming > related code, the latter implements interface functions for access fw & hw > capabilities. > > Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > > <URL:https://patchwork.kernel.org/patch/9976295/> I've pushed the latest set here: <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> You can just say the entire set depends on those going forward; the documentation is needed, too. > > Patch 15 represents the top level that glues all of the other components together, > passing arguments between the components. > > Patch 16 is a recent effort to extend v6 for advanced camera features like > Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > > Link to user space implementation: > > git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > > ImgU media topology print: > > # media-ctl -d /dev/media0 -p > Media controller API version 4.19.0 > > Media device information > ------------------------ > driver ipu3-imgu > model ipu3-imgu > serial > bus info PCI:0000:00:05.0 > hw revision 0x80862015 > driver version 4.19.0 > > Device topology > - entity 1: ipu3-imgu 0 (5 pads, 5 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > pad0: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown This doesn't seem right. Which formats can be enumerated from the pad? > crop:(0,0)/1920x1080 > compose:(0,0)/1920x1080] Does the compose rectangle affect the scaling on all outputs? > <- "ipu3-imgu 0 input":0 [] Are there links that have no useful link configuration? If so, you should set them enabled and immutable in the driver. > pad1: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] I'd suggest to use MEDIA_BUS_FMT_FIXED here. > <- "ipu3-imgu 0 parameters":0 [] > pad2: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 0 output":0 [] > pad3: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 0 viewfinder":0 [] Are there other differences between output and viewfinder? > pad4: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 0 3a stat":0 [] FIXED here, too. > > - entity 7: ipu3-imgu 1 (5 pads, 5 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev1 > pad0: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > crop:(0,0)/1920x1080 > compose:(0,0)/1920x1080] > <- "ipu3-imgu 1 input":0 [] > pad1: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > <- "ipu3-imgu 1 parameters":0 [] > pad2: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 1 output":0 [] > pad3: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 1 viewfinder":0 [] > pad4: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 1 3a stat":0 [] This is a minor matter but --- could you create the second sub-device after the video device nodes related to the first one have been already created? That'd make reading the output easier. > > - entity 17: ipu3-imgu 0 input (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Source > -> "ipu3-imgu 0":0 [] > > - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video1 > pad0: Source > -> "ipu3-imgu 0":1 [] > > - entity 29: ipu3-imgu 0 output (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video2 > pad0: Sink > <- "ipu3-imgu 0":2 [] > > - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video3 > pad0: Sink > <- "ipu3-imgu 0":3 [] > > - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video4 > pad0: Sink > <- "ipu3-imgu 0":4 [] > > - entity 47: ipu3-imgu 1 input (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video5 > pad0: Source > -> "ipu3-imgu 1":0 [] > > - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video6 > pad0: Source > -> "ipu3-imgu 1":1 [] > > - entity 59: ipu3-imgu 1 output (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video7 > pad0: Sink > <- "ipu3-imgu 1":2 [] > > - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video8 > pad0: Sink > <- "ipu3-imgu 1":3 [] > > - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video9 > pad0: Sink > <- "ipu3-imgu 1":4 [] > > > v4l2-compliance utility is built with Sakari's patches for meta data > output support(rebased): > > <URL:https://patchwork.linuxtv.org/patch/43370/> > <URL:https://patchwork.linuxtv.org/patch/43369/> > > The test (v4l2-compliance -m 0) passes without error, outputs are appended at > the end of revision history. > > Note: > > 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be enabled > prior to the test. > 2. Stream tests are not performed since it requires pre-configuration for each case. > > =========== > = history = > =========== > > v7 update: > > 1. Add driver and uAPI documentation. > > Update based on v1 review from Tomasz, Hans, Sokari and Mauro: > https://patchwork.kernel.org/patch/10465663/ > https://patchwork.kernel.org/patch/10465665/ > > 2. Add dual pipe support which includes: > - Extend current IMGU device to contain 2 subdevs and two groups of video nodes. > - Add a v4l2 ctrl to allow user to specify the mode(video or still) of the pipe. > > 3. Kconfig > - Restrict build for X86 arch to fix build error for ia64/sparc. > (fatal error: asm/set_memory.h: No such file or directory) > > 4. ipu3-abi.h > - Change __u32 to u32. > - Use generic __attribute__((aligned(x))) format. (Mauro/Hans) > - Split abi to 2 patches, one for register defines, enums, the other for structs. (Tomasz) > > 5. ipu3-mmu.c > - Fix ipu3-mmu/dmamap exit functions. (Tomasz) > (Port from https://chromium-review.googlesource.com/1084522) > - Use free_page instead of kfree. (Tomasz) > - document struct ipu3_mmu_info. > - Fix copyright information. > > 6. ipu3-dmamap.c (Tomasz) > - Update APIs based on v6 review. > - Replace sizeof(struct page *) with sizeof(*pages). > - Remove un-needed (WARN_ON(!dev)) inside void *ipu3_dmamap_alloc(). > > 7. ipu3.c (Tomasz) > - imgu_video_nodes_init() > Fix the missing call to ipu3_v4l2_unregister() in the error path of > imgu_dummybufs_preallocate(). > - imgu_queue_buffers() > Evaluate loop condition explicitly for code clarity and simplicity. > FW requires all output buffers to be queued at start, so adjust the order of > buffer queuing accordingly. (bufix by Tianshu) > - imgu_isr_threaded() > Fix interrupt handler return value. > (Port from https://chromium-review.googlesource.com/1088539) > - Add back the buf_drain_wq from ("avoid sleep in wait_event condition")' > (Port from https://chromium-review.googlesource.com/875420) > > 8. ipu3-v4l2.c > - ipu3_v4l2_register(). (Tomasz) > Split media initialization and registration, also change media device > register/un-register order. > > - Fix v4l2-compliance fail on sub-devicef for VIDIOC_CREATE_BUFS and > VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT. > > 9. ipu3-css.c, ipu3-css.h, ipu3-css-fw.h, ipu3-abi.h > - Convert macros in structs to enums. (Tomasz) > > 10. ipu3-css-pool.c, ipu3-css-pool.h, ipu3.c > - Document the structs. (Hans/Maruo) > > 11. ipu3-css-params.c > - Fixup for noise reduction parameters processing. (bug fixing) > > version 6: > > - intel-ipu3.h uAPI > Move out the definitions not used by user space. (suggested by Sakari) > - ipu3-abi.h, ipu3-css-fw.h > Clean up the header files. > Remove enum type from ABI structs. > - ipu3-css.h and ipu3-css.c > Disable DVS support and remove related code. > - ipu3-v4l2.c > Fixes of v4l2_compliance test fails on ImgU sub-dev. > - ipu3-css-params.c > Refactor awb/awb_fr/af_ops_calc() functions. (Sakari) > - Build mmu and dmamap driver as part of ImgU ko module; (Sakari) > - Add "ipu3-imgu" prefix to media entity names; (Sakari) > - Fix indentation and white space; (Sakari) > - Rebase to kernel v4.16; > - Use SPDX license identifiers in all drivers; (Sakari) > - Internal fix and performance improvements such as: > Stop fw gracefully during stream off. > Enable irq only after start streaming to avoid unexpected interrupt. > Use spinlock to protect IPU3_CSS_QUEUES access. > Return NULL when dequeuing buffer before streaming. > > TODOs: > - Documentation on ImgU driver programming interface to configure and enable > ISP HW, which will include details on complete V4L2 Kernel driver interface > and IO-Control parameters, except for the ISP internal algorithm and its > parameters (which is Intel proprietary IP). > > version 5: > - ipu3-css-pool.c/ipu3_css_pool_check(). > add handling of the framenum wrap around case in ipu3_css_pool_check(). > - ipu3.c, ipu3-v4l2.c, ipu3.h > merge struct ipu3_mem2mem2_device into imgu_device and update the code > accordingly. (Suggested by Sakari) > - ipu3-mmu.c driver: > use __get_free_page() for page-aligned allocations (Tomasz). > optimize tlb invalidation by calling them at the end of map/unmap. (Tomasz). > remove dependency on iommu. (Sakari) > introduce few new functions from iommu.c. > - ipu3-dmamap.c driver > call mmu directly without IOMMU_SUPPORT (Sakari) > update dmamap APIs. (Suggested by Tomasz) > - ipu3_v4l2.c > move g/s_selection callback to V4l2 sub-device (Sakari) > remove colon from ImgU sub-device name. (Sakari) > - ipu3-css-params.c > fix indentation, 0-day scan warnings etc. > - ipu3-css.c > fix warning about NULL comparison. (Sakari) > - intel-ipu3.h: > remove redundant IPU3_ALIGN attribute (Sakari). > fix up un-needed fields in struct ipu3_uapi_params (Sakari) > re-order this to be 2nd in the patch set. > - Makefile: remove Copyright header. (Sakari) > - Internal fix: > optimize shot-to-shot performance. > update default white balance gains defined in ipu3-tables.c > > TODOs: > > - Documentation on ImgU driver programming interface to configure and enable > ISP HW, which will include details on complete V4L2 Kernel driver interface > and IO-Control parameters, except for the ISP internal algorithm and its > parameters (which is Intel proprietary IP). > > - Review ipu3_css_pool_* group APIs usage. > > version 4: > - Used V4L2_BUF_TYPE_META_OUTPUT for: > - V4L2_META_FMT_IPU3_STAT_PARAMS > > - Used V4L2_BUF_TYPE_META_CAPTURE for: > - V4L2_META_FMT_IPU3_STAT_3A > - V4L2_META_FMT_IPU3_STAT_DVS > - V4L2_META_FMT_IPU3_STAT_LACE > - Supported v4l2 MPLANE format on video nodes. > - ipu3-dmamap.c: Removed dma ops and dependencies on IOMMU_DMA lib. > - ipu3-mmu.c: Restructured the driver. > - intel-ipu3.h: Added __padding qualifier for uapi definitions. > - Internal fix: power and performance related issues. > - Fixed v4l2-compliance test. > - Fixed build failure for x86 with 32bit config. > > version 3: > - ipu3-mmu.c and ipu3-dmamap.c: > Tomasz Figa reworked both drivers and updated related files. > - ipu2-abi.h: > update imgu_abi_binary_info ABI to support latest ipu3-fw.bin. > use __packed qualifier on structs suggested by Sakari Ailus. > - ipu3-css-fw.c/ipu3-css-fw.h: following fix were suggested by Tomasz Figa: > remove pointer type in firmware blob structs. > fix binary_header array in struct imgu_fw_header. > fix calling ipu3_css_fw_show_binary() before proper checking. > fix logic error for valid length checking of blob name. > - ipu3-css-params.c/ipu3_css_scaler_get_exp(): > use lib helper suggested by Andy Shevchenko. > - ipu3-v4l2.c/ipu3_videoc_querycap(): > fill device_caps fix suggested by Hans Verkuil. > add VB2_DMABUF suggested by Tomasz Figa. > - ipu3-css.c: increase IMGU freq from 300MHZ to 450MHZ (internal fix) > - ipu3.c: use vb2_dma_sg_memop for the time being(internal fix). > > version 2: > This version cherry-picked firmware ABI change and other > fix in order to bring the code up-to-date with our internal release. > > I will go over the review comments in v1 and address them in v3 and > future update. > > version 1: > - Initial submission > > -------------------------------------------------------------------------------- > > v4l2-compliance test output: > > ./v4l2-compliance -m 0 > > v4l2-compliance SHA: 7aa151889ffe89b1cd94a8198b0caba1a8c70398, 64 bits > > Compliance test for device /dev/media0: > > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) Is there no revision field for the hardware? We could also use the SoC name in the model if it's known. It might be that there is another SoC that contains the same device but I don't see that as a problem really. > Driver version : 4.19.0 > > Required ioctls: > test MEDIA_IOC_DEVICE_INFO: OK > > Allow for multiple opens: > test second /dev/media0 open: OK > test MEDIA_IOC_DEVICE_INFO: OK > test for unlimited opens: OK > > Media Controller ioctls: > test MEDIA_IOC_G_TOPOLOGY: OK > Entities: 12 Interfaces: 12 Pads: 20 Links: 22 > test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK > test MEDIA_IOC_SETUP_LINK: OK
On 11/01/2018 08:03 PM, Sakari Ailus wrote: > Hi Yong, > > Thanks for the update! > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: >> Hi, >> >> This series adds support for the Intel IPU3 (Image Processing Unit) >> ImgU which is essentially a modern memory-to-memory ISP. It implements >> raw Bayer to YUV image format conversion as well as a large number of >> other pixel processing algorithms for improving the image quality. >> >> Meta data formats are defined for image statistics (3A, i.e. automatic >> white balance, exposure and focus, histogram and local area contrast >> enhancement) as well as for the pixel processing algorithm parameters. >> The documentation for these formats is currently not included in the >> patchset but will be added in a future version of this set. >> >> The algorithm parameters need to be considered specific to a given frame >> and typically a large number of these parameters change on frame to frame >> basis. Additionally, the parameters are highly structured (and not a flat >> space of independent configuration primitives). They also reflect the >> data structures used by the firmware and the hardware. On top of that, >> the algorithms require highly specialized user space to make meaningful >> use of them. For these reasons it has been chosen video buffers to pass >> the parameters to the device. >> >> On individual patches: >> >> The heart of ImgU is the CSS, or Camera Subsystem, which contains the >> image processors and HW accelerators. >> >> The 3A statistics and other firmware parameter computation related >> functions are implemented in patch 11. >> >> All IPU3 pipeline default settings can be found in patch 10. >> >> To access DDR via ImgU's own memory space, IPU3 is also equipped with >> its own MMU unit, the driver is implemented in patch 6. >> >> Patch 7 uses above driver for DMA mapping operation. >> >> The communication between IPU3 firmware and driver is implemented with circular >> queues in patch 8. >> >> Patch 9 provide some utility functions and manage IPU3 fw download and >> install. >> >> The firmware which is called ipu3-fw.bin can be downloaded from: >> >> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) >> >> Firmware ABI is defined in patches 4 and 5. >> >> Patches 12 and 13 are of the same file, the former contains all h/w programming >> related code, the latter implements interface functions for access fw & hw >> capabilities. >> >> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: >> >> <URL:https://patchwork.kernel.org/patch/9976295/> > I've pushed the latest set here: > > <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > > You can just say the entire set depends on those going forward; the > documentation is needed, too. > >> Patch 15 represents the top level that glues all of the other components together, >> passing arguments between the components. >> >> Patch 16 is a recent effort to extend v6 for advanced camera features like >> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. >> >> Link to user space implementation: >> >> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera >> >> ImgU media topology print: >> >> # media-ctl -d /dev/media0 -p >> Media controller API version 4.19.0 >> >> Media device information >> ------------------------ >> driver ipu3-imgu >> model ipu3-imgu >> serial >> bus info PCI:0000:00:05.0 >> hw revision 0x80862015 >> driver version 4.19.0 >> >> Device topology >> - entity 1: ipu3-imgu 0 (5 pads, 5 links) >> type V4L2 subdev subtype Unknown flags 0 >> device node name /dev/v4l-subdev0 >> pad0: Sink >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > This doesn't seem right. Which formats can be enumerated from the pad? > >> crop:(0,0)/1920x1080 >> compose:(0,0)/1920x1080] > Does the compose rectangle affect the scaling on all outputs? Sakari, driver use crop and compose targets to help set input-feeder and BDS output resolutions which are 2 key block of whole imaging pipeline, not the actual ending output, but they will impact the final output. > >> <- "ipu3-imgu 0 input":0 [] > Are there links that have no useful link configuration? If so, you should > set them enabled and immutable in the driver. The enabled status of input pads is used to get which pipe that user is trying to enable (ipu3_link_setup()), so it could not been set as immutable. > >> pad1: Sink >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > I'd suggest to use MEDIA_BUS_FMT_FIXED here. > >> <- "ipu3-imgu 0 parameters":0 [] >> pad2: Source >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> -> "ipu3-imgu 0 output":0 [] >> pad3: Source >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> -> "ipu3-imgu 0 viewfinder":0 [] > Are there other differences between output and viewfinder? output and viewfinder are the main and secondary output of output system. 'main' output is not allowed to be scaled, only support crop. secondary output 'viewfinder' can support both cropping and scaling. User can select different nodes to use as preview and capture flexibly based on the actual use cases. > >> pad4: Source >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> -> "ipu3-imgu 0 3a stat":0 [] > FIXED here, too. > >> - entity 7: ipu3-imgu 1 (5 pads, 5 links) >> type V4L2 subdev subtype Unknown flags 0 >> device node name /dev/v4l-subdev1 >> pad0: Sink >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown >> crop:(0,0)/1920x1080 >> compose:(0,0)/1920x1080] >> <- "ipu3-imgu 1 input":0 [] >> pad1: Sink >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> <- "ipu3-imgu 1 parameters":0 [] >> pad2: Source >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> -> "ipu3-imgu 1 output":0 [] >> pad3: Source >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> -> "ipu3-imgu 1 viewfinder":0 [] >> pad4: Source >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >> -> "ipu3-imgu 1 3a stat":0 [] > This is a minor matter but --- could you create the second sub-device after > the video device nodes related to the first one have been already created? > That'd make reading the output easier. > >> - entity 17: ipu3-imgu 0 input (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video0 >> pad0: Source >> -> "ipu3-imgu 0":0 [] >> >> - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video1 >> pad0: Source >> -> "ipu3-imgu 0":1 [] >> >> - entity 29: ipu3-imgu 0 output (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video2 >> pad0: Sink >> <- "ipu3-imgu 0":2 [] >> >> - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video3 >> pad0: Sink >> <- "ipu3-imgu 0":3 [] >> >> - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video4 >> pad0: Sink >> <- "ipu3-imgu 0":4 [] >> >> - entity 47: ipu3-imgu 1 input (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video5 >> pad0: Source >> -> "ipu3-imgu 1":0 [] >> >> - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video6 >> pad0: Source >> -> "ipu3-imgu 1":1 [] >> >> - entity 59: ipu3-imgu 1 output (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video7 >> pad0: Sink >> <- "ipu3-imgu 1":2 [] >> >> - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video8 >> pad0: Sink >> <- "ipu3-imgu 1":3 [] >> >> - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) >> type Node subtype V4L flags 0 >> device node name /dev/video9 >> pad0: Sink >> <- "ipu3-imgu 1":4 [] >> >> >> v4l2-compliance utility is built with Sakari's patches for meta data >> output support(rebased): >> >> <URL:https://patchwork.linuxtv.org/patch/43370/> >> <URL:https://patchwork.linuxtv.org/patch/43369/> >> >> The test (v4l2-compliance -m 0) passes without error, outputs are appended at >> the end of revision history. >> >> Note: >> >> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be enabled >> prior to the test. >> 2. Stream tests are not performed since it requires pre-configuration for each case. >> >> =========== >> = history = >> =========== >> >> v7 update: >> >> 1. Add driver and uAPI documentation. >> >> Update based on v1 review from Tomasz, Hans, Sokari and Mauro: >> https://patchwork.kernel.org/patch/10465663/ >> https://patchwork.kernel.org/patch/10465665/ >> >> 2. Add dual pipe support which includes: >> - Extend current IMGU device to contain 2 subdevs and two groups of video nodes. >> - Add a v4l2 ctrl to allow user to specify the mode(video or still) of the pipe. >> >> 3. Kconfig >> - Restrict build for X86 arch to fix build error for ia64/sparc. >> (fatal error: asm/set_memory.h: No such file or directory) >> >> 4. ipu3-abi.h >> - Change __u32 to u32. >> - Use generic __attribute__((aligned(x))) format. (Mauro/Hans) >> - Split abi to 2 patches, one for register defines, enums, the other for structs. (Tomasz) >> >> 5. ipu3-mmu.c >> - Fix ipu3-mmu/dmamap exit functions. (Tomasz) >> (Port from https://chromium-review.googlesource.com/1084522) >> - Use free_page instead of kfree. (Tomasz) >> - document struct ipu3_mmu_info. >> - Fix copyright information. >> >> 6. ipu3-dmamap.c (Tomasz) >> - Update APIs based on v6 review. >> - Replace sizeof(struct page *) with sizeof(*pages). >> - Remove un-needed (WARN_ON(!dev)) inside void *ipu3_dmamap_alloc(). >> >> 7. ipu3.c (Tomasz) >> - imgu_video_nodes_init() >> Fix the missing call to ipu3_v4l2_unregister() in the error path of >> imgu_dummybufs_preallocate(). >> - imgu_queue_buffers() >> Evaluate loop condition explicitly for code clarity and simplicity. >> FW requires all output buffers to be queued at start, so adjust the order of >> buffer queuing accordingly. (bufix by Tianshu) >> - imgu_isr_threaded() >> Fix interrupt handler return value. >> (Port from https://chromium-review.googlesource.com/1088539) >> - Add back the buf_drain_wq from ("avoid sleep in wait_event condition")' >> (Port from https://chromium-review.googlesource.com/875420) >> >> 8. ipu3-v4l2.c >> - ipu3_v4l2_register(). (Tomasz) >> Split media initialization and registration, also change media device >> register/un-register order. >> >> - Fix v4l2-compliance fail on sub-devicef for VIDIOC_CREATE_BUFS and >> VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT. >> >> 9. ipu3-css.c, ipu3-css.h, ipu3-css-fw.h, ipu3-abi.h >> - Convert macros in structs to enums. (Tomasz) >> >> 10. ipu3-css-pool.c, ipu3-css-pool.h, ipu3.c >> - Document the structs. (Hans/Maruo) >> >> 11. ipu3-css-params.c >> - Fixup for noise reduction parameters processing. (bug fixing) >> >> version 6: >> >> - intel-ipu3.h uAPI >> Move out the definitions not used by user space. (suggested by Sakari) >> - ipu3-abi.h, ipu3-css-fw.h >> Clean up the header files. >> Remove enum type from ABI structs. >> - ipu3-css.h and ipu3-css.c >> Disable DVS support and remove related code. >> - ipu3-v4l2.c >> Fixes of v4l2_compliance test fails on ImgU sub-dev. >> - ipu3-css-params.c >> Refactor awb/awb_fr/af_ops_calc() functions. (Sakari) >> - Build mmu and dmamap driver as part of ImgU ko module; (Sakari) >> - Add "ipu3-imgu" prefix to media entity names; (Sakari) >> - Fix indentation and white space; (Sakari) >> - Rebase to kernel v4.16; >> - Use SPDX license identifiers in all drivers; (Sakari) >> - Internal fix and performance improvements such as: >> Stop fw gracefully during stream off. >> Enable irq only after start streaming to avoid unexpected interrupt. >> Use spinlock to protect IPU3_CSS_QUEUES access. >> Return NULL when dequeuing buffer before streaming. >> >> TODOs: >> - Documentation on ImgU driver programming interface to configure and enable >> ISP HW, which will include details on complete V4L2 Kernel driver interface >> and IO-Control parameters, except for the ISP internal algorithm and its >> parameters (which is Intel proprietary IP). >> >> version 5: >> - ipu3-css-pool.c/ipu3_css_pool_check(). >> add handling of the framenum wrap around case in ipu3_css_pool_check(). >> - ipu3.c, ipu3-v4l2.c, ipu3.h >> merge struct ipu3_mem2mem2_device into imgu_device and update the code >> accordingly. (Suggested by Sakari) >> - ipu3-mmu.c driver: >> use __get_free_page() for page-aligned allocations (Tomasz). >> optimize tlb invalidation by calling them at the end of map/unmap. (Tomasz). >> remove dependency on iommu. (Sakari) >> introduce few new functions from iommu.c. >> - ipu3-dmamap.c driver >> call mmu directly without IOMMU_SUPPORT (Sakari) >> update dmamap APIs. (Suggested by Tomasz) >> - ipu3_v4l2.c >> move g/s_selection callback to V4l2 sub-device (Sakari) >> remove colon from ImgU sub-device name. (Sakari) >> - ipu3-css-params.c >> fix indentation, 0-day scan warnings etc. >> - ipu3-css.c >> fix warning about NULL comparison. (Sakari) >> - intel-ipu3.h: >> remove redundant IPU3_ALIGN attribute (Sakari). >> fix up un-needed fields in struct ipu3_uapi_params (Sakari) >> re-order this to be 2nd in the patch set. >> - Makefile: remove Copyright header. (Sakari) >> - Internal fix: >> optimize shot-to-shot performance. >> update default white balance gains defined in ipu3-tables.c >> >> TODOs: >> >> - Documentation on ImgU driver programming interface to configure and enable >> ISP HW, which will include details on complete V4L2 Kernel driver interface >> and IO-Control parameters, except for the ISP internal algorithm and its >> parameters (which is Intel proprietary IP). >> >> - Review ipu3_css_pool_* group APIs usage. >> >> version 4: >> - Used V4L2_BUF_TYPE_META_OUTPUT for: >> - V4L2_META_FMT_IPU3_STAT_PARAMS >> >> - Used V4L2_BUF_TYPE_META_CAPTURE for: >> - V4L2_META_FMT_IPU3_STAT_3A >> - V4L2_META_FMT_IPU3_STAT_DVS >> - V4L2_META_FMT_IPU3_STAT_LACE >> - Supported v4l2 MPLANE format on video nodes. >> - ipu3-dmamap.c: Removed dma ops and dependencies on IOMMU_DMA lib. >> - ipu3-mmu.c: Restructured the driver. >> - intel-ipu3.h: Added __padding qualifier for uapi definitions. >> - Internal fix: power and performance related issues. >> - Fixed v4l2-compliance test. >> - Fixed build failure for x86 with 32bit config. >> >> version 3: >> - ipu3-mmu.c and ipu3-dmamap.c: >> Tomasz Figa reworked both drivers and updated related files. >> - ipu2-abi.h: >> update imgu_abi_binary_info ABI to support latest ipu3-fw.bin. >> use __packed qualifier on structs suggested by Sakari Ailus. >> - ipu3-css-fw.c/ipu3-css-fw.h: following fix were suggested by Tomasz Figa: >> remove pointer type in firmware blob structs. >> fix binary_header array in struct imgu_fw_header. >> fix calling ipu3_css_fw_show_binary() before proper checking. >> fix logic error for valid length checking of blob name. >> - ipu3-css-params.c/ipu3_css_scaler_get_exp(): >> use lib helper suggested by Andy Shevchenko. >> - ipu3-v4l2.c/ipu3_videoc_querycap(): >> fill device_caps fix suggested by Hans Verkuil. >> add VB2_DMABUF suggested by Tomasz Figa. >> - ipu3-css.c: increase IMGU freq from 300MHZ to 450MHZ (internal fix) >> - ipu3.c: use vb2_dma_sg_memop for the time being(internal fix). >> >> version 2: >> This version cherry-picked firmware ABI change and other >> fix in order to bring the code up-to-date with our internal release. >> >> I will go over the review comments in v1 and address them in v3 and >> future update. >> >> version 1: >> - Initial submission >> >> -------------------------------------------------------------------------------- >> >> v4l2-compliance test output: >> >> ./v4l2-compliance -m 0 >> >> v4l2-compliance SHA: 7aa151889ffe89b1cd94a8198b0caba1a8c70398, 64 bits >> >> Compliance test for device /dev/media0: >> >> Media Driver Info: >> Driver name : ipu3-imgu >> Model : ipu3-imgu >> Serial : >> Bus info : PCI:0000:00:05.0 >> Media version : 4.19.0 >> Hardware revision: 0x80862015 (2156273685) > Is there no revision field for the hardware? We could also use the SoC name > in the model if it's known. It might be that there is another SoC that > contains the same device but I don't see that as a problem really. > >> Driver version : 4.19.0 >> >> Required ioctls: >> test MEDIA_IOC_DEVICE_INFO: OK >> >> Allow for multiple opens: >> test second /dev/media0 open: OK >> test MEDIA_IOC_DEVICE_INFO: OK >> test for unlimited opens: OK >> >> Media Controller ioctls: >> test MEDIA_IOC_G_TOPOLOGY: OK >> Entities: 12 Interfaces: 12 Pads: 20 Links: 22 >> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK >> test MEDIA_IOC_SETUP_LINK: OK
Hi, Sakari, Thanks for your review and comments. Bingbu has replied to some of your questions, so I will continue with the rest. > -----Original Message----- > From: Bing Bu Cao [mailto:bingbu.cao@linux.intel.com] > Sent: Tuesday, November 6, 2018 10:17 PM > To: Sakari Ailus <sakari.ailus@linux.intel.com>; Zhi, Yong > <yong.zhi@intel.com> > Cc: linux-media@vger.kernel.org; tfiga@chromium.org; > mchehab@kernel.org; hans.verkuil@cisco.com; > laurent.pinchart@ideasonboard.com; Mani, Rajmohan > <rajmohan.mani@intel.com>; Zheng, Jian Xu <jian.xu.zheng@intel.com>; Hu, > Jerry W <jerry.w.hu@intel.com>; Toivonen, Tuukka > <tuukka.toivonen@intel.com>; Qiu, Tian Shu <tian.shu.qiu@intel.com>; Cao, > Bingbu <bingbu.cao@intel.com> > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > Hi Yong, > > > > Thanks for the update! > > > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > >> Hi, > >> > >> This series adds support for the Intel IPU3 (Image Processing Unit) > >> ImgU which is essentially a modern memory-to-memory ISP. It > >> implements raw Bayer to YUV image format conversion as well as a > >> large number of other pixel processing algorithms for improving the image > quality. > >> > >> Meta data formats are defined for image statistics (3A, i.e. > >> automatic white balance, exposure and focus, histogram and local area > >> contrast > >> enhancement) as well as for the pixel processing algorithm parameters. > >> The documentation for these formats is currently not included in the > >> patchset but will be added in a future version of this set. > >> > >> The algorithm parameters need to be considered specific to a given > >> frame and typically a large number of these parameters change on > >> frame to frame basis. Additionally, the parameters are highly > >> structured (and not a flat space of independent configuration > >> primitives). They also reflect the data structures used by the > >> firmware and the hardware. On top of that, the algorithms require > >> highly specialized user space to make meaningful use of them. For > >> these reasons it has been chosen video buffers to pass the parameters to > the device. > >> > >> On individual patches: > >> > >> The heart of ImgU is the CSS, or Camera Subsystem, which contains the > >> image processors and HW accelerators. > >> > >> The 3A statistics and other firmware parameter computation related > >> functions are implemented in patch 11. > >> > >> All IPU3 pipeline default settings can be found in patch 10. > >> > >> To access DDR via ImgU's own memory space, IPU3 is also equipped with > >> its own MMU unit, the driver is implemented in patch 6. > >> > >> Patch 7 uses above driver for DMA mapping operation. > >> > >> The communication between IPU3 firmware and driver is implemented > >> with circular queues in patch 8. > >> > >> Patch 9 provide some utility functions and manage IPU3 fw download > >> and install. > >> > >> The firmware which is called ipu3-fw.bin can be downloaded from: > >> > >> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware > >> .git (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > >> > >> Firmware ABI is defined in patches 4 and 5. > >> > >> Patches 12 and 13 are of the same file, the former contains all h/w > >> programming related code, the latter implements interface functions > >> for access fw & hw capabilities. > >> > >> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT > work: > >> > >> <URL:https://patchwork.kernel.org/patch/9976295/> > > I've pushed the latest set here: > > > > <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > > > > You can just say the entire set depends on those going forward; the > > documentation is needed, too. > > Ack. > >> Patch 15 represents the top level that glues all of the other > >> components together, passing arguments between the components. > >> > >> Patch 16 is a recent effort to extend v6 for advanced camera features > >> like Continuous View Finder (CVF) and Snapshot During Video(SDV) > support. > >> > >> Link to user space implementation: > >> > >> git clone > >> https://chromium.googlesource.com/chromiumos/platform/arc-camera > >> > >> ImgU media topology print: > >> > >> # media-ctl -d /dev/media0 -p > >> Media controller API version 4.19.0 > >> > >> Media device information > >> ------------------------ > >> driver ipu3-imgu > >> model ipu3-imgu > >> serial > >> bus info PCI:0000:00:05.0 > >> hw revision 0x80862015 > >> driver version 4.19.0 > >> > >> Device topology > >> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >> type V4L2 subdev subtype Unknown flags 0 > >> device node name /dev/v4l-subdev0 > >> pad0: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > This doesn't seem right. Which formats can be enumerated from the pad? Supposed to be raw formats for the colorspace field. > > > >> crop:(0,0)/1920x1080 > >> compose:(0,0)/1920x1080] > > Does the compose rectangle affect the scaling on all outputs? > Sakari, driver use crop and compose targets to help set input-feeder and BDS > output resolutions which are 2 key block of whole imaging pipeline, not the > actual ending output, but they will impact the final output. > > > >> <- "ipu3-imgu 0 input":0 [] > > Are there links that have no useful link configuration? If so, you > > should set them enabled and immutable in the driver. > The enabled status of input pads is used to get which pipe that user is trying > to enable (ipu3_link_setup()), so it could not been set as immutable. > > > >> pad1: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > I'd suggest to use MEDIA_BUS_FMT_FIXED here. Make sense as the parameter size is fixed. > > > >> <- "ipu3-imgu 0 parameters":0 [] > >> pad2: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 output":0 [] > >> pad3: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 viewfinder":0 [] > > Are there other differences between output and viewfinder? > output and viewfinder are the main and secondary output of output system. > 'main' output is not allowed to be scaled, only support crop. secondary > output 'viewfinder' > can support both cropping and scaling. User can select different nodes to use > as preview and capture flexibly based on the actual use cases. > > > >> pad4: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 3a stat":0 [] > > FIXED here, too. Sure. > > > >> - entity 7: ipu3-imgu 1 (5 pads, 5 links) > >> type V4L2 subdev subtype Unknown flags 0 > >> device node name /dev/v4l-subdev1 > >> pad0: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >> crop:(0,0)/1920x1080 > >> compose:(0,0)/1920x1080] > >> <- "ipu3-imgu 1 input":0 [] > >> pad1: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> <- "ipu3-imgu 1 parameters":0 [] > >> pad2: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 1 output":0 [] > >> pad3: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 1 viewfinder":0 [] > >> pad4: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 1 3a stat":0 [] > > This is a minor matter but --- could you create the second sub-device > > after the video device nodes related to the first one have been already > created? > > That'd make reading the output easier. > > This can be done in next update. > >> - entity 17: ipu3-imgu 0 input (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video0 > >> pad0: Source > >> -> "ipu3-imgu 0":0 [] > >> > >> - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video1 > >> pad0: Source > >> -> "ipu3-imgu 0":1 [] > >> > >> - entity 29: ipu3-imgu 0 output (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video2 > >> pad0: Sink > >> <- "ipu3-imgu 0":2 [] > >> > >> - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video3 > >> pad0: Sink > >> <- "ipu3-imgu 0":3 [] > >> > >> - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video4 > >> pad0: Sink > >> <- "ipu3-imgu 0":4 [] > >> > >> - entity 47: ipu3-imgu 1 input (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video5 > >> pad0: Source > >> -> "ipu3-imgu 1":0 [] > >> > >> - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video6 > >> pad0: Source > >> -> "ipu3-imgu 1":1 [] > >> > >> - entity 59: ipu3-imgu 1 output (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video7 > >> pad0: Sink > >> <- "ipu3-imgu 1":2 [] > >> > >> - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video8 > >> pad0: Sink > >> <- "ipu3-imgu 1":3 [] > >> > >> - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video9 > >> pad0: Sink > >> <- "ipu3-imgu 1":4 [] > >> > >> > >> v4l2-compliance utility is built with Sakari's patches for meta data > >> output support(rebased): > >> > >> <URL:https://patchwork.linuxtv.org/patch/43370/> > >> <URL:https://patchwork.linuxtv.org/patch/43369/> > >> > >> The test (v4l2-compliance -m 0) passes without error, outputs are > >> appended at the end of revision history. > >> > >> Note: > >> > >> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be > enabled > >> prior to the test. > >> 2. Stream tests are not performed since it requires pre-configuration for > each case. > >> > >> =========== > >> = history = > >> =========== > >> > >> v7 update: > >> > >> 1. Add driver and uAPI documentation. > >> > >> Update based on v1 review from Tomasz, Hans, Sokari and Mauro: > >> https://patchwork.kernel.org/patch/10465663/ > >> https://patchwork.kernel.org/patch/10465665/ > >> > >> 2. Add dual pipe support which includes: > >> - Extend current IMGU device to contain 2 subdevs and two groups of > video nodes. > >> - Add a v4l2 ctrl to allow user to specify the mode(video or still) of the > pipe. > >> > >> 3. Kconfig > >> - Restrict build for X86 arch to fix build error for ia64/sparc. > >> (fatal error: asm/set_memory.h: No such file or directory) > >> > >> 4. ipu3-abi.h > >> - Change __u32 to u32. > >> - Use generic __attribute__((aligned(x))) format. (Mauro/Hans) > >> - Split abi to 2 patches, one for register defines, enums, the other > >> for structs. (Tomasz) > >> > >> 5. ipu3-mmu.c > >> - Fix ipu3-mmu/dmamap exit functions. (Tomasz) > >> (Port from https://chromium-review.googlesource.com/1084522) > >> - Use free_page instead of kfree. (Tomasz) > >> - document struct ipu3_mmu_info. > >> - Fix copyright information. > >> > >> 6. ipu3-dmamap.c (Tomasz) > >> - Update APIs based on v6 review. > >> - Replace sizeof(struct page *) with sizeof(*pages). > >> - Remove un-needed (WARN_ON(!dev)) inside void > *ipu3_dmamap_alloc(). > >> > >> 7. ipu3.c (Tomasz) > >> - imgu_video_nodes_init() > >> Fix the missing call to ipu3_v4l2_unregister() in the error path of > >> imgu_dummybufs_preallocate(). > >> - imgu_queue_buffers() > >> Evaluate loop condition explicitly for code clarity and simplicity. > >> FW requires all output buffers to be queued at start, so adjust the order > of > >> buffer queuing accordingly. (bufix by Tianshu) > >> - imgu_isr_threaded() > >> Fix interrupt handler return value. > >> (Port from https://chromium-review.googlesource.com/1088539) > >> - Add back the buf_drain_wq from ("avoid sleep in wait_event > condition")' > >> (Port from https://chromium-review.googlesource.com/875420) > >> > >> 8. ipu3-v4l2.c > >> - ipu3_v4l2_register(). (Tomasz) > >> Split media initialization and registration, also change media device > >> register/un-register order. > >> > >> - Fix v4l2-compliance fail on sub-devicef for VIDIOC_CREATE_BUFS and > >> VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT. > >> > >> 9. ipu3-css.c, ipu3-css.h, ipu3-css-fw.h, ipu3-abi.h > >> - Convert macros in structs to enums. (Tomasz) > >> > >> 10. ipu3-css-pool.c, ipu3-css-pool.h, ipu3.c > >> - Document the structs. (Hans/Maruo) > >> > >> 11. ipu3-css-params.c > >> - Fixup for noise reduction parameters processing. (bug fixing) > >> > >> version 6: > >> > >> - intel-ipu3.h uAPI > >> Move out the definitions not used by user space. (suggested by > >> Sakari) > >> - ipu3-abi.h, ipu3-css-fw.h > >> Clean up the header files. > >> Remove enum type from ABI structs. > >> - ipu3-css.h and ipu3-css.c > >> Disable DVS support and remove related code. > >> - ipu3-v4l2.c > >> Fixes of v4l2_compliance test fails on ImgU sub-dev. > >> - ipu3-css-params.c > >> Refactor awb/awb_fr/af_ops_calc() functions. (Sakari) > >> - Build mmu and dmamap driver as part of ImgU ko module; (Sakari) > >> - Add "ipu3-imgu" prefix to media entity names; (Sakari) > >> - Fix indentation and white space; (Sakari) > >> - Rebase to kernel v4.16; > >> - Use SPDX license identifiers in all drivers; (Sakari) > >> - Internal fix and performance improvements such as: > >> Stop fw gracefully during stream off. > >> Enable irq only after start streaming to avoid unexpected interrupt. > >> Use spinlock to protect IPU3_CSS_QUEUES access. > >> Return NULL when dequeuing buffer before streaming. > >> > >> TODOs: > >> - Documentation on ImgU driver programming interface to configure and > enable > >> ISP HW, which will include details on complete V4L2 Kernel driver > interface > >> and IO-Control parameters, except for the ISP internal algorithm and its > >> parameters (which is Intel proprietary IP). > >> > >> version 5: > >> - ipu3-css-pool.c/ipu3_css_pool_check(). > >> add handling of the framenum wrap around case in > ipu3_css_pool_check(). > >> - ipu3.c, ipu3-v4l2.c, ipu3.h > >> merge struct ipu3_mem2mem2_device into imgu_device and update the > code > >> accordingly. (Suggested by Sakari) > >> - ipu3-mmu.c driver: > >> use __get_free_page() for page-aligned allocations (Tomasz). > >> optimize tlb invalidation by calling them at the end of map/unmap. > (Tomasz). > >> remove dependency on iommu. (Sakari) > >> introduce few new functions from iommu.c. > >> - ipu3-dmamap.c driver > >> call mmu directly without IOMMU_SUPPORT (Sakari) > >> update dmamap APIs. (Suggested by Tomasz) > >> - ipu3_v4l2.c > >> move g/s_selection callback to V4l2 sub-device (Sakari) > >> remove colon from ImgU sub-device name. (Sakari) > >> - ipu3-css-params.c > >> fix indentation, 0-day scan warnings etc. > >> - ipu3-css.c > >> fix warning about NULL comparison. (Sakari) > >> - intel-ipu3.h: > >> remove redundant IPU3_ALIGN attribute (Sakari). > >> fix up un-needed fields in struct ipu3_uapi_params (Sakari) > >> re-order this to be 2nd in the patch set. > >> - Makefile: remove Copyright header. (Sakari) > >> - Internal fix: > >> optimize shot-to-shot performance. > >> update default white balance gains defined in ipu3-tables.c > >> > >> TODOs: > >> > >> - Documentation on ImgU driver programming interface to configure and > enable > >> ISP HW, which will include details on complete V4L2 Kernel driver > interface > >> and IO-Control parameters, except for the ISP internal algorithm and its > >> parameters (which is Intel proprietary IP). > >> > >> - Review ipu3_css_pool_* group APIs usage. > >> > >> version 4: > >> - Used V4L2_BUF_TYPE_META_OUTPUT for: > >> - V4L2_META_FMT_IPU3_STAT_PARAMS > >> > >> - Used V4L2_BUF_TYPE_META_CAPTURE for: > >> - V4L2_META_FMT_IPU3_STAT_3A > >> - V4L2_META_FMT_IPU3_STAT_DVS > >> - V4L2_META_FMT_IPU3_STAT_LACE > >> - Supported v4l2 MPLANE format on video nodes. > >> - ipu3-dmamap.c: Removed dma ops and dependencies on IOMMU_DMA > lib. > >> - ipu3-mmu.c: Restructured the driver. > >> - intel-ipu3.h: Added __padding qualifier for uapi definitions. > >> - Internal fix: power and performance related issues. > >> - Fixed v4l2-compliance test. > >> - Fixed build failure for x86 with 32bit config. > >> > >> version 3: > >> - ipu3-mmu.c and ipu3-dmamap.c: > >> Tomasz Figa reworked both drivers and updated related files. > >> - ipu2-abi.h: > >> update imgu_abi_binary_info ABI to support latest ipu3-fw.bin. > >> use __packed qualifier on structs suggested by Sakari Ailus. > >> - ipu3-css-fw.c/ipu3-css-fw.h: following fix were suggested by Tomasz Figa: > >> remove pointer type in firmware blob structs. > >> fix binary_header array in struct imgu_fw_header. > >> fix calling ipu3_css_fw_show_binary() before proper checking. > >> fix logic error for valid length checking of blob name. > >> - ipu3-css-params.c/ipu3_css_scaler_get_exp(): > >> use lib helper suggested by Andy Shevchenko. > >> - ipu3-v4l2.c/ipu3_videoc_querycap(): > >> fill device_caps fix suggested by Hans Verkuil. > >> add VB2_DMABUF suggested by Tomasz Figa. > >> - ipu3-css.c: increase IMGU freq from 300MHZ to 450MHZ (internal fix) > >> - ipu3.c: use vb2_dma_sg_memop for the time being(internal fix). > >> > >> version 2: > >> This version cherry-picked firmware ABI change and other fix in order > >> to bring the code up-to-date with our internal release. > >> > >> I will go over the review comments in v1 and address them in v3 and > >> future update. > >> > >> version 1: > >> - Initial submission > >> > >> --------------------------------------------------------------------- > >> ----------- > >> > >> v4l2-compliance test output: > >> > >> ./v4l2-compliance -m 0 > >> > >> v4l2-compliance SHA: 7aa151889ffe89b1cd94a8198b0caba1a8c70398, 64 > >> bits > >> > >> Compliance test for device /dev/media0: > >> > >> Media Driver Info: > >> Driver name : ipu3-imgu > >> Model : ipu3-imgu > >> Serial : > >> Bus info : PCI:0000:00:05.0 > >> Media version : 4.19.0 > >> Hardware revision: 0x80862015 (2156273685) > > Is there no revision field for the hardware? We could also use the SoC > > name in the model if it's known. It might be that there is another SoC > > that contains the same device but I don't see that as a problem really. > > For the first question, do you mean revision field of IPU3? For the Model, do you suggest changing "ipu3-imgu" to "Intel(R) Core(TM) m3-7Y30 CPU ipu3-imgu"? I have not found a handy way to read SoC name yet. Thanks, Yong > >> Driver version : 4.19.0 > >> > >> Required ioctls: > >> test MEDIA_IOC_DEVICE_INFO: OK > >> > >> Allow for multiple opens: > >> test second /dev/media0 open: OK > >> test MEDIA_IOC_DEVICE_INFO: OK > >> test for unlimited opens: OK > >> > >> Media Controller ioctls: > >> test MEDIA_IOC_G_TOPOLOGY: OK > >> Entities: 12 Interfaces: 12 Pads: 20 Links: 22 > >> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK > >> test MEDIA_IOC_SETUP_LINK: OK
Hi Bing Bu, On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > Hi Yong, > > > > Thanks for the update! > > > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > >> Hi, > >> > >> This series adds support for the Intel IPU3 (Image Processing Unit) > >> ImgU which is essentially a modern memory-to-memory ISP. It implements > >> raw Bayer to YUV image format conversion as well as a large number of > >> other pixel processing algorithms for improving the image quality. > >> > >> Meta data formats are defined for image statistics (3A, i.e. automatic > >> white balance, exposure and focus, histogram and local area contrast > >> enhancement) as well as for the pixel processing algorithm parameters. > >> The documentation for these formats is currently not included in the > >> patchset but will be added in a future version of this set. > >> > >> The algorithm parameters need to be considered specific to a given frame > >> and typically a large number of these parameters change on frame to frame > >> basis. Additionally, the parameters are highly structured (and not a flat > >> space of independent configuration primitives). They also reflect the > >> data structures used by the firmware and the hardware. On top of that, > >> the algorithms require highly specialized user space to make meaningful > >> use of them. For these reasons it has been chosen video buffers to pass > >> the parameters to the device. > >> > >> On individual patches: > >> > >> The heart of ImgU is the CSS, or Camera Subsystem, which contains the > >> image processors and HW accelerators. > >> > >> The 3A statistics and other firmware parameter computation related > >> functions are implemented in patch 11. > >> > >> All IPU3 pipeline default settings can be found in patch 10. > >> > >> To access DDR via ImgU's own memory space, IPU3 is also equipped with > >> its own MMU unit, the driver is implemented in patch 6. > >> > >> Patch 7 uses above driver for DMA mapping operation. > >> > >> The communication between IPU3 firmware and driver is implemented with circular > >> queues in patch 8. > >> > >> Patch 9 provide some utility functions and manage IPU3 fw download and > >> install. > >> > >> The firmware which is called ipu3-fw.bin can be downloaded from: > >> > >> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > >> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > >> > >> Firmware ABI is defined in patches 4 and 5. > >> > >> Patches 12 and 13 are of the same file, the former contains all h/w programming > >> related code, the latter implements interface functions for access fw & hw > >> capabilities. > >> > >> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > >> > >> <URL:https://patchwork.kernel.org/patch/9976295/> > > I've pushed the latest set here: > > > > <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > > > > You can just say the entire set depends on those going forward; the > > documentation is needed, too. > > > >> Patch 15 represents the top level that glues all of the other components together, > >> passing arguments between the components. > >> > >> Patch 16 is a recent effort to extend v6 for advanced camera features like > >> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > >> > >> Link to user space implementation: > >> > >> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > >> > >> ImgU media topology print: > >> > >> # media-ctl -d /dev/media0 -p > >> Media controller API version 4.19.0 > >> > >> Media device information > >> ------------------------ > >> driver ipu3-imgu > >> model ipu3-imgu > >> serial > >> bus info PCI:0000:00:05.0 > >> hw revision 0x80862015 > >> driver version 4.19.0 > >> > >> Device topology > >> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >> type V4L2 subdev subtype Unknown flags 0 > >> device node name /dev/v4l-subdev0 > >> pad0: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > This doesn't seem right. Which formats can be enumerated from the pad? Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) format whereas the CAPTURE video nodes always have NV12. Can you confirm? If the OUTPUT video node format selection has no effect on the rest of the pipeline (device capabilities, which processing blocks are in use, CAPTURE video nodes formats etc.), I think you could simply use the FIXED media bus code for each pad. That would actually make sense: this device always works from memory to memory, and thus does not really have a pixel data bus external to the device which is what the media bus codes really are for. > > > >> crop:(0,0)/1920x1080 > >> compose:(0,0)/1920x1080] > > Does the compose rectangle affect the scaling on all outputs? > Sakari, driver use crop and compose targets to help set input-feeder and BDS > output resolutions which are 2 key block of whole imaging pipeline, not the > actual ending output, but they will impact the final output. Ack. Thanks for the clarification. > > > >> <- "ipu3-imgu 0 input":0 [] > > Are there links that have no useful link configuration? If so, you should > > set them enabled and immutable in the driver. > The enabled status of input pads is used to get which pipe that user is > trying to enable (ipu3_link_setup()), so it could not been set as immutable. But the rest of them could be, right? > > > >> pad1: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > I'd suggest to use MEDIA_BUS_FMT_FIXED here. > > > >> <- "ipu3-imgu 0 parameters":0 [] > >> pad2: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 output":0 [] > >> pad3: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 viewfinder":0 [] > > Are there other differences between output and viewfinder? > output and viewfinder are the main and secondary output of output system. > 'main' output is not allowed to be scaled, only support crop. secondary > output 'viewfinder' > can support both cropping and scaling. User can select different nodes > to use > as preview and capture flexibly based on the actual use cases. If there's scaling to be configured, I'd expect to see the COMPOSE target supported.
Hi Yong, On Fri, Nov 09, 2018 at 01:28:27AM +0000, Zhi, Yong wrote: > Hi, Sakari, > > Thanks for your review and comments. > Bingbu has replied to some of your questions, so I will continue with the rest. > > > -----Original Message----- > > From: Bing Bu Cao [mailto:bingbu.cao@linux.intel.com] > > Sent: Tuesday, November 6, 2018 10:17 PM > > To: Sakari Ailus <sakari.ailus@linux.intel.com>; Zhi, Yong > > <yong.zhi@intel.com> > > Cc: linux-media@vger.kernel.org; tfiga@chromium.org; > > mchehab@kernel.org; hans.verkuil@cisco.com; > > laurent.pinchart@ideasonboard.com; Mani, Rajmohan > > <rajmohan.mani@intel.com>; Zheng, Jian Xu <jian.xu.zheng@intel.com>; Hu, > > Jerry W <jerry.w.hu@intel.com>; Toivonen, Tuukka > > <tuukka.toivonen@intel.com>; Qiu, Tian Shu <tian.shu.qiu@intel.com>; Cao, > > Bingbu <bingbu.cao@intel.com> > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > > Hi Yong, > > > > > > Thanks for the update! > > > > > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > >> Hi, > > >> > > >> This series adds support for the Intel IPU3 (Image Processing Unit) > > >> ImgU which is essentially a modern memory-to-memory ISP. It > > >> implements raw Bayer to YUV image format conversion as well as a > > >> large number of other pixel processing algorithms for improving the image > > quality. > > >> > > >> Meta data formats are defined for image statistics (3A, i.e. > > >> automatic white balance, exposure and focus, histogram and local area > > >> contrast > > >> enhancement) as well as for the pixel processing algorithm parameters. > > >> The documentation for these formats is currently not included in the > > >> patchset but will be added in a future version of this set. > > >> > > >> The algorithm parameters need to be considered specific to a given > > >> frame and typically a large number of these parameters change on > > >> frame to frame basis. Additionally, the parameters are highly > > >> structured (and not a flat space of independent configuration > > >> primitives). They also reflect the data structures used by the > > >> firmware and the hardware. On top of that, the algorithms require > > >> highly specialized user space to make meaningful use of them. For > > >> these reasons it has been chosen video buffers to pass the parameters to > > the device. > > >> > > >> On individual patches: > > >> > > >> The heart of ImgU is the CSS, or Camera Subsystem, which contains the > > >> image processors and HW accelerators. > > >> > > >> The 3A statistics and other firmware parameter computation related > > >> functions are implemented in patch 11. > > >> > > >> All IPU3 pipeline default settings can be found in patch 10. > > >> > > >> To access DDR via ImgU's own memory space, IPU3 is also equipped with > > >> its own MMU unit, the driver is implemented in patch 6. > > >> > > >> Patch 7 uses above driver for DMA mapping operation. > > >> > > >> The communication between IPU3 firmware and driver is implemented > > >> with circular queues in patch 8. > > >> > > >> Patch 9 provide some utility functions and manage IPU3 fw download > > >> and install. > > >> > > >> The firmware which is called ipu3-fw.bin can be downloaded from: > > >> > > >> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware > > >> .git (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > > >> > > >> Firmware ABI is defined in patches 4 and 5. > > >> > > >> Patches 12 and 13 are of the same file, the former contains all h/w > > >> programming related code, the latter implements interface functions > > >> for access fw & hw capabilities. > > >> > > >> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT > > work: > > >> > > >> <URL:https://patchwork.kernel.org/patch/9976295/> > > > I've pushed the latest set here: > > > > > > <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > > > > > > You can just say the entire set depends on those going forward; the > > > documentation is needed, too. > > > > > Ack. > > > >> Patch 15 represents the top level that glues all of the other > > >> components together, passing arguments between the components. > > >> > > >> Patch 16 is a recent effort to extend v6 for advanced camera features > > >> like Continuous View Finder (CVF) and Snapshot During Video(SDV) > > support. > > >> > > >> Link to user space implementation: > > >> > > >> git clone > > >> https://chromium.googlesource.com/chromiumos/platform/arc-camera > > >> > > >> ImgU media topology print: > > >> > > >> # media-ctl -d /dev/media0 -p > > >> Media controller API version 4.19.0 > > >> > > >> Media device information > > >> ------------------------ > > >> driver ipu3-imgu > > >> model ipu3-imgu > > >> serial > > >> bus info PCI:0000:00:05.0 > > >> hw revision 0x80862015 > > >> driver version 4.19.0 > > >> > > >> Device topology > > >> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > > >> type V4L2 subdev subtype Unknown flags 0 > > >> device node name /dev/v4l-subdev0 > > >> pad0: Sink > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > > This doesn't seem right. Which formats can be enumerated from the pad? > > Supposed to be raw formats for the colorspace field. I think you could even leave colorspace unchanged. Please also see my reply to Bing Bu. > > > > > > >> crop:(0,0)/1920x1080 > > >> compose:(0,0)/1920x1080] > > > Does the compose rectangle affect the scaling on all outputs? > > Sakari, driver use crop and compose targets to help set input-feeder and BDS > > output resolutions which are 2 key block of whole imaging pipeline, not the > > actual ending output, but they will impact the final output. > > > > > >> <- "ipu3-imgu 0 input":0 [] > > > Are there links that have no useful link configuration? If so, you > > > should set them enabled and immutable in the driver. > > The enabled status of input pads is used to get which pipe that user is trying > > to enable (ipu3_link_setup()), so it could not been set as immutable. > > > > > >> pad1: Sink > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > > I'd suggest to use MEDIA_BUS_FMT_FIXED here. > > Make sense as the parameter size is fixed. > > > > > > >> <- "ipu3-imgu 0 parameters":0 [] > > >> pad2: Source > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> -> "ipu3-imgu 0 output":0 [] > > >> pad3: Source > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> -> "ipu3-imgu 0 viewfinder":0 [] > > > Are there other differences between output and viewfinder? > > output and viewfinder are the main and secondary output of output system. > > 'main' output is not allowed to be scaled, only support crop. secondary > > output 'viewfinder' > > can support both cropping and scaling. User can select different nodes to use > > as preview and capture flexibly based on the actual use cases. > > > > > >> pad4: Source > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> -> "ipu3-imgu 0 3a stat":0 [] > > > FIXED here, too. > > Sure. > > > > > > >> - entity 7: ipu3-imgu 1 (5 pads, 5 links) > > >> type V4L2 subdev subtype Unknown flags 0 > > >> device node name /dev/v4l-subdev1 > > >> pad0: Sink > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > >> crop:(0,0)/1920x1080 > > >> compose:(0,0)/1920x1080] > > >> <- "ipu3-imgu 1 input":0 [] > > >> pad1: Sink > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> <- "ipu3-imgu 1 parameters":0 [] > > >> pad2: Source > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> -> "ipu3-imgu 1 output":0 [] > > >> pad3: Source > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> -> "ipu3-imgu 1 viewfinder":0 [] > > >> pad4: Source > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > >> -> "ipu3-imgu 1 3a stat":0 [] > > > This is a minor matter but --- could you create the second sub-device > > > after the video device nodes related to the first one have been already > > created? > > > That'd make reading the output easier. > > > > > This can be done in next update. Thanks! ... > > >> v4l2-compliance SHA: 7aa151889ffe89b1cd94a8198b0caba1a8c70398, 64 > > >> bits > > >> > > >> Compliance test for device /dev/media0: > > >> > > >> Media Driver Info: > > >> Driver name : ipu3-imgu > > >> Model : ipu3-imgu > > >> Serial : > > >> Bus info : PCI:0000:00:05.0 > > >> Media version : 4.19.0 > > >> Hardware revision: 0x80862015 (2156273685) > > > Is there no revision field for the hardware? We could also use the SoC > > > name in the model if it's known. It might be that there is another SoC > > > that contains the same device but I don't see that as a problem really. > > > > > For the first question, do you mean revision field of IPU3? > For the Model, do you suggest changing "ipu3-imgu" to "Intel(R) Core(TM) > m3-7Y30 CPU ipu3-imgu"? I have not found a handy way to read SoC name > yet. I was thinking of something like "Kaby lake". If the IPU3 appears in other SoCs it would likely have a different PCI ID.
On 11/09/2018 06:09 PM, Sakari Ailus wrote: > Hi Bing Bu, > > On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: >> On 11/01/2018 08:03 PM, Sakari Ailus wrote: >>> Hi Yong, >>> >>> Thanks for the update! >>> >>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: >>>> Hi, >>>> >>>> This series adds support for the Intel IPU3 (Image Processing Unit) >>>> ImgU which is essentially a modern memory-to-memory ISP. It implements >>>> raw Bayer to YUV image format conversion as well as a large number of >>>> other pixel processing algorithms for improving the image quality. >>>> >>>> Meta data formats are defined for image statistics (3A, i.e. automatic >>>> white balance, exposure and focus, histogram and local area contrast >>>> enhancement) as well as for the pixel processing algorithm parameters. >>>> The documentation for these formats is currently not included in the >>>> patchset but will be added in a future version of this set. >>>> >>>> The algorithm parameters need to be considered specific to a given frame >>>> and typically a large number of these parameters change on frame to frame >>>> basis. Additionally, the parameters are highly structured (and not a flat >>>> space of independent configuration primitives). They also reflect the >>>> data structures used by the firmware and the hardware. On top of that, >>>> the algorithms require highly specialized user space to make meaningful >>>> use of them. For these reasons it has been chosen video buffers to pass >>>> the parameters to the device. >>>> >>>> On individual patches: >>>> >>>> The heart of ImgU is the CSS, or Camera Subsystem, which contains the >>>> image processors and HW accelerators. >>>> >>>> The 3A statistics and other firmware parameter computation related >>>> functions are implemented in patch 11. >>>> >>>> All IPU3 pipeline default settings can be found in patch 10. >>>> >>>> To access DDR via ImgU's own memory space, IPU3 is also equipped with >>>> its own MMU unit, the driver is implemented in patch 6. >>>> >>>> Patch 7 uses above driver for DMA mapping operation. >>>> >>>> The communication between IPU3 firmware and driver is implemented with circular >>>> queues in patch 8. >>>> >>>> Patch 9 provide some utility functions and manage IPU3 fw download and >>>> install. >>>> >>>> The firmware which is called ipu3-fw.bin can be downloaded from: >>>> >>>> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >>>> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) >>>> >>>> Firmware ABI is defined in patches 4 and 5. >>>> >>>> Patches 12 and 13 are of the same file, the former contains all h/w programming >>>> related code, the latter implements interface functions for access fw & hw >>>> capabilities. >>>> >>>> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: >>>> >>>> <URL:https://patchwork.kernel.org/patch/9976295/> >>> I've pushed the latest set here: >>> >>> <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> >>> >>> You can just say the entire set depends on those going forward; the >>> documentation is needed, too. >>> >>>> Patch 15 represents the top level that glues all of the other components together, >>>> passing arguments between the components. >>>> >>>> Patch 16 is a recent effort to extend v6 for advanced camera features like >>>> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. >>>> >>>> Link to user space implementation: >>>> >>>> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera >>>> >>>> ImgU media topology print: >>>> >>>> # media-ctl -d /dev/media0 -p >>>> Media controller API version 4.19.0 >>>> >>>> Media device information >>>> ------------------------ >>>> driver ipu3-imgu >>>> model ipu3-imgu >>>> serial >>>> bus info PCI:0000:00:05.0 >>>> hw revision 0x80862015 >>>> driver version 4.19.0 >>>> >>>> Device topology >>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) >>>> type V4L2 subdev subtype Unknown flags 0 >>>> device node name /dev/v4l-subdev0 >>>> pad0: Sink >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown >>> This doesn't seem right. Which formats can be enumerated from the pad? > Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) > format whereas the CAPTURE video nodes always have NV12. Can you confirm? Hi, Sakari, Yes, I think the pad_fmt should also be changed. Yong, could you add some extra code for this and test? like: static int ipu3_v4l2_node_setup(struct imgu_device *imgu, unsigned int pipe, ... Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â V4L2_PIX_FMT_NV12; Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->vdev_fmt.fmt.pix_mp = def_pix_fmt; Â Â Â Â Â Â Â } +Â Â Â Â Â Â if (node->vdev_fmt.type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) +Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->pad_fmt.code = MEDIA_BUS_FMT_SGRBG10_1X10; + Â > > If the OUTPUT video node format selection has no effect on the rest of the > pipeline (device capabilities, which processing blocks are in use, CAPTURE > video nodes formats etc.), I think you could simply use the FIXED media bus > code for each pad. That would actually make sense: this device always works > from memory to memory, and thus does not really have a pixel data bus > external to the device which is what the media bus codes really are for. > >>>> crop:(0,0)/1920x1080 >>>> compose:(0,0)/1920x1080] >>> Does the compose rectangle affect the scaling on all outputs? >> Sakari, driver use crop and compose targets to help set input-feeder and BDS >> output resolutions which are 2 key block of whole imaging pipeline, not the >> actual ending output, but they will impact the final output. > Ack. Thanks for the clarification. > >>>> <- "ipu3-imgu 0 input":0 [] >>> Are there links that have no useful link configuration? If so, you should >>> set them enabled and immutable in the driver. >> The enabled status of input pads is used to get which pipe that user is >> trying to enable (ipu3_link_setup()), so it could not been set as immutable. > But the rest of them could be, right? Yes. > >>>> pad1: Sink >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>> I'd suggest to use MEDIA_BUS_FMT_FIXED here. >>> >>>> <- "ipu3-imgu 0 parameters":0 [] >>>> pad2: Source >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>> -> "ipu3-imgu 0 output":0 [] >>>> pad3: Source >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>> -> "ipu3-imgu 0 viewfinder":0 [] >>> Are there other differences between output and viewfinder? >> output and viewfinder are the main and secondary output of output system. >> 'main' output is not allowed to be scaled, only support crop. secondary >> output 'viewfinder' >> can support both cropping and scaling. User can select different nodes >> to use >> as preview and capture flexibly based on the actual use cases. > If there's scaling to be configured, I'd expect to see the COMPOSE target > supported. Actually the viewfinder is the result of scaling, that means you can not do more scaling. The resolution of output and viewfinder should be fixed once the pipeline is determined. >
Hi Bing Bu, On Mon, Nov 12, 2018 at 12:31:16PM +0800, Bing Bu Cao wrote: > > > On 11/09/2018 06:09 PM, Sakari Ailus wrote: > > Hi Bing Bu, > > > > On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: > >> On 11/01/2018 08:03 PM, Sakari Ailus wrote: > >>> Hi Yong, > >>> > >>> Thanks for the update! > >>> > >>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > >>>> Hi, > >>>> > >>>> This series adds support for the Intel IPU3 (Image Processing Unit) > >>>> ImgU which is essentially a modern memory-to-memory ISP. It implements > >>>> raw Bayer to YUV image format conversion as well as a large number of > >>>> other pixel processing algorithms for improving the image quality. > >>>> > >>>> Meta data formats are defined for image statistics (3A, i.e. automatic > >>>> white balance, exposure and focus, histogram and local area contrast > >>>> enhancement) as well as for the pixel processing algorithm parameters. > >>>> The documentation for these formats is currently not included in the > >>>> patchset but will be added in a future version of this set. > >>>> > >>>> The algorithm parameters need to be considered specific to a given frame > >>>> and typically a large number of these parameters change on frame to frame > >>>> basis. Additionally, the parameters are highly structured (and not a flat > >>>> space of independent configuration primitives). They also reflect the > >>>> data structures used by the firmware and the hardware. On top of that, > >>>> the algorithms require highly specialized user space to make meaningful > >>>> use of them. For these reasons it has been chosen video buffers to pass > >>>> the parameters to the device. > >>>> > >>>> On individual patches: > >>>> > >>>> The heart of ImgU is the CSS, or Camera Subsystem, which contains the > >>>> image processors and HW accelerators. > >>>> > >>>> The 3A statistics and other firmware parameter computation related > >>>> functions are implemented in patch 11. > >>>> > >>>> All IPU3 pipeline default settings can be found in patch 10. > >>>> > >>>> To access DDR via ImgU's own memory space, IPU3 is also equipped with > >>>> its own MMU unit, the driver is implemented in patch 6. > >>>> > >>>> Patch 7 uses above driver for DMA mapping operation. > >>>> > >>>> The communication between IPU3 firmware and driver is implemented with circular > >>>> queues in patch 8. > >>>> > >>>> Patch 9 provide some utility functions and manage IPU3 fw download and > >>>> install. > >>>> > >>>> The firmware which is called ipu3-fw.bin can be downloaded from: > >>>> > >>>> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > >>>> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > >>>> > >>>> Firmware ABI is defined in patches 4 and 5. > >>>> > >>>> Patches 12 and 13 are of the same file, the former contains all h/w programming > >>>> related code, the latter implements interface functions for access fw & hw > >>>> capabilities. > >>>> > >>>> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > >>>> > >>>> <URL:https://patchwork.kernel.org/patch/9976295/> > >>> I've pushed the latest set here: > >>> > >>> <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > >>> > >>> You can just say the entire set depends on those going forward; the > >>> documentation is needed, too. > >>> > >>>> Patch 15 represents the top level that glues all of the other components together, > >>>> passing arguments between the components. > >>>> > >>>> Patch 16 is a recent effort to extend v6 for advanced camera features like > >>>> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > >>>> > >>>> Link to user space implementation: > >>>> > >>>> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > >>>> > >>>> ImgU media topology print: > >>>> > >>>> # media-ctl -d /dev/media0 -p > >>>> Media controller API version 4.19.0 > >>>> > >>>> Media device information > >>>> ------------------------ > >>>> driver ipu3-imgu > >>>> model ipu3-imgu > >>>> serial > >>>> bus info PCI:0000:00:05.0 > >>>> hw revision 0x80862015 > >>>> driver version 4.19.0 > >>>> > >>>> Device topology > >>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >>>> type V4L2 subdev subtype Unknown flags 0 > >>>> device node name /dev/v4l-subdev0 > >>>> pad0: Sink > >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >>> This doesn't seem right. Which formats can be enumerated from the pad? > > Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) > > format whereas the CAPTURE video nodes always have NV12. Can you confirm? > Hi, Sakari, > Yes, I think the pad_fmt should also be changed. > Yong, could you add some extra code for this and test? like: > > static int ipu3_v4l2_node_setup(struct imgu_device *imgu, unsigned int pipe, > ... > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â V4L2_PIX_FMT_NV12; > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->vdev_fmt.fmt.pix_mp = def_pix_fmt; > Â Â Â Â Â Â Â } > > +Â Â Â Â Â Â if (node->vdev_fmt.type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) > +Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->pad_fmt.code = MEDIA_BUS_FMT_SGRBG10_1X10; > + > Â > > > > If the OUTPUT video node format selection has no effect on the rest of the > > pipeline (device capabilities, which processing blocks are in use, CAPTURE > > video nodes formats etc.), I think you could simply use the FIXED media bus > > code for each pad. That would actually make sense: this device always works > > from memory to memory, and thus does not really have a pixel data bus > > external to the device which is what the media bus codes really are for. > > > >>>> crop:(0,0)/1920x1080 > >>>> compose:(0,0)/1920x1080] > >>> Does the compose rectangle affect the scaling on all outputs? > >> Sakari, driver use crop and compose targets to help set input-feeder and BDS > >> output resolutions which are 2 key block of whole imaging pipeline, not the > >> actual ending output, but they will impact the final output. > > Ack. Thanks for the clarification. > > > >>>> <- "ipu3-imgu 0 input":0 [] > >>> Are there links that have no useful link configuration? If so, you should > >>> set them enabled and immutable in the driver. > >> The enabled status of input pads is used to get which pipe that user is > >> trying to enable (ipu3_link_setup()), so it could not been set as immutable. > > But the rest of them could be, right? > Yes. > > > >>>> pad1: Sink > >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>> I'd suggest to use MEDIA_BUS_FMT_FIXED here. > >>> > >>>> <- "ipu3-imgu 0 parameters":0 [] > >>>> pad2: Source > >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>>> -> "ipu3-imgu 0 output":0 [] > >>>> pad3: Source > >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>>> -> "ipu3-imgu 0 viewfinder":0 [] > >>> Are there other differences between output and viewfinder? > >> output and viewfinder are the main and secondary output of output system. > >> 'main' output is not allowed to be scaled, only support crop. secondary > >> output 'viewfinder' > >> can support both cropping and scaling. User can select different nodes > >> to use > >> as preview and capture flexibly based on the actual use cases. > > If there's scaling to be configured, I'd expect to see the COMPOSE target > > supported. > Actually the viewfinder is the result of scaling, that means you can not > do more scaling. How do you configure the scaling of the viewfinder currently? > The resolution of output and viewfinder should be fixed once the > pipeline is determined.
On 11/13/2018 06:31 PM, Sakari Ailus wrote: > Hi Bing Bu, > > On Mon, Nov 12, 2018 at 12:31:16PM +0800, Bing Bu Cao wrote: >> >> On 11/09/2018 06:09 PM, Sakari Ailus wrote: >>> Hi Bing Bu, >>> >>> On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: >>>> On 11/01/2018 08:03 PM, Sakari Ailus wrote: >>>>> Hi Yong, >>>>> >>>>> Thanks for the update! >>>>> >>>>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: >>>>>> Hi, >>>>>> >>>>>> This series adds support for the Intel IPU3 (Image Processing Unit) >>>>>> ImgU which is essentially a modern memory-to-memory ISP. It implements >>>>>> raw Bayer to YUV image format conversion as well as a large number of >>>>>> other pixel processing algorithms for improving the image quality. >>>>>> >>>>>> Meta data formats are defined for image statistics (3A, i.e. automatic >>>>>> white balance, exposure and focus, histogram and local area contrast >>>>>> enhancement) as well as for the pixel processing algorithm parameters. >>>>>> The documentation for these formats is currently not included in the >>>>>> patchset but will be added in a future version of this set. >>>>>> >>>>>> The algorithm parameters need to be considered specific to a given frame >>>>>> and typically a large number of these parameters change on frame to frame >>>>>> basis. Additionally, the parameters are highly structured (and not a flat >>>>>> space of independent configuration primitives). They also reflect the >>>>>> data structures used by the firmware and the hardware. On top of that, >>>>>> the algorithms require highly specialized user space to make meaningful >>>>>> use of them. For these reasons it has been chosen video buffers to pass >>>>>> the parameters to the device. >>>>>> >>>>>> On individual patches: >>>>>> >>>>>> The heart of ImgU is the CSS, or Camera Subsystem, which contains the >>>>>> image processors and HW accelerators. >>>>>> >>>>>> The 3A statistics and other firmware parameter computation related >>>>>> functions are implemented in patch 11. >>>>>> >>>>>> All IPU3 pipeline default settings can be found in patch 10. >>>>>> >>>>>> To access DDR via ImgU's own memory space, IPU3 is also equipped with >>>>>> its own MMU unit, the driver is implemented in patch 6. >>>>>> >>>>>> Patch 7 uses above driver for DMA mapping operation. >>>>>> >>>>>> The communication between IPU3 firmware and driver is implemented with circular >>>>>> queues in patch 8. >>>>>> >>>>>> Patch 9 provide some utility functions and manage IPU3 fw download and >>>>>> install. >>>>>> >>>>>> The firmware which is called ipu3-fw.bin can be downloaded from: >>>>>> >>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >>>>>> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) >>>>>> >>>>>> Firmware ABI is defined in patches 4 and 5. >>>>>> >>>>>> Patches 12 and 13 are of the same file, the former contains all h/w programming >>>>>> related code, the latter implements interface functions for access fw & hw >>>>>> capabilities. >>>>>> >>>>>> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: >>>>>> >>>>>> <URL:https://patchwork.kernel.org/patch/9976295/> >>>>> I've pushed the latest set here: >>>>> >>>>> <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> >>>>> >>>>> You can just say the entire set depends on those going forward; the >>>>> documentation is needed, too. >>>>> >>>>>> Patch 15 represents the top level that glues all of the other components together, >>>>>> passing arguments between the components. >>>>>> >>>>>> Patch 16 is a recent effort to extend v6 for advanced camera features like >>>>>> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. >>>>>> >>>>>> Link to user space implementation: >>>>>> >>>>>> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera >>>>>> >>>>>> ImgU media topology print: >>>>>> >>>>>> # media-ctl -d /dev/media0 -p >>>>>> Media controller API version 4.19.0 >>>>>> >>>>>> Media device information >>>>>> ------------------------ >>>>>> driver ipu3-imgu >>>>>> model ipu3-imgu >>>>>> serial >>>>>> bus info PCI:0000:00:05.0 >>>>>> hw revision 0x80862015 >>>>>> driver version 4.19.0 >>>>>> >>>>>> Device topology >>>>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) >>>>>> type V4L2 subdev subtype Unknown flags 0 >>>>>> device node name /dev/v4l-subdev0 >>>>>> pad0: Sink >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown >>>>> This doesn't seem right. Which formats can be enumerated from the pad? >>> Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) >>> format whereas the CAPTURE video nodes always have NV12. Can you confirm? >> Hi, Sakari, >> Yes, I think the pad_fmt should also be changed. >> Yong, could you add some extra code for this and test? like: >> >> static int ipu3_v4l2_node_setup(struct imgu_device *imgu, unsigned int pipe, >> ... >> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â V4L2_PIX_FMT_NV12; >> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->vdev_fmt.fmt.pix_mp = def_pix_fmt; >> Â Â Â Â Â Â Â } >> >> +Â Â Â Â Â Â if (node->vdev_fmt.type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) >> +Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->pad_fmt.code = MEDIA_BUS_FMT_SGRBG10_1X10; >> + >> Â >>> If the OUTPUT video node format selection has no effect on the rest of the >>> pipeline (device capabilities, which processing blocks are in use, CAPTURE >>> video nodes formats etc.), I think you could simply use the FIXED media bus >>> code for each pad. That would actually make sense: this device always works >>> from memory to memory, and thus does not really have a pixel data bus >>> external to the device which is what the media bus codes really are for. >>> >>>>>> crop:(0,0)/1920x1080 >>>>>> compose:(0,0)/1920x1080] >>>>> Does the compose rectangle affect the scaling on all outputs? >>>> Sakari, driver use crop and compose targets to help set input-feeder and BDS >>>> output resolutions which are 2 key block of whole imaging pipeline, not the >>>> actual ending output, but they will impact the final output. >>> Ack. Thanks for the clarification. >>> >>>>>> <- "ipu3-imgu 0 input":0 [] >>>>> Are there links that have no useful link configuration? If so, you should >>>>> set them enabled and immutable in the driver. >>>> The enabled status of input pads is used to get which pipe that user is >>>> trying to enable (ipu3_link_setup()), so it could not been set as immutable. >>> But the rest of them could be, right? >> Yes. >>>>>> pad1: Sink >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>> I'd suggest to use MEDIA_BUS_FMT_FIXED here. >>>>> >>>>>> <- "ipu3-imgu 0 parameters":0 [] >>>>>> pad2: Source >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>>> -> "ipu3-imgu 0 output":0 [] >>>>>> pad3: Source >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>>> -> "ipu3-imgu 0 viewfinder":0 [] >>>>> Are there other differences between output and viewfinder? >>>> output and viewfinder are the main and secondary output of output system. >>>> 'main' output is not allowed to be scaled, only support crop. secondary >>>> output 'viewfinder' >>>> can support both cropping and scaling. User can select different nodes >>>> to use >>>> as preview and capture flexibly based on the actual use cases. >>> If there's scaling to be configured, I'd expect to see the COMPOSE target >>> supported. >> Actually the viewfinder is the result of scaling, that means you can not >> do more scaling. > How do you configure the scaling of the viewfinder currently? We consider that the viewfinder as a secondary output, and set the format by subdev set_fmt() directly and all pads formats will be used to find binary and build pipeline. > >> The resolution of output and viewfinder should be fixed once the >> pipeline is determined.
On Tue, Nov 13, 2018 at 07:04:01PM +0800, Bing Bu Cao wrote: > > > On 11/13/2018 06:31 PM, Sakari Ailus wrote: > > Hi Bing Bu, > > > > On Mon, Nov 12, 2018 at 12:31:16PM +0800, Bing Bu Cao wrote: > >> > >> On 11/09/2018 06:09 PM, Sakari Ailus wrote: > >>> Hi Bing Bu, > >>> > >>> On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: > >>>> On 11/01/2018 08:03 PM, Sakari Ailus wrote: > >>>>> Hi Yong, > >>>>> > >>>>> Thanks for the update! > >>>>> > >>>>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > >>>>>> Hi, > >>>>>> > >>>>>> This series adds support for the Intel IPU3 (Image Processing Unit) > >>>>>> ImgU which is essentially a modern memory-to-memory ISP. It implements > >>>>>> raw Bayer to YUV image format conversion as well as a large number of > >>>>>> other pixel processing algorithms for improving the image quality. > >>>>>> > >>>>>> Meta data formats are defined for image statistics (3A, i.e. automatic > >>>>>> white balance, exposure and focus, histogram and local area contrast > >>>>>> enhancement) as well as for the pixel processing algorithm parameters. > >>>>>> The documentation for these formats is currently not included in the > >>>>>> patchset but will be added in a future version of this set. > >>>>>> > >>>>>> The algorithm parameters need to be considered specific to a given frame > >>>>>> and typically a large number of these parameters change on frame to frame > >>>>>> basis. Additionally, the parameters are highly structured (and not a flat > >>>>>> space of independent configuration primitives). They also reflect the > >>>>>> data structures used by the firmware and the hardware. On top of that, > >>>>>> the algorithms require highly specialized user space to make meaningful > >>>>>> use of them. For these reasons it has been chosen video buffers to pass > >>>>>> the parameters to the device. > >>>>>> > >>>>>> On individual patches: > >>>>>> > >>>>>> The heart of ImgU is the CSS, or Camera Subsystem, which contains the > >>>>>> image processors and HW accelerators. > >>>>>> > >>>>>> The 3A statistics and other firmware parameter computation related > >>>>>> functions are implemented in patch 11. > >>>>>> > >>>>>> All IPU3 pipeline default settings can be found in patch 10. > >>>>>> > >>>>>> To access DDR via ImgU's own memory space, IPU3 is also equipped with > >>>>>> its own MMU unit, the driver is implemented in patch 6. > >>>>>> > >>>>>> Patch 7 uses above driver for DMA mapping operation. > >>>>>> > >>>>>> The communication between IPU3 firmware and driver is implemented with circular > >>>>>> queues in patch 8. > >>>>>> > >>>>>> Patch 9 provide some utility functions and manage IPU3 fw download and > >>>>>> install. > >>>>>> > >>>>>> The firmware which is called ipu3-fw.bin can be downloaded from: > >>>>>> > >>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > >>>>>> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > >>>>>> > >>>>>> Firmware ABI is defined in patches 4 and 5. > >>>>>> > >>>>>> Patches 12 and 13 are of the same file, the former contains all h/w programming > >>>>>> related code, the latter implements interface functions for access fw & hw > >>>>>> capabilities. > >>>>>> > >>>>>> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > >>>>>> > >>>>>> <URL:https://patchwork.kernel.org/patch/9976295/> > >>>>> I've pushed the latest set here: > >>>>> > >>>>> <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > >>>>> > >>>>> You can just say the entire set depends on those going forward; the > >>>>> documentation is needed, too. > >>>>> > >>>>>> Patch 15 represents the top level that glues all of the other components together, > >>>>>> passing arguments between the components. > >>>>>> > >>>>>> Patch 16 is a recent effort to extend v6 for advanced camera features like > >>>>>> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > >>>>>> > >>>>>> Link to user space implementation: > >>>>>> > >>>>>> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > >>>>>> > >>>>>> ImgU media topology print: > >>>>>> > >>>>>> # media-ctl -d /dev/media0 -p > >>>>>> Media controller API version 4.19.0 > >>>>>> > >>>>>> Media device information > >>>>>> ------------------------ > >>>>>> driver ipu3-imgu > >>>>>> model ipu3-imgu > >>>>>> serial > >>>>>> bus info PCI:0000:00:05.0 > >>>>>> hw revision 0x80862015 > >>>>>> driver version 4.19.0 > >>>>>> > >>>>>> Device topology > >>>>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >>>>>> type V4L2 subdev subtype Unknown flags 0 > >>>>>> device node name /dev/v4l-subdev0 > >>>>>> pad0: Sink > >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >>>>> This doesn't seem right. Which formats can be enumerated from the pad? > >>> Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) > >>> format whereas the CAPTURE video nodes always have NV12. Can you confirm? > >> Hi, Sakari, > >> Yes, I think the pad_fmt should also be changed. > >> Yong, could you add some extra code for this and test? like: > >> > >> static int ipu3_v4l2_node_setup(struct imgu_device *imgu, unsigned int pipe, > >> ... > >> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â V4L2_PIX_FMT_NV12; > >> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->vdev_fmt.fmt.pix_mp = def_pix_fmt; > >> Â Â Â Â Â Â Â } > >> > >> +Â Â Â Â Â Â if (node->vdev_fmt.type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) > >> +Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->pad_fmt.code = MEDIA_BUS_FMT_SGRBG10_1X10; > >> + > >> Â > >>> If the OUTPUT video node format selection has no effect on the rest of the > >>> pipeline (device capabilities, which processing blocks are in use, CAPTURE > >>> video nodes formats etc.), I think you could simply use the FIXED media bus > >>> code for each pad. That would actually make sense: this device always works > >>> from memory to memory, and thus does not really have a pixel data bus > >>> external to the device which is what the media bus codes really are for. > >>> > >>>>>> crop:(0,0)/1920x1080 > >>>>>> compose:(0,0)/1920x1080] > >>>>> Does the compose rectangle affect the scaling on all outputs? > >>>> Sakari, driver use crop and compose targets to help set input-feeder and BDS > >>>> output resolutions which are 2 key block of whole imaging pipeline, not the > >>>> actual ending output, but they will impact the final output. > >>> Ack. Thanks for the clarification. > >>> > >>>>>> <- "ipu3-imgu 0 input":0 [] > >>>>> Are there links that have no useful link configuration? If so, you should > >>>>> set them enabled and immutable in the driver. > >>>> The enabled status of input pads is used to get which pipe that user is > >>>> trying to enable (ipu3_link_setup()), so it could not been set as immutable. > >>> But the rest of them could be, right? > >> Yes. > >>>>>> pad1: Sink > >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>>>> I'd suggest to use MEDIA_BUS_FMT_FIXED here. > >>>>> > >>>>>> <- "ipu3-imgu 0 parameters":0 [] > >>>>>> pad2: Source > >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>>>>> -> "ipu3-imgu 0 output":0 [] > >>>>>> pad3: Source > >>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>>>>> -> "ipu3-imgu 0 viewfinder":0 [] > >>>>> Are there other differences between output and viewfinder? > >>>> output and viewfinder are the main and secondary output of output system. > >>>> 'main' output is not allowed to be scaled, only support crop. secondary > >>>> output 'viewfinder' > >>>> can support both cropping and scaling. User can select different nodes > >>>> to use > >>>> as preview and capture flexibly based on the actual use cases. > >>> If there's scaling to be configured, I'd expect to see the COMPOSE target > >>> supported. > >> Actually the viewfinder is the result of scaling, that means you can not > >> do more scaling. > > How do you configure the scaling of the viewfinder currently? > We consider that the viewfinder as a secondary output, and set the format by > subdev set_fmt() directly and all pads formats will be used to find > binary and > build pipeline. Ok. Could you instead use the compose target to configure the scaling? Setting the format on the source pad would have no effect.
On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > Hi, > > This series adds support for the Intel IPU3 (Image Processing Unit) > ImgU which is essentially a modern memory-to-memory ISP. It implements > raw Bayer to YUV image format conversion as well as a large number of > other pixel processing algorithms for improving the image quality. > > Meta data formats are defined for image statistics (3A, i.e. automatic > white balance, exposure and focus, histogram and local area contrast > enhancement) as well as for the pixel processing algorithm parameters. > The documentation for these formats is currently not included in the > patchset but will be added in a future version of this set. > > The algorithm parameters need to be considered specific to a given frame > and typically a large number of these parameters change on frame to frame > basis. Additionally, the parameters are highly structured (and not a flat > space of independent configuration primitives). They also reflect the > data structures used by the firmware and the hardware. On top of that, > the algorithms require highly specialized user space to make meaningful > use of them. For these reasons it has been chosen video buffers to pass > the parameters to the device. > > On individual patches: > > The heart of ImgU is the CSS, or Camera Subsystem, which contains the > image processors and HW accelerators. > > The 3A statistics and other firmware parameter computation related > functions are implemented in patch 11. > > All IPU3 pipeline default settings can be found in patch 10. > Seems to me that patch 10 didn't make it to the mailing list, am I wrong? I'm pointing it out as the same happened on your v6. Thanks j > To access DDR via ImgU's own memory space, IPU3 is also equipped with > its own MMU unit, the driver is implemented in patch 6. > > Patch 7 uses above driver for DMA mapping operation. > > The communication between IPU3 firmware and driver is implemented with circular > queues in patch 8. > > Patch 9 provide some utility functions and manage IPU3 fw download and > install. > > The firmware which is called ipu3-fw.bin can be downloaded from: > > git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > > Firmware ABI is defined in patches 4 and 5. > > Patches 12 and 13 are of the same file, the former contains all h/w programming > related code, the latter implements interface functions for access fw & hw > capabilities. > > Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > > <URL:https://patchwork.kernel.org/patch/9976295/> > > Patch 15 represents the top level that glues all of the other components together, > passing arguments between the components. > > Patch 16 is a recent effort to extend v6 for advanced camera features like > Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > > Link to user space implementation: > > git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > > ImgU media topology print: > > # media-ctl -d /dev/media0 -p > Media controller API version 4.19.0 > > Media device information > ------------------------ > driver ipu3-imgu > model ipu3-imgu > serial > bus info PCI:0000:00:05.0 > hw revision 0x80862015 > driver version 4.19.0 > > Device topology > - entity 1: ipu3-imgu 0 (5 pads, 5 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > pad0: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > crop:(0,0)/1920x1080 > compose:(0,0)/1920x1080] > <- "ipu3-imgu 0 input":0 [] > pad1: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > <- "ipu3-imgu 0 parameters":0 [] > pad2: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 0 output":0 [] > pad3: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 0 viewfinder":0 [] > pad4: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 0 3a stat":0 [] > > - entity 7: ipu3-imgu 1 (5 pads, 5 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev1 > pad0: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > crop:(0,0)/1920x1080 > compose:(0,0)/1920x1080] > <- "ipu3-imgu 1 input":0 [] > pad1: Sink > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > <- "ipu3-imgu 1 parameters":0 [] > pad2: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 1 output":0 [] > pad3: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 1 viewfinder":0 [] > pad4: Source > [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > -> "ipu3-imgu 1 3a stat":0 [] > > - entity 17: ipu3-imgu 0 input (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Source > -> "ipu3-imgu 0":0 [] > > - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video1 > pad0: Source > -> "ipu3-imgu 0":1 [] > > - entity 29: ipu3-imgu 0 output (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video2 > pad0: Sink > <- "ipu3-imgu 0":2 [] > > - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video3 > pad0: Sink > <- "ipu3-imgu 0":3 [] > > - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video4 > pad0: Sink > <- "ipu3-imgu 0":4 [] > > - entity 47: ipu3-imgu 1 input (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video5 > pad0: Source > -> "ipu3-imgu 1":0 [] > > - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video6 > pad0: Source > -> "ipu3-imgu 1":1 [] > > - entity 59: ipu3-imgu 1 output (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video7 > pad0: Sink > <- "ipu3-imgu 1":2 [] > > - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video8 > pad0: Sink > <- "ipu3-imgu 1":3 [] > > - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video9 > pad0: Sink > <- "ipu3-imgu 1":4 [] > > > v4l2-compliance utility is built with Sakari's patches for meta data > output support(rebased): > > <URL:https://patchwork.linuxtv.org/patch/43370/> > <URL:https://patchwork.linuxtv.org/patch/43369/> > > The test (v4l2-compliance -m 0) passes without error, outputs are appended at > the end of revision history. > > Note: > > 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be enabled > prior to the test. > 2. Stream tests are not performed since it requires pre-configuration for each case. > > =========== > = history = > =========== > > v7 update: > > 1. Add driver and uAPI documentation. > > Update based on v1 review from Tomasz, Hans, Sokari and Mauro: > https://patchwork.kernel.org/patch/10465663/ > https://patchwork.kernel.org/patch/10465665/ > > 2. Add dual pipe support which includes: > - Extend current IMGU device to contain 2 subdevs and two groups of video nodes. > - Add a v4l2 ctrl to allow user to specify the mode(video or still) of the pipe. > > 3. Kconfig > - Restrict build for X86 arch to fix build error for ia64/sparc. > (fatal error: asm/set_memory.h: No such file or directory) > > 4. ipu3-abi.h > - Change __u32 to u32. > - Use generic __attribute__((aligned(x))) format. (Mauro/Hans) > - Split abi to 2 patches, one for register defines, enums, the other for structs. (Tomasz) > > 5. ipu3-mmu.c > - Fix ipu3-mmu/dmamap exit functions. (Tomasz) > (Port from https://chromium-review.googlesource.com/1084522) > - Use free_page instead of kfree. (Tomasz) > - document struct ipu3_mmu_info. > - Fix copyright information. > > 6. ipu3-dmamap.c (Tomasz) > - Update APIs based on v6 review. > - Replace sizeof(struct page *) with sizeof(*pages). > - Remove un-needed (WARN_ON(!dev)) inside void *ipu3_dmamap_alloc(). > > 7. ipu3.c (Tomasz) > - imgu_video_nodes_init() > Fix the missing call to ipu3_v4l2_unregister() in the error path of > imgu_dummybufs_preallocate(). > - imgu_queue_buffers() > Evaluate loop condition explicitly for code clarity and simplicity. > FW requires all output buffers to be queued at start, so adjust the order of > buffer queuing accordingly. (bufix by Tianshu) > - imgu_isr_threaded() > Fix interrupt handler return value. > (Port from https://chromium-review.googlesource.com/1088539) > - Add back the buf_drain_wq from ("avoid sleep in wait_event condition")' > (Port from https://chromium-review.googlesource.com/875420) > > 8. ipu3-v4l2.c > - ipu3_v4l2_register(). (Tomasz) > Split media initialization and registration, also change media device > register/un-register order. > > - Fix v4l2-compliance fail on sub-devicef for VIDIOC_CREATE_BUFS and > VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT. > > 9. ipu3-css.c, ipu3-css.h, ipu3-css-fw.h, ipu3-abi.h > - Convert macros in structs to enums. (Tomasz) > > 10. ipu3-css-pool.c, ipu3-css-pool.h, ipu3.c > - Document the structs. (Hans/Maruo) > > 11. ipu3-css-params.c > - Fixup for noise reduction parameters processing. (bug fixing) > > version 6: > > - intel-ipu3.h uAPI > Move out the definitions not used by user space. (suggested by Sakari) > - ipu3-abi.h, ipu3-css-fw.h > Clean up the header files. > Remove enum type from ABI structs. > - ipu3-css.h and ipu3-css.c > Disable DVS support and remove related code. > - ipu3-v4l2.c > Fixes of v4l2_compliance test fails on ImgU sub-dev. > - ipu3-css-params.c > Refactor awb/awb_fr/af_ops_calc() functions. (Sakari) > - Build mmu and dmamap driver as part of ImgU ko module; (Sakari) > - Add "ipu3-imgu" prefix to media entity names; (Sakari) > - Fix indentation and white space; (Sakari) > - Rebase to kernel v4.16; > - Use SPDX license identifiers in all drivers; (Sakari) > - Internal fix and performance improvements such as: > Stop fw gracefully during stream off. > Enable irq only after start streaming to avoid unexpected interrupt. > Use spinlock to protect IPU3_CSS_QUEUES access. > Return NULL when dequeuing buffer before streaming. > > TODOs: > - Documentation on ImgU driver programming interface to configure and enable > ISP HW, which will include details on complete V4L2 Kernel driver interface > and IO-Control parameters, except for the ISP internal algorithm and its > parameters (which is Intel proprietary IP). > > version 5: > - ipu3-css-pool.c/ipu3_css_pool_check(). > add handling of the framenum wrap around case in ipu3_css_pool_check(). > - ipu3.c, ipu3-v4l2.c, ipu3.h > merge struct ipu3_mem2mem2_device into imgu_device and update the code > accordingly. (Suggested by Sakari) > - ipu3-mmu.c driver: > use __get_free_page() for page-aligned allocations (Tomasz). > optimize tlb invalidation by calling them at the end of map/unmap. (Tomasz). > remove dependency on iommu. (Sakari) > introduce few new functions from iommu.c. > - ipu3-dmamap.c driver > call mmu directly without IOMMU_SUPPORT (Sakari) > update dmamap APIs. (Suggested by Tomasz) > - ipu3_v4l2.c > move g/s_selection callback to V4l2 sub-device (Sakari) > remove colon from ImgU sub-device name. (Sakari) > - ipu3-css-params.c > fix indentation, 0-day scan warnings etc. > - ipu3-css.c > fix warning about NULL comparison. (Sakari) > - intel-ipu3.h: > remove redundant IPU3_ALIGN attribute (Sakari). > fix up un-needed fields in struct ipu3_uapi_params (Sakari) > re-order this to be 2nd in the patch set. > - Makefile: remove Copyright header. (Sakari) > - Internal fix: > optimize shot-to-shot performance. > update default white balance gains defined in ipu3-tables.c > > TODOs: > > - Documentation on ImgU driver programming interface to configure and enable > ISP HW, which will include details on complete V4L2 Kernel driver interface > and IO-Control parameters, except for the ISP internal algorithm and its > parameters (which is Intel proprietary IP). > > - Review ipu3_css_pool_* group APIs usage. > > version 4: > - Used V4L2_BUF_TYPE_META_OUTPUT for: > - V4L2_META_FMT_IPU3_STAT_PARAMS > > - Used V4L2_BUF_TYPE_META_CAPTURE for: > - V4L2_META_FMT_IPU3_STAT_3A > - V4L2_META_FMT_IPU3_STAT_DVS > - V4L2_META_FMT_IPU3_STAT_LACE > - Supported v4l2 MPLANE format on video nodes. > - ipu3-dmamap.c: Removed dma ops and dependencies on IOMMU_DMA lib. > - ipu3-mmu.c: Restructured the driver. > - intel-ipu3.h: Added __padding qualifier for uapi definitions. > - Internal fix: power and performance related issues. > - Fixed v4l2-compliance test. > - Fixed build failure for x86 with 32bit config. > > version 3: > - ipu3-mmu.c and ipu3-dmamap.c: > Tomasz Figa reworked both drivers and updated related files. > - ipu2-abi.h: > update imgu_abi_binary_info ABI to support latest ipu3-fw.bin. > use __packed qualifier on structs suggested by Sakari Ailus. > - ipu3-css-fw.c/ipu3-css-fw.h: following fix were suggested by Tomasz Figa: > remove pointer type in firmware blob structs. > fix binary_header array in struct imgu_fw_header. > fix calling ipu3_css_fw_show_binary() before proper checking. > fix logic error for valid length checking of blob name. > - ipu3-css-params.c/ipu3_css_scaler_get_exp(): > use lib helper suggested by Andy Shevchenko. > - ipu3-v4l2.c/ipu3_videoc_querycap(): > fill device_caps fix suggested by Hans Verkuil. > add VB2_DMABUF suggested by Tomasz Figa. > - ipu3-css.c: increase IMGU freq from 300MHZ to 450MHZ (internal fix) > - ipu3.c: use vb2_dma_sg_memop for the time being(internal fix). > > version 2: > This version cherry-picked firmware ABI change and other > fix in order to bring the code up-to-date with our internal release. > > I will go over the review comments in v1 and address them in v3 and > future update. > > version 1: > - Initial submission > > -------------------------------------------------------------------------------- > > v4l2-compliance test output: > > ./v4l2-compliance -m 0 > > v4l2-compliance SHA: 7aa151889ffe89b1cd94a8198b0caba1a8c70398, 64 bits > > Compliance test for device /dev/media0: > > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > > Required ioctls: > test MEDIA_IOC_DEVICE_INFO: OK > > Allow for multiple opens: > test second /dev/media0 open: OK > test MEDIA_IOC_DEVICE_INFO: OK > test for unlimited opens: OK > > Media Controller ioctls: > test MEDIA_IOC_G_TOPOLOGY: OK > Entities: 12 Interfaces: 12 Pads: 20 Links: 22 > test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK > test MEDIA_IOC_SETUP_LINK: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/v4l-subdev0: > > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x0300000d > Type : V4L Sub-Device > Entity Info: > ID : 0x00000001 (1) > Name : ipu3-imgu 0 > Function : Video Statistics > Pad 0x01000002 : 0: Sink > Link 0x02000015: from remote pad 0x1000012 of entity 'ipu3-imgu 0 input': Data, Enabled > Pad 0x01000003 : 1: Sink > Link 0x0200001b: from remote pad 0x1000018 of entity 'ipu3-imgu 0 parameters': Data > Pad 0x01000004 : 2: Source > Link 0x02000021: to remote pad 0x100001e of entity 'ipu3-imgu 0 output': Data, Enabled > Pad 0x01000005 : 3: Source > Link 0x02000027: to remote pad 0x1000024 of entity 'ipu3-imgu 0 viewfinder': Data, Enabled > Pad 0x01000006 : 4: Source > Link 0x0200002d: to remote pad 0x100002a of entity 'ipu3-imgu 0 3a stat': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > > Allow for multiple opens: > test second /dev/v4l-subdev0 open: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Sub-Device ioctls (Sink Pad 0): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Sink Pad 1): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Source Pad 2): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Source Pad 3): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Source Pad 4): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > test VIDIOC_QUERYCTRL: OK > test VIDIOC_G/S_CTRL: OK > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 1 Private Controls: 1 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported) > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK (Not Supported) > test VIDIOC_TRY_FMT: OK (Not Supported) > test VIDIOC_S_FMT: OK (Not Supported) > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported) > test VIDIOC_EXPBUF: OK (Not Supported) > > -------------------------------------------------------------------------------- > Compliance test for device /dev/v4l-subdev1: > > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x0300000f > Type : V4L Sub-Device > Entity Info: > ID : 0x00000007 (7) > Name : ipu3-imgu 1 > Function : Video Statistics > Pad 0x01000008 : 0: Sink > Link 0x02000033: from remote pad 0x1000030 of entity 'ipu3-imgu 1 input': Data, Enabled > Pad 0x01000009 : 1: Sink > Link 0x02000039: from remote pad 0x1000036 of entity 'ipu3-imgu 1 parameters': Data > Pad 0x0100000a : 2: Source > Link 0x0200003f: to remote pad 0x100003c of entity 'ipu3-imgu 1 output': Data, Enabled > Pad 0x0100000b : 3: Source > Link 0x02000045: to remote pad 0x1000042 of entity 'ipu3-imgu 1 viewfinder': Data, Enabled > Pad 0x0100000c : 4: Source > Link 0x0200004b: to remote pad 0x1000048 of entity 'ipu3-imgu 1 3a stat': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > > Allow for multiple opens: > test second /dev/v4l-subdev1 open: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Sub-Device ioctls (Sink Pad 0): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Sink Pad 1): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Source Pad 2): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Source Pad 3): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Sub-Device ioctls (Source Pad 4): > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Try VIDIOC_SUBDEV_G/S_FMT: OK > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) > test Active VIDIOC_SUBDEV_G/S_FMT: OK > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK > test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > test VIDIOC_QUERYCTRL: OK > test VIDIOC_G/S_CTRL: OK > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 1 Private Controls: 1 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported) > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK (Not Supported) > test VIDIOC_TRY_FMT: OK (Not Supported) > test VIDIOC_S_FMT: OK (Not Supported) > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported) > test VIDIOC_EXPBUF: OK (Not Supported) > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video0: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:input > Driver version : 4.19.0 > Capabilities : 0x84202000 > Video Output Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04202000 > Video Output Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000013 > Type : V4L Video > Entity Info: > ID : 0x00000011 (17) > Name : ipu3-imgu 0 input > Function : V4L2 I/O > Pad 0x01000012 : 0: Source > Link 0x02000015: to remote pad 0x1000002 of entity 'ipu3-imgu 0': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video0 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 1 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Output 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Output 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK > > Codec ioctls (Output 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Output 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video1: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:parameters > Driver version : 4.19.0 > Capabilities : 0x8c200000 > Metadata Output > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x0c200000 > Metadata Output > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000019 > Type : V4L Video > Entity Info: > ID : 0x00000017 (23) > Name : ipu3-imgu 0 parameters > Function : V4L2 I/O > Pad 0x01000018 : 0: Source > Link 0x0200001b: to remote pad 0x1000003 of entity 'ipu3-imgu 0': Data > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video1 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video2: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:output > Driver version : 4.19.0 > Capabilities : 0x84201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x0300001f > Type : V4L Video > Entity Info: > ID : 0x0000001d (29) > Name : ipu3-imgu 0 output > Function : V4L2 I/O > Pad 0x0100001e : 0: Sink > Link 0x02000021: from remote pad 0x1000004 of entity 'ipu3-imgu 0': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video2 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 1 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Input 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Input 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK > > Codec ioctls (Input 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Input 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video3: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:viewfinder > Driver version : 4.19.0 > Capabilities : 0x84201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000025 > Type : V4L Video > Entity Info: > ID : 0x00000023 (35) > Name : ipu3-imgu 0 viewfinder > Function : V4L2 I/O > Pad 0x01000024 : 0: Sink > Link 0x02000027: from remote pad 0x1000005 of entity 'ipu3-imgu 0': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video3 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 1 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Input 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Input 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK > > Codec ioctls (Input 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Input 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video4: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:3a stat > Driver version : 4.19.0 > Capabilities : 0x84a00000 > Metadata Capture > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04a00000 > Metadata Capture > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x0300002b > Type : V4L Video > Entity Info: > ID : 0x00000029 (41) > Name : ipu3-imgu 0 3a stat > Function : V4L2 I/O > Pad 0x0100002a : 0: Sink > Link 0x0200002d: from remote pad 0x1000006 of entity 'ipu3-imgu 0': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video4 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video5: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:input > Driver version : 4.19.0 > Capabilities : 0x84202000 > Video Output Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04202000 > Video Output Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000031 > Type : V4L Video > Entity Info: > ID : 0x0000002f (47) > Name : ipu3-imgu 1 input > Function : V4L2 I/O > Pad 0x01000030 : 0: Source > Link 0x02000033: to remote pad 0x1000008 of entity 'ipu3-imgu 1': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video5 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 1 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Output 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Output 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK > > Codec ioctls (Output 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Output 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video6: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:parameters > Driver version : 4.19.0 > Capabilities : 0x8c200000 > Metadata Output > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x0c200000 > Metadata Output > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000037 > Type : V4L Video > Entity Info: > ID : 0x00000035 (53) > Name : ipu3-imgu 1 parameters > Function : V4L2 I/O > Pad 0x01000036 : 0: Source > Link 0x02000039: to remote pad 0x1000009 of entity 'ipu3-imgu 1': Data > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video6 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video7: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:output > Driver version : 4.19.0 > Capabilities : 0x84201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x0300003d > Type : V4L Video > Entity Info: > ID : 0x0000003b (59) > Name : ipu3-imgu 1 output > Function : V4L2 I/O > Pad 0x0100003c : 0: Sink > Link 0x0200003f: from remote pad 0x100000a of entity 'ipu3-imgu 1': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video7 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 1 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Input 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Input 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK > > Codec ioctls (Input 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Input 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video8: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:viewfinder > Driver version : 4.19.0 > Capabilities : 0x84201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000043 > Type : V4L Video > Entity Info: > ID : 0x00000041 (65) > Name : ipu3-imgu 1 viewfinder > Function : V4L2 I/O > Pad 0x01000042 : 0: Sink > Link 0x02000045: from remote pad 0x100000b of entity 'ipu3-imgu 1': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video8 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 1 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Input 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Input 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK > > Codec ioctls (Input 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Input 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > -------------------------------------------------------------------------------- > Compliance test for device /dev/video9: > > Driver Info: > Driver name : ipu3-imgu > Card type : ipu3-imgu > Bus info : PCI:3a stat > Driver version : 4.19.0 > Capabilities : 0x84a00000 > Metadata Capture > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04a00000 > Metadata Capture > Streaming > Extended Pix Format > Media Driver Info: > Driver name : ipu3-imgu > Model : ipu3-imgu > Serial : > Bus info : PCI:0000:00:05.0 > Media version : 4.19.0 > Hardware revision: 0x80862015 (2156273685) > Driver version : 4.19.0 > Interface Info: > ID : 0x03000049 > Type : V4L Video > Entity Info: > ID : 0x00000047 (71) > Name : ipu3-imgu 1 3a stat > Function : V4L2 I/O > Pad 0x01000048 : 0: Sink > Link 0x0200004b: from remote pad 0x100000c of entity 'ipu3-imgu 1': Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > > Allow for multiple opens: > test second /dev/video9 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > > Total: 597, Succeeded: 597, Failed: 0, Warnings: 0 > > -------------------------------------------------------------------------------- > > Thanks, > > Yong > > Cao,Bing Bu (1): > intel-ipu3: add dual pipe support > > Rajmohan Mani (1): > doc-rst: Add Intel IPU3 documentation > > Tomasz Figa (2): > intel-ipu3: mmu: Implement driver > intel-ipu3: Implement DMA mapping functions > > Yong Zhi (12): > v4l: Add Intel IPU3 meta buffer formats > v4l: Add Intel IPU3 meta data uAPI > intel-ipu3: abi: Add register definitions and enum > intel-ipu3: abi: Add structs > intel-ipu3: css: Add dma buff pool utility functions > intel-ipu3: css: Add support for firmware management > intel-ipu3: css: Add static settings for image pipeline > intel-ipu3: css: Compute and program ccs > intel-ipu3: css: Initialize css hardware > intel-ipu3: Add css pipeline programming > intel-ipu3: Add v4l2 driver based on media framework > intel-ipu3: Add imgu top level pci device driver > > Documentation/media/uapi/v4l/meta-formats.rst | 1 + > .../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 181 + > Documentation/media/v4l-drivers/index.rst | 1 + > Documentation/media/v4l-drivers/ipu3.rst | 326 + > drivers/media/pci/intel/ipu3/Kconfig | 16 + > drivers/media/pci/intel/ipu3/Makefile | 12 + > drivers/media/pci/intel/ipu3/ipu3-abi.h | 2011 ++++ > drivers/media/pci/intel/ipu3/ipu3-css-fw.c | 265 + > drivers/media/pci/intel/ipu3/ipu3-css-fw.h | 188 + > drivers/media/pci/intel/ipu3/ipu3-css-params.c | 2936 ++++++ > drivers/media/pci/intel/ipu3/ipu3-css-params.h | 28 + > drivers/media/pci/intel/ipu3/ipu3-css-pool.c | 136 + > drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 56 + > drivers/media/pci/intel/ipu3/ipu3-css.c | 2407 +++++ > drivers/media/pci/intel/ipu3/ipu3-css.h | 215 + > drivers/media/pci/intel/ipu3/ipu3-dmamap.c | 270 + > drivers/media/pci/intel/ipu3/ipu3-dmamap.h | 22 + > drivers/media/pci/intel/ipu3/ipu3-mmu.c | 560 ++ > drivers/media/pci/intel/ipu3/ipu3-mmu.h | 35 + > drivers/media/pci/intel/ipu3/ipu3-tables.c | 9609 ++++++++++++++++++++ > drivers/media/pci/intel/ipu3/ipu3-tables.h | 66 + > drivers/media/pci/intel/ipu3/ipu3-v4l2.c | 1436 +++ > drivers/media/pci/intel/ipu3/ipu3.c | 830 ++ > drivers/media/pci/intel/ipu3/ipu3.h | 169 + > drivers/media/v4l2-core/v4l2-ioctl.c | 2 + > include/uapi/linux/intel-ipu3.h | 2827 ++++++ > include/uapi/linux/videodev2.h | 4 + > 27 files changed, 24609 insertions(+) > create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst > create mode 100644 Documentation/media/v4l-drivers/ipu3.rst > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-abi.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-tables.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-tables.h > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-v4l2.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3.c > create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h > create mode 100644 include/uapi/linux/intel-ipu3.h > > -- > 2.7.4 >
On 11/14/2018 05:58 AM, Sakari Ailus wrote: > On Tue, Nov 13, 2018 at 07:04:01PM +0800, Bing Bu Cao wrote: >> >> On 11/13/2018 06:31 PM, Sakari Ailus wrote: >>> Hi Bing Bu, >>> >>> On Mon, Nov 12, 2018 at 12:31:16PM +0800, Bing Bu Cao wrote: >>>> On 11/09/2018 06:09 PM, Sakari Ailus wrote: >>>>> Hi Bing Bu, >>>>> >>>>> On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: >>>>>> On 11/01/2018 08:03 PM, Sakari Ailus wrote: >>>>>>> Hi Yong, >>>>>>> >>>>>>> Thanks for the update! >>>>>>> >>>>>>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> This series adds support for the Intel IPU3 (Image Processing Unit) >>>>>>>> ImgU which is essentially a modern memory-to-memory ISP. It implements >>>>>>>> raw Bayer to YUV image format conversion as well as a large number of >>>>>>>> other pixel processing algorithms for improving the image quality. >>>>>>>> >>>>>>>> Meta data formats are defined for image statistics (3A, i.e. automatic >>>>>>>> white balance, exposure and focus, histogram and local area contrast >>>>>>>> enhancement) as well as for the pixel processing algorithm parameters. >>>>>>>> The documentation for these formats is currently not included in the >>>>>>>> patchset but will be added in a future version of this set. >>>>>>>> >>>>>>>> The algorithm parameters need to be considered specific to a given frame >>>>>>>> and typically a large number of these parameters change on frame to frame >>>>>>>> basis. Additionally, the parameters are highly structured (and not a flat >>>>>>>> space of independent configuration primitives). They also reflect the >>>>>>>> data structures used by the firmware and the hardware. On top of that, >>>>>>>> the algorithms require highly specialized user space to make meaningful >>>>>>>> use of them. For these reasons it has been chosen video buffers to pass >>>>>>>> the parameters to the device. >>>>>>>> >>>>>>>> On individual patches: >>>>>>>> >>>>>>>> The heart of ImgU is the CSS, or Camera Subsystem, which contains the >>>>>>>> image processors and HW accelerators. >>>>>>>> >>>>>>>> The 3A statistics and other firmware parameter computation related >>>>>>>> functions are implemented in patch 11. >>>>>>>> >>>>>>>> All IPU3 pipeline default settings can be found in patch 10. >>>>>>>> >>>>>>>> To access DDR via ImgU's own memory space, IPU3 is also equipped with >>>>>>>> its own MMU unit, the driver is implemented in patch 6. >>>>>>>> >>>>>>>> Patch 7 uses above driver for DMA mapping operation. >>>>>>>> >>>>>>>> The communication between IPU3 firmware and driver is implemented with circular >>>>>>>> queues in patch 8. >>>>>>>> >>>>>>>> Patch 9 provide some utility functions and manage IPU3 fw download and >>>>>>>> install. >>>>>>>> >>>>>>>> The firmware which is called ipu3-fw.bin can be downloaded from: >>>>>>>> >>>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >>>>>>>> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) >>>>>>>> >>>>>>>> Firmware ABI is defined in patches 4 and 5. >>>>>>>> >>>>>>>> Patches 12 and 13 are of the same file, the former contains all h/w programming >>>>>>>> related code, the latter implements interface functions for access fw & hw >>>>>>>> capabilities. >>>>>>>> >>>>>>>> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: >>>>>>>> >>>>>>>> <URL:https://patchwork.kernel.org/patch/9976295/> >>>>>>> I've pushed the latest set here: >>>>>>> >>>>>>> <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> >>>>>>> >>>>>>> You can just say the entire set depends on those going forward; the >>>>>>> documentation is needed, too. >>>>>>> >>>>>>>> Patch 15 represents the top level that glues all of the other components together, >>>>>>>> passing arguments between the components. >>>>>>>> >>>>>>>> Patch 16 is a recent effort to extend v6 for advanced camera features like >>>>>>>> Continuous View Finder (CVF) and Snapshot During Video(SDV) support. >>>>>>>> >>>>>>>> Link to user space implementation: >>>>>>>> >>>>>>>> git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera >>>>>>>> >>>>>>>> ImgU media topology print: >>>>>>>> >>>>>>>> # media-ctl -d /dev/media0 -p >>>>>>>> Media controller API version 4.19.0 >>>>>>>> >>>>>>>> Media device information >>>>>>>> ------------------------ >>>>>>>> driver ipu3-imgu >>>>>>>> model ipu3-imgu >>>>>>>> serial >>>>>>>> bus info PCI:0000:00:05.0 >>>>>>>> hw revision 0x80862015 >>>>>>>> driver version 4.19.0 >>>>>>>> >>>>>>>> Device topology >>>>>>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) >>>>>>>> type V4L2 subdev subtype Unknown flags 0 >>>>>>>> device node name /dev/v4l-subdev0 >>>>>>>> pad0: Sink >>>>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown >>>>>>> This doesn't seem right. Which formats can be enumerated from the pad? >>>>> Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) >>>>> format whereas the CAPTURE video nodes always have NV12. Can you confirm? >>>> Hi, Sakari, >>>> Yes, I think the pad_fmt should also be changed. >>>> Yong, could you add some extra code for this and test? like: >>>> >>>> static int ipu3_v4l2_node_setup(struct imgu_device *imgu, unsigned int pipe, >>>> ... >>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â V4L2_PIX_FMT_NV12; >>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->vdev_fmt.fmt.pix_mp = def_pix_fmt; >>>> Â Â Â Â Â Â Â } >>>> >>>> +Â Â Â Â Â Â if (node->vdev_fmt.type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) >>>> +Â Â Â Â Â Â Â Â Â Â Â Â Â Â node->pad_fmt.code = MEDIA_BUS_FMT_SGRBG10_1X10; >>>> + >>>> Â >>>>> If the OUTPUT video node format selection has no effect on the rest of the >>>>> pipeline (device capabilities, which processing blocks are in use, CAPTURE >>>>> video nodes formats etc.), I think you could simply use the FIXED media bus >>>>> code for each pad. That would actually make sense: this device always works >>>>> from memory to memory, and thus does not really have a pixel data bus >>>>> external to the device which is what the media bus codes really are for. >>>>> >>>>>>>> crop:(0,0)/1920x1080 >>>>>>>> compose:(0,0)/1920x1080] >>>>>>> Does the compose rectangle affect the scaling on all outputs? >>>>>> Sakari, driver use crop and compose targets to help set input-feeder and BDS >>>>>> output resolutions which are 2 key block of whole imaging pipeline, not the >>>>>> actual ending output, but they will impact the final output. >>>>> Ack. Thanks for the clarification. >>>>> >>>>>>>> <- "ipu3-imgu 0 input":0 [] >>>>>>> Are there links that have no useful link configuration? If so, you should >>>>>>> set them enabled and immutable in the driver. >>>>>> The enabled status of input pads is used to get which pipe that user is >>>>>> trying to enable (ipu3_link_setup()), so it could not been set as immutable. >>>>> But the rest of them could be, right? >>>> Yes. >>>>>>>> pad1: Sink >>>>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>>>> I'd suggest to use MEDIA_BUS_FMT_FIXED here. >>>>>>> >>>>>>>> <- "ipu3-imgu 0 parameters":0 [] >>>>>>>> pad2: Source >>>>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>>>>> -> "ipu3-imgu 0 output":0 [] >>>>>>>> pad3: Source >>>>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>>>>> -> "ipu3-imgu 0 viewfinder":0 [] >>>>>>> Are there other differences between output and viewfinder? >>>>>> output and viewfinder are the main and secondary output of output system. >>>>>> 'main' output is not allowed to be scaled, only support crop. secondary >>>>>> output 'viewfinder' >>>>>> can support both cropping and scaling. User can select different nodes >>>>>> to use >>>>>> as preview and capture flexibly based on the actual use cases. >>>>> If there's scaling to be configured, I'd expect to see the COMPOSE target >>>>> supported. >>>> Actually the viewfinder is the result of scaling, that means you can not >>>> do more scaling. >>> How do you configure the scaling of the viewfinder currently? >> We consider that the viewfinder as a secondary output, and set the format by >> subdev set_fmt() directly and all pads formats will be used to find >> binary and >> build pipeline. > Ok. > > Could you instead use the compose target to configure the scaling? Setting > the format on the source pad would have no effect. Hi, Sakari, For the secondary output (viewfinder), it support both cropping and scaling, in order to keep the aspect ratio, system will do [crop --> compose], sounds like it should have 2 selection targets, but firmware/driver did not provide clear interfaces to allow user to configure the cropping, driver calculate the scaler and cropping parameters based on the output-system input and output resolution to keep aspect ratio and field of view. I think it is more succinct to set the actual output by set format instead of using selection targets. >
Hi Jacopo, On Wed, Nov 14, 2018 at 01:25:11AM +0100, jacopo mondi wrote: > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > Hi, > > > > This series adds support for the Intel IPU3 (Image Processing Unit) > > ImgU which is essentially a modern memory-to-memory ISP. It implements > > raw Bayer to YUV image format conversion as well as a large number of > > other pixel processing algorithms for improving the image quality. > > > > Meta data formats are defined for image statistics (3A, i.e. automatic > > white balance, exposure and focus, histogram and local area contrast > > enhancement) as well as for the pixel processing algorithm parameters. > > The documentation for these formats is currently not included in the > > patchset but will be added in a future version of this set. > > > > The algorithm parameters need to be considered specific to a given frame > > and typically a large number of these parameters change on frame to frame > > basis. Additionally, the parameters are highly structured (and not a flat > > space of independent configuration primitives). They also reflect the > > data structures used by the firmware and the hardware. On top of that, > > the algorithms require highly specialized user space to make meaningful > > use of them. For these reasons it has been chosen video buffers to pass > > the parameters to the device. > > > > On individual patches: > > > > The heart of ImgU is the CSS, or Camera Subsystem, which contains the > > image processors and HW accelerators. > > > > The 3A statistics and other firmware parameter computation related > > functions are implemented in patch 11. > > > > All IPU3 pipeline default settings can be found in patch 10. > > > > Seems to me that patch 10 didn't make it to the mailing list, am I > wrong? > > I'm pointing it out as the same happened on your v6. Thanks for pointing this out. I've uploaded the entire set here: <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=ipu3-v7> including the 10th patch: <URL:https://git.linuxtv.org/sailus/media_tree.git/commit/?h=ipu3-v7&id=41e2f0d114dbc195efed079202d22748ddedbe83> It's too big to get through the list server. :-( Luckily, it's mostly tables so there's not much to review there. Default settings, effectively.
Hi Sakari, On Wed, Nov 14, 2018 at 09:40:50AM +0200, Sakari Ailus wrote: > Hi Jacopo, > > On Wed, Nov 14, 2018 at 01:25:11AM +0100, jacopo mondi wrote: > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > > Hi, > > > > > > This series adds support for the Intel IPU3 (Image Processing Unit) > > > ImgU which is essentially a modern memory-to-memory ISP. It implements > > > raw Bayer to YUV image format conversion as well as a large number of > > > other pixel processing algorithms for improving the image quality. > > > > > > Meta data formats are defined for image statistics (3A, i.e. automatic > > > white balance, exposure and focus, histogram and local area contrast > > > enhancement) as well as for the pixel processing algorithm parameters. > > > The documentation for these formats is currently not included in the > > > patchset but will be added in a future version of this set. > > > > > > The algorithm parameters need to be considered specific to a given frame > > > and typically a large number of these parameters change on frame to frame > > > basis. Additionally, the parameters are highly structured (and not a flat > > > space of independent configuration primitives). They also reflect the > > > data structures used by the firmware and the hardware. On top of that, > > > the algorithms require highly specialized user space to make meaningful > > > use of them. For these reasons it has been chosen video buffers to pass > > > the parameters to the device. > > > > > > On individual patches: > > > > > > The heart of ImgU is the CSS, or Camera Subsystem, which contains the > > > image processors and HW accelerators. > > > > > > The 3A statistics and other firmware parameter computation related > > > functions are implemented in patch 11. > > > > > > All IPU3 pipeline default settings can be found in patch 10. > > > > > > > Seems to me that patch 10 didn't make it to the mailing list, am I > > wrong? > > > > I'm pointing it out as the same happened on your v6. > > Thanks for pointing this out. I've uploaded the entire set here: > > <URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=ipu3-v7> > > including the 10th patch: > > <URL:https://git.linuxtv.org/sailus/media_tree.git/commit/?h=ipu3-v7&id=41e2f0d114dbc195efed079202d22748ddedbe83> > > It's too big to get through the list server. :-( > > Luckily, it's mostly tables so there's not much to review there. Default > settings, effectively. Thanks, I've now received all patches. Thanks j > > -- > Regards, > > Sakari Ailus > sakari.ailus@linux.intel.com
Hello Yong, On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > Hi, > > This series adds support for the Intel IPU3 (Image Processing Unit) > ImgU which is essentially a modern memory-to-memory ISP. It implements > raw Bayer to YUV image format conversion as well as a large number of > other pixel processing algorithms for improving the image quality. > > Meta data formats are defined for image statistics (3A, i.e. automatic > white balance, exposure and focus, histogram and local area contrast > enhancement) as well as for the pixel processing algorithm parameters. > The documentation for these formats is currently not included in the > patchset but will be added in a future version of this set. > > The algorithm parameters need to be considered specific to a given frame > and typically a large number of these parameters change on frame to frame > basis. Additionally, the parameters are highly structured (and not a flat > space of independent configuration primitives). They also reflect the > data structures used by the firmware and the hardware. On top of that, > the algorithms require highly specialized user space to make meaningful > use of them. For these reasons it has been chosen video buffers to pass > the parameters to the device. > > On individual patches: > > The heart of ImgU is the CSS, or Camera Subsystem, which contains the > image processors and HW accelerators. > > The 3A statistics and other firmware parameter computation related > functions are implemented in patch 11. > > All IPU3 pipeline default settings can be found in patch 10. > > To access DDR via ImgU's own memory space, IPU3 is also equipped with > its own MMU unit, the driver is implemented in patch 6. > > Patch 7 uses above driver for DMA mapping operation. > > The communication between IPU3 firmware and driver is implemented with > circular queues in patch 8. > > Patch 9 provide some utility functions and manage IPU3 fw download and > install. > > The firmware which is called ipu3-fw.bin can be downloaded from: > > git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > > Firmware ABI is defined in patches 4 and 5. > > Patches 12 and 13 are of the same file, the former contains all h/w > programming related code, the latter implements interface functions for > access fw & hw capabilities. > > Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > > <URL:https://patchwork.kernel.org/patch/9976295/> > > Patch 15 represents the top level that glues all of the other components > together, passing arguments between the components. > > Patch 16 is a recent effort to extend v6 for advanced camera features like > Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > > Link to user space implementation: > > git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > > ImgU media topology print: [snip] > v4l2-compliance utility is built with Sakari's patches for meta data > output support(rebased): > > <URL:https://patchwork.linuxtv.org/patch/43370/> > <URL:https://patchwork.linuxtv.org/patch/43369/> > > The test (v4l2-compliance -m 0) passes without error, outputs are appended > at the end of revision history. > > Note: > > 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be enabled > prior to the test. > 2. Stream tests are not performed since it requires pre-configuration for > each case. And that's a bit of an issue. I've tested the driver with a small script based on media-ctl to configure links and yavta to interface with the video nodes, and got the following oops: [ 136.927788] divide error: 0000 [#1] PREEMPT SMP PTI [ 136.927801] CPU: 2 PID: 2069 Comm: yavta Not tainted 4.20.0-rc1+ #9 [ 136.927806] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ 136.927820] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] [ 136.927825] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 <f7> fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 [ 136.927830] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 [ 136.927835] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: ffff9af2c3e353c0 [ 136.927839] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: ffff9af2c3e353c0 [ 136.927843] RBP: 0000000000000001 R08: 0000000000000000 R09: ffff9af2c0b83880 [ 136.927846] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: 00000000000003a0 [ 136.927849] R13: 0000000000025a0a R14: 0000000000000000 R15: 0000000000000000 [ 136.927854] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) knlGS: 0000000000000000 [ 136.927858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 136.927862] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: 00000000003606e0 [ 136.927865] Call Trace: [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a [ 136.927892] ? __switch_to_asm+0x40/0x70 [ 136.927899] ? alloc_vmap_area+0x78/0x2f6 [ 136.927903] ? __switch_to_asm+0x40/0x70 [ 136.927907] ? __switch_to_asm+0x34/0x70 [ 136.927911] ? __switch_to_asm+0x40/0x70 [ 136.927915] ? __switch_to_asm+0x34/0x70 [ 136.927923] ? __inc_numa_state+0x28/0x70 [ 136.927929] ? preempt_latency_start+0x1e/0x3d [ 136.927936] ? get_page_from_freelist+0x821/0xb62 [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b [ 136.927948] ? kmem_cache_alloc_node_trace+0xf6/0x108 [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] [ 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] [ 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] [ 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] [ 136.928022] ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] [ 136.928034] vb2_start_streaming+0x6c/0xf2 [videobuf2_common] [ 136.928044] vb2_core_streamon+0xcf/0x109 [videobuf2_common] [ 136.928061] __video_do_ioctl+0x239/0x388 [videodev] [ 136.928081] video_usercopy+0x25d/0x47a [videodev] [ 136.928097] ? copy_overflow+0x14/0x14 [videodev] [ 136.928115] v4l2_ioctl+0x4d/0x58 [videodev] [ 136.928123] vfs_ioctl+0x1b/0x28 [ 136.928130] do_vfs_ioctl+0x4de/0x566 [ 136.928139] ksys_ioctl+0x50/0x70 [ 136.928146] __x64_sys_ioctl+0x16/0x19 [ 136.928152] do_syscall_64+0x4d/0x5a [ 136.928158] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 136.928164] RIP: 0033:0x7f1ec9a84f47 [ 136.928169] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 136.928173] RSP: 002b:00007ffe279e6188 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 136.928178] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f1ec9a84f47 [ 136.928181] RDX: 00007ffe279e6194 RSI: 0000000040045612 RDI: 0000000000000003 [ 136.928184] RBP: 0000000000000000 R08: 00007f1ec776d000 R09: 0000000000000000 [ 136.928188] R10: 0000000000000020 R11: 0000000000000246 R12: 00007ffe279e6360 [ 136.928191] R13: 0000000000000004 R14: 00007ffe279e6360 R15: 00007ffe279e8826 [ 136.928198] Modules linked in: ccm zram arc4 iwlmvm mac80211 intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp iwlwifi cfg80211 hid_multitouch ipu3_imgu ipu3_cio2 8250_dw videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 processor_thermal_device intel_soc_dts_iosf videobuf2_common ov5670 ov13858 dw9714 v4l2_fwnode v4l2_common videodev media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel chromeos_pstore mac_hid autofs4 usbhid mmc_block hid_generic i915 sdhci_pci video cqhci i2c_algo_bit sdhci drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm drm_panel_orientation_quirks i2c_hid hid [ 136.928273] ---[ end trace 4ec6c2ce09e06d9d ]--- [ 136.928288] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] [ 136.928293] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 <f7> fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 [ 136.928297] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 [ 136.928302] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: ffff9af2c3e353c0 [ 136.928307] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: ffff9af2c3e353c0 [ 136.928311] RBP: 0000000000000001 R08: 0000000000000000 R09: ffff9af2c0b83880 [ 136.928320] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: 00000000000003a0 [ 136.928324] R13: 0000000000025a0a R14: 0000000000000000 R15: 0000000000000000 [ 136.928330] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) knlGS: 0000000000000000 [ 136.928349] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 136.928364] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: 00000000003606e0 The script can be found at https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000040.html. I may be doing something wrong (and I probably am), but in any case, the driver shouldn't crash. Could you please have a look ? Please note that, as I couldn't get the IMGU to process frames, the part of the script that converts output files to PPM is untested and thus probably incorrect. [snip]
On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hello Yong, > > On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > > Hi, > > > > This series adds support for the Intel IPU3 (Image Processing Unit) > > ImgU which is essentially a modern memory-to-memory ISP. It implements > > raw Bayer to YUV image format conversion as well as a large number of > > other pixel processing algorithms for improving the image quality. > > > > Meta data formats are defined for image statistics (3A, i.e. automatic > > white balance, exposure and focus, histogram and local area contrast > > enhancement) as well as for the pixel processing algorithm parameters. > > The documentation for these formats is currently not included in the > > patchset but will be added in a future version of this set. > > > > The algorithm parameters need to be considered specific to a given frame > > and typically a large number of these parameters change on frame to frame > > basis. Additionally, the parameters are highly structured (and not a flat > > space of independent configuration primitives). They also reflect the > > data structures used by the firmware and the hardware. On top of that, > > the algorithms require highly specialized user space to make meaningful > > use of them. For these reasons it has been chosen video buffers to pass > > the parameters to the device. > > > > On individual patches: > > > > The heart of ImgU is the CSS, or Camera Subsystem, which contains the > > image processors and HW accelerators. > > > > The 3A statistics and other firmware parameter computation related > > functions are implemented in patch 11. > > > > All IPU3 pipeline default settings can be found in patch 10. > > > > To access DDR via ImgU's own memory space, IPU3 is also equipped with > > its own MMU unit, the driver is implemented in patch 6. > > > > Patch 7 uses above driver for DMA mapping operation. > > > > The communication between IPU3 firmware and driver is implemented with > > circular queues in patch 8. > > > > Patch 9 provide some utility functions and manage IPU3 fw download and > > install. > > > > The firmware which is called ipu3-fw.bin can be downloaded from: > > > > git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > > (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > > > > Firmware ABI is defined in patches 4 and 5. > > > > Patches 12 and 13 are of the same file, the former contains all h/w > > programming related code, the latter implements interface functions for > > access fw & hw capabilities. > > > > Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: > > > > <URL:https://patchwork.kernel.org/patch/9976295/> > > > > Patch 15 represents the top level that glues all of the other components > > together, passing arguments between the components. > > > > Patch 16 is a recent effort to extend v6 for advanced camera features like > > Continuous View Finder (CVF) and Snapshot During Video(SDV) support. > > > > Link to user space implementation: > > > > git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera > > > > ImgU media topology print: > > [snip] > > > v4l2-compliance utility is built with Sakari's patches for meta data > > output support(rebased): > > > > <URL:https://patchwork.linuxtv.org/patch/43370/> > > <URL:https://patchwork.linuxtv.org/patch/43369/> > > > > The test (v4l2-compliance -m 0) passes without error, outputs are appended > > at the end of revision history. > > > > Note: > > > > 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be enabled > > prior to the test. > > 2. Stream tests are not performed since it requires pre-configuration for > > each case. > > And that's a bit of an issue. I've tested the driver with a small script based > on media-ctl to configure links and yavta to interface with the video nodes, > and got the following oops: > > [ 136.927788] divide error: 0000 [#1] PREEMPT SMP PTI > [ 136.927801] CPU: 2 PID: 2069 Comm: yavta Not tainted 4.20.0-rc1+ #9 > [ 136.927806] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 136.927820] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] > [ 136.927825] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd > ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 <f7> fb > 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 > [ 136.927830] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 > [ 136.927835] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > ffff9af2c3e353c0 > [ 136.927839] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > ffff9af2c3e353c0 > [ 136.927843] RBP: 0000000000000001 R08: 0000000000000000 R09: > ffff9af2c0b83880 > [ 136.927846] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > 00000000000003a0 > [ 136.927849] R13: 0000000000025a0a R14: 0000000000000000 R15: > 0000000000000000 > [ 136.927854] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) knlGS: > 0000000000000000 > [ 136.927858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 136.927862] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > 00000000003606e0 > [ 136.927865] Call Trace: > [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a > [ 136.927892] ? __switch_to_asm+0x40/0x70 > [ 136.927899] ? alloc_vmap_area+0x78/0x2f6 > [ 136.927903] ? __switch_to_asm+0x40/0x70 > [ 136.927907] ? __switch_to_asm+0x34/0x70 > [ 136.927911] ? __switch_to_asm+0x40/0x70 > [ 136.927915] ? __switch_to_asm+0x34/0x70 > [ 136.927923] ? __inc_numa_state+0x28/0x70 > [ 136.927929] ? preempt_latency_start+0x1e/0x3d > [ 136.927936] ? get_page_from_freelist+0x821/0xb62 > [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b > [ 136.927948] ? kmem_cache_alloc_node_trace+0xf6/0x108 > [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 Is it just me or the backtrace above doesn't seem to make sense? I don't see any allocations inside ipu3_css_cfg_acc(). > [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] > [ 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] > [ 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] > [ 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] > [ 136.928022] ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] > [ 136.928034] vb2_start_streaming+0x6c/0xf2 [videobuf2_common] > [ 136.928044] vb2_core_streamon+0xcf/0x109 [videobuf2_common] > [ 136.928061] __video_do_ioctl+0x239/0x388 [videodev] > [ 136.928081] video_usercopy+0x25d/0x47a [videodev] > [ 136.928097] ? copy_overflow+0x14/0x14 [videodev] > [ 136.928115] v4l2_ioctl+0x4d/0x58 [videodev] > [ 136.928123] vfs_ioctl+0x1b/0x28 > [ 136.928130] do_vfs_ioctl+0x4de/0x566 > [ 136.928139] ksys_ioctl+0x50/0x70 > [ 136.928146] __x64_sys_ioctl+0x16/0x19 > [ 136.928152] do_syscall_64+0x4d/0x5a > [ 136.928158] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 136.928164] RIP: 0033:0x7f1ec9a84f47 > [ 136.928169] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 > c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d > 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 136.928173] RSP: 002b:00007ffe279e6188 EFLAGS: 00000246 ORIG_RAX: > 0000000000000010 > [ 136.928178] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > 00007f1ec9a84f47 > [ 136.928181] RDX: 00007ffe279e6194 RSI: 0000000040045612 RDI: > 0000000000000003 > [ 136.928184] RBP: 0000000000000000 R08: 00007f1ec776d000 R09: > 0000000000000000 > [ 136.928188] R10: 0000000000000020 R11: 0000000000000246 R12: > 00007ffe279e6360 > [ 136.928191] R13: 0000000000000004 R14: 00007ffe279e6360 R15: > 00007ffe279e8826 > [ 136.928198] Modules linked in: ccm zram arc4 iwlmvm mac80211 intel_rapl > x86_pkg_temp_thermal intel_powerclamp coretemp iwlwifi cfg80211 hid_multitouch > ipu3_imgu ipu3_cio2 8250_dw videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 > processor_thermal_device intel_soc_dts_iosf videobuf2_common ov5670 ov13858 > dw9714 v4l2_fwnode v4l2_common videodev media at24 cros_ec_lpcs cros_ec_core > int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel > chromeos_pstore mac_hid autofs4 usbhid mmc_block hid_generic i915 sdhci_pci > video cqhci i2c_algo_bit sdhci drm_kms_helper syscopyarea sysfillrect > sysimgblt fb_sys_fops drm drm_panel_orientation_quirks i2c_hid hid > [ 136.928273] ---[ end trace 4ec6c2ce09e06d9d ]--- > [ 136.928288] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] > [ 136.928293] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd > ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 <f7> fb > 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 > [ 136.928297] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 > [ 136.928302] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > ffff9af2c3e353c0 > [ 136.928307] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > ffff9af2c3e353c0 > [ 136.928311] RBP: 0000000000000001 R08: 0000000000000000 R09: > ffff9af2c0b83880 > [ 136.928320] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > 00000000000003a0 > [ 136.928324] R13: 0000000000025a0a R14: 0000000000000000 R15: > 0000000000000000 > [ 136.928330] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) knlGS: > 0000000000000000 > [ 136.928349] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 136.928364] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > 00000000003606e0 > > The script can be found at https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000040.html. > > I may be doing something wrong (and I probably am), but in any case, the > driver shouldn't crash. Could you please have a look ? It looks like the driver doesn't have the default state initialized correctly somewhere and it ends up using 0 as the divisor in some calculation? Something to fix indeed. Best regards, Tomasz
Hi Tomasz, On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > > On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: [snip] > >> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be > >> enabled prior to the test. > >> 2. Stream tests are not performed since it requires pre-configuration > >> for each case. > > > > And that's a bit of an issue. I've tested the driver with a small script > > based on media-ctl to configure links and yavta to interface with the > > video nodes, and got the following oops: > > > > [ 136.927788] divide error: 0000 [#1] PREEMPT SMP PTI > > [ 136.927801] CPU: 2 PID: 2069 Comm: yavta Not tainted 4.20.0-rc1+ #9 > > [ 136.927806] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 136.927820] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] > > [ 136.927825] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 > > fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 > > <f7> fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 > > [ 136.927830] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 > > [ 136.927835] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > > ffff9af2c3e353c0 > > [ 136.927839] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > > ffff9af2c3e353c0 > > [ 136.927843] RBP: 0000000000000001 R08: 0000000000000000 R09: > > ffff9af2c0b83880 > > [ 136.927846] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > > 00000000000003a0 > > [ 136.927849] R13: 0000000000025a0a R14: 0000000000000000 R15: > > 0000000000000000 > > [ 136.927854] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) > > knlGS: > > 0000000000000000 > > [ 136.927858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > [ 136.927862] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > > 00000000003606e0 > > [ 136.927865] Call Trace: > > [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a > > [ 136.927892] ? __switch_to_asm+0x40/0x70 > > [ 136.927899] ? alloc_vmap_area+0x78/0x2f6 > > [ 136.927903] ? __switch_to_asm+0x40/0x70 > > [ 136.927907] ? __switch_to_asm+0x34/0x70 > > [ 136.927911] ? __switch_to_asm+0x40/0x70 > > [ 136.927915] ? __switch_to_asm+0x34/0x70 > > [ 136.927923] ? __inc_numa_state+0x28/0x70 > > [ 136.927929] ? preempt_latency_start+0x1e/0x3d > > [ 136.927936] ? get_page_from_freelist+0x821/0xb62 > > [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b > > [ 136.927948] ? kmem_cache_alloc_node_trace+0xf6/0x108 > > [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 > > Is it just me or the backtrace above doesn't seem to make sense? I > don't see any allocations inside ipu3_css_cfg_acc(). I suppose that's why it's prefixed with '?' :-) > > [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] > > [ 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] > > [ 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] > > [ 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] > > [ 136.928022] ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] > > [ 136.928034] vb2_start_streaming+0x6c/0xf2 [videobuf2_common] > > [ 136.928044] vb2_core_streamon+0xcf/0x109 [videobuf2_common] > > [ 136.928061] __video_do_ioctl+0x239/0x388 [videodev] > > [ 136.928081] video_usercopy+0x25d/0x47a [videodev] > > [ 136.928097] ? copy_overflow+0x14/0x14 [videodev] > > [ 136.928115] v4l2_ioctl+0x4d/0x58 [videodev] > > [ 136.928123] vfs_ioctl+0x1b/0x28 > > [ 136.928130] do_vfs_ioctl+0x4de/0x566 > > [ 136.928139] ksys_ioctl+0x50/0x70 > > [ 136.928146] __x64_sys_ioctl+0x16/0x19 > > [ 136.928152] do_syscall_64+0x4d/0x5a > > [ 136.928158] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 136.928164] RIP: 0033:0x7f1ec9a84f47 > > [ 136.928169] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 136.928173] RSP: 002b:00007ffe279e6188 EFLAGS: 00000246 ORIG_RAX: > > 0000000000000010 > > [ 136.928178] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > 00007f1ec9a84f47 > > [ 136.928181] RDX: 00007ffe279e6194 RSI: 0000000040045612 RDI: > > 0000000000000003 > > [ 136.928184] RBP: 0000000000000000 R08: 00007f1ec776d000 R09: > > 0000000000000000 > > [ 136.928188] R10: 0000000000000020 R11: 0000000000000246 R12: > > 00007ffe279e6360 > > [ 136.928191] R13: 0000000000000004 R14: 00007ffe279e6360 R15: > > 00007ffe279e8826 > > [ 136.928198] Modules linked in: ccm zram arc4 iwlmvm mac80211 intel_rapl > > x86_pkg_temp_thermal intel_powerclamp coretemp iwlwifi cfg80211 > > hid_multitouch ipu3_imgu ipu3_cio2 8250_dw videobuf2_dma_sg > > videobuf2_memops videobuf2_v4l2 processor_thermal_device > > intel_soc_dts_iosf videobuf2_common ov5670 ov13858 dw9714 v4l2_fwnode > > v4l2_common videodev media at24 cros_ec_lpcs cros_ec_core int3403_thermal > > int340x_thermal_zone int3400_thermal acpi_thermal_rel chromeos_pstore > > mac_hid autofs4 usbhid mmc_block hid_generic i915 sdhci_pci video cqhci > > i2c_algo_bit sdhci drm_kms_helper syscopyarea sysfillrect sysimgblt > > fb_sys_fops drm drm_panel_orientation_quirks i2c_hid hid [ 136.928273] > > ---[ end trace 4ec6c2ce09e06d9d ]--- > > [ 136.928288] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] > > [ 136.928293] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 > > fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 > > <f7> fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 > > [ 136.928297] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 > > [ 136.928302] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > > ffff9af2c3e353c0 > > [ 136.928307] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > > ffff9af2c3e353c0 > > [ 136.928311] RBP: 0000000000000001 R08: 0000000000000000 R09: > > ffff9af2c0b83880 > > [ 136.928320] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > > 00000000000003a0 > > [ 136.928324] R13: 0000000000025a0a R14: 0000000000000000 R15: > > 0000000000000000 > > [ 136.928330] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) > > knlGS: > > 0000000000000000 > > [ 136.928349] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > [ 136.928364] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > > 00000000003606e0 > > > > The script can be found at > > https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/00004 > > 0.html. > > > > I may be doing something wrong (and I probably am), but in any case, the > > driver shouldn't crash. Could you please have a look ? > > It looks like the driver doesn't have the default state initialized > correctly somewhere and it ends up using 0 as the divisor in some > calculation? Something to fix indeed. That's probably the case. I'll trust Intel to fix that in v8 :-)
Hi Laurent, Tomasz, Thanks for the reviews. > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hi Tomasz, > > On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > > On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > > > On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > > [snip] > > > >> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to > > >> be enabled prior to the test. > > >> 2. Stream tests are not performed since it requires > > >> pre-configuration for each case. > > > > > > And that's a bit of an issue. I've tested the driver with a small > > > script based on media-ctl to configure links and yavta to interface > > > with the video nodes, and got the following oops: > > > > > > [ 136.927788] divide error: 0000 [#1] PREEMPT SMP PTI [ > > > 136.927801] CPU: 2 PID: 2069 Comm: yavta Not tainted 4.20.0-rc1+ #9 > > > [ 136.927806] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ > > > 136.927820] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] [ > > > 136.927825] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 > > > 00 fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 > > > 00 99 <f7> fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 > > > 19 d2 [ 136.927830] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 [ > > > 136.927835] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > > > ffff9af2c3e353c0 > > > [ 136.927839] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > > > ffff9af2c3e353c0 > > > [ 136.927843] RBP: 0000000000000001 R08: 0000000000000000 R09: > > > ffff9af2c0b83880 > > > [ 136.927846] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > > > 00000000000003a0 > > > [ 136.927849] R13: 0000000000025a0a R14: 0000000000000000 R15: > > > 0000000000000000 > > > [ 136.927854] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) > > > knlGS: > > > 0000000000000000 > > > [ 136.927858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ > > > 136.927862] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > > > 00000000003606e0 > > > [ 136.927865] Call Trace: > > > [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a > > > [ 136.927892] ? __switch_to_asm+0x40/0x70 [ 136.927899] ? > > > alloc_vmap_area+0x78/0x2f6 [ 136.927903] ? > > > __switch_to_asm+0x40/0x70 [ 136.927907] ? > > > __switch_to_asm+0x34/0x70 [ 136.927911] ? > > > __switch_to_asm+0x40/0x70 [ 136.927915] ? > > > __switch_to_asm+0x34/0x70 [ 136.927923] ? > > > __inc_numa_state+0x28/0x70 [ 136.927929] ? > > > preempt_latency_start+0x1e/0x3d [ 136.927936] ? > > > get_page_from_freelist+0x821/0xb62 > > > [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b [ 136.927948] ? > > > kmem_cache_alloc_node_trace+0xf6/0x108 > > > [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 > > > > Is it just me or the backtrace above doesn't seem to make sense? I > > don't see any allocations inside ipu3_css_cfg_acc(). > > I suppose that's why it's prefixed with '?' :-) > > > > [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] [ > > > 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] [ > > > 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] [ > > > 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] [ 136.928022] > > > ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] [ 136.928034] > > > vb2_start_streaming+0x6c/0xf2 [videobuf2_common] [ 136.928044] > > > vb2_core_streamon+0xcf/0x109 [videobuf2_common] [ 136.928061] > > > __video_do_ioctl+0x239/0x388 [videodev] [ 136.928081] > > > video_usercopy+0x25d/0x47a [videodev] [ 136.928097] ? > > > copy_overflow+0x14/0x14 [videodev] [ 136.928115] > > > v4l2_ioctl+0x4d/0x58 [videodev] [ 136.928123] vfs_ioctl+0x1b/0x28 > > > [ 136.928130] do_vfs_ioctl+0x4de/0x566 [ 136.928139] > > > ksys_ioctl+0x50/0x70 [ 136.928146] __x64_sys_ioctl+0x16/0x19 [ > > > 136.928152] do_syscall_64+0x4d/0x5a [ 136.928158] > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > [ 136.928164] RIP: 0033:0x7f1ec9a84f47 [ 136.928169] Code: 00 00 > > > 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f > > > 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 > > > 48 [ 136.928173] RSP: 002b:00007ffe279e6188 EFLAGS: 00000246 > ORIG_RAX: > > > 0000000000000010 > > > [ 136.928178] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > > 00007f1ec9a84f47 > > > [ 136.928181] RDX: 00007ffe279e6194 RSI: 0000000040045612 RDI: > > > 0000000000000003 > > > [ 136.928184] RBP: 0000000000000000 R08: 00007f1ec776d000 R09: > > > 0000000000000000 > > > [ 136.928188] R10: 0000000000000020 R11: 0000000000000246 R12: > > > 00007ffe279e6360 > > > [ 136.928191] R13: 0000000000000004 R14: 00007ffe279e6360 R15: > > > 00007ffe279e8826 > > > [ 136.928198] Modules linked in: ccm zram arc4 iwlmvm mac80211 > > > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp iwlwifi > > > cfg80211 hid_multitouch ipu3_imgu ipu3_cio2 8250_dw > videobuf2_dma_sg > > > videobuf2_memops videobuf2_v4l2 processor_thermal_device > > > intel_soc_dts_iosf videobuf2_common ov5670 ov13858 dw9714 > > > v4l2_fwnode v4l2_common videodev media at24 cros_ec_lpcs > > > cros_ec_core int3403_thermal int340x_thermal_zone int3400_thermal > > > acpi_thermal_rel chromeos_pstore mac_hid autofs4 usbhid mmc_block > > > hid_generic i915 sdhci_pci video cqhci i2c_algo_bit sdhci > > > drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm > > > drm_panel_orientation_quirks i2c_hid hid [ 136.928273] ---[ end > > > trace 4ec6c2ce09e06d9d ]--- [ 136.928288] RIP: > > > 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] [ 136.928293] Code: > > > 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd ff ff 81 64 > > > 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 <f7> fb 0f > > > af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 [ > > > 136.928297] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 [ 136.928302] > RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > > > ffff9af2c3e353c0 > > > [ 136.928307] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > > > ffff9af2c3e353c0 > > > [ 136.928311] RBP: 0000000000000001 R08: 0000000000000000 R09: > > > ffff9af2c0b83880 > > > [ 136.928320] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > > > 00000000000003a0 > > > [ 136.928324] R13: 0000000000025a0a R14: 0000000000000000 R15: > > > 0000000000000000 > > > [ 136.928330] FS: 00007f1eca167700(0000) GS:ffff8c19fab00000(0000) > > > knlGS: > > > 0000000000000000 > > > [ 136.928349] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ > > > 136.928364] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > > > 00000000003606e0 > > > > > > The script can be found at > > > https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/ > > > 00004 > > > 0.html. > > > > > > I may be doing something wrong (and I probably am), but in any case, > > > the driver shouldn't crash. Could you please have a look ? > > > > It looks like the driver doesn't have the default state initialized > > correctly somewhere and it ends up using 0 as the divisor in some > > calculation? Something to fix indeed. > > That's probably the case. I'll trust Intel to fix that in v8 :-) > Ack. > -- > Regards, > > Laurent Pinchart > >
Hello Bing, On Wednesday, 7 November 2018 06:16:47 EET Bing Bu Cao wrote: > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: [snip] > >> ImgU media topology print: > >> > >> # media-ctl -d /dev/media0 -p > >> Media controller API version 4.19.0 > >> > >> Media device information > >> ------------------------ > >> driver ipu3-imgu > >> model ipu3-imgu > >> serial > >> bus info PCI:0000:00:05.0 > >> hw revision 0x80862015 > >> driver version 4.19.0 > >> > >> Device topology > >> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >> type V4L2 subdev subtype Unknown flags 0 > >> device node name /dev/v4l-subdev0 > >> pad0: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > > > This doesn't seem right. Which formats can be enumerated from the pad? > > > >> crop:(0,0)/1920x1080 > >> compose:(0,0)/1920x1080] > > > > Does the compose rectangle affect the scaling on all outputs? > > Sakari, driver use crop and compose targets to help set input-feeder and BDS > output resolutions which are 2 key block of whole imaging pipeline, not the > actual ending output, but they will impact the final output. > > >> <- "ipu3-imgu 0 input":0 [] > > > > Are there links that have no useful link configuration? If so, you should > > set them enabled and immutable in the driver. > > The enabled status of input pads is used to get which pipe that user is > trying to enable (ipu3_link_setup()), so it could not been set as immutable. Each pipe needs an input in order to operate, so from that point of view the input is mandatory. Why can't we make this link immutable, and use the stream state (VIDIOC_STREAMON/VIDIOC_STREAMOFF) to enable/disable the pipes ? > >> pad1: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > > > > I'd suggest to use MEDIA_BUS_FMT_FIXED here. > > > >> <- "ipu3-imgu 0 parameters":0 [] > >> > >> pad2: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 output":0 [] > >> > >> pad3: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 viewfinder":0 [] > > > > Are there other differences between output and viewfinder? > > output and viewfinder are the main and secondary output of output system. > 'main' output is not allowed to be scaled, only support crop. secondary > output 'viewfinder' can support both cropping and scaling. User can select > different nodes to use as preview and capture flexibly based on the actual > use cases. This is very useful information, thank you. Could it be added to the documentation (patch 02/16) with a block diagram ? > >> pad4: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 0 3a stat":0 [] > > > > FIXED here, too. > > > >> - entity 7: ipu3-imgu 1 (5 pads, 5 links) > >> type V4L2 subdev subtype Unknown flags 0 > >> device node name /dev/v4l-subdev1 > >> > >> pad0: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >> crop:(0,0)/1920x1080 > >> compose:(0,0)/1920x1080] > >> <- "ipu3-imgu 1 input":0 [] > >> > >> pad1: Sink > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> <- "ipu3-imgu 1 parameters":0 [] > >> > >> pad2: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 1 output":0 [] > >> > >> pad3: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 1 viewfinder":0 [] > >> > >> pad4: Source > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >> -> "ipu3-imgu 1 3a stat":0 [] > > > > This is a minor matter but --- could you create the second sub-device > > after the video device nodes related to the first one have been already > > created? That'd make reading the output easier. > > > >> - entity 17: ipu3-imgu 0 input (1 pad, 1 link) > >> > >> type Node subtype V4L flags 0 > >> device node name /dev/video0 > >> > >> pad0: Source > >> -> "ipu3-imgu 0":0 [] > >> > >> - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video1 > >> > >> pad0: Source > >> -> "ipu3-imgu 0":1 [] > >> > >> - entity 29: ipu3-imgu 0 output (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video2 > >> > >> pad0: Sink > >> <- "ipu3-imgu 0":2 [] > >> > >> - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video3 > >> > >> pad0: Sink > >> <- "ipu3-imgu 0":3 [] > >> > >> - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video4 > >> > >> pad0: Sink > >> <- "ipu3-imgu 0":4 [] > >> > >> - entity 47: ipu3-imgu 1 input (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video5 > >> > >> pad0: Source > >> -> "ipu3-imgu 1":0 [] > >> > >> - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video6 > >> > >> pad0: Source > >> -> "ipu3-imgu 1":1 [] > >> > >> - entity 59: ipu3-imgu 1 output (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video7 > >> > >> pad0: Sink > >> <- "ipu3-imgu 1":2 [] > >> > >> - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video8 > >> > >> pad0: Sink > >> <- "ipu3-imgu 1":3 [] > >> > >> - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video9 > >> > >> pad0: Sink > >> <- "ipu3-imgu 1":4 [] [snip]
Hi Sakari, On Friday, 9 November 2018 12:09:54 EET Sakari Ailus wrote: > On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > >> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: [snip] > >>> ImgU media topology print: > >>> > >>> # media-ctl -d /dev/media0 -p > >>> Media controller API version 4.19.0 > >>> > >>> Media device information > >>> ------------------------ > >>> driver ipu3-imgu > >>> model ipu3-imgu > >>> serial > >>> bus info PCI:0000:00:05.0 > >>> hw revision 0x80862015 > >>> driver version 4.19.0 > >>> > >>> Device topology > >>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >>> type V4L2 subdev subtype Unknown flags 0 > >>> device node name /dev/v4l-subdev0 > >>> pad0: Sink > >>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >> > >> This doesn't seem right. Which formats can be enumerated from the pad? > > Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) > format whereas the CAPTURE video nodes always have NV12. Can you confirm? > > If the OUTPUT video node format selection has no effect on the rest of the > pipeline (device capabilities, which processing blocks are in use, CAPTURE > video nodes formats etc.), I think you could simply use the FIXED media bus > code for each pad. That would actually make sense: this device always works > from memory to memory, and thus does not really have a pixel data bus > external to the device which is what the media bus codes really are for. Isn't the Bayer variant useful information to configure debayering ? I would expect it to be passed through the format on pad 0.
Hi Laurent, On Fri, Nov 30, 2018 at 01:09:37AM +0200, Laurent Pinchart wrote: > Hi Sakari, > > > On Friday, 9 November 2018 12:09:54 EET Sakari Ailus wrote: > > On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: > > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > >> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > [snip] > > > >>> ImgU media topology print: > > >>> > > >>> # media-ctl -d /dev/media0 -p > > >>> Media controller API version 4.19.0 > > >>> > > >>> Media device information > > >>> ------------------------ > > >>> driver ipu3-imgu > > >>> model ipu3-imgu > > >>> serial > > >>> bus info PCI:0000:00:05.0 > > >>> hw revision 0x80862015 > > >>> driver version 4.19.0 > > >>> > > >>> Device topology > > >>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > > >>> type V4L2 subdev subtype Unknown flags 0 > > >>> device node name /dev/v4l-subdev0 > > >>> pad0: Sink > > >>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > >> > > >> This doesn't seem right. Which formats can be enumerated from the pad? > > > > Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) > > format whereas the CAPTURE video nodes always have NV12. Can you confirm? > > > > If the OUTPUT video node format selection has no effect on the rest of the > > pipeline (device capabilities, which processing blocks are in use, CAPTURE > > video nodes formats etc.), I think you could simply use the FIXED media bus > > code for each pad. That would actually make sense: this device always works > > from memory to memory, and thus does not really have a pixel data bus > > external to the device which is what the media bus codes really are for. > > Isn't the Bayer variant useful information to configure debayering ? I would > expect it to be passed through the format on pad 0. That's already configured on the video node. The FIXED media bus code is intended for links where there's nothing to configure --- which is the case here.
Hi Laurent, On Fri, Nov 30, 2018 at 01:07:53AM +0200, Laurent Pinchart wrote: > Hello Bing, > > On Wednesday, 7 November 2018 06:16:47 EET Bing Bu Cao wrote: > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > [snip] > > > >> ImgU media topology print: > > >> > > >> # media-ctl -d /dev/media0 -p > > >> Media controller API version 4.19.0 > > >> > > >> Media device information > > >> ------------------------ > > >> driver ipu3-imgu > > >> model ipu3-imgu > > >> serial > > >> bus info PCI:0000:00:05.0 > > >> hw revision 0x80862015 > > >> driver version 4.19.0 > > >> > > >> Device topology > > >> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > > >> type V4L2 subdev subtype Unknown flags 0 > > >> device node name /dev/v4l-subdev0 > > >> pad0: Sink > > >> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > > > > > > This doesn't seem right. Which formats can be enumerated from the pad? > > > > > >> crop:(0,0)/1920x1080 > > >> compose:(0,0)/1920x1080] > > > > > > Does the compose rectangle affect the scaling on all outputs? > > > > Sakari, driver use crop and compose targets to help set input-feeder and BDS > > output resolutions which are 2 key block of whole imaging pipeline, not the > > actual ending output, but they will impact the final output. > > > > >> <- "ipu3-imgu 0 input":0 [] > > > > > > Are there links that have no useful link configuration? If so, you should > > > set them enabled and immutable in the driver. > > > > The enabled status of input pads is used to get which pipe that user is > > trying to enable (ipu3_link_setup()), so it could not been set as immutable. > > Each pipe needs an input in order to operate, so from that point of view the > input is mandatory. Why can't we make this link immutable, and use the stream > state (VIDIOC_STREAMON/VIDIOC_STREAMOFF) to enable/disable the pipes ? There are only two options (AFAIK) in choosing the firmware, and by configuring the links this is better visible to the user: the links the state of which can be changed are not immutable. The driver can also obtain the explicit pipeline configuration, which makes the implementation more simple.
Hi Sakari, On Monday, 3 December 2018 11:51:01 EET Sakari Ailus wrote: > On Fri, Nov 30, 2018 at 01:07:53AM +0200, Laurent Pinchart wrote: > > On Wednesday, 7 November 2018 06:16:47 EET Bing Bu Cao wrote: > >> On 11/01/2018 08:03 PM, Sakari Ailus wrote: > >>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > > > [snip] > > > >>>> ImgU media topology print: > >>>> > >>>> # media-ctl -d /dev/media0 -p > >>>> Media controller API version 4.19.0 > >>>> > >>>> Media device information > >>>> ------------------------ > >>>> driver ipu3-imgu > >>>> model ipu3-imgu > >>>> serial > >>>> bus info PCI:0000:00:05.0 > >>>> hw revision 0x80862015 > >>>> driver version 4.19.0 > >>>> > >>>> Device topology > >>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >>>> type V4L2 subdev subtype Unknown flags 0 > >>>> device node name /dev/v4l-subdev0 > >>>> pad0: Sink > >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >>> > >>> This doesn't seem right. Which formats can be enumerated from the pad? > >>> > >>>> crop:(0,0)/1920x1080 > >>>> compose:(0,0)/1920x1080] > >>> > >>> Does the compose rectangle affect the scaling on all outputs? > >> > >> Sakari, driver use crop and compose targets to help set input-feeder and > >> BDS output resolutions which are 2 key block of whole imaging pipeline, > >> not the actual ending output, but they will impact the final output. > >> > >>>> <- "ipu3-imgu 0 input":0 [] > >>> > >>> Are there links that have no useful link configuration? If so, you > >>> should set them enabled and immutable in the driver. > >> > >> The enabled status of input pads is used to get which pipe that user is > >> trying to enable (ipu3_link_setup()), so it could not been set as > >> immutable. > > > > Each pipe needs an input in order to operate, so from that point of view > > the input is mandatory. Why can't we make this link immutable, and use > > the stream state (VIDIOC_STREAMON/VIDIOC_STREAMOFF) to enable/disable the > > pipes ? > > There are only two options (AFAIK) in choosing the firmware, and by > configuring the links this is better visible to the user: the links the > state of which can be changed are not immutable. The driver can also obtain > the explicit pipeline configuration, which makes the implementation more > simple. Do you mean that different firmwares are loaded based on link configuration ? Does it also mean that once we start using the first pipeline the configuration of the second pipeline can't be changed anymore ? If so, what's the reason for such a limitation ?
Hi Laurent, Tomasz, > > Thanks for the reviews. > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > Hi Tomasz, > > > > On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > > > On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > > > > On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > > > > [snip] > > > > > >> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to > > > >> be enabled prior to the test. > > > >> 2. Stream tests are not performed since it requires > > > >> pre-configuration for each case. > > > > > > > > And that's a bit of an issue. I've tested the driver with a small > > > > script based on media-ctl to configure links and yavta to > > > > interface with the video nodes, and got the following oops: > > > > > > > > [ 136.927788] divide error: 0000 [#1] PREEMPT SMP PTI [ > > > > 136.927801] CPU: 2 PID: 2069 Comm: yavta Not tainted 4.20.0-rc1+ > > > > #9 [ 136.927806] Hardware name: HP Soraka/Soraka, BIOS > > > > 08/30/2018 [ 136.927820] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 > > > > [ipu3_imgu] [ 136.927825] Code: 89 44 24 28 42 8b 44 86 6c f7 54 > > > > 24 04 81 64 24 28 > > > > 00 fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 > > > > 00 > > > > 00 99 <f7> fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd > > > > 01 > > > > 19 d2 [ 136.927830] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 [ > > > > 136.927835] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > > > > ffff9af2c3e353c0 > > > > [ 136.927839] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > > > > ffff9af2c3e353c0 > > > > [ 136.927843] RBP: 0000000000000001 R08: 0000000000000000 R09: > > > > ffff9af2c0b83880 > > > > [ 136.927846] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > > > > 00000000000003a0 > > > > [ 136.927849] R13: 0000000000025a0a R14: 0000000000000000 R15: > > > > 0000000000000000 > > > > [ 136.927854] FS: 00007f1eca167700(0000) > > > > GS:ffff8c19fab00000(0000) > > > > knlGS: > > > > 0000000000000000 > > > > [ 136.927858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ > > > > 136.927862] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > > > > 00000000003606e0 > > > > [ 136.927865] Call Trace: > > > > [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a > > > > [ 136.927892] ? __switch_to_asm+0x40/0x70 [ 136.927899] ? > > > > alloc_vmap_area+0x78/0x2f6 [ 136.927903] ? > > > > __switch_to_asm+0x40/0x70 [ 136.927907] ? > > > > __switch_to_asm+0x34/0x70 [ 136.927911] ? > > > > __switch_to_asm+0x40/0x70 [ 136.927915] ? > > > > __switch_to_asm+0x34/0x70 [ 136.927923] ? > > > > __inc_numa_state+0x28/0x70 [ 136.927929] ? > > > > preempt_latency_start+0x1e/0x3d [ 136.927936] ? > > > > get_page_from_freelist+0x821/0xb62 > > > > [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b [ 136.927948] ? > > > > kmem_cache_alloc_node_trace+0xf6/0x108 > > > > [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 > > > > > > Is it just me or the backtrace above doesn't seem to make sense? I > > > don't see any allocations inside ipu3_css_cfg_acc(). > > > > I suppose that's why it's prefixed with '?' :-) > > > > > > [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] [ > > > > 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] [ > > > > 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] [ > > > > 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] [ 136.928022] > > > > ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] [ 136.928034] > > > > vb2_start_streaming+0x6c/0xf2 [videobuf2_common] [ 136.928044] > > > > vb2_core_streamon+0xcf/0x109 [videobuf2_common] [ 136.928061] > > > > __video_do_ioctl+0x239/0x388 [videodev] [ 136.928081] > > > > video_usercopy+0x25d/0x47a [videodev] [ 136.928097] ? > > > > copy_overflow+0x14/0x14 [videodev] [ 136.928115] > > > > v4l2_ioctl+0x4d/0x58 [videodev] [ 136.928123] > > > > vfs_ioctl+0x1b/0x28 [ 136.928130] do_vfs_ioctl+0x4de/0x566 [ > > > > 136.928139] > > > > ksys_ioctl+0x50/0x70 [ 136.928146] __x64_sys_ioctl+0x16/0x19 [ > > > > 136.928152] do_syscall_64+0x4d/0x5a [ 136.928158] > > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 136.928164] RIP: 0033:0x7f1ec9a84f47 [ 136.928169] Code: 00 00 > > > > 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 > > > > 0f > > > > 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 > > > > 01 > > > > 48 [ 136.928173] RSP: 002b:00007ffe279e6188 EFLAGS: 00000246 > > ORIG_RAX: > > > > 0000000000000010 > > > > [ 136.928178] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > > > 00007f1ec9a84f47 > > > > [ 136.928181] RDX: 00007ffe279e6194 RSI: 0000000040045612 RDI: > > > > 0000000000000003 > > > > [ 136.928184] RBP: 0000000000000000 R08: 00007f1ec776d000 R09: > > > > 0000000000000000 > > > > [ 136.928188] R10: 0000000000000020 R11: 0000000000000246 R12: > > > > 00007ffe279e6360 > > > > [ 136.928191] R13: 0000000000000004 R14: 00007ffe279e6360 R15: > > > > 00007ffe279e8826 > > > > [ 136.928198] Modules linked in: ccm zram arc4 iwlmvm mac80211 > > > > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp iwlwifi > > > > cfg80211 hid_multitouch ipu3_imgu ipu3_cio2 8250_dw > > videobuf2_dma_sg > > > > videobuf2_memops videobuf2_v4l2 processor_thermal_device > > > > intel_soc_dts_iosf videobuf2_common ov5670 ov13858 dw9714 > > > > v4l2_fwnode v4l2_common videodev media at24 cros_ec_lpcs > > > > cros_ec_core int3403_thermal int340x_thermal_zone int3400_thermal > > > > acpi_thermal_rel chromeos_pstore mac_hid autofs4 usbhid mmc_block > > > > hid_generic i915 sdhci_pci video cqhci i2c_algo_bit sdhci > > > > drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm > > > > drm_panel_orientation_quirks i2c_hid hid [ 136.928273] ---[ end > > > > trace 4ec6c2ce09e06d9d ]--- [ 136.928288] RIP: > > > > 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] [ 136.928293] Code: > > > > 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd ff ff 81 > > > > 64 > > > > 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 <f7> fb > > > > 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 [ > > > > 136.928297] RSP: 0018:ffff9af2c0b837c8 EFLAGS: 00010202 [ > > > > 136.928302] > > RAX: 00000000ffffffff RBX: 0000000000000000 RCX: > > > > ffff9af2c3e353c0 > > > > [ 136.928307] RDX: 00000000ffffffff RSI: ffff9af2c0b838e0 RDI: > > > > ffff9af2c3e353c0 > > > > [ 136.928311] RBP: 0000000000000001 R08: 0000000000000000 R09: > > > > ffff9af2c0b83880 > > > > [ 136.928320] R10: ffff9af2c3e353c0 R11: ffff9af2c3e357c0 R12: > > > > 00000000000003a0 > > > > [ 136.928324] R13: 0000000000025a0a R14: 0000000000000000 R15: > > > > 0000000000000000 > > > > [ 136.928330] FS: 00007f1eca167700(0000) > > > > GS:ffff8c19fab00000(0000) > > > > knlGS: > > > > 0000000000000000 > > > > [ 136.928349] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ > > > > 136.928364] CR2: 00007f1ec776c000 CR3: 00000001312a4003 CR4: > > > > 00000000003606e0 > > > > > > > > The script can be found at > > > > https://lists.libcamera.org/pipermail/libcamera-devel/2018-Novembe > > > > r/ > > > > 00004 > > > > 0.html. > > > > > > > > I may be doing something wrong (and I probably am), but in any > > > > case, the driver shouldn't crash. Could you please have a look ? > > > > > > It looks like the driver doesn't have the default state initialized > > > correctly somewhere and it ends up using 0 as the divisor in some > > > calculation? Something to fix indeed. > > > > That's probably the case. I'll trust Intel to fix that in v8 :-) > > > > Ack. > Thanks for catching this. I was able to reproduce this error and I see that error handling is missing, leading to the panic. https://git.linuxtv.org/sailus/media_tree.git/tree/drivers/media /pci/intel/ipu3/ipu3-css-params.c?h=ipu3-v7&id= 19cee7329ca2d0156043cac6afcde535e93310af#n433 is where the -EINVAL is returned. Setting the return type as int for the following function and all its callers to use the return value properly to error out, makes the panic go away. ipu3_css_osys_calc_frame_and_stripe_params() Will include the fix in v8. Thanks for catching this. Raj > > -- > > Regards, > > > > Laurent Pinchart > > > >
Hi Rajmohan, On Tuesday, 4 December 2018 18:07:16 EET Mani, Rajmohan wrote: > >> On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > >>> On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > >>>> On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > >> > >> [snip] > >> > >>>>> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to > >>>>> be enabled prior to the test. > >>>>> 2. Stream tests are not performed since it requires > >>>>> pre-configuration for each case. > >>>> > >>>> And that's a bit of an issue. I've tested the driver with a small > >>>> script based on media-ctl to configure links and yavta to > >>>> interface with the video nodes, and got the following oops: [snip] > >>>> The script can be found at > >>>> https://lists.libcamera.org/pipermail/libcamera-devel/2018-Novembe > >>>> r/000040.html. > >>>> > >>>> I may be doing something wrong (and I probably am), but in any > >>>> case, the driver shouldn't crash. Could you please have a look ? > >>> > >>> It looks like the driver doesn't have the default state initialized > >>> correctly somewhere and it ends up using 0 as the divisor in some > >>> calculation? Something to fix indeed. > >> > >> That's probably the case. I'll trust Intel to fix that in v8 :-) > > > > Ack. > > Thanks for catching this. > I was able to reproduce this error and I see that error handling > is missing, leading to the panic. > > https://git.linuxtv.org/sailus/media_tree.git/tree/drivers/media > /pci/intel/ipu3/ipu3-css-params.c?h=ipu3-v7&id= > 19cee7329ca2d0156043cac6afcde535e93310af#n433 > > is where the -EINVAL is returned. > > Setting the return type as int for the following function and all > its callers to use the return value properly to error out, makes > the panic go away. I assume that I will still not be able to process frames through the pipe then, as I'll get an error :-) Could you tell me why the configuration is incorrect and how I can fix it ? > ipu3_css_osys_calc_frame_and_stripe_params() > > Will include the fix in v8. Thank you. > Thanks for catching this. You're welcome.
Hi Laurent, > -----Original Message----- > From: Laurent Pinchart [mailto:laurent.pinchart@ideasonboard.com] > Sent: Tuesday, December 04, 2018 8:42 AM > To: Mani, Rajmohan <rajmohan.mani@intel.com> > Cc: Tomasz Figa <tfiga@chromium.org>; Zhi, Yong <yong.zhi@intel.com>; > Linux Media Mailing List <linux-media@vger.kernel.org>; Sakari Ailus > <sakari.ailus@linux.intel.com>; Mauro Carvalho Chehab > <mchehab@kernel.org>; Hans Verkuil <hans.verkuil@cisco.com>; Zheng, Jian > Xu <jian.xu.zheng@intel.com>; Hu, Jerry W <jerry.w.hu@intel.com>; > Toivonen, Tuukka <tuukka.toivonen@intel.com>; Qiu, Tian Shu > <tian.shu.qiu@intel.com>; Cao, Bingbu <bingbu.cao@intel.com> > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hi Rajmohan, > > On Tuesday, 4 December 2018 18:07:16 EET Mani, Rajmohan wrote: > > >> On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > > >>> On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > > >>>> On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > > >> > > >> [snip] > > >> > > >>>>> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need > > >>>>> to be enabled prior to the test. > > >>>>> 2. Stream tests are not performed since it requires > > >>>>> pre-configuration for each case. > > >>>> > > >>>> And that's a bit of an issue. I've tested the driver with a small > > >>>> script based on media-ctl to configure links and yavta to > > >>>> interface with the video nodes, and got the following oops: > > [snip] > > > >>>> The script can be found at > > >>>> https://lists.libcamera.org/pipermail/libcamera-devel/2018-Novemb > > >>>> e > > >>>> r/000040.html. > > >>>> > > >>>> I may be doing something wrong (and I probably am), but in any > > >>>> case, the driver shouldn't crash. Could you please have a look ? > > >>> > > >>> It looks like the driver doesn't have the default state > > >>> initialized correctly somewhere and it ends up using 0 as the > > >>> divisor in some calculation? Something to fix indeed. > > >> > > >> That's probably the case. I'll trust Intel to fix that in v8 :-) > > > > > > Ack. > > > > Thanks for catching this. > > I was able to reproduce this error and I see that error handling is > > missing, leading to the panic. > > > > https://git.linuxtv.org/sailus/media_tree.git/tree/drivers/media > > /pci/intel/ipu3/ipu3-css-params.c?h=ipu3-v7&id= > > 19cee7329ca2d0156043cac6afcde535e93310af#n433 > > > > is where the -EINVAL is returned. > > > > Setting the return type as int for the following function and all its > > callers to use the return value properly to error out, makes the panic > > go away. > > I assume that I will still not be able to process frames through the pipe then, as > I'll get an error :-) Could you tell me why the configuration is incorrect and how > I can fix it ? > :) Let me look into this more and get back. Thanks for your patience. Raj > > ipu3_css_osys_calc_frame_and_stripe_params() > > > > Will include the fix in v8. > > Thank you. > > > Thanks for catching this. > > You're welcome. > > -- > Regards, > > Laurent Pinchart > >
Hi Laurent, > > Hi Rajmohan, > > > > On Tuesday, 4 December 2018 18:07:16 EET Mani, Rajmohan wrote: > > > >> On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > > > >>> On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > > > >>>> On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > > > >> > > > >> [snip] > > > >> > > > >>>>> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need > > > >>>>> to be enabled prior to the test. > > > >>>>> 2. Stream tests are not performed since it requires > > > >>>>> pre-configuration for each case. > > > >>>> > > > >>>> And that's a bit of an issue. I've tested the driver with a > > > >>>> small script based on media-ctl to configure links and yavta to > > > >>>> interface with the video nodes, and got the following oops: > > > > [snip] > > > > > >>>> The script can be found at > > > >>>> https://lists.libcamera.org/pipermail/libcamera-devel/2018-Nove > > > >>>> mb > > > >>>> e > > > >>>> r/000040.html. > > > >>>> > > > >>>> I may be doing something wrong (and I probably am), but in any > > > >>>> case, the driver shouldn't crash. Could you please have a look ? > > > >>> > > > >>> It looks like the driver doesn't have the default state > > > >>> initialized correctly somewhere and it ends up using 0 as the > > > >>> divisor in some calculation? Something to fix indeed. > > > >> > > > >> That's probably the case. I'll trust Intel to fix that in v8 :-) > > > > > > > > Ack. > > > > > > Thanks for catching this. > > > I was able to reproduce this error and I see that error handling is > > > missing, leading to the panic. > > > > > > https://git.linuxtv.org/sailus/media_tree.git/tree/drivers/media > > > /pci/intel/ipu3/ipu3-css-params.c?h=ipu3-v7&id= > > > 19cee7329ca2d0156043cac6afcde535e93310af#n433 > > > > > > is where the -EINVAL is returned. > > > > > > Setting the return type as int for the following function and all > > > its callers to use the return value properly to error out, makes the > > > panic go away. > > > > I assume that I will still not be able to process frames through the > > pipe then, as I'll get an error :-) Could you tell me why the > > configuration is incorrect and how I can fix it ? > > > > :) > Let me look into this more and get back. > Thanks for your patience. I can see a couple of steps missing in the script below. (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000040.html) From patch 02 of this v7 series "doc-rst: Add Intel IPU3 documentation", under section "Configuring ImgU V4L2 subdev for image processing"... 1. The pipe mode needs to be configured for the V4L2 subdev. Also the pipe mode of the corresponding V4L2 subdev should be set as desired (e.g 0 for video mode or 1 for still mode) through the control id 0x009819a1 as below. e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 2. ImgU pipeline needs to be configured for image processing as below. RAW bayer frames go through the following ISP pipeline HW blocks to have the processed image output to the DDR memory. RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> Geometric Distortion Correction (GDC) -> DDR The ImgU V4L2 subdev has to be configured with the supported resolutions in all the above HW blocks, for a given input resolution. For a given supported resolution for an input frame, the Input Feeder, Bayer Down Scaling and GDC blocks should be configured with the supported resolutions. This information can be obtained by looking at the following IPU3 ISP configuration table for ov5670 sensor. https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/master /baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss /graph_settings_ov5670.xml For the ov5670 example, for an input frame with a resolution of 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and 2560x1920 respectively. The following steps prepare the ImgU ISP pipeline for the image processing. 1. The ImgU V4L2 subdev data format should be set by using the VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained above. 2. The ImgU V4L2 subdev cropping should be set by using the VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the target, using the input feeder height and width. 3. The ImgU V4L2 subdev composing should be set by using the VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the target, using the BDS height and width. Once these 2 steps are done, the raw bayer frames can be input to the ImgU V4L2 subdev for processing. > Raj > > > > ipu3_css_osys_calc_frame_and_stripe_params() > > > > > > Will include the fix in v8. > > > > Thank you. > > > > > Thanks for catching this. > > > > You're welcome. > > > > -- > > Regards, > > > > Laurent Pinchart > > > >
Hi Rajmohan, On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: [snip] > I can see a couple of steps missing in the script below. > (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000040. > html) > > From patch 02 of this v7 series "doc-rst: Add Intel IPU3 documentation", > under section "Configuring ImgU V4L2 subdev for image processing"... > > 1. The pipe mode needs to be configured for the V4L2 subdev. > > Also the pipe mode of the corresponding V4L2 subdev should be set as > desired (e.g 0 for video mode or 1 for still mode) through the control > id 0x009819a1 as below. > > e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 I assume the control takes a valid default value ? It's better to set it explicitly anyway, so I'll do so. > 2. ImgU pipeline needs to be configured for image processing as below. > > RAW bayer frames go through the following ISP pipeline HW blocks to > have the processed image output to the DDR memory. > > RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > Geometric Distortion Correction (GDC) -> DDR > > The ImgU V4L2 subdev has to be configured with the supported > resolutions in all the above HW blocks, for a given input resolution. > > For a given supported resolution for an input frame, the Input Feeder, > Bayer Down Scaling and GDC blocks should be configured with the > supported resolutions. This information can be obtained by looking at > the following IPU3 ISP configuration table for ov5670 sensor. > > https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/maste > r /baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss > /graph_settings_ov5670.xml > > For the ov5670 example, for an input frame with a resolution of > 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > 2560x1920 respectively. How is the GDC output resolution computed from the input resolution ? Does the GDC always consume 32 columns and 22 lines ? > The following steps prepare the ImgU ISP pipeline for the image processing. > > 1. The ImgU V4L2 subdev data format should be set by using the > VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained above. If I understand things correctly, the GDC resolution is the pipeline output resolution. Why is it configured on pad 0 ? > 2. The ImgU V4L2 subdev cropping should be set by using the > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > target, using the input feeder height and width. > > 3. The ImgU V4L2 subdev composing should be set by using the > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > target, using the BDS height and width. > > Once these 2 steps are done, the raw bayer frames can be input to the > ImgU V4L2 subdev for processing. Do I need to capture from both the output and viewfinder nodes ? How are they related to the IF -> BDS -> GDC pipeline, are they both fed from the GDC output ? If so, how does the viewfinder scaler fit in that picture ? I have tried the above configuration with the IPU3 v8 driver, and while the kernel doesn't crash, no images get processed. The userspace processes wait forever for buffers to be ready. I then configured pad 2 to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ There's one problem though: during capture, or very soon after it, the machine locks up completely. I suspect a memory corruption, as when it doesn't log immediately commands such as dmesg will not produce any output and just block, until the system freezes soon after (especially when moving the mouse). I would still call this an improvement to some extent, but there's definitely room for more improvements :-) To reproduce the issue, you can run the ipu3-process.sh script (attached to this e-mail) with the following arguments: $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in the IPU3-specific Bayer format (for a total of 6469632 bytes).
Hello Rajmohan, On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > [snip] > > > I can see a couple of steps missing in the script below. > > (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/00004 > > 0. html) > > > > From patch 02 of this v7 series "doc-rst: Add Intel IPU3 documentation", > > under section "Configuring ImgU V4L2 subdev for image processing"... > > > > 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > Also the pipe mode of the corresponding V4L2 subdev should be set as > > desired (e.g 0 for video mode or 1 for still mode) through the control > > id 0x009819a1 as below. > > > > e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > I assume the control takes a valid default value ? It's better to set it > explicitly anyway, so I'll do so. > > > 2. ImgU pipeline needs to be configured for image processing as below. > > > > RAW bayer frames go through the following ISP pipeline HW blocks to > > have the processed image output to the DDR memory. > > > > RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > Geometric Distortion Correction (GDC) -> DDR > > > > The ImgU V4L2 subdev has to be configured with the supported > > resolutions in all the above HW blocks, for a given input resolution. > > > > For a given supported resolution for an input frame, the Input Feeder, > > Bayer Down Scaling and GDC blocks should be configured with the > > supported resolutions. This information can be obtained by looking at > > the following IPU3 ISP configuration table for ov5670 sensor. > > > > https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/mas > > te r /baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss > > /graph_settings_ov5670.xml > > > > For the ov5670 example, for an input frame with a resolution of > > 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > 2560x1920 respectively. > > How is the GDC output resolution computed from the input resolution ? Does > the GDC always consume 32 columns and 22 lines ? > > > The following steps prepare the ImgU ISP pipeline for the image > > processing. > > > > 1. The ImgU V4L2 subdev data format should be set by using the > > VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > > above. > > If I understand things correctly, the GDC resolution is the pipeline output > resolution. Why is it configured on pad 0 ? > > > 2. The ImgU V4L2 subdev cropping should be set by using the > > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > target, using the input feeder height and width. > > > > 3. The ImgU V4L2 subdev composing should be set by using the > > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > target, using the BDS height and width. > > > > Once these 2 steps are done, the raw bayer frames can be input to the > > ImgU V4L2 subdev for processing. > > Do I need to capture from both the output and viewfinder nodes ? How are > they related to the IF -> BDS -> GDC pipeline, are they both fed from the > GDC output ? If so, how does the viewfinder scaler fit in that picture ? > > I have tried the above configuration with the IPU3 v8 driver, and while the > kernel doesn't crash, no images get processed. The userspace processes wait > forever for buffers to be ready. I then configured pad 2 to 2560x1920 and > pad 3 to 1920x1080, and managed to capture images \o/ > > There's one problem though: during capture, or very soon after it, the > machine locks up completely. I suspect a memory corruption, as when it > doesn't log immediately commands such as dmesg will not produce any output > and just block, until the system freezes soon after (especially when moving > the mouse). > > I would still call this an improvement to some extent, but there's > definitely room for more improvements :-) > > To reproduce the issue, you can run the ipu3-process.sh script (attached to > this e-mail) with the following arguments: > > $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > > frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in the > IPU3-specific Bayer format (for a total of 6469632 bytes). I managed to get the dmesg output, and it doesn't look pretty. [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm mac80211 iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm drm_panel_orientation_quirks i2c_hid hid [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C 4.20.0-rc6+ #2 [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: 000000000000000c [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: 00000000ffffffff [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: ffff8f5cfaba16f0 [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: ffff8f5cf58f0028 [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: ffff8f5cf58f04e8 [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) knlGS: 0000000000000000 [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: 00000000003606e0 [ 571.217301] Call Trace: [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] [ 571.217480] vfs_ioctl+0x1e/0x2b [ 571.217486] do_vfs_ioctl+0x531/0x559 [ 571.217494] ? vfs_write+0xd1/0xdf [ 571.217500] ksys_ioctl+0x50/0x70 [ 571.217506] __x64_sys_ioctl+0x16/0x19 [ 571.217512] do_syscall_64+0x53/0x60 [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 571.217524] RIP: 0033:0x7f85cf9b9f47 [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f85cf9b9f47 [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: 0000000000000003 [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: 00007f85d009c700 [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: 000055f4c4dc0b06 [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: 00007ffc59057825 [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout
Hello again, On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > > [snip] > > > > > I can see a couple of steps missing in the script below. > > > (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000 > > > 040.html) > > > > > > From patch 02 of this v7 series "doc-rst: Add Intel IPU3 documentation", > > > under section "Configuring ImgU V4L2 subdev for image processing"... > > > > > > 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > > > Also the pipe mode of the corresponding V4L2 subdev should be set as > > > desired (e.g 0 for video mode or 1 for still mode) through the control > > > id 0x009819a1 as below. > > > > > > e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > > I assume the control takes a valid default value ? It's better to set it > > explicitly anyway, so I'll do so. > > > > > 2. ImgU pipeline needs to be configured for image processing as below. > > > > > > RAW bayer frames go through the following ISP pipeline HW blocks to > > > have the processed image output to the DDR memory. > > > > > > RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > Geometric Distortion Correction (GDC) -> DDR > > > > > > The ImgU V4L2 subdev has to be configured with the supported > > > resolutions in all the above HW blocks, for a given input resolution. > > > > > > For a given supported resolution for an input frame, the Input Feeder, > > > Bayer Down Scaling and GDC blocks should be configured with the > > > supported resolutions. This information can be obtained by looking at > > > the following IPU3 ISP configuration table for ov5670 sensor. > > > > > > https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/m > > > aster/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/ > > > gcss/graph_settings_ov5670.xml > > > > > > For the ov5670 example, for an input frame with a resolution of > > > 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > > resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > > 2560x1920 respectively. > > > > How is the GDC output resolution computed from the input resolution ? Does > > the GDC always consume 32 columns and 22 lines ? > > > > > The following steps prepare the ImgU ISP pipeline for the image > > > processing. > > > > > > 1. The ImgU V4L2 subdev data format should be set by using the > > > VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > > > above. > > > > If I understand things correctly, the GDC resolution is the pipeline > > output resolution. Why is it configured on pad 0 ? > > > > > 2. The ImgU V4L2 subdev cropping should be set by using the > > > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > > target, using the input feeder height and width. > > > > > > 3. The ImgU V4L2 subdev composing should be set by using the > > > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > > target, using the BDS height and width. > > > > > > Once these 2 steps are done, the raw bayer frames can be input to the > > > ImgU V4L2 subdev for processing. > > > > Do I need to capture from both the output and viewfinder nodes ? How are > > they related to the IF -> BDS -> GDC pipeline, are they both fed from the > > GDC output ? If so, how does the viewfinder scaler fit in that picture ? > > > > I have tried the above configuration with the IPU3 v8 driver, and while > > the kernel doesn't crash, no images get processed. The userspace processes > > wait forever for buffers to be ready. I then configured pad 2 to 2560x1920 > > and pad 3 to 1920x1080, and managed to capture images \o/ > > > > There's one problem though: during capture, or very soon after it, the > > machine locks up completely. I suspect a memory corruption, as when it > > doesn't log immediately commands such as dmesg will not produce any output > > and just block, until the system freezes soon after (especially when > > moving the mouse). > > > > I would still call this an improvement to some extent, but there's > > definitely room for more improvements :-) > > > > To reproduce the issue, you can run the ipu3-process.sh script (attached > > to this e-mail) with the following arguments: > > > > $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 This should have read $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 Without the --vf argument no images are processed. It seems that the Intel mail server blocked the mail that contained the script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in the > > IPU3-specific Bayer format (for a total of 6469632 bytes). > > I managed to get the dmesg output, and it doesn't look pretty. > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm mac80211 > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > drm_panel_orientation_quirks i2c_hid hid > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > 4.20.0-rc6+ #2 > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > 000000000000000c > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > 00000000ffffffff > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > ffff8f5cfaba16f0 > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > ffff8f5cf58f0028 > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > ffff8f5cf58f04e8 > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) knlGS: > 0000000000000000 > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > 00000000003606e0 > [ 571.217301] Call Trace: > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > [ 571.217480] vfs_ioctl+0x1e/0x2b > [ 571.217486] do_vfs_ioctl+0x531/0x559 > [ 571.217494] ? vfs_write+0xd1/0xdf > [ 571.217500] ksys_ioctl+0x50/0x70 > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > [ 571.217512] do_syscall_64+0x53/0x60 > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > 0000000000000010 > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > 00007f85cf9b9f47 > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > 0000000000000003 > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > 00007f85d009c700 > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > 000055f4c4dc0b06 > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > 00007ffc59057825 > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout And after fixing another issue in the capture script (which was setting the format on the ImgU subdev pad 3 to 2560x1920 but capture in 1920x1080), I now get plenty of the following messages: [ 221.366131] BUG: Bad page state in process yavta pfn:14a4ff [ 221.366134] page:ffffde5d45293fc0 count:-1 mapcount:0 mapping: 0000000000000000 index:0x0 [ 221.366137] flags: 0x200000000000000() [ 221.366140] raw: 0200000000000000 dead000000000100 dead000000000200 0000000000000000 [ 221.366143] raw: 0000000000000000 0000000000000000 ffffffffffffffff 0000000000000000 [ 221.366145] page dumped because: nonzero _refcount [ 221.366147] Modules linked in: asix usbnet mii zram arc4 iwlmvm intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp mac80211 iwlwifi cfg80211 hid_multitouch 8250_dw ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 processor_thermal_device videobuf2_common intel_soc_dts_iosf ov13858 ov5670 dw9714 v4l2_fwnode v4l2_common videodev media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone chromeos_pstore mac_hid int3400_thermal acpi_thermal_rel autofs4 usbhid mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops sdhci_pci cqhci sdhci drm drm_panel_orientation_quirks i2c_hid hid [ 221.366172] CPU: 3 PID: 1022 Comm: yavta Tainted: G B WC 4.20.0-rc6+ #2 [ 221.366173] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ 221.366173] Call Trace: [ 221.366176] dump_stack+0x46/0x59 [ 221.366179] bad_page+0xf2/0x10c [ 221.366182] free_pages_check+0x78/0x81 [ 221.366186] free_pcppages_bulk+0xa6/0x236 [ 221.366190] free_unref_page+0x4b/0x53 [ 221.366193] vb2_dma_sg_put+0x95/0xb5 [videobuf2_dma_sg] [ 221.366197] __vb2_buf_mem_free+0x3a/0x6e [videobuf2_common] [ 221.366202] __vb2_queue_free+0xe3/0x1be [videobuf2_common] [ 221.366207] vb2_core_reqbufs+0xe9/0x2cc [videobuf2_common] [ 221.366212] vb2_ioctl_reqbufs+0x78/0x9e [videobuf2_v4l2] [ 221.366220] __video_do_ioctl+0x258/0x38e [videodev] [ 221.366229] video_usercopy+0x25f/0x4e5 [videodev] [ 221.366237] ? copy_overflow+0x14/0x14 [videodev] [ 221.366240] ? unmap_region+0xe0/0x10a [ 221.366250] v4l2_ioctl+0x4d/0x58 [videodev] [ 221.366253] vfs_ioctl+0x1e/0x2b [ 221.366255] do_vfs_ioctl+0x531/0x559 [ 221.366260] ksys_ioctl+0x50/0x70 [ 221.366263] __x64_sys_ioctl+0x16/0x19 [ 221.366266] do_syscall_64+0x53/0x60 [ 221.366269] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 221.366270] RIP: 0033:0x7fbe39f6af47 [ 221.366272] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 221.366273] RSP: 002b:00007fff05638e68 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 221.366275] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fbe39f6af47 [ 221.366279] RDX: 00007fff05638f90 RSI: 00000000c0145608 RDI: 0000000000000003 [ 221.366283] RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000045 [ 221.366287] R10: 0000000000000557 R11: 0000000000000246 R12: 000055c83bd76750 [ 221.366290] R13: 000055c83b6b26a0 R14: 0000000000000001 R15: 00007fff0563a825
On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > Hello Rajmohan, > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: >> >> [snip] >> >>> I can see a couple of steps missing in the script below. >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/00004 >>> 0. html) >>> >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 documentation", >>> under section "Configuring ImgU V4L2 subdev for image processing"... >>> >>> 1. The pipe mode needs to be configured for the V4L2 subdev. >>> >>> Also the pipe mode of the corresponding V4L2 subdev should be set as >>> desired (e.g 0 for video mode or 1 for still mode) through the control >>> id 0x009819a1 as below. >>> >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 >> I assume the control takes a valid default value ? It's better to set it >> explicitly anyway, so I'll do so. The video mode is set by default. If you want to set to still mode or change mode, you need set the subdev control. >> >>> 2. ImgU pipeline needs to be configured for image processing as below. >>> >>> RAW bayer frames go through the following ISP pipeline HW blocks to >>> have the processed image output to the DDR memory. >>> >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> >>> Geometric Distortion Correction (GDC) -> DDR >>> >>> The ImgU V4L2 subdev has to be configured with the supported >>> resolutions in all the above HW blocks, for a given input resolution. >>> >>> For a given supported resolution for an input frame, the Input Feeder, >>> Bayer Down Scaling and GDC blocks should be configured with the >>> supported resolutions. This information can be obtained by looking at >>> the following IPU3 ISP configuration table for ov5670 sensor. >>> >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/mas >>> te r /baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss >>> /graph_settings_ov5670.xml >>> >>> For the ov5670 example, for an input frame with a resolution of >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and >>> 2560x1920 respectively. >> How is the GDC output resolution computed from the input resolution ? Does >> the GDC always consume 32 columns and 22 lines ? All the intermediate resolutions in the pipeline are determined by the actual use case, in other word determined by the IMGU input resolution(sensor output) and the final output and viewfinder resolution. BDS mainly do Bayer downscaling, it has limitation that the downscaling factor must be a value a integer multiple of 1/32. GDC output depends on the input and width should be x8 and height x4 alignment. >> >>> The following steps prepare the ImgU ISP pipeline for the image >>> processing. >>> >>> 1. The ImgU V4L2 subdev data format should be set by using the >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained >>> above. >> If I understand things correctly, the GDC resolution is the pipeline output >> resolution. Why is it configured on pad 0 ? We see the GDC output resolution as the input of output system, the sink pad format is used for output and viewfinder resolutions. >> >>> 2. The ImgU V4L2 subdev cropping should be set by using the >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the >>> target, using the input feeder height and width. >>> >>> 3. The ImgU V4L2 subdev composing should be set by using the >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>> target, using the BDS height and width. >>> >>> Once these 2 steps are done, the raw bayer frames can be input to the >>> ImgU V4L2 subdev for processing. >> Do I need to capture from both the output and viewfinder nodes ? How are >> they related to the IF -> BDS -> GDC pipeline, are they both fed from the >> GDC output ? If so, how does the viewfinder scaler fit in that picture ? The output capture should be set, the viewfinder can be disabled. The IF and BDS are seen as crop and compose of the imgu input video device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source pads. >> >> I have tried the above configuration with the IPU3 v8 driver, and while the >> kernel doesn't crash, no images get processed. The userspace processes wait >> forever for buffers to be ready. I then configured pad 2 to 2560x1920 and >> pad 3 to 1920x1080, and managed to capture images \o/ >> >> There's one problem though: during capture, or very soon after it, the >> machine locks up completely. I suspect a memory corruption, as when it >> doesn't log immediately commands such as dmesg will not produce any output >> and just block, until the system freezes soon after (especially when moving >> the mouse). >> >> I would still call this an improvement to some extent, but there's >> definitely room for more improvements :-) >> >> To reproduce the issue, you can run the ipu3-process.sh script (attached to >> this e-mail) with the following arguments: >> >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 >> >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in the >> IPU3-specific Bayer format (for a total of 6469632 bytes). > I managed to get the dmesg output, and it doesn't look pretty. > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm mac80211 > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev at24 > media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > drm_panel_orientation_quirks i2c_hid hid > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > 4.20.0-rc6+ #2 > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 48 > 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 <0f> 0b > 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > 000000000000000c > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > 00000000ffffffff > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > ffff8f5cfaba16f0 > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > ffff8f5cf58f0028 > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > ffff8f5cf58f04e8 > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) knlGS: > 0000000000000000 > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > 00000000003606e0 > [ 571.217301] Call Trace: > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > [ 571.217480] vfs_ioctl+0x1e/0x2b > [ 571.217486] do_vfs_ioctl+0x531/0x559 > [ 571.217494] ? vfs_write+0xd1/0xdf > [ 571.217500] ksys_ioctl+0x50/0x70 > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > [ 571.217512] do_syscall_64+0x53/0x60 > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 > c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d > 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > 0000000000000010 > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > 00007f85cf9b9f47 > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > 0000000000000003 > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > 00007f85d009c700 > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > 000055f4c4dc0b06 > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > 00007ffc59057825 > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout >
Hello Bingbu, On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > >> > >> [snip] > >> > >>> I can see a couple of steps missing in the script below. > >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000 > >>> 040.html) > >>> > >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > >>> documentation", under section "Configuring ImgU V4L2 subdev for image > >>> processing"... > >>> > >>> 1. The pipe mode needs to be configured for the V4L2 subdev. > >>> > >>> Also the pipe mode of the corresponding V4L2 subdev should be set as > >>> desired (e.g 0 for video mode or 1 for still mode) through the control > >>> id 0x009819a1 as below. > >>> > >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > >> > >> I assume the control takes a valid default value ? It's better to set it > >> explicitly anyway, so I'll do so. > > The video mode is set by default. If you want to set to still mode or change > mode, you need set the subdev control. > > >>> 2. ImgU pipeline needs to be configured for image processing as below. > >>> > >>> RAW bayer frames go through the following ISP pipeline HW blocks to > >>> have the processed image output to the DDR memory. > >>> > >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > >>> Geometric Distortion Correction (GDC) -> DDR > >>> > >>> The ImgU V4L2 subdev has to be configured with the supported > >>> resolutions in all the above HW blocks, for a given input resolution. > >>> > >>> For a given supported resolution for an input frame, the Input Feeder, > >>> Bayer Down Scaling and GDC blocks should be configured with the > >>> supported resolutions. This information can be obtained by looking at > >>> the following IPU3 ISP configuration table for ov5670 sensor. > >>> > >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/m > >>> aster/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/ > >>> gcss/graph_settings_ov5670.xml > >>> > >>> For the ov5670 example, for an input frame with a resolution of > >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > >>> 2560x1920 respectively. > >> > >> How is the GDC output resolution computed from the input resolution ? > >> Does the GDC always consume 32 columns and 22 lines ? > > All the intermediate resolutions in the pipeline are determined by the > actual use case, in other word determined by the IMGU input > resolution(sensor output) and the final output and viewfinder resolution. > BDS mainly do Bayer downscaling, it has limitation that the downscaling > factor must be a value a integer multiple of 1/32. > GDC output depends on the input and width should be x8 and height x4 > alignment. Thank you for the information. This will need to be captured in the documentation, along with information related to how each block in the hardware pipeline interacts with the image size. It should be possible for a developer to compute the output and viewfinder resolutions based on the parameters of the image processing algorithms just with the information contained in the driver documentation. > >>> The following steps prepare the ImgU ISP pipeline for the image > >>> processing. > >>> > >>> 1. The ImgU V4L2 subdev data format should be set by using the > >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > >>> above. > >> > >> If I understand things correctly, the GDC resolution is the pipeline > >> output resolution. Why is it configured on pad 0 ? > > We see the GDC output resolution as the input of output system, the sink pad > format is used for output and viewfinder resolutions. The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be the ImgU input, the format configured there should correspond to the format on the connected video node, and should thus be the sensor format. You can then use the crop and compose rectangles on pad 0, along with the format, crop and compose rectangles on the output and viewfinder pads, to configure the device. This should be fixed in the driver, and the documentation should then be updated accordingly. > >>> 2. The ImgU V4L2 subdev cropping should be set by using the > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > >>> target, using the input feeder height and width. > >>> > >>> 3. The ImgU V4L2 subdev composing should be set by using the > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > >>> target, using the BDS height and width. > >>> > >>> Once these 2 steps are done, the raw bayer frames can be input to the > >>> ImgU V4L2 subdev for processing. > >> > >> Do I need to capture from both the output and viewfinder nodes ? How are > >> they related to the IF -> BDS -> GDC pipeline, are they both fed from the > >> GDC output ? If so, how does the viewfinder scaler fit in that picture ? > > The output capture should be set, the viewfinder can be disabled. > The IF and BDS are seen as crop and compose of the imgu input video > device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > pads. The GDC is the last block in the pipeline according to the information provided above. How can it be seen as the subdev sink pad ? That doesn't make sense to me. I'm not asking for the MC graph to expose all internal blocks of the ImgU, but if you want to retain a single subdev model, the format on the sink pad needs to correspond to what is provided to the ImgU. Please see figure 4.6 of https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for more information regarding how you can use the sink crop, sink compose and source crop rectangles. > >> I have tried the above configuration with the IPU3 v8 driver, and while > >> the kernel doesn't crash, no images get processed. The userspace > >> processes wait forever for buffers to be ready. I then configured pad 2 > >> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ > >> > >> There's one problem though: during capture, or very soon after it, the > >> machine locks up completely. I suspect a memory corruption, as when it > >> doesn't log immediately commands such as dmesg will not produce any > >> output and just block, until the system freezes soon after (especially > >> when moving the mouse). > >> > >> I would still call this an improvement to some extent, but there's > >> definitely room for more improvements :-) > >> > >> To reproduce the issue, you can run the ipu3-process.sh script (attached > >> to this e-mail) with the following arguments: > >> > >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > >> > >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in > >> the IPU3-specific Bayer format (for a total of 6469632 bytes). > > > > I managed to get the dmesg output, and it doesn't look pretty. > > > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > mac80211 > > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 > > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device > > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev > > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core > > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > > drm_panel_orientation_quirks i2c_hid hid > > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > > 4.20.0-rc6+ #2 > > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 > > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > > 000000000000000c > > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > > 00000000ffffffff > > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > > ffff8f5cfaba16f0 > > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > > ffff8f5cf58f0028 > > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > > ffff8f5cf58f04e8 > > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) > > knlGS: > > 0000000000000000 > > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > > 00000000003606e0 > > [ 571.217301] Call Trace: > > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > > [ 571.217480] vfs_ioctl+0x1e/0x2b > > [ 571.217486] do_vfs_ioctl+0x531/0x559 > > [ 571.217494] ? vfs_write+0xd1/0xdf > > [ 571.217500] ksys_ioctl+0x50/0x70 > > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > > [ 571.217512] do_syscall_64+0x53/0x60 > > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > > 0000000000000010 > > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > > 00007f85cf9b9f47 > > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > > 0000000000000003 > > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > > 00007f85d009c700 > > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > > 000055f4c4dc0b06 > > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > > 00007ffc59057825 > > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout
On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > Hello Bingbu, > > On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: >> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: >>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: >>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: >>>> >>>> [snip] >>>> >>>>> I can see a couple of steps missing in the script below. >>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000 >>>>> 040.html) >>>>> >>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 >>>>> documentation", under section "Configuring ImgU V4L2 subdev for image >>>>> processing"... >>>>> >>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. >>>>> >>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as >>>>> desired (e.g 0 for video mode or 1 for still mode) through the control >>>>> id 0x009819a1 as below. >>>>> >>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 >>>> I assume the control takes a valid default value ? It's better to set it >>>> explicitly anyway, so I'll do so. >> The video mode is set by default. If you want to set to still mode or change >> mode, you need set the subdev control. >> >>>>> 2. ImgU pipeline needs to be configured for image processing as below. >>>>> >>>>> RAW bayer frames go through the following ISP pipeline HW blocks to >>>>> have the processed image output to the DDR memory. >>>>> >>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> >>>>> Geometric Distortion Correction (GDC) -> DDR >>>>> >>>>> The ImgU V4L2 subdev has to be configured with the supported >>>>> resolutions in all the above HW blocks, for a given input resolution. >>>>> >>>>> For a given supported resolution for an input frame, the Input Feeder, >>>>> Bayer Down Scaling and GDC blocks should be configured with the >>>>> supported resolutions. This information can be obtained by looking at >>>>> the following IPU3 ISP configuration table for ov5670 sensor. >>>>> >>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/m >>>>> aster/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/ >>>>> gcss/graph_settings_ov5670.xml >>>>> >>>>> For the ov5670 example, for an input frame with a resolution of >>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding >>>>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and >>>>> 2560x1920 respectively. >>>> How is the GDC output resolution computed from the input resolution ? >>>> Does the GDC always consume 32 columns and 22 lines ? >> All the intermediate resolutions in the pipeline are determined by the >> actual use case, in other word determined by the IMGU input >> resolution(sensor output) and the final output and viewfinder resolution. >> BDS mainly do Bayer downscaling, it has limitation that the downscaling >> factor must be a value a integer multiple of 1/32. >> GDC output depends on the input and width should be x8 and height x4 >> alignment. > Thank you for the information. This will need to be captured in the > documentation, along with information related to how each block in the > hardware pipeline interacts with the image size. It should be possible for a > developer to compute the output and viewfinder resolutions based on the > parameters of the image processing algorithms just with the information > contained in the driver documentation. > >>>>> The following steps prepare the ImgU ISP pipeline for the image >>>>> processing. >>>>> >>>>> 1. The ImgU V4L2 subdev data format should be set by using the >>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained >>>>> above. >>>> If I understand things correctly, the GDC resolution is the pipeline >>>> output resolution. Why is it configured on pad 0 ? >> We see the GDC output resolution as the input of output system, the sink pad >> format is used for output and viewfinder resolutions. > The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be the > ImgU input, the format configured there should correspond to the format on the > connected video node, and should thus be the sensor format. You can then use > the crop and compose rectangles on pad 0, along with the format, crop and > compose rectangles on the output and viewfinder pads, to configure the device. > This should be fixed in the driver, and the documentation should then be > updated accordingly. Ack. > >>>>> 2. The ImgU V4L2 subdev cropping should be set by using the >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the >>>>> target, using the input feeder height and width. >>>>> >>>>> 3. The ImgU V4L2 subdev composing should be set by using the >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>>>> target, using the BDS height and width. >>>>> >>>>> Once these 2 steps are done, the raw bayer frames can be input to the >>>>> ImgU V4L2 subdev for processing. >>>> Do I need to capture from both the output and viewfinder nodes ? How are >>>> they related to the IF -> BDS -> GDC pipeline, are they both fed from the >>>> GDC output ? If so, how does the viewfinder scaler fit in that picture ? >> The output capture should be set, the viewfinder can be disabled. >> The IF and BDS are seen as crop and compose of the imgu input video >> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source >> pads. > The GDC is the last block in the pipeline according to the information > provided above. How can it be seen as the subdev sink pad ? That doesn't make > sense to me. I'm not asking for the MC graph to expose all internal blocks of > the ImgU, but if you want to retain a single subdev model, the format on the > sink pad needs to correspond to what is provided to the ImgU. Please see > figure 4.6 of https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for more information regarding how you can use the sink crop, sink > compose and source crop rectangles. Ack, thanks! > >>>> I have tried the above configuration with the IPU3 v8 driver, and while >>>> the kernel doesn't crash, no images get processed. The userspace >>>> processes wait forever for buffers to be ready. I then configured pad 2 >>>> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ >>>> >>>> There's one problem though: during capture, or very soon after it, the >>>> machine locks up completely. I suspect a memory corruption, as when it >>>> doesn't log immediately commands such as dmesg will not produce any >>>> output and just block, until the system freezes soon after (especially >>>> when moving the mouse). >>>> >>>> I would still call this an improvement to some extent, but there's >>>> definitely room for more improvements :-) >>>> >>>> To reproduce the issue, you can run the ipu3-process.sh script (attached >>>> to this e-mail) with the following arguments: >>>> >>>> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 >>>> >>>> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in >>>> the IPU3-specific Bayer format (for a total of 6469632 bytes). >>> I managed to get the dmesg output, and it doesn't look pretty. >>> >>> [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ >>> libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 >>> ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] >>> [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm >>> mac80211 >>> iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 >>> 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg >>> videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device >>> intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev >>> at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core >>> int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid >>> mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea >>> sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm >>> drm_panel_orientation_quirks i2c_hid hid >>> [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C >>> 4.20.0-rc6+ #2 >>> [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 >>> [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] >>> [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 >>> 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 >>> <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 >>> [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 >>> [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: >>> 000000000000000c >>> [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: >>> 00000000ffffffff >>> [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: >>> ffff8f5cfaba16f0 >>> [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: >>> ffff8f5cf58f0028 >>> [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: >>> ffff8f5cf58f04e8 >>> [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) >>> knlGS: >>> 0000000000000000 >>> [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>> [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: >>> 00000000003606e0 >>> [ 571.217301] Call Trace: >>> [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] >>> [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] >>> [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] >>> [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] >>> [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] >>> [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] >>> [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] >>> [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] >>> [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] >>> [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] >>> [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] >>> [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] >>> [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] >>> [ 571.217480] vfs_ioctl+0x1e/0x2b >>> [ 571.217486] do_vfs_ioctl+0x531/0x559 >>> [ 571.217494] ? vfs_write+0xd1/0xdf >>> [ 571.217500] ksys_ioctl+0x50/0x70 >>> [ 571.217506] __x64_sys_ioctl+0x16/0x19 >>> [ 571.217512] do_syscall_64+0x53/0x60 >>> [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 >>> [ 571.217524] RIP: 0033:0x7f85cf9b9f47 >>> [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 >>> c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 >>> <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 >>> [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: >>> 0000000000000010 >>> [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: >>> 00007f85cf9b9f47 >>> [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: >>> 0000000000000003 >>> [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: >>> 00007f85d009c700 >>> [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: >>> 000055f4c4dc0b06 >>> [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: >>> 00007ffc59057825 >>> [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- >>> [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout
Hello Yong, Could you please have a look at the crash reported below ? On Tuesday, 11 December 2018 16:20:43 EET Laurent Pinchart wrote: > On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > >> > >> [snip] > >> > >>> I can see a couple of steps missing in the script below. > >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 > >>> 00040.html) > >>> > >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > >>> documentation", > >>> under section "Configuring ImgU V4L2 subdev for image processing"... > >>> > >>> 1. The pipe mode needs to be configured for the V4L2 subdev. > >>> > >>> Also the pipe mode of the corresponding V4L2 subdev should be set as > >>> desired (e.g 0 for video mode or 1 for still mode) through the control > >>> id 0x009819a1 as below. > >>> > >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > >> > >> I assume the control takes a valid default value ? It's better to set it > >> explicitly anyway, so I'll do so. > >> > >>> 2. ImgU pipeline needs to be configured for image processing as below. > >>> > >>> RAW bayer frames go through the following ISP pipeline HW blocks to > >>> have the processed image output to the DDR memory. > >>> > >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > >>> Geometric Distortion Correction (GDC) -> DDR > >>> > >>> The ImgU V4L2 subdev has to be configured with the supported > >>> resolutions in all the above HW blocks, for a given input resolution. > >>> > >>> For a given supported resolution for an input frame, the Input Feeder, > >>> Bayer Down Scaling and GDC blocks should be configured with the > >>> supported resolutions. This information can be obtained by looking at > >>> the following IPU3 ISP configuration table for ov5670 sensor. > >>> > >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > >>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/ > >>> gcss/graph_settings_ov5670.xml > >>> > >>> For the ov5670 example, for an input frame with a resolution of > >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > >>> 2560x1920 respectively. > >> > >> How is the GDC output resolution computed from the input resolution ? > >> Does the GDC always consume 32 columns and 22 lines ? > >> > >>> The following steps prepare the ImgU ISP pipeline for the image > >>> processing. > >>> > >>> 1. The ImgU V4L2 subdev data format should be set by using the > >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > >>> above. > >> > >> If I understand things correctly, the GDC resolution is the pipeline > >> output resolution. Why is it configured on pad 0 ? > >> > >>> 2. The ImgU V4L2 subdev cropping should be set by using the > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > >>> target, using the input feeder height and width. > >>> > >>> 3. The ImgU V4L2 subdev composing should be set by using the > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > >>> target, using the BDS height and width. > >>> > >>> Once these 2 steps are done, the raw bayer frames can be input to the > >>> ImgU V4L2 subdev for processing. > >> > >> Do I need to capture from both the output and viewfinder nodes ? How are > >> they related to the IF -> BDS -> GDC pipeline, are they both fed from > >> the GDC output ? If so, how does the viewfinder scaler fit in that > >> picture ? > >> > >> I have tried the above configuration with the IPU3 v8 driver, and while > >> the kernel doesn't crash, no images get processed. The userspace > >> processes wait forever for buffers to be ready. I then configured pad 2 > >> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ > >> > >> There's one problem though: during capture, or very soon after it, the > >> machine locks up completely. I suspect a memory corruption, as when it > >> doesn't log immediately commands such as dmesg will not produce any > >> output and just block, until the system freezes soon after (especially > >> when moving the mouse). > >> > >> I would still call this an improvement to some extent, but there's > >> definitely room for more improvements :-) > >> > >> To reproduce the issue, you can run the ipu3-process.sh script (attached > >> to this e-mail) with the following arguments: > >> > >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > > This should have read > > $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > Without the --vf argument no images are processed. > > It seems that the Intel mail server blocked the mail that contained the > script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in > >> the IPU3-specific Bayer format (for a total of 6469632 bytes). > > > > I managed to get the dmesg output, and it doesn't look pretty. > > > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > mac80211 > > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 > > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device > > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev > > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core > > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > > drm_panel_orientation_quirks i2c_hid hid > > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > > 4.20.0-rc6+ #2 > > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 > > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > > 000000000000000c > > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > > 00000000ffffffff > > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > > ffff8f5cfaba16f0 > > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > > ffff8f5cf58f0028 > > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > > ffff8f5cf58f04e8 > > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) > > knlGS: > > 0000000000000000 > > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > > 00000000003606e0 > > [ 571.217301] Call Trace: > > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > > [ 571.217480] vfs_ioctl+0x1e/0x2b > > [ 571.217486] do_vfs_ioctl+0x531/0x559 > > [ 571.217494] ? vfs_write+0xd1/0xdf > > [ 571.217500] ksys_ioctl+0x50/0x70 > > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > > [ 571.217512] do_syscall_64+0x53/0x60 > > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > > 0000000000000010 > > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > > 00007f85cf9b9f47 > > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > > 0000000000000003 > > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > > 00007f85d009c700 > > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > > 000055f4c4dc0b06 > > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > > 00007ffc59057825 > > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > And after fixing another issue in the capture script (which was setting the > format on the ImgU subdev pad 3 to 2560x1920 but capture in 1920x1080), I > now get plenty of the following messages: > > [ 221.366131] BUG: Bad page state in process yavta pfn:14a4ff > [ 221.366134] page:ffffde5d45293fc0 count:-1 mapcount:0 mapping: > 0000000000000000 index:0x0 > [ 221.366137] flags: 0x200000000000000() > [ 221.366140] raw: 0200000000000000 dead000000000100 dead000000000200 > 0000000000000000 > [ 221.366143] raw: 0000000000000000 0000000000000000 ffffffffffffffff > 0000000000000000 > [ 221.366145] page dumped because: nonzero _refcount > [ 221.366147] Modules linked in: asix usbnet mii zram arc4 iwlmvm > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp mac80211 iwlwifi > cfg80211 hid_multitouch 8250_dw ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > videobuf2_memops videobuf2_v4l2 processor_thermal_device videobuf2_common > intel_soc_dts_iosf ov13858 ov5670 dw9714 v4l2_fwnode v4l2_common videodev > media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone > chromeos_pstore mac_hid int3400_thermal acpi_thermal_rel autofs4 usbhid > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > sysfillrect sysimgblt fb_sys_fops sdhci_pci cqhci sdhci drm > drm_panel_orientation_quirks i2c_hid hid > [ 221.366172] CPU: 3 PID: 1022 Comm: yavta Tainted: G B WC > 4.20.0-rc6+ #2 > [ 221.366173] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 221.366173] Call Trace: > [ 221.366176] dump_stack+0x46/0x59 > [ 221.366179] bad_page+0xf2/0x10c > [ 221.366182] free_pages_check+0x78/0x81 > [ 221.366186] free_pcppages_bulk+0xa6/0x236 > [ 221.366190] free_unref_page+0x4b/0x53 > [ 221.366193] vb2_dma_sg_put+0x95/0xb5 [videobuf2_dma_sg] > [ 221.366197] __vb2_buf_mem_free+0x3a/0x6e [videobuf2_common] > [ 221.366202] __vb2_queue_free+0xe3/0x1be [videobuf2_common] > [ 221.366207] vb2_core_reqbufs+0xe9/0x2cc [videobuf2_common] > [ 221.366212] vb2_ioctl_reqbufs+0x78/0x9e [videobuf2_v4l2] > [ 221.366220] __video_do_ioctl+0x258/0x38e [videodev] > [ 221.366229] video_usercopy+0x25f/0x4e5 [videodev] > [ 221.366237] ? copy_overflow+0x14/0x14 [videodev] > [ 221.366240] ? unmap_region+0xe0/0x10a > [ 221.366250] v4l2_ioctl+0x4d/0x58 [videodev] > [ 221.366253] vfs_ioctl+0x1e/0x2b > [ 221.366255] do_vfs_ioctl+0x531/0x559 > [ 221.366260] ksys_ioctl+0x50/0x70 > [ 221.366263] __x64_sys_ioctl+0x16/0x19 > [ 221.366266] do_syscall_64+0x53/0x60 > [ 221.366269] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 221.366270] RIP: 0033:0x7fbe39f6af47 > [ 221.366272] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 221.366273] RSP: 002b:00007fff05638e68 EFLAGS: 00000246 ORIG_RAX: > 0000000000000010 > [ 221.366275] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > 00007fbe39f6af47 > [ 221.366279] RDX: 00007fff05638f90 RSI: 00000000c0145608 RDI: > 0000000000000003 > [ 221.366283] RBP: 0000000000000004 R08: 0000000000000000 R09: > 0000000000000045 > [ 221.366287] R10: 0000000000000557 R11: 0000000000000246 R12: > 000055c83bd76750 > [ 221.366290] R13: 000055c83b6b26a0 R14: 0000000000000001 R15: > 00007fff0563a825
On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > Hello Bingbu, > > On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: >> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: >>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: >>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: >>>> >>>> [snip] >>>> >>>>> I can see a couple of steps missing in the script below. >>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/000 >>>>> 040.html) >>>>> >>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 >>>>> documentation", under section "Configuring ImgU V4L2 subdev for image >>>>> processing"... >>>>> >>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. >>>>> >>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as >>>>> desired (e.g 0 for video mode or 1 for still mode) through the control >>>>> id 0x009819a1 as below. >>>>> >>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 >>>> I assume the control takes a valid default value ? It's better to set it >>>> explicitly anyway, so I'll do so. >> The video mode is set by default. If you want to set to still mode or change >> mode, you need set the subdev control. >> >>>>> 2. ImgU pipeline needs to be configured for image processing as below. >>>>> >>>>> RAW bayer frames go through the following ISP pipeline HW blocks to >>>>> have the processed image output to the DDR memory. >>>>> >>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> >>>>> Geometric Distortion Correction (GDC) -> DDR >>>>> >>>>> The ImgU V4L2 subdev has to be configured with the supported >>>>> resolutions in all the above HW blocks, for a given input resolution. >>>>> >>>>> For a given supported resolution for an input frame, the Input Feeder, >>>>> Bayer Down Scaling and GDC blocks should be configured with the >>>>> supported resolutions. This information can be obtained by looking at >>>>> the following IPU3 ISP configuration table for ov5670 sensor. >>>>> >>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/m >>>>> aster/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/ >>>>> gcss/graph_settings_ov5670.xml >>>>> >>>>> For the ov5670 example, for an input frame with a resolution of >>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding >>>>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and >>>>> 2560x1920 respectively. >>>> How is the GDC output resolution computed from the input resolution ? >>>> Does the GDC always consume 32 columns and 22 lines ? >> All the intermediate resolutions in the pipeline are determined by the >> actual use case, in other word determined by the IMGU input >> resolution(sensor output) and the final output and viewfinder resolution. >> BDS mainly do Bayer downscaling, it has limitation that the downscaling >> factor must be a value a integer multiple of 1/32. >> GDC output depends on the input and width should be x8 and height x4 >> alignment. > Thank you for the information. This will need to be captured in the > documentation, along with information related to how each block in the > hardware pipeline interacts with the image size. It should be possible for a > developer to compute the output and viewfinder resolutions based on the > parameters of the image processing algorithms just with the information > contained in the driver documentation. > >>>>> The following steps prepare the ImgU ISP pipeline for the image >>>>> processing. >>>>> >>>>> 1. The ImgU V4L2 subdev data format should be set by using the >>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained >>>>> above. >>>> If I understand things correctly, the GDC resolution is the pipeline >>>> output resolution. Why is it configured on pad 0 ? >> We see the GDC output resolution as the input of output system, the sink pad >> format is used for output and viewfinder resolutions. > The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be the > ImgU input, the format configured there should correspond to the format on the > connected video node, and should thus be the sensor format. You can then use > the crop and compose rectangles on pad 0, along with the format, crop and > compose rectangles on the output and viewfinder pads, to configure the device. > This should be fixed in the driver, and the documentation should then be > updated accordingly. Hi, Laurent, Thanks for your review. I think it make sense for me that using Pad 0 as the ImgU input(IF). However, I prefer using the 2 source pads for output and viewfinder. It makes more sense because the output and viewfinder are independent output. The whole pipeline in ImgU looks like: IF --> BDS --> GDC ---> OUTPUT Â Â Â Â Â Â Â Â Â Â Â Â Â | Â Â Â Â Â Â Â Â Â Â Â Â |-----> VF The BDS is used to do Bayer downscaling and GDC can do cropping. My understanding is that scaled size is configured on the CROP rectangle by COMPOSE selection target, the order seems like not aligned with the actual processing in ImgU if we set the crop/compose on sink pad. Is there some rules for the order of the configuration in the subdev API? Could I use crop selection based on the scaled size? >>>>> 2. The ImgU V4L2 subdev cropping should be set by using the >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the >>>>> target, using the input feeder height and width. >>>>> >>>>> 3. The ImgU V4L2 subdev composing should be set by using the >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>>>> target, using the BDS height and width. >>>>> >>>>> Once these 2 steps are done, the raw bayer frames can be input to the >>>>> ImgU V4L2 subdev for processing. >>>> Do I need to capture from both the output and viewfinder nodes ? How are >>>> they related to the IF -> BDS -> GDC pipeline, are they both fed from the >>>> GDC output ? If so, how does the viewfinder scaler fit in that picture ? >> The output capture should be set, the viewfinder can be disabled. >> The IF and BDS are seen as crop and compose of the imgu input video >> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source >> pads. > The GDC is the last block in the pipeline according to the information > provided above. How can it be seen as the subdev sink pad ? That doesn't make > sense to me. I'm not asking for the MC graph to expose all internal blocks of > the ImgU, but if you want to retain a single subdev model, the format on the > sink pad needs to correspond to what is provided to the ImgU. Please see > figure 4.6 of https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for more information regarding how you can use the sink crop, sink > compose and source crop rectangles. > >>>> I have tried the above configuration with the IPU3 v8 driver, and while >>>> the kernel doesn't crash, no images get processed. The userspace >>>> processes wait forever for buffers to be ready. I then configured pad 2 >>>> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ >>>> >>>> There's one problem though: during capture, or very soon after it, the >>>> machine locks up completely. I suspect a memory corruption, as when it >>>> doesn't log immediately commands such as dmesg will not produce any >>>> output and just block, until the system freezes soon after (especially >>>> when moving the mouse). >>>> >>>> I would still call this an improvement to some extent, but there's >>>> definitely room for more improvements :-) >>>> >>>> To reproduce the issue, you can run the ipu3-process.sh script (attached >>>> to this e-mail) with the following arguments: >>>> >>>> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 >>>> >>>> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in >>>> the IPU3-specific Bayer format (for a total of 6469632 bytes). >>> I managed to get the dmesg output, and it doesn't look pretty. >>> >>> [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ >>> libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 >>> ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] >>> [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm >>> mac80211 >>> iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp cfg80211 >>> 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg >>> videobuf2_memops videobuf2_v4l2 videobuf2_common processor_thermal_device >>> intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common videodev >>> at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs cros_ec_core >>> int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid >>> mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea >>> sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm >>> drm_panel_orientation_quirks i2c_hid hid >>> [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C >>> 4.20.0-rc6+ #2 >>> [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 >>> [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] >>> [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc f3 >>> 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 >>> <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 >>> [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 >>> [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: >>> 000000000000000c >>> [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: >>> 00000000ffffffff >>> [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: >>> ffff8f5cfaba16f0 >>> [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: >>> ffff8f5cf58f0028 >>> [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: >>> ffff8f5cf58f04e8 >>> [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) >>> knlGS: >>> 0000000000000000 >>> [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >>> [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: >>> 00000000003606e0 >>> [ 571.217301] Call Trace: >>> [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] >>> [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] >>> [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] >>> [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] >>> [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] >>> [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] >>> [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] >>> [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] >>> [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] >>> [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] >>> [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] >>> [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] >>> [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] >>> [ 571.217480] vfs_ioctl+0x1e/0x2b >>> [ 571.217486] do_vfs_ioctl+0x531/0x559 >>> [ 571.217494] ? vfs_write+0xd1/0xdf >>> [ 571.217500] ksys_ioctl+0x50/0x70 >>> [ 571.217506] __x64_sys_ioctl+0x16/0x19 >>> [ 571.217512] do_syscall_64+0x53/0x60 >>> [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 >>> [ 571.217524] RIP: 0033:0x7f85cf9b9f47 >>> [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 >>> c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 >>> <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 >>> [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: >>> 0000000000000010 >>> [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: >>> 00007f85cf9b9f47 >>> [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: >>> 0000000000000003 >>> [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: >>> 00007f85d009c700 >>> [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: >>> 000055f4c4dc0b06 >>> [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: >>> 00007ffc59057825 >>> [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- >>> [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout
Hellon On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > Hello Yong, > > Could you please have a look at the crash reported below ? A bit more information to help you debugging this. I've enabled KASAN in the kernel configuration, and get the following use-after-free reports. [ 166.332920] ================================================================== [ 166.332937] BUG: KASAN: use-after-free in __cached_rbnode_delete_update+0x36/0x202 [ 166.332944] Read of size 8 at addr ffff888133823718 by task yavta/1305 [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C 4.20.0-rc6+ #3 [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ 166.332959] Call Trace: [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] print_address_description+0x65/0x227 [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] __cached_rbnode_delete_update+0x36/0x202 [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] __free_iova+0x23/0x31 [ 166.333011] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ 166.333022] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.333032] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.333043] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.333056] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333067] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.333079] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333088] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333096] ? __mutex_lock_interruptible_slowpath+0xf/0xf [ 166.333104] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.333123] __video_do_ioctl+0x625/0x887 [videodev] [ 166.333142] ? copy_overflow+0x14/0x14 [videodev] [ 166.333147] ? slab_free_freelist_hook+0x46/0x94 [ 166.333151] ? kfree+0x107/0x1a0 [ 166.333169] video_usercopy+0x3a3/0x8ae [videodev] [ 166.333187] ? copy_overflow+0x14/0x14 [videodev] [ 166.333203] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.333207] ? __wake_up_common+0x342/0x342 [ 166.333215] ? atomic_long_add_return+0x15/0x24 [ 166.333219] ? ldsem_up_read+0x15/0x29 [ 166.333223] ? tty_write+0x4c6/0x4d8 [ 166.333227] ? n_tty_receive_char_special+0x1152/0x1152 [ 166.333244] ? video_usercopy+0x8ae/0x8ae [videodev] [ 166.333260] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333266] vfs_ioctl+0x76/0x89 [ 166.333271] do_vfs_ioctl+0xb33/0xb7e [ 166.333275] ? __switch_to_asm+0x40/0x70 [ 166.333279] ? __switch_to_asm+0x40/0x70 [ 166.333282] ? __switch_to_asm+0x34/0x70 [ 166.333286] ? __switch_to_asm+0x40/0x70 [ 166.333290] ? ioctl_preallocate+0x174/0x174 [ 166.333294] ? __switch_to+0x71c/0xb00 [ 166.333299] ? compat_start_thread+0x6b/0x6b [ 166.333302] ? __switch_to_asm+0x34/0x70 [ 166.333305] ? __switch_to_asm+0x40/0x70 [ 166.333309] ? mmdrop+0x12/0x23 [ 166.333313] ? finish_task_switch+0x34d/0x3de [ 166.333319] ? __schedule+0x1004/0x1045 [ 166.333325] ? firmware_map_remove+0x119/0x119 [ 166.333330] ksys_ioctl+0x50/0x70 [ 166.333335] __x64_sys_ioctl+0x82/0x89 [ 166.333340] do_syscall_64+0xa0/0xd2 [ 166.333345] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.333349] RIP: 0033:0x7f2481541f47 [ 166.333354] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 166.333357] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 166.333362] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 [ 166.333364] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ 166.333367] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 [ 166.333369] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 [ 166.333372] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 [ 166.333383] Allocated by task 1305: [ 166.333389] kasan_kmalloc+0x8a/0x98 [ 166.333392] slab_post_alloc_hook+0x31/0x51 [ 166.333396] kmem_cache_alloc+0xd7/0x174 [ 166.333399] alloc_iova+0x24/0x2ea [ 166.333407] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] [ 166.333415] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] [ 166.333424] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] [ 166.333433] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] [ 166.333442] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] [ 166.333449] vb2_start_streaming+0x164/0x33b [videobuf2_common] [ 166.333455] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] [ 166.333471] __video_do_ioctl+0x625/0x887 [videodev] [ 166.333487] video_usercopy+0x3a3/0x8ae [videodev] [ 166.333501] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333505] vfs_ioctl+0x76/0x89 [ 166.333508] do_vfs_ioctl+0xb33/0xb7e [ 166.333511] ksys_ioctl+0x50/0x70 [ 166.333514] __x64_sys_ioctl+0x82/0x89 [ 166.333518] do_syscall_64+0xa0/0xd2 [ 166.333521] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.333526] Freed by task 1301: [ 166.333532] __kasan_slab_free+0xfa/0x11c [ 166.333535] slab_free_freelist_hook+0x46/0x94 [ 166.333538] kmem_cache_free+0x7b/0x172 [ 166.333542] __free_iova+0x23/0x31 [ 166.333550] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ 166.333557] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.333566] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.333574] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.333584] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333593] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333599] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333606] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.333621] __video_do_ioctl+0x625/0x887 [videodev] [ 166.333637] video_usercopy+0x3a3/0x8ae [videodev] [ 166.333652] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333655] vfs_ioctl+0x76/0x89 [ 166.333658] do_vfs_ioctl+0xb33/0xb7e [ 166.333662] ksys_ioctl+0x50/0x70 [ 166.333665] __x64_sys_ioctl+0x82/0x89 [ 166.333668] do_syscall_64+0xa0/0xd2 [ 166.333671] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.333678] The buggy address belongs to the object at ffff888133823700 which belongs to the cache iommu_iova of size 40 [ 166.333685] The buggy address is located 24 bytes inside of 40-byte region [ffff888133823700, ffff888133823728) [ 166.333690] The buggy address belongs to the page: [ 166.333696] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 [ 166.333703] flags: 0x200000000010200(slab|head) [ 166.333710] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 [ 166.333717] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 [ 166.333720] page dumped because: kasan: bad access detected [ 166.333726] Memory state around the buggy address: [ 166.333732] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.333737] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.333742] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc [ 166.333745] ^ [ 166.333750] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.333755] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.333759] ================================================================== [ 166.333762] Disabling lock debugging due to kernel taint [ 166.333764] ================================================================== [ 166.333770] BUG: KASAN: double-free or invalid-free in kmem_cache_free+0x7b/0x172 [ 166.333780] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 [ 166.333782] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ 166.333783] Call Trace: [ 166.333789] dump_stack+0x5b/0x81 [ 166.333795] print_address_description+0x65/0x227 [ 166.333799] ? kmem_cache_free+0x7b/0x172 [ 166.333803] kasan_report_invalid_free+0x67/0xa0 [ 166.333807] ? kmem_cache_free+0x7b/0x172 [ 166.333812] __kasan_slab_free+0x86/0x11c [ 166.333817] slab_free_freelist_hook+0x46/0x94 [ 166.333822] kmem_cache_free+0x7b/0x172 [ 166.333826] ? __free_iova+0x23/0x31 [ 166.333831] __free_iova+0x23/0x31 [ 166.333840] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ 166.333851] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.333861] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.333872] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.333885] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333896] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.333908] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333917] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333923] ? __mutex_lock_interruptible_slowpath+0xf/0xf [ 166.333932] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.333950] __video_do_ioctl+0x625/0x887 [videodev] [ 166.333970] ? copy_overflow+0x14/0x14 [videodev] [ 166.333974] ? slab_free_freelist_hook+0x46/0x94 [ 166.333979] ? kfree+0x107/0x1a0 [ 166.333997] video_usercopy+0x3a3/0x8ae [videodev] [ 166.334015] ? copy_overflow+0x14/0x14 [videodev] [ 166.334031] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.334035] ? __wake_up_common+0x342/0x342 [ 166.334042] ? atomic_long_add_return+0x15/0x24 [ 166.334046] ? ldsem_up_read+0x15/0x29 [ 166.334050] ? tty_write+0x4c6/0x4d8 [ 166.334054] ? n_tty_receive_char_special+0x1152/0x1152 [ 166.334071] ? video_usercopy+0x8ae/0x8ae [videodev] [ 166.334087] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.334092] vfs_ioctl+0x76/0x89 [ 166.334097] do_vfs_ioctl+0xb33/0xb7e [ 166.334101] ? __switch_to_asm+0x40/0x70 [ 166.334105] ? __switch_to_asm+0x40/0x70 [ 166.334108] ? __switch_to_asm+0x34/0x70 [ 166.334111] ? __switch_to_asm+0x40/0x70 [ 166.334116] ? ioctl_preallocate+0x174/0x174 [ 166.334120] ? __switch_to+0x71c/0xb00 [ 166.334124] ? compat_start_thread+0x6b/0x6b [ 166.334127] ? __switch_to_asm+0x34/0x70 [ 166.334130] ? __switch_to_asm+0x40/0x70 [ 166.334134] ? mmdrop+0x12/0x23 [ 166.334137] ? finish_task_switch+0x34d/0x3de [ 166.334143] ? __schedule+0x1004/0x1045 [ 166.334148] ? firmware_map_remove+0x119/0x119 [ 166.334153] ksys_ioctl+0x50/0x70 [ 166.334158] __x64_sys_ioctl+0x82/0x89 [ 166.334163] do_syscall_64+0xa0/0xd2 [ 166.334167] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.334171] RIP: 0033:0x7f2481541f47 [ 166.334175] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 166.334177] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 166.334181] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 [ 166.334184] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ 166.334186] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 [ 166.334189] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 [ 166.334191] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 [ 166.334201] Allocated by task 1305: [ 166.334207] kasan_kmalloc+0x8a/0x98 [ 166.334210] slab_post_alloc_hook+0x31/0x51 [ 166.334213] kmem_cache_alloc+0xd7/0x174 [ 166.334216] alloc_iova+0x24/0x2ea [ 166.334225] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] [ 166.334233] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] [ 166.334241] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] [ 166.334250] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] [ 166.334259] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] [ 166.334266] vb2_start_streaming+0x164/0x33b [videobuf2_common] [ 166.334273] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] [ 166.334288] __video_do_ioctl+0x625/0x887 [videodev] [ 166.334304] video_usercopy+0x3a3/0x8ae [videodev] [ 166.334319] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.334322] vfs_ioctl+0x76/0x89 [ 166.334325] do_vfs_ioctl+0xb33/0xb7e [ 166.334328] ksys_ioctl+0x50/0x70 [ 166.334332] __x64_sys_ioctl+0x82/0x89 [ 166.334335] do_syscall_64+0xa0/0xd2 [ 166.334338] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.334343] Freed by task 1301: [ 166.334349] __kasan_slab_free+0xfa/0x11c [ 166.334352] slab_free_freelist_hook+0x46/0x94 [ 166.334355] kmem_cache_free+0x7b/0x172 [ 166.334359] __free_iova+0x23/0x31 [ 166.334367] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ 166.334375] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.334383] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.334392] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.334401] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.334410] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.334416] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.334423] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.334438] __video_do_ioctl+0x625/0x887 [videodev] [ 166.334454] video_usercopy+0x3a3/0x8ae [videodev] [ 166.334469] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.334472] vfs_ioctl+0x76/0x89 [ 166.334475] do_vfs_ioctl+0xb33/0xb7e [ 166.334479] ksys_ioctl+0x50/0x70 [ 166.334482] __x64_sys_ioctl+0x82/0x89 [ 166.334485] do_syscall_64+0xa0/0xd2 [ 166.334488] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.334494] The buggy address belongs to the object at ffff888133823700 which belongs to the cache iommu_iova of size 40 [ 166.334501] The buggy address is located 0 bytes inside of 40-byte region [ffff888133823700, ffff888133823728) [ 166.334506] The buggy address belongs to the page: [ 166.334511] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 [ 166.334517] flags: 0x200000000010200(slab|head) [ 166.334524] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 [ 166.334530] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 [ 166.334533] page dumped because: kasan: bad access detected [ 166.334539] Memory state around the buggy address: [ 166.334544] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.334549] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.334554] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc [ 166.334558] ^ [ 166.334562] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.334567] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.334571] ================================================================== [ 166.340377] ================================================================== [ 166.340388] BUG: KASAN: double-free or invalid-free in kfree+0x107/0x1a0 [ 166.340399] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 [ 166.340401] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ 166.340403] Call Trace: [ 166.340410] dump_stack+0x5b/0x81 [ 166.340416] print_address_description+0x65/0x227 [ 166.340420] ? kfree+0x107/0x1a0 [ 166.340425] kasan_report_invalid_free+0x67/0xa0 [ 166.340428] ? kfree+0x107/0x1a0 [ 166.340433] __kasan_slab_free+0x86/0x11c [ 166.340438] slab_free_freelist_hook+0x46/0x94 [ 166.340443] kfree+0x107/0x1a0 [ 166.340454] ? ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] [ 166.340464] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] [ 166.340475] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.340485] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.340495] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.340509] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.340520] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.340531] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.340541] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.340548] ? __mutex_lock_interruptible_slowpath+0xf/0xf [ 166.340557] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.340575] __video_do_ioctl+0x625/0x887 [videodev] [ 166.340595] ? copy_overflow+0x14/0x14 [videodev] [ 166.340600] ? slab_free_freelist_hook+0x46/0x94 [ 166.340604] ? kfree+0x107/0x1a0 [ 166.340622] video_usercopy+0x3a3/0x8ae [videodev] [ 166.340640] ? copy_overflow+0x14/0x14 [videodev] [ 166.340657] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.340660] ? __wake_up_common+0x342/0x342 [ 166.340668] ? atomic_long_add_return+0x15/0x24 [ 166.340672] ? ldsem_up_read+0x15/0x29 [ 166.340677] ? tty_write+0x4c6/0x4d8 [ 166.340681] ? n_tty_receive_char_special+0x1152/0x1152 [ 166.340698] ? video_usercopy+0x8ae/0x8ae [videodev] [ 166.340714] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.340720] vfs_ioctl+0x76/0x89 [ 166.340725] do_vfs_ioctl+0xb33/0xb7e [ 166.340729] ? __switch_to_asm+0x40/0x70 [ 166.340733] ? __switch_to_asm+0x40/0x70 [ 166.340736] ? __switch_to_asm+0x34/0x70 [ 166.340739] ? __switch_to_asm+0x40/0x70 [ 166.340743] ? ioctl_preallocate+0x174/0x174 [ 166.340748] ? __switch_to+0x71c/0xb00 [ 166.340752] ? compat_start_thread+0x6b/0x6b [ 166.340756] ? __switch_to_asm+0x34/0x70 [ 166.340759] ? __switch_to_asm+0x40/0x70 [ 166.340762] ? mmdrop+0x12/0x23 [ 166.340766] ? finish_task_switch+0x34d/0x3de [ 166.340772] ? __schedule+0x1004/0x1045 [ 166.340777] ? firmware_map_remove+0x119/0x119 [ 166.340782] ksys_ioctl+0x50/0x70 [ 166.340788] __x64_sys_ioctl+0x82/0x89 [ 166.340793] do_syscall_64+0xa0/0xd2 [ 166.340797] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.340802] RIP: 0033:0x7f2481541f47 [ 166.340806] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 166.340809] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 166.340813] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 [ 166.340816] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ 166.340819] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 [ 166.340821] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 [ 166.340824] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 [ 166.340834] Allocated by task 1305: [ 166.340840] kasan_kmalloc+0x8a/0x98 [ 166.340844] __kmalloc_node+0x193/0x1ba [ 166.340848] kvmalloc_node+0x44/0x6d [ 166.340856] ipu3_dmamap_alloc+0x1c9/0x83f [ipu3_imgu] [ 166.340864] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] [ 166.340873] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] [ 166.340882] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] [ 166.340891] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] [ 166.340897] vb2_start_streaming+0x164/0x33b [videobuf2_common] [ 166.340904] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] [ 166.340920] __video_do_ioctl+0x625/0x887 [videodev] [ 166.340935] video_usercopy+0x3a3/0x8ae [videodev] [ 166.340950] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.340954] vfs_ioctl+0x76/0x89 [ 166.340957] do_vfs_ioctl+0xb33/0xb7e [ 166.340960] ksys_ioctl+0x50/0x70 [ 166.340963] __x64_sys_ioctl+0x82/0x89 [ 166.340966] do_syscall_64+0xa0/0xd2 [ 166.340969] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.340974] Freed by task 1301: [ 166.340980] __kasan_slab_free+0xfa/0x11c [ 166.340983] slab_free_freelist_hook+0x46/0x94 [ 166.340986] kfree+0x107/0x1a0 [ 166.340994] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] [ 166.341002] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.341010] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.341019] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.341028] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.341037] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.341043] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.341050] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.341066] __video_do_ioctl+0x625/0x887 [videodev] [ 166.341081] video_usercopy+0x3a3/0x8ae [videodev] [ 166.341096] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.341100] vfs_ioctl+0x76/0x89 [ 166.341103] do_vfs_ioctl+0xb33/0xb7e [ 166.341106] ksys_ioctl+0x50/0x70 [ 166.341109] __x64_sys_ioctl+0x82/0x89 [ 166.341112] do_syscall_64+0xa0/0xd2 [ 166.341116] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 166.341122] The buggy address belongs to the object at ffff88811d228440 which belongs to the cache kmalloc-8 of size 8 [ 166.341129] The buggy address is located 0 bytes inside of 8-byte region [ffff88811d228440, ffff88811d228448) [ 166.341134] The buggy address belongs to the page: [ 166.341140] page:ffffea0004748a00 count:1 mapcount:0 mapping:ffff88815a80c340 index:0xffff88811d228f80 compound_mapcount: 0 [ 166.341146] flags: 0x200000000010200(slab|head) [ 166.341153] raw: 0200000000010200 ffffea000564b288 ffffea00049ef708 ffff88815a80c340 [ 166.341159] raw: ffff88811d228f80 0000000000160013 00000001ffffffff 0000000000000000 [ 166.341163] page dumped because: kasan: bad access detected [ 166.341169] Memory state around the buggy address: [ 166.341174] ffff88811d228300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.341179] ffff88811d228380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.341184] >ffff88811d228400: fc fc fc fc fc fc fc fc fb fc fc fc fc fc fc fc [ 166.341188] ^ [ 166.341192] ffff88811d228480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.341197] ffff88811d228500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 166.341201] ================================================================== > On Tuesday, 11 December 2018 16:20:43 EET Laurent Pinchart wrote: > > On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > >> > > >> [snip] > > >> > > >>> I can see a couple of steps missing in the script below. > > >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 > > >>> 00040.html) > > >>> > > >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > >>> documentation", > > >>> under section "Configuring ImgU V4L2 subdev for image processing"... > > >>> > > >>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > >>> > > >>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > >>> desired (e.g 0 for video mode or 1 for still mode) through the control > > >>> id 0x009819a1 as below. > > >>> > > >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > >> > > >> I assume the control takes a valid default value ? It's better to set > > >> it > > >> explicitly anyway, so I'll do so. > > >> > > >>> 2. ImgU pipeline needs to be configured for image processing as below. > > >>> > > >>> RAW bayer frames go through the following ISP pipeline HW blocks to > > >>> have the processed image output to the DDR memory. > > >>> > > >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > >>> Geometric Distortion Correction (GDC) -> DDR > > >>> > > >>> The ImgU V4L2 subdev has to be configured with the supported > > >>> resolutions in all the above HW blocks, for a given input resolution. > > >>> > > >>> For a given supported resolution for an input frame, the Input Feeder, > > >>> Bayer Down Scaling and GDC blocks should be configured with the > > >>> supported resolutions. This information can be obtained by looking at > > >>> the following IPU3 ISP configuration table for ov5670 sensor. > > >>> > > >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > > >>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files > > >>> / > > >>> gcss/graph_settings_ov5670.xml > > >>> > > >>> For the ov5670 example, for an input frame with a resolution of > > >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > >>> 2560x1920 respectively. > > >> > > >> How is the GDC output resolution computed from the input resolution ? > > >> Does the GDC always consume 32 columns and 22 lines ? > > >> > > >>> The following steps prepare the ImgU ISP pipeline for the image > > >>> processing. > > >>> > > >>> 1. The ImgU V4L2 subdev data format should be set by using the > > >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > > >>> above. > > >> > > >> If I understand things correctly, the GDC resolution is the pipeline > > >> output resolution. Why is it configured on pad 0 ? > > >> > > >>> 2. The ImgU V4L2 subdev cropping should be set by using the > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > >>> target, using the input feeder height and width. > > >>> > > >>> 3. The ImgU V4L2 subdev composing should be set by using the > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > >>> target, using the BDS height and width. > > >>> > > >>> Once these 2 steps are done, the raw bayer frames can be input to the > > >>> ImgU V4L2 subdev for processing. > > >> > > >> Do I need to capture from both the output and viewfinder nodes ? How > > >> are > > >> they related to the IF -> BDS -> GDC pipeline, are they both fed from > > >> the GDC output ? If so, how does the viewfinder scaler fit in that > > >> picture ? > > >> > > >> I have tried the above configuration with the IPU3 v8 driver, and while > > >> the kernel doesn't crash, no images get processed. The userspace > > >> processes wait forever for buffers to be ready. I then configured pad 2 > > >> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ > > >> > > >> There's one problem though: during capture, or very soon after it, the > > >> machine locks up completely. I suspect a memory corruption, as when it > > >> doesn't log immediately commands such as dmesg will not produce any > > >> output and just block, until the system freezes soon after (especially > > >> when moving the mouse). > > >> > > >> I would still call this an improvement to some extent, but there's > > >> definitely room for more improvements :-) > > >> > > >> To reproduce the issue, you can run the ipu3-process.sh script > > >> (attached > > >> to this e-mail) with the following arguments: > > >> > > >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > > > > This should have read > > > > $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > > > Without the --vf argument no images are processed. > > > > It seems that the Intel mail server blocked the mail that contained the > > script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > > > >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in > > >> the IPU3-specific Bayer format (for a total of 6469632 bytes). > > > > > > I managed to get the dmesg output, and it doesn't look pretty. > > > > > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > > > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > > > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > mac80211 > > > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp > > > cfg80211 > > > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > videobuf2_memops videobuf2_v4l2 videobuf2_common > > > processor_thermal_device > > > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common > > > videodev > > > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs > > > cros_ec_core > > > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > > > drm_panel_orientation_quirks i2c_hid hid > > > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > > > 4.20.0-rc6+ #2 > > > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc > > > f3 > > > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > > > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > > > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > > > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > > > 000000000000000c > > > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > > > 00000000ffffffff > > > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > > > ffff8f5cfaba16f0 > > > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > > > ffff8f5cf58f0028 > > > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > > > ffff8f5cf58f04e8 > > > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) > > > knlGS: > > > 0000000000000000 > > > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > > > 00000000003606e0 > > > [ 571.217301] Call Trace: > > > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > > > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > > > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > > > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > > > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > > > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > > > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > > > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > > > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > > > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > > > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > > > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > > > [ 571.217480] vfs_ioctl+0x1e/0x2b > > > [ 571.217486] do_vfs_ioctl+0x531/0x559 > > > [ 571.217494] ? vfs_write+0xd1/0xdf > > > [ 571.217500] ksys_ioctl+0x50/0x70 > > > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > > > [ 571.217512] do_syscall_64+0x53/0x60 > > > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > > > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 > > > 48 > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > > > 0000000000000010 > > > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > > > 00007f85cf9b9f47 > > > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > > > 0000000000000003 > > > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > > > 00007f85d009c700 > > > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > > > 000055f4c4dc0b06 > > > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > > > 00007ffc59057825 > > > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > > > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > > > And after fixing another issue in the capture script (which was setting > > the > > format on the ImgU subdev pad 3 to 2560x1920 but capture in 1920x1080), I > > now get plenty of the following messages: > > > > [ 221.366131] BUG: Bad page state in process yavta pfn:14a4ff > > [ 221.366134] page:ffffde5d45293fc0 count:-1 mapcount:0 mapping: > > 0000000000000000 index:0x0 > > [ 221.366137] flags: 0x200000000000000() > > [ 221.366140] raw: 0200000000000000 dead000000000100 dead000000000200 > > 0000000000000000 > > [ 221.366143] raw: 0000000000000000 0000000000000000 ffffffffffffffff > > 0000000000000000 > > [ 221.366145] page dumped because: nonzero _refcount > > [ 221.366147] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp mac80211 iwlwifi > > cfg80211 hid_multitouch 8250_dw ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > videobuf2_memops videobuf2_v4l2 processor_thermal_device videobuf2_common > > intel_soc_dts_iosf ov13858 ov5670 dw9714 v4l2_fwnode v4l2_common videodev > > media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone > > chromeos_pstore mac_hid int3400_thermal acpi_thermal_rel autofs4 usbhid > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > sysfillrect sysimgblt fb_sys_fops sdhci_pci cqhci sdhci drm > > drm_panel_orientation_quirks i2c_hid hid > > [ 221.366172] CPU: 3 PID: 1022 Comm: yavta Tainted: G B WC > > 4.20.0-rc6+ #2 > > [ 221.366173] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 221.366173] Call Trace: > > [ 221.366176] dump_stack+0x46/0x59 > > [ 221.366179] bad_page+0xf2/0x10c > > [ 221.366182] free_pages_check+0x78/0x81 > > [ 221.366186] free_pcppages_bulk+0xa6/0x236 > > [ 221.366190] free_unref_page+0x4b/0x53 > > [ 221.366193] vb2_dma_sg_put+0x95/0xb5 [videobuf2_dma_sg] > > [ 221.366197] __vb2_buf_mem_free+0x3a/0x6e [videobuf2_common] > > [ 221.366202] __vb2_queue_free+0xe3/0x1be [videobuf2_common] > > [ 221.366207] vb2_core_reqbufs+0xe9/0x2cc [videobuf2_common] > > [ 221.366212] vb2_ioctl_reqbufs+0x78/0x9e [videobuf2_v4l2] > > [ 221.366220] __video_do_ioctl+0x258/0x38e [videodev] > > [ 221.366229] video_usercopy+0x25f/0x4e5 [videodev] > > [ 221.366237] ? copy_overflow+0x14/0x14 [videodev] > > [ 221.366240] ? unmap_region+0xe0/0x10a > > [ 221.366250] v4l2_ioctl+0x4d/0x58 [videodev] > > [ 221.366253] vfs_ioctl+0x1e/0x2b > > [ 221.366255] do_vfs_ioctl+0x531/0x559 > > [ 221.366260] ksys_ioctl+0x50/0x70 > > [ 221.366263] __x64_sys_ioctl+0x16/0x19 > > [ 221.366266] do_syscall_64+0x53/0x60 > > [ 221.366269] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 221.366270] RIP: 0033:0x7fbe39f6af47 > > [ 221.366272] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 221.366273] RSP: 002b:00007fff05638e68 EFLAGS: 00000246 ORIG_RAX: > > 0000000000000010 > > [ 221.366275] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > 00007fbe39f6af47 > > [ 221.366279] RDX: 00007fff05638f90 RSI: 00000000c0145608 RDI: > > 0000000000000003 > > [ 221.366283] RBP: 0000000000000004 R08: 0000000000000000 R09: > > 0000000000000045 > > [ 221.366287] R10: 0000000000000557 R11: 0000000000000246 R12: > > 000055c83bd76750 > > [ 221.366290] R13: 000055c83b6b26a0 R14: 0000000000000001 R15: > > 00007fff0563a825
On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hellon > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > Hello Yong, > > > > Could you please have a look at the crash reported below ? > > A bit more information to help you debugging this. I've enabled KASAN in the > kernel configuration, and get the following use-after-free reports. > > [ 166.332920] ================================================================== > [ 166.332937] BUG: KASAN: use-after-free in __cached_rbnode_delete_update+0x36/0x202 > [ 166.332944] Read of size 8 at addr ffff888133823718 by task yavta/1305 > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C 4.20.0-rc6+ #3 > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 166.332959] Call Trace: > [ 166.332967] dump_stack+0x5b/0x81 > [ 166.332974] print_address_description+0x65/0x227 > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > [ 166.332983] kasan_report+0x247/0x285 > [ 166.332989] __cached_rbnode_delete_update+0x36/0x202 > [ 166.332995] private_free_iova+0x57/0x6d > [ 166.332999] __free_iova+0x23/0x31 > [ 166.333011] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] Thanks Laurent, I think this is a very good hint. It looks like we're basically freeing and already freed IOVA and corrupting some allocator state? > [ 166.333022] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 166.333032] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > [ 166.333043] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > [ 166.333056] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > [ 166.333067] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > [ 166.333079] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > [ 166.333088] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > [ 166.333096] ? __mutex_lock_interruptible_slowpath+0xf/0xf > [ 166.333104] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > [ 166.333123] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.333142] ? copy_overflow+0x14/0x14 [videodev] > [ 166.333147] ? slab_free_freelist_hook+0x46/0x94 > [ 166.333151] ? kfree+0x107/0x1a0 > [ 166.333169] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.333187] ? copy_overflow+0x14/0x14 [videodev] > [ 166.333203] ? v4l_enumstd+0x49/0x49 [videodev] > [ 166.333207] ? __wake_up_common+0x342/0x342 > [ 166.333215] ? atomic_long_add_return+0x15/0x24 > [ 166.333219] ? ldsem_up_read+0x15/0x29 > [ 166.333223] ? tty_write+0x4c6/0x4d8 > [ 166.333227] ? n_tty_receive_char_special+0x1152/0x1152 > [ 166.333244] ? video_usercopy+0x8ae/0x8ae [videodev] > [ 166.333260] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.333266] vfs_ioctl+0x76/0x89 > [ 166.333271] do_vfs_ioctl+0xb33/0xb7e > [ 166.333275] ? __switch_to_asm+0x40/0x70 > [ 166.333279] ? __switch_to_asm+0x40/0x70 > [ 166.333282] ? __switch_to_asm+0x34/0x70 > [ 166.333286] ? __switch_to_asm+0x40/0x70 > [ 166.333290] ? ioctl_preallocate+0x174/0x174 > [ 166.333294] ? __switch_to+0x71c/0xb00 > [ 166.333299] ? compat_start_thread+0x6b/0x6b > [ 166.333302] ? __switch_to_asm+0x34/0x70 > [ 166.333305] ? __switch_to_asm+0x40/0x70 > [ 166.333309] ? mmdrop+0x12/0x23 > [ 166.333313] ? finish_task_switch+0x34d/0x3de > [ 166.333319] ? __schedule+0x1004/0x1045 > [ 166.333325] ? firmware_map_remove+0x119/0x119 > [ 166.333330] ksys_ioctl+0x50/0x70 > [ 166.333335] __x64_sys_ioctl+0x82/0x89 > [ 166.333340] do_syscall_64+0xa0/0xd2 > [ 166.333345] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 166.333349] RIP: 0033:0x7f2481541f47 > [ 166.333354] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 166.333357] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > [ 166.333362] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > [ 166.333364] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > [ 166.333367] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > [ 166.333369] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > [ 166.333372] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > [ 166.333383] Allocated by task 1305: > [ 166.333389] kasan_kmalloc+0x8a/0x98 > [ 166.333392] slab_post_alloc_hook+0x31/0x51 > [ 166.333396] kmem_cache_alloc+0xd7/0x174 > [ 166.333399] alloc_iova+0x24/0x2ea > [ 166.333407] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] > [ 166.333415] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > [ 166.333424] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > [ 166.333433] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > [ 166.333442] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > [ 166.333449] vb2_start_streaming+0x164/0x33b [videobuf2_common] > [ 166.333455] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > [ 166.333471] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.333487] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.333501] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.333505] vfs_ioctl+0x76/0x89 > [ 166.333508] do_vfs_ioctl+0xb33/0xb7e > [ 166.333511] ksys_ioctl+0x50/0x70 > [ 166.333514] __x64_sys_ioctl+0x82/0x89 > [ 166.333518] do_syscall_64+0xa0/0xd2 > [ 166.333521] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.333526] Freed by task 1301: > [ 166.333532] __kasan_slab_free+0xfa/0x11c > [ 166.333535] slab_free_freelist_hook+0x46/0x94 > [ 166.333538] kmem_cache_free+0x7b/0x172 > [ 166.333542] __free_iova+0x23/0x31 > [ 166.333550] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > [ 166.333557] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 166.333566] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > [ 166.333574] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > [ 166.333584] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > [ 166.333593] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > [ 166.333599] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > [ 166.333606] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > [ 166.333621] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.333637] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.333652] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.333655] vfs_ioctl+0x76/0x89 > [ 166.333658] do_vfs_ioctl+0xb33/0xb7e > [ 166.333662] ksys_ioctl+0x50/0x70 > [ 166.333665] __x64_sys_ioctl+0x82/0x89 > [ 166.333668] do_syscall_64+0xa0/0xd2 > [ 166.333671] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.333678] The buggy address belongs to the object at ffff888133823700 > which belongs to the cache iommu_iova of size 40 > [ 166.333685] The buggy address is located 24 bytes inside of > 40-byte region [ffff888133823700, ffff888133823728) > [ 166.333690] The buggy address belongs to the page: > [ 166.333696] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 > [ 166.333703] flags: 0x200000000010200(slab|head) > [ 166.333710] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 > [ 166.333717] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 > [ 166.333720] page dumped because: kasan: bad access detected > > [ 166.333726] Memory state around the buggy address: > [ 166.333732] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.333737] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.333742] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc > [ 166.333745] ^ > [ 166.333750] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.333755] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.333759] ================================================================== > [ 166.333762] Disabling lock debugging due to kernel taint > [ 166.333764] ================================================================== > [ 166.333770] BUG: KASAN: double-free or invalid-free in kmem_cache_free+0x7b/0x172 > > [ 166.333780] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 > [ 166.333782] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 166.333783] Call Trace: > [ 166.333789] dump_stack+0x5b/0x81 > [ 166.333795] print_address_description+0x65/0x227 > [ 166.333799] ? kmem_cache_free+0x7b/0x172 > [ 166.333803] kasan_report_invalid_free+0x67/0xa0 > [ 166.333807] ? kmem_cache_free+0x7b/0x172 > [ 166.333812] __kasan_slab_free+0x86/0x11c > [ 166.333817] slab_free_freelist_hook+0x46/0x94 > [ 166.333822] kmem_cache_free+0x7b/0x172 > [ 166.333826] ? __free_iova+0x23/0x31 > [ 166.333831] __free_iova+0x23/0x31 > [ 166.333840] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > [ 166.333851] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 166.333861] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > [ 166.333872] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > [ 166.333885] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > [ 166.333896] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > [ 166.333908] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > [ 166.333917] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > [ 166.333923] ? __mutex_lock_interruptible_slowpath+0xf/0xf > [ 166.333932] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > [ 166.333950] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.333970] ? copy_overflow+0x14/0x14 [videodev] > [ 166.333974] ? slab_free_freelist_hook+0x46/0x94 > [ 166.333979] ? kfree+0x107/0x1a0 > [ 166.333997] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.334015] ? copy_overflow+0x14/0x14 [videodev] > [ 166.334031] ? v4l_enumstd+0x49/0x49 [videodev] > [ 166.334035] ? __wake_up_common+0x342/0x342 > [ 166.334042] ? atomic_long_add_return+0x15/0x24 > [ 166.334046] ? ldsem_up_read+0x15/0x29 > [ 166.334050] ? tty_write+0x4c6/0x4d8 > [ 166.334054] ? n_tty_receive_char_special+0x1152/0x1152 > [ 166.334071] ? video_usercopy+0x8ae/0x8ae [videodev] > [ 166.334087] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.334092] vfs_ioctl+0x76/0x89 > [ 166.334097] do_vfs_ioctl+0xb33/0xb7e > [ 166.334101] ? __switch_to_asm+0x40/0x70 > [ 166.334105] ? __switch_to_asm+0x40/0x70 > [ 166.334108] ? __switch_to_asm+0x34/0x70 > [ 166.334111] ? __switch_to_asm+0x40/0x70 > [ 166.334116] ? ioctl_preallocate+0x174/0x174 > [ 166.334120] ? __switch_to+0x71c/0xb00 > [ 166.334124] ? compat_start_thread+0x6b/0x6b > [ 166.334127] ? __switch_to_asm+0x34/0x70 > [ 166.334130] ? __switch_to_asm+0x40/0x70 > [ 166.334134] ? mmdrop+0x12/0x23 > [ 166.334137] ? finish_task_switch+0x34d/0x3de > [ 166.334143] ? __schedule+0x1004/0x1045 > [ 166.334148] ? firmware_map_remove+0x119/0x119 > [ 166.334153] ksys_ioctl+0x50/0x70 > [ 166.334158] __x64_sys_ioctl+0x82/0x89 > [ 166.334163] do_syscall_64+0xa0/0xd2 > [ 166.334167] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 166.334171] RIP: 0033:0x7f2481541f47 > [ 166.334175] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 166.334177] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > [ 166.334181] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > [ 166.334184] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > [ 166.334186] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > [ 166.334189] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > [ 166.334191] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > [ 166.334201] Allocated by task 1305: > [ 166.334207] kasan_kmalloc+0x8a/0x98 > [ 166.334210] slab_post_alloc_hook+0x31/0x51 > [ 166.334213] kmem_cache_alloc+0xd7/0x174 > [ 166.334216] alloc_iova+0x24/0x2ea > [ 166.334225] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] > [ 166.334233] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > [ 166.334241] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > [ 166.334250] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > [ 166.334259] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > [ 166.334266] vb2_start_streaming+0x164/0x33b [videobuf2_common] > [ 166.334273] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > [ 166.334288] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.334304] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.334319] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.334322] vfs_ioctl+0x76/0x89 > [ 166.334325] do_vfs_ioctl+0xb33/0xb7e > [ 166.334328] ksys_ioctl+0x50/0x70 > [ 166.334332] __x64_sys_ioctl+0x82/0x89 > [ 166.334335] do_syscall_64+0xa0/0xd2 > [ 166.334338] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.334343] Freed by task 1301: > [ 166.334349] __kasan_slab_free+0xfa/0x11c > [ 166.334352] slab_free_freelist_hook+0x46/0x94 > [ 166.334355] kmem_cache_free+0x7b/0x172 > [ 166.334359] __free_iova+0x23/0x31 > [ 166.334367] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > [ 166.334375] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 166.334383] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > [ 166.334392] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > [ 166.334401] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > [ 166.334410] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > [ 166.334416] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > [ 166.334423] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > [ 166.334438] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.334454] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.334469] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.334472] vfs_ioctl+0x76/0x89 > [ 166.334475] do_vfs_ioctl+0xb33/0xb7e > [ 166.334479] ksys_ioctl+0x50/0x70 > [ 166.334482] __x64_sys_ioctl+0x82/0x89 > [ 166.334485] do_syscall_64+0xa0/0xd2 > [ 166.334488] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.334494] The buggy address belongs to the object at ffff888133823700 > which belongs to the cache iommu_iova of size 40 > [ 166.334501] The buggy address is located 0 bytes inside of > 40-byte region [ffff888133823700, ffff888133823728) > [ 166.334506] The buggy address belongs to the page: > [ 166.334511] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 > [ 166.334517] flags: 0x200000000010200(slab|head) > [ 166.334524] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 > [ 166.334530] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 > [ 166.334533] page dumped because: kasan: bad access detected > > [ 166.334539] Memory state around the buggy address: > [ 166.334544] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.334549] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.334554] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc > [ 166.334558] ^ > [ 166.334562] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.334567] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.334571] ================================================================== > [ 166.340377] ================================================================== > [ 166.340388] BUG: KASAN: double-free or invalid-free in kfree+0x107/0x1a0 > > [ 166.340399] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 > [ 166.340401] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > [ 166.340403] Call Trace: > [ 166.340410] dump_stack+0x5b/0x81 > [ 166.340416] print_address_description+0x65/0x227 > [ 166.340420] ? kfree+0x107/0x1a0 > [ 166.340425] kasan_report_invalid_free+0x67/0xa0 > [ 166.340428] ? kfree+0x107/0x1a0 > [ 166.340433] __kasan_slab_free+0x86/0x11c > [ 166.340438] slab_free_freelist_hook+0x46/0x94 > [ 166.340443] kfree+0x107/0x1a0 > [ 166.340454] ? ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > [ 166.340464] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > [ 166.340475] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 166.340485] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > [ 166.340495] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > [ 166.340509] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > [ 166.340520] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > [ 166.340531] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > [ 166.340541] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > [ 166.340548] ? __mutex_lock_interruptible_slowpath+0xf/0xf > [ 166.340557] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > [ 166.340575] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.340595] ? copy_overflow+0x14/0x14 [videodev] > [ 166.340600] ? slab_free_freelist_hook+0x46/0x94 > [ 166.340604] ? kfree+0x107/0x1a0 > [ 166.340622] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.340640] ? copy_overflow+0x14/0x14 [videodev] > [ 166.340657] ? v4l_enumstd+0x49/0x49 [videodev] > [ 166.340660] ? __wake_up_common+0x342/0x342 > [ 166.340668] ? atomic_long_add_return+0x15/0x24 > [ 166.340672] ? ldsem_up_read+0x15/0x29 > [ 166.340677] ? tty_write+0x4c6/0x4d8 > [ 166.340681] ? n_tty_receive_char_special+0x1152/0x1152 > [ 166.340698] ? video_usercopy+0x8ae/0x8ae [videodev] > [ 166.340714] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.340720] vfs_ioctl+0x76/0x89 > [ 166.340725] do_vfs_ioctl+0xb33/0xb7e > [ 166.340729] ? __switch_to_asm+0x40/0x70 > [ 166.340733] ? __switch_to_asm+0x40/0x70 > [ 166.340736] ? __switch_to_asm+0x34/0x70 > [ 166.340739] ? __switch_to_asm+0x40/0x70 > [ 166.340743] ? ioctl_preallocate+0x174/0x174 > [ 166.340748] ? __switch_to+0x71c/0xb00 > [ 166.340752] ? compat_start_thread+0x6b/0x6b > [ 166.340756] ? __switch_to_asm+0x34/0x70 > [ 166.340759] ? __switch_to_asm+0x40/0x70 > [ 166.340762] ? mmdrop+0x12/0x23 > [ 166.340766] ? finish_task_switch+0x34d/0x3de > [ 166.340772] ? __schedule+0x1004/0x1045 > [ 166.340777] ? firmware_map_remove+0x119/0x119 > [ 166.340782] ksys_ioctl+0x50/0x70 > [ 166.340788] __x64_sys_ioctl+0x82/0x89 > [ 166.340793] do_syscall_64+0xa0/0xd2 > [ 166.340797] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 166.340802] RIP: 0033:0x7f2481541f47 > [ 166.340806] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > [ 166.340809] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > [ 166.340813] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > [ 166.340816] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > [ 166.340819] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > [ 166.340821] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > [ 166.340824] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > [ 166.340834] Allocated by task 1305: > [ 166.340840] kasan_kmalloc+0x8a/0x98 > [ 166.340844] __kmalloc_node+0x193/0x1ba > [ 166.340848] kvmalloc_node+0x44/0x6d > [ 166.340856] ipu3_dmamap_alloc+0x1c9/0x83f [ipu3_imgu] > [ 166.340864] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > [ 166.340873] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > [ 166.340882] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > [ 166.340891] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > [ 166.340897] vb2_start_streaming+0x164/0x33b [videobuf2_common] > [ 166.340904] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > [ 166.340920] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.340935] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.340950] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.340954] vfs_ioctl+0x76/0x89 > [ 166.340957] do_vfs_ioctl+0xb33/0xb7e > [ 166.340960] ksys_ioctl+0x50/0x70 > [ 166.340963] __x64_sys_ioctl+0x82/0x89 > [ 166.340966] do_syscall_64+0xa0/0xd2 > [ 166.340969] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.340974] Freed by task 1301: > [ 166.340980] __kasan_slab_free+0xfa/0x11c > [ 166.340983] slab_free_freelist_hook+0x46/0x94 > [ 166.340986] kfree+0x107/0x1a0 > [ 166.340994] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > [ 166.341002] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > [ 166.341010] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > [ 166.341019] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > [ 166.341028] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > [ 166.341037] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > [ 166.341043] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > [ 166.341050] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > [ 166.341066] __video_do_ioctl+0x625/0x887 [videodev] > [ 166.341081] video_usercopy+0x3a3/0x8ae [videodev] > [ 166.341096] v4l2_ioctl+0xb7/0xc5 [videodev] > [ 166.341100] vfs_ioctl+0x76/0x89 > [ 166.341103] do_vfs_ioctl+0xb33/0xb7e > [ 166.341106] ksys_ioctl+0x50/0x70 > [ 166.341109] __x64_sys_ioctl+0x82/0x89 > [ 166.341112] do_syscall_64+0xa0/0xd2 > [ 166.341116] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.341122] The buggy address belongs to the object at ffff88811d228440 > which belongs to the cache kmalloc-8 of size 8 > [ 166.341129] The buggy address is located 0 bytes inside of > 8-byte region [ffff88811d228440, ffff88811d228448) > [ 166.341134] The buggy address belongs to the page: > [ 166.341140] page:ffffea0004748a00 count:1 mapcount:0 mapping:ffff88815a80c340 index:0xffff88811d228f80 compound_mapcount: 0 > [ 166.341146] flags: 0x200000000010200(slab|head) > [ 166.341153] raw: 0200000000010200 ffffea000564b288 ffffea00049ef708 ffff88815a80c340 > [ 166.341159] raw: ffff88811d228f80 0000000000160013 00000001ffffffff 0000000000000000 > [ 166.341163] page dumped because: kasan: bad access detected > > [ 166.341169] Memory state around the buggy address: > [ 166.341174] ffff88811d228300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.341179] ffff88811d228380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.341184] >ffff88811d228400: fc fc fc fc fc fc fc fc fb fc fc fc fc fc fc fc > [ 166.341188] ^ > [ 166.341192] ffff88811d228480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.341197] ffff88811d228500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > [ 166.341201] ================================================================== > > > On Tuesday, 11 December 2018 16:20:43 EET Laurent Pinchart wrote: > > > On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > > > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > >> > > > >> [snip] > > > >> > > > >>> I can see a couple of steps missing in the script below. > > > >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 > > > >>> 00040.html) > > > >>> > > > >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > >>> documentation", > > > >>> under section "Configuring ImgU V4L2 subdev for image processing"... > > > >>> > > > >>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > >>> > > > >>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > >>> desired (e.g 0 for video mode or 1 for still mode) through the control > > > >>> id 0x009819a1 as below. > > > >>> > > > >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > >> > > > >> I assume the control takes a valid default value ? It's better to set > > > >> it > > > >> explicitly anyway, so I'll do so. > > > >> > > > >>> 2. ImgU pipeline needs to be configured for image processing as below. > > > >>> > > > >>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > >>> have the processed image output to the DDR memory. > > > >>> > > > >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > >>> Geometric Distortion Correction (GDC) -> DDR > > > >>> > > > >>> The ImgU V4L2 subdev has to be configured with the supported > > > >>> resolutions in all the above HW blocks, for a given input resolution. > > > >>> > > > >>> For a given supported resolution for an input frame, the Input Feeder, > > > >>> Bayer Down Scaling and GDC blocks should be configured with the > > > >>> supported resolutions. This information can be obtained by looking at > > > >>> the following IPU3 ISP configuration table for ov5670 sensor. > > > >>> > > > >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > > > >>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files > > > >>> / > > > >>> gcss/graph_settings_ov5670.xml > > > >>> > > > >>> For the ov5670 example, for an input frame with a resolution of > > > >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > > >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > > >>> 2560x1920 respectively. > > > >> > > > >> How is the GDC output resolution computed from the input resolution ? > > > >> Does the GDC always consume 32 columns and 22 lines ? > > > >> > > > >>> The following steps prepare the ImgU ISP pipeline for the image > > > >>> processing. > > > >>> > > > >>> 1. The ImgU V4L2 subdev data format should be set by using the > > > >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > > > >>> above. > > > >> > > > >> If I understand things correctly, the GDC resolution is the pipeline > > > >> output resolution. Why is it configured on pad 0 ? > > > >> > > > >>> 2. The ImgU V4L2 subdev cropping should be set by using the > > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > > >>> target, using the input feeder height and width. > > > >>> > > > >>> 3. The ImgU V4L2 subdev composing should be set by using the > > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > > >>> target, using the BDS height and width. > > > >>> > > > >>> Once these 2 steps are done, the raw bayer frames can be input to the > > > >>> ImgU V4L2 subdev for processing. > > > >> > > > >> Do I need to capture from both the output and viewfinder nodes ? How > > > >> are > > > >> they related to the IF -> BDS -> GDC pipeline, are they both fed from > > > >> the GDC output ? If so, how does the viewfinder scaler fit in that > > > >> picture ? > > > >> > > > >> I have tried the above configuration with the IPU3 v8 driver, and while > > > >> the kernel doesn't crash, no images get processed. The userspace > > > >> processes wait forever for buffers to be ready. I then configured pad 2 > > > >> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ > > > >> > > > >> There's one problem though: during capture, or very soon after it, the > > > >> machine locks up completely. I suspect a memory corruption, as when it > > > >> doesn't log immediately commands such as dmesg will not produce any > > > >> output and just block, until the system freezes soon after (especially > > > >> when moving the mouse). > > > >> > > > >> I would still call this an improvement to some extent, but there's > > > >> definitely room for more improvements :-) > > > >> > > > >> To reproduce the issue, you can run the ipu3-process.sh script > > > >> (attached > > > >> to this e-mail) with the following arguments: > > > >> > > > >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > > > > > > This should have read > > > > > > $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > > > > > Without the --vf argument no images are processed. > > > > > > It seems that the Intel mail server blocked the mail that contained the > > > script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > > > > > >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in > > > >> the IPU3-specific Bayer format (for a total of 6469632 bytes). > > > > > > > > I managed to get the dmesg output, and it doesn't look pretty. > > > > > > > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > > > > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > > > > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > > mac80211 > > > > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp > > > > cfg80211 > > > > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > > videobuf2_memops videobuf2_v4l2 videobuf2_common > > > > processor_thermal_device > > > > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common > > > > videodev > > > > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs > > > > cros_ec_core > > > > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > > > > drm_panel_orientation_quirks i2c_hid hid > > > > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > > > > 4.20.0-rc6+ #2 > > > > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc > > > > f3 > > > > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > > > > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > > > > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > > > > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > > > > 000000000000000c > > > > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > > > > 00000000ffffffff > > > > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > > > > ffff8f5cfaba16f0 > > > > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > > > > ffff8f5cf58f0028 > > > > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > > > > ffff8f5cf58f04e8 > > > > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) > > > > knlGS: > > > > 0000000000000000 > > > > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > > > > 00000000003606e0 > > > > [ 571.217301] Call Trace: > > > > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > > > > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > > > > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > > > > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > > > > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > > > > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > > > > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > > > > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > > > > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > > > > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > > > > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > > > > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > > > > [ 571.217480] vfs_ioctl+0x1e/0x2b > > > > [ 571.217486] do_vfs_ioctl+0x531/0x559 > > > > [ 571.217494] ? vfs_write+0xd1/0xdf > > > > [ 571.217500] ksys_ioctl+0x50/0x70 > > > > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > > > > [ 571.217512] do_syscall_64+0x53/0x60 > > > > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > > > > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 > > > > 48 > > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > > > > 0000000000000010 > > > > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > > > > 00007f85cf9b9f47 > > > > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > > > > 0000000000000003 > > > > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > > > > 00007f85d009c700 > > > > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > > > > 000055f4c4dc0b06 > > > > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > > > > 00007ffc59057825 > > > > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > > > > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > > > > > And after fixing another issue in the capture script (which was setting > > > the > > > format on the ImgU subdev pad 3 to 2560x1920 but capture in 1920x1080), I > > > now get plenty of the following messages: > > > > > > [ 221.366131] BUG: Bad page state in process yavta pfn:14a4ff > > > [ 221.366134] page:ffffde5d45293fc0 count:-1 mapcount:0 mapping: > > > 0000000000000000 index:0x0 > > > [ 221.366137] flags: 0x200000000000000() > > > [ 221.366140] raw: 0200000000000000 dead000000000100 dead000000000200 > > > 0000000000000000 > > > [ 221.366143] raw: 0000000000000000 0000000000000000 ffffffffffffffff > > > 0000000000000000 > > > [ 221.366145] page dumped because: nonzero _refcount > > > [ 221.366147] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp mac80211 iwlwifi > > > cfg80211 hid_multitouch 8250_dw ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > videobuf2_memops videobuf2_v4l2 processor_thermal_device videobuf2_common > > > intel_soc_dts_iosf ov13858 ov5670 dw9714 v4l2_fwnode v4l2_common videodev > > > media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone > > > chromeos_pstore mac_hid int3400_thermal acpi_thermal_rel autofs4 usbhid > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > sysfillrect sysimgblt fb_sys_fops sdhci_pci cqhci sdhci drm > > > drm_panel_orientation_quirks i2c_hid hid > > > [ 221.366172] CPU: 3 PID: 1022 Comm: yavta Tainted: G B WC > > > 4.20.0-rc6+ #2 > > > [ 221.366173] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > [ 221.366173] Call Trace: > > > [ 221.366176] dump_stack+0x46/0x59 > > > [ 221.366179] bad_page+0xf2/0x10c > > > [ 221.366182] free_pages_check+0x78/0x81 > > > [ 221.366186] free_pcppages_bulk+0xa6/0x236 > > > [ 221.366190] free_unref_page+0x4b/0x53 > > > [ 221.366193] vb2_dma_sg_put+0x95/0xb5 [videobuf2_dma_sg] > > > [ 221.366197] __vb2_buf_mem_free+0x3a/0x6e [videobuf2_common] > > > [ 221.366202] __vb2_queue_free+0xe3/0x1be [videobuf2_common] > > > [ 221.366207] vb2_core_reqbufs+0xe9/0x2cc [videobuf2_common] > > > [ 221.366212] vb2_ioctl_reqbufs+0x78/0x9e [videobuf2_v4l2] > > > [ 221.366220] __video_do_ioctl+0x258/0x38e [videodev] > > > [ 221.366229] video_usercopy+0x25f/0x4e5 [videodev] > > > [ 221.366237] ? copy_overflow+0x14/0x14 [videodev] > > > [ 221.366240] ? unmap_region+0xe0/0x10a > > > [ 221.366250] v4l2_ioctl+0x4d/0x58 [videodev] > > > [ 221.366253] vfs_ioctl+0x1e/0x2b > > > [ 221.366255] do_vfs_ioctl+0x531/0x559 > > > [ 221.366260] ksys_ioctl+0x50/0x70 > > > [ 221.366263] __x64_sys_ioctl+0x16/0x19 > > > [ 221.366266] do_syscall_64+0x53/0x60 > > > [ 221.366269] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > [ 221.366270] RIP: 0033:0x7fbe39f6af47 > > > [ 221.366272] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > [ 221.366273] RSP: 002b:00007fff05638e68 EFLAGS: 00000246 ORIG_RAX: > > > 0000000000000010 > > > [ 221.366275] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > > 00007fbe39f6af47 > > > [ 221.366279] RDX: 00007fff05638f90 RSI: 00000000c0145608 RDI: > > > 0000000000000003 > > > [ 221.366283] RBP: 0000000000000004 R08: 0000000000000000 R09: > > > 0000000000000045 > > > [ 221.366287] R10: 0000000000000557 R11: 0000000000000246 R12: > > > 000055c83bd76750 > > > [ 221.366290] R13: 000055c83b6b26a0 R14: 0000000000000001 R15: > > > 00007fff0563a825 > > -- > Regards, > > Laurent Pinchart > > >
Hello Bingbu, On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > > On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > >> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > >>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > >>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > >>>> > >>>> [snip] > >>>> > >>>>> I can see a couple of steps missing in the script below. > >>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 > >>>>> 00040.html) > >>>>> > >>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > >>>>> documentation", under section "Configuring ImgU V4L2 subdev for > >>>>> image processing"... > >>>>> > >>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > >>>>> > >>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > >>>>> desired (e.g 0 for video mode or 1 for still mode) through the control > >>>>> id 0x009819a1 as below. > >>>>> > >>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > >>>> > >>>> I assume the control takes a valid default value ? It's better to set > >>>> it explicitly anyway, so I'll do so. > >> > >> The video mode is set by default. If you want to set to still mode or > >> change mode, you need set the subdev control. > >> > >>>>> 2. ImgU pipeline needs to be configured for image processing as below. > >>>>> > >>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > >>>>> have the processed image output to the DDR memory. > >>>>> > >>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > >>>>> Geometric Distortion Correction (GDC) -> DDR > >>>>> > >>>>> The ImgU V4L2 subdev has to be configured with the supported > >>>>> resolutions in all the above HW blocks, for a given input resolution. > >>>>> > >>>>> For a given supported resolution for an input frame, the Input Feeder, > >>>>> Bayer Down Scaling and GDC blocks should be configured with the > >>>>> supported resolutions. This information can be obtained by looking at > >>>>> the following IPU3 ISP configuration table for ov5670 sensor. > >>>>> > >>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > >>>>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > >>>>> files/gcss/graph_settings_ov5670.xml > >>>>> > >>>>> For the ov5670 example, for an input frame with a resolution of > >>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > >>>>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > >>>>> 2560x1920 respectively. > >>>> > >>>> How is the GDC output resolution computed from the input resolution ? > >>>> Does the GDC always consume 32 columns and 22 lines ? > >> > >> All the intermediate resolutions in the pipeline are determined by the > >> actual use case, in other word determined by the IMGU input > >> resolution(sensor output) and the final output and viewfinder resolution. > >> BDS mainly do Bayer downscaling, it has limitation that the downscaling > >> factor must be a value a integer multiple of 1/32. > >> GDC output depends on the input and width should be x8 and height x4 > >> alignment. > > > > Thank you for the information. This will need to be captured in the > > documentation, along with information related to how each block in the > > hardware pipeline interacts with the image size. It should be possible for > > a developer to compute the output and viewfinder resolutions based on the > > parameters of the image processing algorithms just with the information > > contained in the driver documentation. > > > >>>>> The following steps prepare the ImgU ISP pipeline for the image > >>>>> processing. > >>>>> > >>>>> 1. The ImgU V4L2 subdev data format should be set by using the > >>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > >>>>> above. > >>>> > >>>> If I understand things correctly, the GDC resolution is the pipeline > >>>> output resolution. Why is it configured on pad 0 ? > >> > >> We see the GDC output resolution as the input of output system, the sink > >> pad format is used for output and viewfinder resolutions. > > > > The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > > the ImgU input, the format configured there should correspond to the > > format on the connected video node, and should thus be the sensor format. > > You can then use the crop and compose rectangles on pad 0, along with the > > format, crop and compose rectangles on the output and viewfinder pads, to > > configure the device. This should be fixed in the driver, and the > > documentation should then be updated accordingly. > > Hi, Laurent, > > Thanks for your review. > > I think it make sense for me that using Pad 0 as the ImgU input(IF). > However, I prefer using the 2 source pads for output and viewfinder. > It makes more sense because the output and viewfinder are independent > output. > > The whole pipeline in ImgU looks like: > IF --> BDS --> GDC ---> OUTPUT > | > |-----> VF > > The BDS is used to do Bayer downscaling and GDC can do cropping. Does this mean that the main output and the viewfinder output share the same scaler, and that the only difference in size between the two outputs is solely due to cropping ? > My understanding is that scaled size is configured on the CROP rectangle > by COMPOSE selection target, the order seems like not aligned with the > actual processing in ImgU if we set the crop/compose on sink pad. > > Is there some rules for the order of the configuration in the subdev API? > Could I use crop selection based on the scaled size? Please see figure 4.6 in https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/ dev-subdev.html. Scaling is configured on the sink pad through the crop and compose rectangles, while the source crop rectangle is used to perform cropping on the output. If you have a single scaler in the hardware pipeline you can thus configure it on the sink pad, with output and viewfinder separate cropping configure on the source pad. > >>>>> 2. The ImgU V4L2 subdev cropping should be set by using the > >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > >>>>> target, using the input feeder height and width. > >>>>> > >>>>> 3. The ImgU V4L2 subdev composing should be set by using the > >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > >>>>> target, using the BDS height and width. > >>>>> > >>>>> Once these 2 steps are done, the raw bayer frames can be input to the > >>>>> ImgU V4L2 subdev for processing. > >>>> > >>>> Do I need to capture from both the output and viewfinder nodes ? How > >>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > >>>> from the GDC output ? If so, how does the viewfinder scaler fit in that > >>>> picture ? > >> > >> The output capture should be set, the viewfinder can be disabled. > >> The IF and BDS are seen as crop and compose of the imgu input video > >> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > >> pads. > > > > The GDC is the last block in the pipeline according to the information > > provided above. How can it be seen as the subdev sink pad ? That doesn't > > make sense to me. I'm not asking for the MC graph to expose all internal > > blocks of the ImgU, but if you want to retain a single subdev model, the > > format on the sink pad needs to correspond to what is provided to the > > ImgU. Please see figure 4.6 of > > https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for > > more information regarding how you can use the sink crop, sink compose > > and source crop rectangles. [snip]
On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > Hello Bingbu, > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: >>>>>> >>>>>> [snip] >>>>>> >>>>>>> I can see a couple of steps missing in the script below. >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 >>>>>>> 00040.html) >>>>>>> >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for >>>>>>> image processing"... >>>>>>> >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. >>>>>>> >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the control >>>>>>> id 0x009819a1 as below. >>>>>>> >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 >>>>>> I assume the control takes a valid default value ? It's better to set >>>>>> it explicitly anyway, so I'll do so. >>>> The video mode is set by default. If you want to set to still mode or >>>> change mode, you need set the subdev control. >>>> >>>>>>> 2. ImgU pipeline needs to be configured for image processing as below. >>>>>>> >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to >>>>>>> have the processed image output to the DDR memory. >>>>>>> >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> >>>>>>> Geometric Distortion Correction (GDC) -> DDR >>>>>>> >>>>>>> The ImgU V4L2 subdev has to be configured with the supported >>>>>>> resolutions in all the above HW blocks, for a given input resolution. >>>>>>> >>>>>>> For a given supported resolution for an input frame, the Input Feeder, >>>>>>> Bayer Down Scaling and GDC blocks should be configured with the >>>>>>> supported resolutions. This information can be obtained by looking at >>>>>>> the following IPU3 ISP configuration table for ov5670 sensor. >>>>>>> >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ >>>>>>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ >>>>>>> files/gcss/graph_settings_ov5670.xml >>>>>>> >>>>>>> For the ov5670 example, for an input frame with a resolution of >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding >>>>>>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and >>>>>>> 2560x1920 respectively. >>>>>> How is the GDC output resolution computed from the input resolution ? >>>>>> Does the GDC always consume 32 columns and 22 lines ? >>>> All the intermediate resolutions in the pipeline are determined by the >>>> actual use case, in other word determined by the IMGU input >>>> resolution(sensor output) and the final output and viewfinder resolution. >>>> BDS mainly do Bayer downscaling, it has limitation that the downscaling >>>> factor must be a value a integer multiple of 1/32. >>>> GDC output depends on the input and width should be x8 and height x4 >>>> alignment. >>> Thank you for the information. This will need to be captured in the >>> documentation, along with information related to how each block in the >>> hardware pipeline interacts with the image size. It should be possible for >>> a developer to compute the output and viewfinder resolutions based on the >>> parameters of the image processing algorithms just with the information >>> contained in the driver documentation. >>> >>>>>>> The following steps prepare the ImgU ISP pipeline for the image >>>>>>> processing. >>>>>>> >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained >>>>>>> above. >>>>>> If I understand things correctly, the GDC resolution is the pipeline >>>>>> output resolution. Why is it configured on pad 0 ? >>>> We see the GDC output resolution as the input of output system, the sink >>>> pad format is used for output and viewfinder resolutions. >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be >>> the ImgU input, the format configured there should correspond to the >>> format on the connected video node, and should thus be the sensor format. >>> You can then use the crop and compose rectangles on pad 0, along with the >>> format, crop and compose rectangles on the output and viewfinder pads, to >>> configure the device. This should be fixed in the driver, and the >>> documentation should then be updated accordingly. >> Hi, Laurent, >> >> Thanks for your review. >> >> I think it make sense for me that using Pad 0 as the ImgU input(IF). >> However, I prefer using the 2 source pads for output and viewfinder. >> It makes more sense because the output and viewfinder are independent >> output. >> >> The whole pipeline in ImgU looks like: >> IF --> BDS --> GDC ---> OUTPUT >> | >> |-----> VF >> >> The BDS is used to do Bayer downscaling and GDC can do cropping. > Does this mean that the main output and the viewfinder output share the same > scaler, and that the only difference in size between the two outputs is solely > due to cropping ? Laurent, No, output only can do crop and viewfinder support crop and scaling, they share same input. > >> My understanding is that scaled size is configured on the CROP rectangle >> by COMPOSE selection target, the order seems like not aligned with the >> actual processing in ImgU if we set the crop/compose on sink pad. >> >> Is there some rules for the order of the configuration in the subdev API? >> Could I use crop selection based on the scaled size? > Please see figure 4.6 in https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/ > dev-subdev.html. Scaling is configured on the sink pad through the crop and > compose rectangles, while the source crop rectangle is used to perform > cropping on the output. If you have a single scaler in the hardware pipeline > you can thus configure it on the sink pad, with output and viewfinder separate > cropping configure on the source pad. > >>>>>>> 2. The ImgU V4L2 subdev cropping should be set by using the >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the >>>>>>> target, using the input feeder height and width. >>>>>>> >>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>>>>>> target, using the BDS height and width. >>>>>>> >>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the >>>>>>> ImgU V4L2 subdev for processing. >>>>>> Do I need to capture from both the output and viewfinder nodes ? How >>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed >>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that >>>>>> picture ? >>>> The output capture should be set, the viewfinder can be disabled. >>>> The IF and BDS are seen as crop and compose of the imgu input video >>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source >>>> pads. >>> The GDC is the last block in the pipeline according to the information >>> provided above. How can it be seen as the subdev sink pad ? That doesn't >>> make sense to me. I'm not asking for the MC graph to expose all internal >>> blocks of the ImgU, but if you want to retain a single subdev model, the >>> format on the sink pad needs to correspond to what is provided to the >>> ImgU. Please see figure 4.6 of >>> https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for >>> more information regarding how you can use the sink crop, sink compose >>> and source crop rectangles. > [snip] >
Hello Bingbu, On Wednesday, 2 January 2019 04:38:33 EET Bingbu Cao wrote: > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > >>>>>> > >>>>>> [snip] > >>>>>> > >>>>>>> I can see a couple of steps missing in the script below. > >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November > >>>>>>> /000040.html) > >>>>>>> > >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for > >>>>>>> image processing"... > >>>>>>> > >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > >>>>>>> > >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the > >>>>>>> control id 0x009819a1 as below. > >>>>>>> > >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > >>>>>> > >>>>>> I assume the control takes a valid default value ? It's better to set > >>>>>> it explicitly anyway, so I'll do so. > >>>> > >>>> The video mode is set by default. If you want to set to still mode or > >>>> change mode, you need set the subdev control. > >>>> > >>>>>>> 2. ImgU pipeline needs to be configured for image processing as > >>>>>>> below. > >>>>>>> > >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > >>>>>>> have the processed image output to the DDR memory. > >>>>>>> > >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > >>>>>>> Geometric Distortion Correction (GDC) -> DDR > >>>>>>> > >>>>>>> The ImgU V4L2 subdev has to be configured with the supported > >>>>>>> resolutions in all the above HW blocks, for a given input > >>>>>>> resolution. > >>>>>>> > >>>>>>> For a given supported resolution for an input frame, the Input > >>>>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured with > >>>>>>> the supported resolutions. This information can be obtained by > >>>>>>> looking at the following IPU3 ISP configuration table for ov5670 > >>>>>>> sensor. > >>>>>>> > >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays > >>>>>>> /+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > >>>>>>> files/gcss/graph_settings_ov5670.xml > >>>>>>> > >>>>>>> For the ov5670 example, for an input frame with a resolution of > >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > >>>>>>> corresponding resolutions for input feeder, BDS and GDC are > >>>>>>> 2592x1944, 2592x1944 and 2560x1920 respectively. > >>>>>> > >>>>>> How is the GDC output resolution computed from the input resolution ? > >>>>>> Does the GDC always consume 32 columns and 22 lines ? > >>>> > >>>> All the intermediate resolutions in the pipeline are determined by the > >>>> actual use case, in other word determined by the IMGU input > >>>> resolution(sensor output) and the final output and viewfinder > >>>> resolution. BDS mainly do Bayer downscaling, it has limitation that the > >>>> downscaling factor must be a value a integer multiple of 1/32. > >>>> GDC output depends on the input and width should be x8 and height x4 > >>>> alignment. > >>> > >>> Thank you for the information. This will need to be captured in the > >>> documentation, along with information related to how each block in the > >>> hardware pipeline interacts with the image size. It should be possible > >>> for a developer to compute the output and viewfinder resolutions based > >>> on the parameters of the image processing algorithms just with the > >>> information contained in the driver documentation. > >>> > >>>>>>> The following steps prepare the ImgU ISP pipeline for the image > >>>>>>> processing. > >>>>>>> > >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the > >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > >>>>>>> obtained > >>>>>>> above. > >>>>>> > >>>>>> If I understand things correctly, the GDC resolution is the pipeline > >>>>>> output resolution. Why is it configured on pad 0 ? > >>>> > >>>> We see the GDC output resolution as the input of output system, the > >>>> sink pad format is used for output and viewfinder resolutions. > >>> > >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > >>> the ImgU input, the format configured there should correspond to the > >>> format on the connected video node, and should thus be the sensor > >>> format. You can then use the crop and compose rectangles on pad 0, along > >>> with the format, crop and compose rectangles on the output and > >>> viewfinder pads, to configure the device. This should be fixed in the > >>> driver, and the documentation should then be updated accordingly. > >> > >> Hi, Laurent, > >> > >> Thanks for your review. > >> > >> I think it make sense for me that using Pad 0 as the ImgU input(IF). > >> However, I prefer using the 2 source pads for output and viewfinder. > >> It makes more sense because the output and viewfinder are independent > >> output. > >> > >> The whole pipeline in ImgU looks like: > >> IF --> BDS --> GDC ---> OUTPUT > >> |-----> VF > >> > >> The BDS is used to do Bayer downscaling and GDC can do cropping. > > > > Does this mean that the main output and the viewfinder output share the > > same scaler, and that the only difference in size between the two outputs > > is solely due to cropping ? > > Laurent, > No, output only can do crop and viewfinder support crop and scaling, they > share same input. Then you can't support this with a single subdev for the ImgU, you need at least two subdevs. I can offer more guidance, but I'll need more information about the GDC. > >> My understanding is that scaled size is configured on the CROP rectangle > >> by COMPOSE selection target, the order seems like not aligned with the > >> actual processing in ImgU if we set the crop/compose on sink pad. > >> > >> Is there some rules for the order of the configuration in the subdev API? > >> Could I use crop selection based on the scaled size? > > > > Please see figure 4.6 in > > https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html. > > Scaling is configured on the sink pad through the crop and compose > > rectangles, while the source crop rectangle is used to perform cropping > > on the output. If you have a single scaler in the hardware pipeline you > > can thus configure it on the sink pad, with output and viewfinder > > separate cropping configure on the source pad. > > > >>>>>>> 2. The ImgU V4L2 subdev cropping should be set by using the > >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > >>>>>>> target, using the input feeder height and width. > >>>>>>> > >>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the > >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > >>>>>>> target, using the BDS height and width. > >>>>>>> > >>>>>>> Once these 2 steps are done, the raw bayer frames can be input to > >>>>>>> the ImgU V4L2 subdev for processing. > >>>>>> > >>>>>> Do I need to capture from both the output and viewfinder nodes ? How > >>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > >>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in > >>>>>> that picture ? > >>>> > >>>> The output capture should be set, the viewfinder can be disabled. > >>>> The IF and BDS are seen as crop and compose of the imgu input video > >>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > >>>> pads. > >>> > >>> The GDC is the last block in the pipeline according to the information > >>> provided above. How can it be seen as the subdev sink pad ? That doesn't > >>> make sense to me. I'm not asking for the MC graph to expose all internal > >>> blocks of the ImgU, but if you want to retain a single subdev model, the > >>> format on the sink pad needs to correspond to what is provided to the > >>> ImgU. Please see figure 4.6 of > >>> https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for > >>> more information regarding how you can use the sink crop, sink compose > >>> and source crop rectangles. > > > > [snip]
Hi Laurent, On Wed, Jan 02, 2019 at 10:20:13AM +0200, Laurent Pinchart wrote: > Hello Bingbu, > > On Wednesday, 2 January 2019 04:38:33 EET Bingbu Cao wrote: > > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > > >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > > >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > > >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > > >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > >>>>>> > > >>>>>> [snip] > > >>>>>> > > >>>>>>> I can see a couple of steps missing in the script below. > > >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November > > >>>>>>> /000040.html) > > >>>>>>> > > >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for > > >>>>>>> image processing"... > > >>>>>>> > > >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > >>>>>>> > > >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the > > >>>>>>> control id 0x009819a1 as below. > > >>>>>>> > > >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > >>>>>> > > >>>>>> I assume the control takes a valid default value ? It's better to set > > >>>>>> it explicitly anyway, so I'll do so. > > >>>> > > >>>> The video mode is set by default. If you want to set to still mode or > > >>>> change mode, you need set the subdev control. > > >>>> > > >>>>>>> 2. ImgU pipeline needs to be configured for image processing as > > >>>>>>> below. > > >>>>>>> > > >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > > >>>>>>> have the processed image output to the DDR memory. > > >>>>>>> > > >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > >>>>>>> Geometric Distortion Correction (GDC) -> DDR > > >>>>>>> > > >>>>>>> The ImgU V4L2 subdev has to be configured with the supported > > >>>>>>> resolutions in all the above HW blocks, for a given input > > >>>>>>> resolution. > > >>>>>>> > > >>>>>>> For a given supported resolution for an input frame, the Input > > >>>>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured with > > >>>>>>> the supported resolutions. This information can be obtained by > > >>>>>>> looking at the following IPU3 ISP configuration table for ov5670 > > >>>>>>> sensor. > > >>>>>>> > > >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays > > >>>>>>> /+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > > >>>>>>> files/gcss/graph_settings_ov5670.xml > > >>>>>>> > > >>>>>>> For the ov5670 example, for an input frame with a resolution of > > >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > > >>>>>>> corresponding resolutions for input feeder, BDS and GDC are > > >>>>>>> 2592x1944, 2592x1944 and 2560x1920 respectively. > > >>>>>> > > >>>>>> How is the GDC output resolution computed from the input resolution ? > > >>>>>> Does the GDC always consume 32 columns and 22 lines ? > > >>>> > > >>>> All the intermediate resolutions in the pipeline are determined by the > > >>>> actual use case, in other word determined by the IMGU input > > >>>> resolution(sensor output) and the final output and viewfinder > > >>>> resolution. BDS mainly do Bayer downscaling, it has limitation that the > > >>>> downscaling factor must be a value a integer multiple of 1/32. > > >>>> GDC output depends on the input and width should be x8 and height x4 > > >>>> alignment. > > >>> > > >>> Thank you for the information. This will need to be captured in the > > >>> documentation, along with information related to how each block in the > > >>> hardware pipeline interacts with the image size. It should be possible > > >>> for a developer to compute the output and viewfinder resolutions based > > >>> on the parameters of the image processing algorithms just with the > > >>> information contained in the driver documentation. > > >>> > > >>>>>>> The following steps prepare the ImgU ISP pipeline for the image > > >>>>>>> processing. > > >>>>>>> > > >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the > > >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > > >>>>>>> obtained > > >>>>>>> above. > > >>>>>> > > >>>>>> If I understand things correctly, the GDC resolution is the pipeline > > >>>>>> output resolution. Why is it configured on pad 0 ? > > >>>> > > >>>> We see the GDC output resolution as the input of output system, the > > >>>> sink pad format is used for output and viewfinder resolutions. > > >>> > > >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > > >>> the ImgU input, the format configured there should correspond to the > > >>> format on the connected video node, and should thus be the sensor > > >>> format. You can then use the crop and compose rectangles on pad 0, along > > >>> with the format, crop and compose rectangles on the output and > > >>> viewfinder pads, to configure the device. This should be fixed in the > > >>> driver, and the documentation should then be updated accordingly. > > >> > > >> Hi, Laurent, > > >> > > >> Thanks for your review. > > >> > > >> I think it make sense for me that using Pad 0 as the ImgU input(IF). > > >> However, I prefer using the 2 source pads for output and viewfinder. > > >> It makes more sense because the output and viewfinder are independent > > >> output. > > >> > > >> The whole pipeline in ImgU looks like: > > >> IF --> BDS --> GDC ---> OUTPUT > > >> |-----> VF > > >> > > >> The BDS is used to do Bayer downscaling and GDC can do cropping. > > > > > > Does this mean that the main output and the viewfinder output share the > > > same scaler, and that the only difference in size between the two outputs > > > is solely due to cropping ? > > > > Laurent, > > No, output only can do crop and viewfinder support crop and scaling, they > > share same input. > > Then you can't support this with a single subdev for the ImgU, you need at > least two subdevs. I can offer more guidance, but I'll need more information > about the GDC. While the current documentation only defines the functionality of the compose target for sink pads, there are a few sensor drivers supporting it on source pads already. Some drivers such as the OMAP3 ISP also use the format on source pads to configure scaling. The current API certainly allows exposing the compose rectangle also on the source pads, but to make that generic we'd need to amend the API to tell in which order these steps take place. In the meantime the behaviour remains device specific. > > > >> My understanding is that scaled size is configured on the CROP rectangle > > >> by COMPOSE selection target, the order seems like not aligned with the > > >> actual processing in ImgU if we set the crop/compose on sink pad. > > >> > > >> Is there some rules for the order of the configuration in the subdev API? > > >> Could I use crop selection based on the scaled size? > > > > > > Please see figure 4.6 in > > > https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html. > > > Scaling is configured on the sink pad through the crop and compose > > > rectangles, while the source crop rectangle is used to perform cropping > > > on the output. If you have a single scaler in the hardware pipeline you > > > can thus configure it on the sink pad, with output and viewfinder > > > separate cropping configure on the source pad. > > > > > >>>>>>> 2. The ImgU V4L2 subdev cropping should be set by using the > > >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > >>>>>>> target, using the input feeder height and width. > > >>>>>>> > > >>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the > > >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > >>>>>>> target, using the BDS height and width. > > >>>>>>> > > >>>>>>> Once these 2 steps are done, the raw bayer frames can be input to > > >>>>>>> the ImgU V4L2 subdev for processing. > > >>>>>> > > >>>>>> Do I need to capture from both the output and viewfinder nodes ? How > > >>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > > >>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in > > >>>>>> that picture ? > > >>>> > > >>>> The output capture should be set, the viewfinder can be disabled. > > >>>> The IF and BDS are seen as crop and compose of the imgu input video > > >>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > > >>>> pads. > > >>> > > >>> The GDC is the last block in the pipeline according to the information > > >>> provided above. How can it be seen as the subdev sink pad ? That doesn't > > >>> make sense to me. I'm not asking for the MC graph to expose all internal > > >>> blocks of the ImgU, but if you want to retain a single subdev model, the > > >>> format on the sink pad needs to correspond to what is provided to the > > >>> ImgU. Please see figure 4.6 of > > >>> https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for > > >>> more information regarding how you can use the sink crop, sink compose > > >>> and source crop rectangles. > > > > > > [snip] > > -- > Regards, > > Laurent Pinchart > > >
Hi Raj, Yong, Bingbu, Tianshu, On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa <tfiga@chromium.org> wrote: > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > <laurent.pinchart@ideasonboard.com> wrote: > > > > Hellon > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > > Hello Yong, > > > > > > Could you please have a look at the crash reported below ? > > > > A bit more information to help you debugging this. I've enabled KASAN in the > > kernel configuration, and get the following use-after-free reports. > > > > [ 166.332920] ================================================================== > > [ 166.332937] BUG: KASAN: use-after-free in __cached_rbnode_delete_update+0x36/0x202 > > [ 166.332944] Read of size 8 at addr ffff888133823718 by task yavta/1305 > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C 4.20.0-rc6+ #3 > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 166.332959] Call Trace: > > [ 166.332967] dump_stack+0x5b/0x81 > > [ 166.332974] print_address_description+0x65/0x227 > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > [ 166.332983] kasan_report+0x247/0x285 > > [ 166.332989] __cached_rbnode_delete_update+0x36/0x202 > > [ 166.332995] private_free_iova+0x57/0x6d > > [ 166.332999] __free_iova+0x23/0x31 > > [ 166.333011] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > Thanks Laurent, I think this is a very good hint. It looks like we're > basically freeing and already freed IOVA and corrupting some allocator > state? Did you have any luck in reproducing and fixing this double free issue? Best regards, Tomasz > > > [ 166.333022] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 166.333032] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > [ 166.333043] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > [ 166.333056] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > [ 166.333067] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > > [ 166.333079] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > [ 166.333088] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > [ 166.333096] ? __mutex_lock_interruptible_slowpath+0xf/0xf > > [ 166.333104] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > [ 166.333123] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.333142] ? copy_overflow+0x14/0x14 [videodev] > > [ 166.333147] ? slab_free_freelist_hook+0x46/0x94 > > [ 166.333151] ? kfree+0x107/0x1a0 > > [ 166.333169] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.333187] ? copy_overflow+0x14/0x14 [videodev] > > [ 166.333203] ? v4l_enumstd+0x49/0x49 [videodev] > > [ 166.333207] ? __wake_up_common+0x342/0x342 > > [ 166.333215] ? atomic_long_add_return+0x15/0x24 > > [ 166.333219] ? ldsem_up_read+0x15/0x29 > > [ 166.333223] ? tty_write+0x4c6/0x4d8 > > [ 166.333227] ? n_tty_receive_char_special+0x1152/0x1152 > > [ 166.333244] ? video_usercopy+0x8ae/0x8ae [videodev] > > [ 166.333260] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.333266] vfs_ioctl+0x76/0x89 > > [ 166.333271] do_vfs_ioctl+0xb33/0xb7e > > [ 166.333275] ? __switch_to_asm+0x40/0x70 > > [ 166.333279] ? __switch_to_asm+0x40/0x70 > > [ 166.333282] ? __switch_to_asm+0x34/0x70 > > [ 166.333286] ? __switch_to_asm+0x40/0x70 > > [ 166.333290] ? ioctl_preallocate+0x174/0x174 > > [ 166.333294] ? __switch_to+0x71c/0xb00 > > [ 166.333299] ? compat_start_thread+0x6b/0x6b > > [ 166.333302] ? __switch_to_asm+0x34/0x70 > > [ 166.333305] ? __switch_to_asm+0x40/0x70 > > [ 166.333309] ? mmdrop+0x12/0x23 > > [ 166.333313] ? finish_task_switch+0x34d/0x3de > > [ 166.333319] ? __schedule+0x1004/0x1045 > > [ 166.333325] ? firmware_map_remove+0x119/0x119 > > [ 166.333330] ksys_ioctl+0x50/0x70 > > [ 166.333335] __x64_sys_ioctl+0x82/0x89 > > [ 166.333340] do_syscall_64+0xa0/0xd2 > > [ 166.333345] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.333349] RIP: 0033:0x7f2481541f47 > > [ 166.333354] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 166.333357] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > > [ 166.333362] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > > [ 166.333364] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > > [ 166.333367] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > > [ 166.333369] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > > [ 166.333372] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > [ 166.333383] Allocated by task 1305: > > [ 166.333389] kasan_kmalloc+0x8a/0x98 > > [ 166.333392] slab_post_alloc_hook+0x31/0x51 > > [ 166.333396] kmem_cache_alloc+0xd7/0x174 > > [ 166.333399] alloc_iova+0x24/0x2ea > > [ 166.333407] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] > > [ 166.333415] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > > [ 166.333424] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > > [ 166.333433] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > > [ 166.333442] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > > [ 166.333449] vb2_start_streaming+0x164/0x33b [videobuf2_common] > > [ 166.333455] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > > [ 166.333471] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.333487] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.333501] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.333505] vfs_ioctl+0x76/0x89 > > [ 166.333508] do_vfs_ioctl+0xb33/0xb7e > > [ 166.333511] ksys_ioctl+0x50/0x70 > > [ 166.333514] __x64_sys_ioctl+0x82/0x89 > > [ 166.333518] do_syscall_64+0xa0/0xd2 > > [ 166.333521] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.333526] Freed by task 1301: > > [ 166.333532] __kasan_slab_free+0xfa/0x11c > > [ 166.333535] slab_free_freelist_hook+0x46/0x94 > > [ 166.333538] kmem_cache_free+0x7b/0x172 > > [ 166.333542] __free_iova+0x23/0x31 > > [ 166.333550] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > [ 166.333557] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 166.333566] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > [ 166.333574] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > [ 166.333584] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > [ 166.333593] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > [ 166.333599] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > [ 166.333606] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > [ 166.333621] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.333637] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.333652] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.333655] vfs_ioctl+0x76/0x89 > > [ 166.333658] do_vfs_ioctl+0xb33/0xb7e > > [ 166.333662] ksys_ioctl+0x50/0x70 > > [ 166.333665] __x64_sys_ioctl+0x82/0x89 > > [ 166.333668] do_syscall_64+0xa0/0xd2 > > [ 166.333671] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.333678] The buggy address belongs to the object at ffff888133823700 > > which belongs to the cache iommu_iova of size 40 > > [ 166.333685] The buggy address is located 24 bytes inside of > > 40-byte region [ffff888133823700, ffff888133823728) > > [ 166.333690] The buggy address belongs to the page: > > [ 166.333696] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 > > [ 166.333703] flags: 0x200000000010200(slab|head) > > [ 166.333710] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 > > [ 166.333717] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 > > [ 166.333720] page dumped because: kasan: bad access detected > > > > [ 166.333726] Memory state around the buggy address: > > [ 166.333732] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.333737] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.333742] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc > > [ 166.333745] ^ > > [ 166.333750] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.333755] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.333759] ================================================================== > > [ 166.333762] Disabling lock debugging due to kernel taint > > [ 166.333764] ================================================================== > > [ 166.333770] BUG: KASAN: double-free or invalid-free in kmem_cache_free+0x7b/0x172 > > > > [ 166.333780] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 > > [ 166.333782] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 166.333783] Call Trace: > > [ 166.333789] dump_stack+0x5b/0x81 > > [ 166.333795] print_address_description+0x65/0x227 > > [ 166.333799] ? kmem_cache_free+0x7b/0x172 > > [ 166.333803] kasan_report_invalid_free+0x67/0xa0 > > [ 166.333807] ? kmem_cache_free+0x7b/0x172 > > [ 166.333812] __kasan_slab_free+0x86/0x11c > > [ 166.333817] slab_free_freelist_hook+0x46/0x94 > > [ 166.333822] kmem_cache_free+0x7b/0x172 > > [ 166.333826] ? __free_iova+0x23/0x31 > > [ 166.333831] __free_iova+0x23/0x31 > > [ 166.333840] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > [ 166.333851] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 166.333861] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > [ 166.333872] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > [ 166.333885] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > [ 166.333896] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > > [ 166.333908] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > [ 166.333917] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > [ 166.333923] ? __mutex_lock_interruptible_slowpath+0xf/0xf > > [ 166.333932] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > [ 166.333950] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.333970] ? copy_overflow+0x14/0x14 [videodev] > > [ 166.333974] ? slab_free_freelist_hook+0x46/0x94 > > [ 166.333979] ? kfree+0x107/0x1a0 > > [ 166.333997] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.334015] ? copy_overflow+0x14/0x14 [videodev] > > [ 166.334031] ? v4l_enumstd+0x49/0x49 [videodev] > > [ 166.334035] ? __wake_up_common+0x342/0x342 > > [ 166.334042] ? atomic_long_add_return+0x15/0x24 > > [ 166.334046] ? ldsem_up_read+0x15/0x29 > > [ 166.334050] ? tty_write+0x4c6/0x4d8 > > [ 166.334054] ? n_tty_receive_char_special+0x1152/0x1152 > > [ 166.334071] ? video_usercopy+0x8ae/0x8ae [videodev] > > [ 166.334087] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.334092] vfs_ioctl+0x76/0x89 > > [ 166.334097] do_vfs_ioctl+0xb33/0xb7e > > [ 166.334101] ? __switch_to_asm+0x40/0x70 > > [ 166.334105] ? __switch_to_asm+0x40/0x70 > > [ 166.334108] ? __switch_to_asm+0x34/0x70 > > [ 166.334111] ? __switch_to_asm+0x40/0x70 > > [ 166.334116] ? ioctl_preallocate+0x174/0x174 > > [ 166.334120] ? __switch_to+0x71c/0xb00 > > [ 166.334124] ? compat_start_thread+0x6b/0x6b > > [ 166.334127] ? __switch_to_asm+0x34/0x70 > > [ 166.334130] ? __switch_to_asm+0x40/0x70 > > [ 166.334134] ? mmdrop+0x12/0x23 > > [ 166.334137] ? finish_task_switch+0x34d/0x3de > > [ 166.334143] ? __schedule+0x1004/0x1045 > > [ 166.334148] ? firmware_map_remove+0x119/0x119 > > [ 166.334153] ksys_ioctl+0x50/0x70 > > [ 166.334158] __x64_sys_ioctl+0x82/0x89 > > [ 166.334163] do_syscall_64+0xa0/0xd2 > > [ 166.334167] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.334171] RIP: 0033:0x7f2481541f47 > > [ 166.334175] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 166.334177] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > > [ 166.334181] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > > [ 166.334184] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > > [ 166.334186] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > > [ 166.334189] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > > [ 166.334191] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > [ 166.334201] Allocated by task 1305: > > [ 166.334207] kasan_kmalloc+0x8a/0x98 > > [ 166.334210] slab_post_alloc_hook+0x31/0x51 > > [ 166.334213] kmem_cache_alloc+0xd7/0x174 > > [ 166.334216] alloc_iova+0x24/0x2ea > > [ 166.334225] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] > > [ 166.334233] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > > [ 166.334241] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > > [ 166.334250] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > > [ 166.334259] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > > [ 166.334266] vb2_start_streaming+0x164/0x33b [videobuf2_common] > > [ 166.334273] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > > [ 166.334288] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.334304] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.334319] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.334322] vfs_ioctl+0x76/0x89 > > [ 166.334325] do_vfs_ioctl+0xb33/0xb7e > > [ 166.334328] ksys_ioctl+0x50/0x70 > > [ 166.334332] __x64_sys_ioctl+0x82/0x89 > > [ 166.334335] do_syscall_64+0xa0/0xd2 > > [ 166.334338] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.334343] Freed by task 1301: > > [ 166.334349] __kasan_slab_free+0xfa/0x11c > > [ 166.334352] slab_free_freelist_hook+0x46/0x94 > > [ 166.334355] kmem_cache_free+0x7b/0x172 > > [ 166.334359] __free_iova+0x23/0x31 > > [ 166.334367] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > [ 166.334375] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 166.334383] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > [ 166.334392] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > [ 166.334401] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > [ 166.334410] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > [ 166.334416] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > [ 166.334423] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > [ 166.334438] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.334454] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.334469] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.334472] vfs_ioctl+0x76/0x89 > > [ 166.334475] do_vfs_ioctl+0xb33/0xb7e > > [ 166.334479] ksys_ioctl+0x50/0x70 > > [ 166.334482] __x64_sys_ioctl+0x82/0x89 > > [ 166.334485] do_syscall_64+0xa0/0xd2 > > [ 166.334488] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.334494] The buggy address belongs to the object at ffff888133823700 > > which belongs to the cache iommu_iova of size 40 > > [ 166.334501] The buggy address is located 0 bytes inside of > > 40-byte region [ffff888133823700, ffff888133823728) > > [ 166.334506] The buggy address belongs to the page: > > [ 166.334511] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 > > [ 166.334517] flags: 0x200000000010200(slab|head) > > [ 166.334524] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 > > [ 166.334530] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 > > [ 166.334533] page dumped because: kasan: bad access detected > > > > [ 166.334539] Memory state around the buggy address: > > [ 166.334544] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.334549] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.334554] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc > > [ 166.334558] ^ > > [ 166.334562] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.334567] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.334571] ================================================================== > > [ 166.340377] ================================================================== > > [ 166.340388] BUG: KASAN: double-free or invalid-free in kfree+0x107/0x1a0 > > > > [ 166.340399] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 > > [ 166.340401] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 166.340403] Call Trace: > > [ 166.340410] dump_stack+0x5b/0x81 > > [ 166.340416] print_address_description+0x65/0x227 > > [ 166.340420] ? kfree+0x107/0x1a0 > > [ 166.340425] kasan_report_invalid_free+0x67/0xa0 > > [ 166.340428] ? kfree+0x107/0x1a0 > > [ 166.340433] __kasan_slab_free+0x86/0x11c > > [ 166.340438] slab_free_freelist_hook+0x46/0x94 > > [ 166.340443] kfree+0x107/0x1a0 > > [ 166.340454] ? ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > > [ 166.340464] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > > [ 166.340475] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 166.340485] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > [ 166.340495] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > [ 166.340509] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > [ 166.340520] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > > [ 166.340531] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > [ 166.340541] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > [ 166.340548] ? __mutex_lock_interruptible_slowpath+0xf/0xf > > [ 166.340557] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > [ 166.340575] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.340595] ? copy_overflow+0x14/0x14 [videodev] > > [ 166.340600] ? slab_free_freelist_hook+0x46/0x94 > > [ 166.340604] ? kfree+0x107/0x1a0 > > [ 166.340622] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.340640] ? copy_overflow+0x14/0x14 [videodev] > > [ 166.340657] ? v4l_enumstd+0x49/0x49 [videodev] > > [ 166.340660] ? __wake_up_common+0x342/0x342 > > [ 166.340668] ? atomic_long_add_return+0x15/0x24 > > [ 166.340672] ? ldsem_up_read+0x15/0x29 > > [ 166.340677] ? tty_write+0x4c6/0x4d8 > > [ 166.340681] ? n_tty_receive_char_special+0x1152/0x1152 > > [ 166.340698] ? video_usercopy+0x8ae/0x8ae [videodev] > > [ 166.340714] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.340720] vfs_ioctl+0x76/0x89 > > [ 166.340725] do_vfs_ioctl+0xb33/0xb7e > > [ 166.340729] ? __switch_to_asm+0x40/0x70 > > [ 166.340733] ? __switch_to_asm+0x40/0x70 > > [ 166.340736] ? __switch_to_asm+0x34/0x70 > > [ 166.340739] ? __switch_to_asm+0x40/0x70 > > [ 166.340743] ? ioctl_preallocate+0x174/0x174 > > [ 166.340748] ? __switch_to+0x71c/0xb00 > > [ 166.340752] ? compat_start_thread+0x6b/0x6b > > [ 166.340756] ? __switch_to_asm+0x34/0x70 > > [ 166.340759] ? __switch_to_asm+0x40/0x70 > > [ 166.340762] ? mmdrop+0x12/0x23 > > [ 166.340766] ? finish_task_switch+0x34d/0x3de > > [ 166.340772] ? __schedule+0x1004/0x1045 > > [ 166.340777] ? firmware_map_remove+0x119/0x119 > > [ 166.340782] ksys_ioctl+0x50/0x70 > > [ 166.340788] __x64_sys_ioctl+0x82/0x89 > > [ 166.340793] do_syscall_64+0xa0/0xd2 > > [ 166.340797] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 166.340802] RIP: 0033:0x7f2481541f47 > > [ 166.340806] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 166.340809] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > > [ 166.340813] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > > [ 166.340816] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > > [ 166.340819] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > > [ 166.340821] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > > [ 166.340824] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > [ 166.340834] Allocated by task 1305: > > [ 166.340840] kasan_kmalloc+0x8a/0x98 > > [ 166.340844] __kmalloc_node+0x193/0x1ba > > [ 166.340848] kvmalloc_node+0x44/0x6d > > [ 166.340856] ipu3_dmamap_alloc+0x1c9/0x83f [ipu3_imgu] > > [ 166.340864] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > > [ 166.340873] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > > [ 166.340882] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > > [ 166.340891] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > > [ 166.340897] vb2_start_streaming+0x164/0x33b [videobuf2_common] > > [ 166.340904] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > > [ 166.340920] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.340935] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.340950] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.340954] vfs_ioctl+0x76/0x89 > > [ 166.340957] do_vfs_ioctl+0xb33/0xb7e > > [ 166.340960] ksys_ioctl+0x50/0x70 > > [ 166.340963] __x64_sys_ioctl+0x82/0x89 > > [ 166.340966] do_syscall_64+0xa0/0xd2 > > [ 166.340969] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.340974] Freed by task 1301: > > [ 166.340980] __kasan_slab_free+0xfa/0x11c > > [ 166.340983] slab_free_freelist_hook+0x46/0x94 > > [ 166.340986] kfree+0x107/0x1a0 > > [ 166.340994] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > > [ 166.341002] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > [ 166.341010] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > [ 166.341019] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > [ 166.341028] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > [ 166.341037] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > [ 166.341043] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > [ 166.341050] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > [ 166.341066] __video_do_ioctl+0x625/0x887 [videodev] > > [ 166.341081] video_usercopy+0x3a3/0x8ae [videodev] > > [ 166.341096] v4l2_ioctl+0xb7/0xc5 [videodev] > > [ 166.341100] vfs_ioctl+0x76/0x89 > > [ 166.341103] do_vfs_ioctl+0xb33/0xb7e > > [ 166.341106] ksys_ioctl+0x50/0x70 > > [ 166.341109] __x64_sys_ioctl+0x82/0x89 > > [ 166.341112] do_syscall_64+0xa0/0xd2 > > [ 166.341116] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.341122] The buggy address belongs to the object at ffff88811d228440 > > which belongs to the cache kmalloc-8 of size 8 > > [ 166.341129] The buggy address is located 0 bytes inside of > > 8-byte region [ffff88811d228440, ffff88811d228448) > > [ 166.341134] The buggy address belongs to the page: > > [ 166.341140] page:ffffea0004748a00 count:1 mapcount:0 mapping:ffff88815a80c340 index:0xffff88811d228f80 compound_mapcount: 0 > > [ 166.341146] flags: 0x200000000010200(slab|head) > > [ 166.341153] raw: 0200000000010200 ffffea000564b288 ffffea00049ef708 ffff88815a80c340 > > [ 166.341159] raw: ffff88811d228f80 0000000000160013 00000001ffffffff 0000000000000000 > > [ 166.341163] page dumped because: kasan: bad access detected > > > > [ 166.341169] Memory state around the buggy address: > > [ 166.341174] ffff88811d228300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.341179] ffff88811d228380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.341184] >ffff88811d228400: fc fc fc fc fc fc fc fc fb fc fc fc fc fc fc fc > > [ 166.341188] ^ > > [ 166.341192] ffff88811d228480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.341197] ffff88811d228500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > [ 166.341201] ================================================================== > > > > > On Tuesday, 11 December 2018 16:20:43 EET Laurent Pinchart wrote: > > > > On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > > > > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > > >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > > >> > > > > >> [snip] > > > > >> > > > > >>> I can see a couple of steps missing in the script below. > > > > >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 > > > > >>> 00040.html) > > > > >>> > > > > >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > > >>> documentation", > > > > >>> under section "Configuring ImgU V4L2 subdev for image processing"... > > > > >>> > > > > >>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > >>> > > > > >>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > > >>> desired (e.g 0 for video mode or 1 for still mode) through the control > > > > >>> id 0x009819a1 as below. > > > > >>> > > > > >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > > >> > > > > >> I assume the control takes a valid default value ? It's better to set > > > > >> it > > > > >> explicitly anyway, so I'll do so. > > > > >> > > > > >>> 2. ImgU pipeline needs to be configured for image processing as below. > > > > >>> > > > > >>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > > >>> have the processed image output to the DDR memory. > > > > >>> > > > > >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > > >>> Geometric Distortion Correction (GDC) -> DDR > > > > >>> > > > > >>> The ImgU V4L2 subdev has to be configured with the supported > > > > >>> resolutions in all the above HW blocks, for a given input resolution. > > > > >>> > > > > >>> For a given supported resolution for an input frame, the Input Feeder, > > > > >>> Bayer Down Scaling and GDC blocks should be configured with the > > > > >>> supported resolutions. This information can be obtained by looking at > > > > >>> the following IPU3 ISP configuration table for ov5670 sensor. > > > > >>> > > > > >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > > > > >>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files > > > > >>> / > > > > >>> gcss/graph_settings_ov5670.xml > > > > >>> > > > > >>> For the ov5670 example, for an input frame with a resolution of > > > > >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > > > >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > > > >>> 2560x1920 respectively. > > > > >> > > > > >> How is the GDC output resolution computed from the input resolution ? > > > > >> Does the GDC always consume 32 columns and 22 lines ? > > > > >> > > > > >>> The following steps prepare the ImgU ISP pipeline for the image > > > > >>> processing. > > > > >>> > > > > >>> 1. The ImgU V4L2 subdev data format should be set by using the > > > > >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > > > > >>> above. > > > > >> > > > > >> If I understand things correctly, the GDC resolution is the pipeline > > > > >> output resolution. Why is it configured on pad 0 ? > > > > >> > > > > >>> 2. The ImgU V4L2 subdev cropping should be set by using the > > > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > > > >>> target, using the input feeder height and width. > > > > >>> > > > > >>> 3. The ImgU V4L2 subdev composing should be set by using the > > > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > > > >>> target, using the BDS height and width. > > > > >>> > > > > >>> Once these 2 steps are done, the raw bayer frames can be input to the > > > > >>> ImgU V4L2 subdev for processing. > > > > >> > > > > >> Do I need to capture from both the output and viewfinder nodes ? How > > > > >> are > > > > >> they related to the IF -> BDS -> GDC pipeline, are they both fed from > > > > >> the GDC output ? If so, how does the viewfinder scaler fit in that > > > > >> picture ? > > > > >> > > > > >> I have tried the above configuration with the IPU3 v8 driver, and while > > > > >> the kernel doesn't crash, no images get processed. The userspace > > > > >> processes wait forever for buffers to be ready. I then configured pad 2 > > > > >> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ > > > > >> > > > > >> There's one problem though: during capture, or very soon after it, the > > > > >> machine locks up completely. I suspect a memory corruption, as when it > > > > >> doesn't log immediately commands such as dmesg will not produce any > > > > >> output and just block, until the system freezes soon after (especially > > > > >> when moving the mouse). > > > > >> > > > > >> I would still call this an improvement to some extent, but there's > > > > >> definitely room for more improvements :-) > > > > >> > > > > >> To reproduce the issue, you can run the ipu3-process.sh script > > > > >> (attached > > > > >> to this e-mail) with the following arguments: > > > > >> > > > > >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > > > > > > > > This should have read > > > > > > > > $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > > > > > > > Without the --vf argument no images are processed. > > > > > > > > It seems that the Intel mail server blocked the mail that contained the > > > > script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > > > > > > > >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in > > > > >> the IPU3-specific Bayer format (for a total of 6469632 bytes). > > > > > > > > > > I managed to get the dmesg output, and it doesn't look pretty. > > > > > > > > > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > > > > > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > > > > > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > > > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > > > mac80211 > > > > > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp > > > > > cfg80211 > > > > > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > > > videobuf2_memops videobuf2_v4l2 videobuf2_common > > > > > processor_thermal_device > > > > > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common > > > > > videodev > > > > > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs > > > > > cros_ec_core > > > > > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > > > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > > > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > > > > > drm_panel_orientation_quirks i2c_hid hid > > > > > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > > > > > 4.20.0-rc6+ #2 > > > > > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > > > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > > > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc > > > > > f3 > > > > > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > > > > > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > > > > > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > > > > > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > > > > > 000000000000000c > > > > > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > > > > > 00000000ffffffff > > > > > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > > > > > ffff8f5cfaba16f0 > > > > > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > > > > > ffff8f5cf58f0028 > > > > > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > > > > > ffff8f5cf58f04e8 > > > > > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) > > > > > knlGS: > > > > > 0000000000000000 > > > > > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > > > > > 00000000003606e0 > > > > > [ 571.217301] Call Trace: > > > > > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > > > > > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > > > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > > > > > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > > > > > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > > > > > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > > > > > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > > > > > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > > > > > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > > > > > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > > > > > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > > > > > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > > > > > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > > > > > [ 571.217480] vfs_ioctl+0x1e/0x2b > > > > > [ 571.217486] do_vfs_ioctl+0x531/0x559 > > > > > [ 571.217494] ? vfs_write+0xd1/0xdf > > > > > [ 571.217500] ksys_ioctl+0x50/0x70 > > > > > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > > > > > [ 571.217512] do_syscall_64+0x53/0x60 > > > > > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > > > > > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 > > > > > 48 > > > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > > > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > > > > > 0000000000000010 > > > > > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > > > > > 00007f85cf9b9f47 > > > > > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > > > > > 0000000000000003 > > > > > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > > > > > 00007f85d009c700 > > > > > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > > > > > 000055f4c4dc0b06 > > > > > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > > > > > 00007ffc59057825 > > > > > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > > > > > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > > > > > > > And after fixing another issue in the capture script (which was setting > > > > the > > > > format on the ImgU subdev pad 3 to 2560x1920 but capture in 1920x1080), I > > > > now get plenty of the following messages: > > > > > > > > [ 221.366131] BUG: Bad page state in process yavta pfn:14a4ff > > > > [ 221.366134] page:ffffde5d45293fc0 count:-1 mapcount:0 mapping: > > > > 0000000000000000 index:0x0 > > > > [ 221.366137] flags: 0x200000000000000() > > > > [ 221.366140] raw: 0200000000000000 dead000000000100 dead000000000200 > > > > 0000000000000000 > > > > [ 221.366143] raw: 0000000000000000 0000000000000000 ffffffffffffffff > > > > 0000000000000000 > > > > [ 221.366145] page dumped because: nonzero _refcount > > > > [ 221.366147] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp mac80211 iwlwifi > > > > cfg80211 hid_multitouch 8250_dw ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > > videobuf2_memops videobuf2_v4l2 processor_thermal_device videobuf2_common > > > > intel_soc_dts_iosf ov13858 ov5670 dw9714 v4l2_fwnode v4l2_common videodev > > > > media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone > > > > chromeos_pstore mac_hid int3400_thermal acpi_thermal_rel autofs4 usbhid > > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > > sysfillrect sysimgblt fb_sys_fops sdhci_pci cqhci sdhci drm > > > > drm_panel_orientation_quirks i2c_hid hid > > > > [ 221.366172] CPU: 3 PID: 1022 Comm: yavta Tainted: G B WC > > > > 4.20.0-rc6+ #2 > > > > [ 221.366173] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > > [ 221.366173] Call Trace: > > > > [ 221.366176] dump_stack+0x46/0x59 > > > > [ 221.366179] bad_page+0xf2/0x10c > > > > [ 221.366182] free_pages_check+0x78/0x81 > > > > [ 221.366186] free_pcppages_bulk+0xa6/0x236 > > > > [ 221.366190] free_unref_page+0x4b/0x53 > > > > [ 221.366193] vb2_dma_sg_put+0x95/0xb5 [videobuf2_dma_sg] > > > > [ 221.366197] __vb2_buf_mem_free+0x3a/0x6e [videobuf2_common] > > > > [ 221.366202] __vb2_queue_free+0xe3/0x1be [videobuf2_common] > > > > [ 221.366207] vb2_core_reqbufs+0xe9/0x2cc [videobuf2_common] > > > > [ 221.366212] vb2_ioctl_reqbufs+0x78/0x9e [videobuf2_v4l2] > > > > [ 221.366220] __video_do_ioctl+0x258/0x38e [videodev] > > > > [ 221.366229] video_usercopy+0x25f/0x4e5 [videodev] > > > > [ 221.366237] ? copy_overflow+0x14/0x14 [videodev] > > > > [ 221.366240] ? unmap_region+0xe0/0x10a > > > > [ 221.366250] v4l2_ioctl+0x4d/0x58 [videodev] > > > > [ 221.366253] vfs_ioctl+0x1e/0x2b > > > > [ 221.366255] do_vfs_ioctl+0x531/0x559 > > > > [ 221.366260] ksys_ioctl+0x50/0x70 > > > > [ 221.366263] __x64_sys_ioctl+0x16/0x19 > > > > [ 221.366266] do_syscall_64+0x53/0x60 > > > > [ 221.366269] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 221.366270] RIP: 0033:0x7fbe39f6af47 > > > > [ 221.366272] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > > [ 221.366273] RSP: 002b:00007fff05638e68 EFLAGS: 00000246 ORIG_RAX: > > > > 0000000000000010 > > > > [ 221.366275] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > > > 00007fbe39f6af47 > > > > [ 221.366279] RDX: 00007fff05638f90 RSI: 00000000c0145608 RDI: > > > > 0000000000000003 > > > > [ 221.366283] RBP: 0000000000000004 R08: 0000000000000000 R09: > > > > 0000000000000045 > > > > [ 221.366287] R10: 0000000000000557 R11: 0000000000000246 R12: > > > > 000055c83bd76750 > > > > [ 221.366290] R13: 000055c83b6b26a0 R14: 0000000000000001 R15: > > > > 00007fff0563a825 > > > > -- > > Regards, > > > > Laurent Pinchart > > > > > >
Hi Raj, (CC'ing Jacopo Mondi) On Saturday, 5 January 2019 04:26:16 EET Mani, Rajmohan wrote: > >> On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > >>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > >>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > >>>> > >>>> [snip] > >>>> > >>>>> I can see a couple of steps missing in the script below. > >>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-Nove > >>>>> mber/000040.html) > >>>>> > >>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > >>>>> documentation", under section "Configuring ImgU V4L2 subdev for > >>>>> image processing"... > >>>>> > >>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > >>>>> > >>>>> Also the pipe mode of the corresponding V4L2 subdev should be > >>>>> set as desired (e.g 0 for video mode or 1 for still mode) > >>>>> through the control id 0x009819a1 as below. > >>>>> > >>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > >>>> > >>>> I assume the control takes a valid default value ? It's better to > >>>> set it explicitly anyway, so I'll do so. > >>>> > >>>>> 2. ImgU pipeline needs to be configured for image processing as > >>>>> below. > >>>>> > >>>>> RAW bayer frames go through the following ISP pipeline HW blocks > >>>>> to have the processed image output to the DDR memory. > >>>>> > >>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > >>>>> Geometric Distortion Correction (GDC) -> DDR > >>>>> > >>>>> The ImgU V4L2 subdev has to be configured with the supported > >>>>> resolutions in all the above HW blocks, for a given input > >>>>> resolution. > >>>>> > >>>>> For a given supported resolution for an input frame, the Input > >>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured > >>>>> with the supported resolutions. This information can be obtained > >>>>> by looking at the following IPU3 ISP configuration table for > >>>>> ov5670 sensor. > >>>>> > >>>>> https://chromium.googlesource.com/chromiumos/overlays/board-over > >>>>> lays/+/master/baseboard-poppy/media-libs/cros-camera-hal-configs- > >>>>> poppy/files/gcss/graph_settings_ov5670.xml > >>>>> > >>>>> For the ov5670 example, for an input frame with a resolution of > >>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > >>>>> corresponding resolutions for input feeder, BDS and GDC are > >>>>> 2592x1944, 2592x1944 and > >>>>> 2560x1920 respectively. > >>>> > >>>> How is the GDC output resolution computed from the input > >>>> resolution ? Does the GDC always consume 32 columns and 22 lines ? > >>>> > >>>>> The following steps prepare the ImgU ISP pipeline for the image > >>>>> processing. > >>>>> > >>>>> 1. The ImgU V4L2 subdev data format should be set by using the > >>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > >>>>> obtained above. > >>>> > >>>> If I understand things correctly, the GDC resolution is the > >>>> pipeline output resolution. Why is it configured on pad 0 ? > >>>> > >>>>> 2. The ImgU V4L2 subdev cropping should be set by using the > >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as > >>>>> the target, using the input feeder height and width. > >>>>> > >>>>> 3. The ImgU V4L2 subdev composing should be set by using the > >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with > >>>>> V4L2_SEL_TGT_COMPOSE as the target, using the BDS height and width. > >>>>> > >>>>> Once these 2 steps are done, the raw bayer frames can be input > >>>>> to the ImgU V4L2 subdev for processing. > >>>> > >>>> Do I need to capture from both the output and viewfinder nodes ? > >>>> How are they related to the IF -> BDS -> GDC pipeline, are they > >>>> both fed from the GDC output ? If so, how does the viewfinder > >>>> scaler fit in that picture ? > >>>> > >>>> I have tried the above configuration with the IPU3 v8 driver, and > >>>> while the kernel doesn't crash, no images get processed. The > >>>> userspace processes wait forever for buffers to be ready. I then > >>>> configured pad 2 to 2560x1920 and pad 3 to 1920x1080, and managed > >>>> to capture images \o/ > >>>> > >>>> There's one problem though: during capture, or very soon after it, > >>>> the machine locks up completely. I suspect a memory corruption, as > >>>> when it doesn't log immediately commands such as dmesg will not > >>>> produce any output and just block, until the system freezes soon > >>>> after (especially when moving the mouse). > >>>> > >>>> I would still call this an improvement to some extent, but there's > >>>> definitely room for more improvements :-) > >>>> > >>>> To reproduce the issue, you can run the ipu3-process.sh script > >>>> (attached to this e-mail) with the following arguments: > >>>> > >>>> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > >> > >> This should have read > >> > >> $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > >> > >> Without the --vf argument no images are processed. > >> > >> It seems that the Intel mail server blocked the mail that contained the > >> script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > Here's what I have so far. > > I made the following two changes to get image capture and processing > working with the above script. I can see the output and vf ppm files to come > out well. > > 1. Remove the "# Set formats" inside the configure_pipeline() from the > above script and used a simple application (which I invoked from your > script) to set pad 0 of the "ipu3-imgu 0" V4L2 subdev with these 3 sub > steps. > > a. Set the data format using VIDIOC_SUBDEV_S_FMT on pad 0 > GDC width and height (2560x1920) > > b. Set cropping using VIDIOC_SUBDEV_S_SELECTION on pad 0 > with V4L2_SEL_TGT_CROP as target, using the input feeder width > and height (2592x1944) > > c. Set composing using VIDIOC_SUBDEV_S_SELECTION on pad 0 > with V4L2_SEL_TGT_COMPOSE as the target, using > BDS width and height (2592x1944) This is exactly what the first media-ctl -V call just below the "Set formats" comment does. > 2. Modify yavta app to ignore the error (on "3A stat" node) from > cap_get_buf_type(), so all 4 nodes can be running at the same time. I don't get that error with the latest yavta version. > Can you advise why the " # Set formats" part of configure_pipeline() > in your script is not achieving the same results as the above 3 sub steps? I tried removing the three media-ctl -V lines that configure format on pads 2, 3 and 4. The crash then didn't occur when running the script. However, after rebooting the system and retrying the exact same sequence of operation, the driver crashed again, and I haven't been able to process images without crashing since then. The driver is clearly very sensitive to how formats and selection rectangles are configured, and crashes when userspace doesn't operate exactly as expected. This should all be fixed, likely by rewriting the format and selection rectangle configuration code, especially given that it doesn't comply with how V4L2 configures scaling on subdevs. We can discuss this issue with Sakari to decide on how the code should be architectured. How long do you think it would then take to implement the solution ? > Per Bingbu, the driver might be lacking some implementation around this. > > >>>>> 1. The ImgU V4L2 subdev data format should be set by using the > >>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > >>>>> obtained above. > >>>>> 2. The ImgU V4L2 subdev cropping should be set by using the > >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as > >>>>> the target, using the input feeder height and width. > >>>>> 3. The ImgU V4L2 subdev composing should be set by using the > >>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE > >>>>> as the target, using the BDS height and width. > > > > Sorry for the delay. I got interrupted with some other work. > > I will have a look at this and get back soon.
Hello, On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > Hi Raj, Yong, Bingbu, Tianshu, > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa <tfiga@chromium.org> wrote: > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > Hellon > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > > > Hello Yong, > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > A bit more information to help you debugging this. I've enabled KASAN in the > > > kernel configuration, and get the following use-after-free reports. I tested as well using the ipu-process.sh script shared by Laurent, with the following command line: ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 and I got a very similar trace available at: https://paste.debian.net/hidden/5855e15a/ Please note I have been able to process a set of images (with KASAN enabled the machine does not freeze) but the kernel log gets flooded and it is not possible to process any other frame after this. The issue is currently quite annoying and it's a blocker for libcamera development on IPU3. Please let me know if I can support with more testing. Thanks j > > > > > > [ 166.332920] ================================================================== > > > [ 166.332937] BUG: KASAN: use-after-free in __cached_rbnode_delete_update+0x36/0x202 > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by task yavta/1305 > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C 4.20.0-rc6+ #3 > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > [ 166.332959] Call Trace: > > > [ 166.332967] dump_stack+0x5b/0x81 > > > [ 166.332974] print_address_description+0x65/0x227 > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > [ 166.332983] kasan_report+0x247/0x285 > > > [ 166.332989] __cached_rbnode_delete_update+0x36/0x202 > > > [ 166.332995] private_free_iova+0x57/0x6d > > > [ 166.332999] __free_iova+0x23/0x31 > > > [ 166.333011] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > Thanks Laurent, I think this is a very good hint. It looks like we're > > basically freeing and already freed IOVA and corrupting some allocator > > state? > > Did you have any luck in reproducing and fixing this double free issue? > > Best regards, > Tomasz > > > > > > [ 166.333022] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 166.333032] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > > [ 166.333043] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > > [ 166.333056] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > > [ 166.333067] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > > > [ 166.333079] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > > [ 166.333088] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > > [ 166.333096] ? __mutex_lock_interruptible_slowpath+0xf/0xf > > > [ 166.333104] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > > [ 166.333123] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.333142] ? copy_overflow+0x14/0x14 [videodev] > > > [ 166.333147] ? slab_free_freelist_hook+0x46/0x94 > > > [ 166.333151] ? kfree+0x107/0x1a0 > > > [ 166.333169] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.333187] ? copy_overflow+0x14/0x14 [videodev] > > > [ 166.333203] ? v4l_enumstd+0x49/0x49 [videodev] > > > [ 166.333207] ? __wake_up_common+0x342/0x342 > > > [ 166.333215] ? atomic_long_add_return+0x15/0x24 > > > [ 166.333219] ? ldsem_up_read+0x15/0x29 > > > [ 166.333223] ? tty_write+0x4c6/0x4d8 > > > [ 166.333227] ? n_tty_receive_char_special+0x1152/0x1152 > > > [ 166.333244] ? video_usercopy+0x8ae/0x8ae [videodev] > > > [ 166.333260] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.333266] vfs_ioctl+0x76/0x89 > > > [ 166.333271] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.333275] ? __switch_to_asm+0x40/0x70 > > > [ 166.333279] ? __switch_to_asm+0x40/0x70 > > > [ 166.333282] ? __switch_to_asm+0x34/0x70 > > > [ 166.333286] ? __switch_to_asm+0x40/0x70 > > > [ 166.333290] ? ioctl_preallocate+0x174/0x174 > > > [ 166.333294] ? __switch_to+0x71c/0xb00 > > > [ 166.333299] ? compat_start_thread+0x6b/0x6b > > > [ 166.333302] ? __switch_to_asm+0x34/0x70 > > > [ 166.333305] ? __switch_to_asm+0x40/0x70 > > > [ 166.333309] ? mmdrop+0x12/0x23 > > > [ 166.333313] ? finish_task_switch+0x34d/0x3de > > > [ 166.333319] ? __schedule+0x1004/0x1045 > > > [ 166.333325] ? firmware_map_remove+0x119/0x119 > > > [ 166.333330] ksys_ioctl+0x50/0x70 > > > [ 166.333335] __x64_sys_ioctl+0x82/0x89 > > > [ 166.333340] do_syscall_64+0xa0/0xd2 > > > [ 166.333345] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > [ 166.333349] RIP: 0033:0x7f2481541f47 > > > [ 166.333354] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > [ 166.333357] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > > > [ 166.333362] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > > > [ 166.333364] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > > > [ 166.333367] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > > > [ 166.333369] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > > > [ 166.333372] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > > > [ 166.333383] Allocated by task 1305: > > > [ 166.333389] kasan_kmalloc+0x8a/0x98 > > > [ 166.333392] slab_post_alloc_hook+0x31/0x51 > > > [ 166.333396] kmem_cache_alloc+0xd7/0x174 > > > [ 166.333399] alloc_iova+0x24/0x2ea > > > [ 166.333407] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] > > > [ 166.333415] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > > > [ 166.333424] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > > > [ 166.333433] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > > > [ 166.333442] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > > > [ 166.333449] vb2_start_streaming+0x164/0x33b [videobuf2_common] > > > [ 166.333455] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > > > [ 166.333471] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.333487] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.333501] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.333505] vfs_ioctl+0x76/0x89 > > > [ 166.333508] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.333511] ksys_ioctl+0x50/0x70 > > > [ 166.333514] __x64_sys_ioctl+0x82/0x89 > > > [ 166.333518] do_syscall_64+0xa0/0xd2 > > > [ 166.333521] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 166.333526] Freed by task 1301: > > > [ 166.333532] __kasan_slab_free+0xfa/0x11c > > > [ 166.333535] slab_free_freelist_hook+0x46/0x94 > > > [ 166.333538] kmem_cache_free+0x7b/0x172 > > > [ 166.333542] __free_iova+0x23/0x31 > > > [ 166.333550] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > [ 166.333557] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 166.333566] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > > [ 166.333574] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > > [ 166.333584] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > > [ 166.333593] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > > [ 166.333599] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > > [ 166.333606] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > > [ 166.333621] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.333637] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.333652] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.333655] vfs_ioctl+0x76/0x89 > > > [ 166.333658] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.333662] ksys_ioctl+0x50/0x70 > > > [ 166.333665] __x64_sys_ioctl+0x82/0x89 > > > [ 166.333668] do_syscall_64+0xa0/0xd2 > > > [ 166.333671] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 166.333678] The buggy address belongs to the object at ffff888133823700 > > > which belongs to the cache iommu_iova of size 40 > > > [ 166.333685] The buggy address is located 24 bytes inside of > > > 40-byte region [ffff888133823700, ffff888133823728) > > > [ 166.333690] The buggy address belongs to the page: > > > [ 166.333696] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 > > > [ 166.333703] flags: 0x200000000010200(slab|head) > > > [ 166.333710] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 > > > [ 166.333717] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 > > > [ 166.333720] page dumped because: kasan: bad access detected > > > > > > [ 166.333726] Memory state around the buggy address: > > > [ 166.333732] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.333737] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.333742] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.333745] ^ > > > [ 166.333750] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.333755] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.333759] ================================================================== > > > [ 166.333762] Disabling lock debugging due to kernel taint > > > [ 166.333764] ================================================================== > > > [ 166.333770] BUG: KASAN: double-free or invalid-free in kmem_cache_free+0x7b/0x172 > > > > > > [ 166.333780] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 > > > [ 166.333782] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > [ 166.333783] Call Trace: > > > [ 166.333789] dump_stack+0x5b/0x81 > > > [ 166.333795] print_address_description+0x65/0x227 > > > [ 166.333799] ? kmem_cache_free+0x7b/0x172 > > > [ 166.333803] kasan_report_invalid_free+0x67/0xa0 > > > [ 166.333807] ? kmem_cache_free+0x7b/0x172 > > > [ 166.333812] __kasan_slab_free+0x86/0x11c > > > [ 166.333817] slab_free_freelist_hook+0x46/0x94 > > > [ 166.333822] kmem_cache_free+0x7b/0x172 > > > [ 166.333826] ? __free_iova+0x23/0x31 > > > [ 166.333831] __free_iova+0x23/0x31 > > > [ 166.333840] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > [ 166.333851] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 166.333861] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > > [ 166.333872] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > > [ 166.333885] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > > [ 166.333896] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > > > [ 166.333908] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > > [ 166.333917] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > > [ 166.333923] ? __mutex_lock_interruptible_slowpath+0xf/0xf > > > [ 166.333932] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > > [ 166.333950] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.333970] ? copy_overflow+0x14/0x14 [videodev] > > > [ 166.333974] ? slab_free_freelist_hook+0x46/0x94 > > > [ 166.333979] ? kfree+0x107/0x1a0 > > > [ 166.333997] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.334015] ? copy_overflow+0x14/0x14 [videodev] > > > [ 166.334031] ? v4l_enumstd+0x49/0x49 [videodev] > > > [ 166.334035] ? __wake_up_common+0x342/0x342 > > > [ 166.334042] ? atomic_long_add_return+0x15/0x24 > > > [ 166.334046] ? ldsem_up_read+0x15/0x29 > > > [ 166.334050] ? tty_write+0x4c6/0x4d8 > > > [ 166.334054] ? n_tty_receive_char_special+0x1152/0x1152 > > > [ 166.334071] ? video_usercopy+0x8ae/0x8ae [videodev] > > > [ 166.334087] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.334092] vfs_ioctl+0x76/0x89 > > > [ 166.334097] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.334101] ? __switch_to_asm+0x40/0x70 > > > [ 166.334105] ? __switch_to_asm+0x40/0x70 > > > [ 166.334108] ? __switch_to_asm+0x34/0x70 > > > [ 166.334111] ? __switch_to_asm+0x40/0x70 > > > [ 166.334116] ? ioctl_preallocate+0x174/0x174 > > > [ 166.334120] ? __switch_to+0x71c/0xb00 > > > [ 166.334124] ? compat_start_thread+0x6b/0x6b > > > [ 166.334127] ? __switch_to_asm+0x34/0x70 > > > [ 166.334130] ? __switch_to_asm+0x40/0x70 > > > [ 166.334134] ? mmdrop+0x12/0x23 > > > [ 166.334137] ? finish_task_switch+0x34d/0x3de > > > [ 166.334143] ? __schedule+0x1004/0x1045 > > > [ 166.334148] ? firmware_map_remove+0x119/0x119 > > > [ 166.334153] ksys_ioctl+0x50/0x70 > > > [ 166.334158] __x64_sys_ioctl+0x82/0x89 > > > [ 166.334163] do_syscall_64+0xa0/0xd2 > > > [ 166.334167] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > [ 166.334171] RIP: 0033:0x7f2481541f47 > > > [ 166.334175] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > [ 166.334177] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > > > [ 166.334181] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > > > [ 166.334184] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > > > [ 166.334186] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > > > [ 166.334189] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > > > [ 166.334191] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > > > [ 166.334201] Allocated by task 1305: > > > [ 166.334207] kasan_kmalloc+0x8a/0x98 > > > [ 166.334210] slab_post_alloc_hook+0x31/0x51 > > > [ 166.334213] kmem_cache_alloc+0xd7/0x174 > > > [ 166.334216] alloc_iova+0x24/0x2ea > > > [ 166.334225] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] > > > [ 166.334233] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > > > [ 166.334241] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > > > [ 166.334250] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > > > [ 166.334259] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > > > [ 166.334266] vb2_start_streaming+0x164/0x33b [videobuf2_common] > > > [ 166.334273] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > > > [ 166.334288] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.334304] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.334319] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.334322] vfs_ioctl+0x76/0x89 > > > [ 166.334325] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.334328] ksys_ioctl+0x50/0x70 > > > [ 166.334332] __x64_sys_ioctl+0x82/0x89 > > > [ 166.334335] do_syscall_64+0xa0/0xd2 > > > [ 166.334338] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 166.334343] Freed by task 1301: > > > [ 166.334349] __kasan_slab_free+0xfa/0x11c > > > [ 166.334352] slab_free_freelist_hook+0x46/0x94 > > > [ 166.334355] kmem_cache_free+0x7b/0x172 > > > [ 166.334359] __free_iova+0x23/0x31 > > > [ 166.334367] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > [ 166.334375] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 166.334383] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > > [ 166.334392] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > > [ 166.334401] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > > [ 166.334410] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > > [ 166.334416] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > > [ 166.334423] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > > [ 166.334438] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.334454] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.334469] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.334472] vfs_ioctl+0x76/0x89 > > > [ 166.334475] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.334479] ksys_ioctl+0x50/0x70 > > > [ 166.334482] __x64_sys_ioctl+0x82/0x89 > > > [ 166.334485] do_syscall_64+0xa0/0xd2 > > > [ 166.334488] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 166.334494] The buggy address belongs to the object at ffff888133823700 > > > which belongs to the cache iommu_iova of size 40 > > > [ 166.334501] The buggy address is located 0 bytes inside of > > > 40-byte region [ffff888133823700, ffff888133823728) > > > [ 166.334506] The buggy address belongs to the page: > > > [ 166.334511] page:ffffea0004ce0880 count:1 mapcount:0 mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 > > > [ 166.334517] flags: 0x200000000010200(slab|head) > > > [ 166.334524] raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 ffff8881519e8640 > > > [ 166.334530] raw: 0000000000000000 0000000000120012 00000001ffffffff 0000000000000000 > > > [ 166.334533] page dumped because: kasan: bad access detected > > > > > > [ 166.334539] Memory state around the buggy address: > > > [ 166.334544] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.334549] ffff888133823680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.334554] >ffff888133823700: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.334558] ^ > > > [ 166.334562] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.334567] ffff888133823800: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.334571] ================================================================== > > > [ 166.340377] ================================================================== > > > [ 166.340388] BUG: KASAN: double-free or invalid-free in kfree+0x107/0x1a0 > > > > > > [ 166.340399] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0-rc6+ #3 > > > [ 166.340401] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > [ 166.340403] Call Trace: > > > [ 166.340410] dump_stack+0x5b/0x81 > > > [ 166.340416] print_address_description+0x65/0x227 > > > [ 166.340420] ? kfree+0x107/0x1a0 > > > [ 166.340425] kasan_report_invalid_free+0x67/0xa0 > > > [ 166.340428] ? kfree+0x107/0x1a0 > > > [ 166.340433] __kasan_slab_free+0x86/0x11c > > > [ 166.340438] slab_free_freelist_hook+0x46/0x94 > > > [ 166.340443] kfree+0x107/0x1a0 > > > [ 166.340454] ? ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > > > [ 166.340464] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > > > [ 166.340475] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 166.340485] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > > [ 166.340495] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > > [ 166.340509] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > > [ 166.340520] ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] > > > [ 166.340531] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > > [ 166.340541] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > > [ 166.340548] ? __mutex_lock_interruptible_slowpath+0xf/0xf > > > [ 166.340557] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > > [ 166.340575] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.340595] ? copy_overflow+0x14/0x14 [videodev] > > > [ 166.340600] ? slab_free_freelist_hook+0x46/0x94 > > > [ 166.340604] ? kfree+0x107/0x1a0 > > > [ 166.340622] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.340640] ? copy_overflow+0x14/0x14 [videodev] > > > [ 166.340657] ? v4l_enumstd+0x49/0x49 [videodev] > > > [ 166.340660] ? __wake_up_common+0x342/0x342 > > > [ 166.340668] ? atomic_long_add_return+0x15/0x24 > > > [ 166.340672] ? ldsem_up_read+0x15/0x29 > > > [ 166.340677] ? tty_write+0x4c6/0x4d8 > > > [ 166.340681] ? n_tty_receive_char_special+0x1152/0x1152 > > > [ 166.340698] ? video_usercopy+0x8ae/0x8ae [videodev] > > > [ 166.340714] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.340720] vfs_ioctl+0x76/0x89 > > > [ 166.340725] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.340729] ? __switch_to_asm+0x40/0x70 > > > [ 166.340733] ? __switch_to_asm+0x40/0x70 > > > [ 166.340736] ? __switch_to_asm+0x34/0x70 > > > [ 166.340739] ? __switch_to_asm+0x40/0x70 > > > [ 166.340743] ? ioctl_preallocate+0x174/0x174 > > > [ 166.340748] ? __switch_to+0x71c/0xb00 > > > [ 166.340752] ? compat_start_thread+0x6b/0x6b > > > [ 166.340756] ? __switch_to_asm+0x34/0x70 > > > [ 166.340759] ? __switch_to_asm+0x40/0x70 > > > [ 166.340762] ? mmdrop+0x12/0x23 > > > [ 166.340766] ? finish_task_switch+0x34d/0x3de > > > [ 166.340772] ? __schedule+0x1004/0x1045 > > > [ 166.340777] ? firmware_map_remove+0x119/0x119 > > > [ 166.340782] ksys_ioctl+0x50/0x70 > > > [ 166.340788] __x64_sys_ioctl+0x82/0x89 > > > [ 166.340793] do_syscall_64+0xa0/0xd2 > > > [ 166.340797] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > [ 166.340802] RIP: 0033:0x7f2481541f47 > > > [ 166.340806] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > [ 166.340809] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > > > [ 166.340813] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2481541f47 > > > [ 166.340816] RDX: 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 > > > [ 166.340819] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: 00007f2481c24700 > > > [ 166.340821] R10: 0000000000000020 R11: 0000000000000246 R12: 0000555f1c494b06 > > > [ 166.340824] R13: 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > > > [ 166.340834] Allocated by task 1305: > > > [ 166.340840] kasan_kmalloc+0x8a/0x98 > > > [ 166.340844] __kmalloc_node+0x193/0x1ba > > > [ 166.340848] kvmalloc_node+0x44/0x6d > > > [ 166.340856] ipu3_dmamap_alloc+0x1c9/0x83f [ipu3_imgu] > > > [ 166.340864] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] > > > [ 166.340873] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] > > > [ 166.340882] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] > > > [ 166.340891] ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] > > > [ 166.340897] vb2_start_streaming+0x164/0x33b [videobuf2_common] > > > [ 166.340904] vb2_core_streamon+0x1a1/0x208 [videobuf2_common] > > > [ 166.340920] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.340935] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.340950] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.340954] vfs_ioctl+0x76/0x89 > > > [ 166.340957] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.340960] ksys_ioctl+0x50/0x70 > > > [ 166.340963] __x64_sys_ioctl+0x82/0x89 > > > [ 166.340966] do_syscall_64+0xa0/0xd2 > > > [ 166.340969] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 166.340974] Freed by task 1301: > > > [ 166.340980] __kasan_slab_free+0xfa/0x11c > > > [ 166.340983] slab_free_freelist_hook+0x46/0x94 > > > [ 166.340986] kfree+0x107/0x1a0 > > > [ 166.340994] ipu3_dmamap_free+0x17b/0x1d6 [ipu3_imgu] > > > [ 166.341002] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > [ 166.341010] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] > > > [ 166.341019] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] > > > [ 166.341028] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] > > > [ 166.341037] ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] > > > [ 166.341043] __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] > > > [ 166.341050] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] > > > [ 166.341066] __video_do_ioctl+0x625/0x887 [videodev] > > > [ 166.341081] video_usercopy+0x3a3/0x8ae [videodev] > > > [ 166.341096] v4l2_ioctl+0xb7/0xc5 [videodev] > > > [ 166.341100] vfs_ioctl+0x76/0x89 > > > [ 166.341103] do_vfs_ioctl+0xb33/0xb7e > > > [ 166.341106] ksys_ioctl+0x50/0x70 > > > [ 166.341109] __x64_sys_ioctl+0x82/0x89 > > > [ 166.341112] do_syscall_64+0xa0/0xd2 > > > [ 166.341116] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 166.341122] The buggy address belongs to the object at ffff88811d228440 > > > which belongs to the cache kmalloc-8 of size 8 > > > [ 166.341129] The buggy address is located 0 bytes inside of > > > 8-byte region [ffff88811d228440, ffff88811d228448) > > > [ 166.341134] The buggy address belongs to the page: > > > [ 166.341140] page:ffffea0004748a00 count:1 mapcount:0 mapping:ffff88815a80c340 index:0xffff88811d228f80 compound_mapcount: 0 > > > [ 166.341146] flags: 0x200000000010200(slab|head) > > > [ 166.341153] raw: 0200000000010200 ffffea000564b288 ffffea00049ef708 ffff88815a80c340 > > > [ 166.341159] raw: ffff88811d228f80 0000000000160013 00000001ffffffff 0000000000000000 > > > [ 166.341163] page dumped because: kasan: bad access detected > > > > > > [ 166.341169] Memory state around the buggy address: > > > [ 166.341174] ffff88811d228300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.341179] ffff88811d228380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.341184] >ffff88811d228400: fc fc fc fc fc fc fc fc fb fc fc fc fc fc fc fc > > > [ 166.341188] ^ > > > [ 166.341192] ffff88811d228480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.341197] ffff88811d228500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc > > > [ 166.341201] ================================================================== > > > > > > > On Tuesday, 11 December 2018 16:20:43 EET Laurent Pinchart wrote: > > > > > On Tuesday, 11 December 2018 15:43:53 EET Laurent Pinchart wrote: > > > > > > On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > > > >> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > > > >> > > > > > >> [snip] > > > > > >> > > > > > >>> I can see a couple of steps missing in the script below. > > > > > >>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/0 > > > > > >>> 00040.html) > > > > > >>> > > > > > >>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > > > >>> documentation", > > > > > >>> under section "Configuring ImgU V4L2 subdev for image processing"... > > > > > >>> > > > > > >>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > > >>> > > > > > >>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > > > >>> desired (e.g 0 for video mode or 1 for still mode) through the control > > > > > >>> id 0x009819a1 as below. > > > > > >>> > > > > > >>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > > > >> > > > > > >> I assume the control takes a valid default value ? It's better to set > > > > > >> it > > > > > >> explicitly anyway, so I'll do so. > > > > > >> > > > > > >>> 2. ImgU pipeline needs to be configured for image processing as below. > > > > > >>> > > > > > >>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > > > >>> have the processed image output to the DDR memory. > > > > > >>> > > > > > >>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > > > >>> Geometric Distortion Correction (GDC) -> DDR > > > > > >>> > > > > > >>> The ImgU V4L2 subdev has to be configured with the supported > > > > > >>> resolutions in all the above HW blocks, for a given input resolution. > > > > > >>> > > > > > >>> For a given supported resolution for an input frame, the Input Feeder, > > > > > >>> Bayer Down Scaling and GDC blocks should be configured with the > > > > > >>> supported resolutions. This information can be obtained by looking at > > > > > >>> the following IPU3 ISP configuration table for ov5670 sensor. > > > > > >>> > > > > > >>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > > > > > >>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files > > > > > >>> / > > > > > >>> gcss/graph_settings_ov5670.xml > > > > > >>> > > > > > >>> For the ov5670 example, for an input frame with a resolution of > > > > > >>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > > > > >>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > > > > >>> 2560x1920 respectively. > > > > > >> > > > > > >> How is the GDC output resolution computed from the input resolution ? > > > > > >> Does the GDC always consume 32 columns and 22 lines ? > > > > > >> > > > > > >>> The following steps prepare the ImgU ISP pipeline for the image > > > > > >>> processing. > > > > > >>> > > > > > >>> 1. The ImgU V4L2 subdev data format should be set by using the > > > > > >>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > > > > > >>> above. > > > > > >> > > > > > >> If I understand things correctly, the GDC resolution is the pipeline > > > > > >> output resolution. Why is it configured on pad 0 ? > > > > > >> > > > > > >>> 2. The ImgU V4L2 subdev cropping should be set by using the > > > > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > > > > >>> target, using the input feeder height and width. > > > > > >>> > > > > > >>> 3. The ImgU V4L2 subdev composing should be set by using the > > > > > >>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > > > > >>> target, using the BDS height and width. > > > > > >>> > > > > > >>> Once these 2 steps are done, the raw bayer frames can be input to the > > > > > >>> ImgU V4L2 subdev for processing. > > > > > >> > > > > > >> Do I need to capture from both the output and viewfinder nodes ? How > > > > > >> are > > > > > >> they related to the IF -> BDS -> GDC pipeline, are they both fed from > > > > > >> the GDC output ? If so, how does the viewfinder scaler fit in that > > > > > >> picture ? > > > > > >> > > > > > >> I have tried the above configuration with the IPU3 v8 driver, and while > > > > > >> the kernel doesn't crash, no images get processed. The userspace > > > > > >> processes wait forever for buffers to be ready. I then configured pad 2 > > > > > >> to 2560x1920 and pad 3 to 1920x1080, and managed to capture images \o/ > > > > > >> > > > > > >> There's one problem though: during capture, or very soon after it, the > > > > > >> machine locks up completely. I suspect a memory corruption, as when it > > > > > >> doesn't log immediately commands such as dmesg will not produce any > > > > > >> output and just block, until the system freezes soon after (especially > > > > > >> when moving the mouse). > > > > > >> > > > > > >> I would still call this an improvement to some extent, but there's > > > > > >> definitely room for more improvements :-) > > > > > >> > > > > > >> To reproduce the issue, you can run the ipu3-process.sh script > > > > > >> (attached > > > > > >> to this e-mail) with the following arguments: > > > > > >> > > > > > >> $ ipu3-process.sh --out 2560x1920 frame-2592x1944.cio2 > > > > > > > > > > This should have read > > > > > > > > > > $ ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > > > > > > > > > Without the --vf argument no images are processed. > > > > > > > > > > It seems that the Intel mail server blocked the mail that contained the > > > > > script. You can find a copy at http://paste.debian.net/hidden/fd5bb8df/. > > > > > > > > > > >> frame-2592x1944.cio2 is a binary file containing a 2592x1944 images in > > > > > >> the IPU3-specific Bayer format (for a total of 6469632 bytes). > > > > > > > > > > > > I managed to get the dmesg output, and it doesn't look pretty. > > > > > > > > > > > > [ 571.217192] WARNING: CPU: 3 PID: 1303 at /home/laurent/src/iob/oss/ > > > > > > libcamera/linux/drivers/staging/media/ipu3/ipu3-dmamap.c:172 > > > > > > ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > > > > [ 571.217196] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > > > > mac80211 > > > > > > iwlwifi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp > > > > > > cfg80211 > > > > > > 8250_dw hid_multitouch ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > > > > videobuf2_memops videobuf2_v4l2 videobuf2_common > > > > > > processor_thermal_device > > > > > > intel_soc_dts_iosf ov13858 dw9714 ov5670 v4l2_fwnode v4l2_common > > > > > > videodev > > > > > > at24 media int3403_thermal int340x_thermal_zone cros_ec_lpcs > > > > > > cros_ec_core > > > > > > int3400_thermal chromeos_pstore mac_hid acpi_thermal_rel autofs4 usbhid > > > > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > > > > sysfillrect sdhci_pci sysimgblt fb_sys_fops cqhci sdhci drm > > > > > > drm_panel_orientation_quirks i2c_hid hid > > > > > > [ 571.217254] CPU: 3 PID: 1303 Comm: yavta Tainted: G C > > > > > > 4.20.0-rc6+ #2 > > > > > > [ 571.217256] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > > > > [ 571.217267] RIP: 0010:ipu3_dmamap_unmap+0x30/0x75 [ipu3_imgu] > > > > > > [ 571.217271] Code: 54 55 48 8d af d0 6e 00 00 53 48 8b 76 10 49 89 fc > > > > > > f3 > > > > > > 48 0f bc 8f f0 6e 00 00 48 89 ef 48 d3 ee e8 e6 73 d9 e6 48 85 c0 75 07 > > > > > > <0f> 0b 5b 5d 41 5c c3 48 8b 70 20 48 89 c3 48 8b 40 18 49 8b bc 24 > > > > > > [ 571.217274] RSP: 0018:ffffb675021c7b38 EFLAGS: 00010246 > > > > > > [ 571.217278] RAX: 0000000000000000 RBX: ffff8f5cf58f8448 RCX: > > > > > > 000000000000000c > > > > > > [ 571.217280] RDX: 0000000000000000 RSI: 0000000000000202 RDI: > > > > > > 00000000ffffffff > > > > > > [ 571.217283] RBP: ffff8f5cf58f6ef8 R08: 00000000000006c5 R09: > > > > > > ffff8f5cfaba16f0 > > > > > > [ 571.217286] R10: ffff8f5cbf508f98 R11: 000000e03da27aba R12: > > > > > > ffff8f5cf58f0028 > > > > > > [ 571.217289] R13: ffff8f5cf58f0028 R14: 0000000000000000 R15: > > > > > > ffff8f5cf58f04e8 > > > > > > [ 571.217293] FS: 00007f85d009c700(0000) GS:ffff8f5cfab80000(0000) > > > > > > knlGS: > > > > > > 0000000000000000 > > > > > > [ 571.217296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > > > [ 571.217299] CR2: 00007f3440fce4b0 CR3: 000000014abf2001 CR4: > > > > > > 00000000003606e0 > > > > > > [ 571.217301] Call Trace: > > > > > > [ 571.217316] ipu3_dmamap_free+0x5b/0x8f [ipu3_imgu] > > > > > > [ 571.217326] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] > > > > > > [ 571.217338] ipu3_css_pipeline_cleanup+0x59/0x8f [ipu3_imgu] > > > > > > [ 571.217348] ipu3_css_stop_streaming+0x15b/0x20f [ipu3_imgu] > > > > > > [ 571.217360] imgu_s_stream+0x5a/0x30a [ipu3_imgu] > > > > > > [ 571.217371] ? ipu3_all_nodes_streaming+0x14f/0x16b [ipu3_imgu] > > > > > > [ 571.217382] ipu3_vb2_stop_streaming+0xe4/0x10f [ipu3_imgu] > > > > > > [ 571.217392] __vb2_queue_cancel+0x2b/0x1b8 [videobuf2_common] > > > > > > [ 571.217402] vb2_core_streamoff+0x30/0x71 [videobuf2_common] > > > > > > [ 571.217418] __video_do_ioctl+0x258/0x38e [videodev] > > > > > > [ 571.217438] video_usercopy+0x25f/0x4e5 [videodev] > > > > > > [ 571.217453] ? copy_overflow+0x14/0x14 [videodev] > > > > > > [ 571.217471] v4l2_ioctl+0x4d/0x58 [videodev] > > > > > > [ 571.217480] vfs_ioctl+0x1e/0x2b > > > > > > [ 571.217486] do_vfs_ioctl+0x531/0x559 > > > > > > [ 571.217494] ? vfs_write+0xd1/0xdf > > > > > > [ 571.217500] ksys_ioctl+0x50/0x70 > > > > > > [ 571.217506] __x64_sys_ioctl+0x16/0x19 > > > > > > [ 571.217512] do_syscall_64+0x53/0x60 > > > > > > [ 571.217519] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > [ 571.217524] RIP: 0033:0x7f85cf9b9f47 > > > > > > [ 571.217528] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 > > > > > > 48 > > > > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > > > > [ 571.217531] RSP: 002b:00007ffc59056b78 EFLAGS: 00000246 ORIG_RAX: > > > > > > 0000000000000010 > > > > > > [ 571.217535] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > > > > > > 00007f85cf9b9f47 > > > > > > [ 571.217537] RDX: 00007ffc59056b84 RSI: 0000000040045613 RDI: > > > > > > 0000000000000003 > > > > > > [ 571.217540] RBP: 000055f4c4dc0af8 R08: 00007f85cd7c4000 R09: > > > > > > 00007f85d009c700 > > > > > > [ 571.217542] R10: 0000000000000020 R11: 0000000000000246 R12: > > > > > > 000055f4c4dc0b06 > > > > > > [ 571.217545] R13: 0000000000000004 R14: 00007ffc59056d50 R15: > > > > > > 00007ffc59057825 > > > > > > [ 571.217553] ---[ end trace 4b42bd84953eff53 ]--- > > > > > > [ 571.318645] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > > > > > > > > > And after fixing another issue in the capture script (which was setting > > > > > the > > > > > format on the ImgU subdev pad 3 to 2560x1920 but capture in 1920x1080), I > > > > > now get plenty of the following messages: > > > > > > > > > > [ 221.366131] BUG: Bad page state in process yavta pfn:14a4ff > > > > > [ 221.366134] page:ffffde5d45293fc0 count:-1 mapcount:0 mapping: > > > > > 0000000000000000 index:0x0 > > > > > [ 221.366137] flags: 0x200000000000000() > > > > > [ 221.366140] raw: 0200000000000000 dead000000000100 dead000000000200 > > > > > 0000000000000000 > > > > > [ 221.366143] raw: 0000000000000000 0000000000000000 ffffffffffffffff > > > > > 0000000000000000 > > > > > [ 221.366145] page dumped because: nonzero _refcount > > > > > [ 221.366147] Modules linked in: asix usbnet mii zram arc4 iwlmvm > > > > > intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp mac80211 iwlwifi > > > > > cfg80211 hid_multitouch 8250_dw ipu3_cio2 ipu3_imgu(C) videobuf2_dma_sg > > > > > videobuf2_memops videobuf2_v4l2 processor_thermal_device videobuf2_common > > > > > intel_soc_dts_iosf ov13858 ov5670 dw9714 v4l2_fwnode v4l2_common videodev > > > > > media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone > > > > > chromeos_pstore mac_hid int3400_thermal acpi_thermal_rel autofs4 usbhid > > > > > mmc_block hid_generic i915 video i2c_algo_bit drm_kms_helper syscopyarea > > > > > sysfillrect sysimgblt fb_sys_fops sdhci_pci cqhci sdhci drm > > > > > drm_panel_orientation_quirks i2c_hid hid > > > > > [ 221.366172] CPU: 3 PID: 1022 Comm: yavta Tainted: G B WC > > > > > 4.20.0-rc6+ #2 > > > > > [ 221.366173] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > > > > [ 221.366173] Call Trace: > > > > > [ 221.366176] dump_stack+0x46/0x59 > > > > > [ 221.366179] bad_page+0xf2/0x10c > > > > > [ 221.366182] free_pages_check+0x78/0x81 > > > > > [ 221.366186] free_pcppages_bulk+0xa6/0x236 > > > > > [ 221.366190] free_unref_page+0x4b/0x53 > > > > > [ 221.366193] vb2_dma_sg_put+0x95/0xb5 [videobuf2_dma_sg] > > > > > [ 221.366197] __vb2_buf_mem_free+0x3a/0x6e [videobuf2_common] > > > > > [ 221.366202] __vb2_queue_free+0xe3/0x1be [videobuf2_common] > > > > > [ 221.366207] vb2_core_reqbufs+0xe9/0x2cc [videobuf2_common] > > > > > [ 221.366212] vb2_ioctl_reqbufs+0x78/0x9e [videobuf2_v4l2] > > > > > [ 221.366220] __video_do_ioctl+0x258/0x38e [videodev] > > > > > [ 221.366229] video_usercopy+0x25f/0x4e5 [videodev] > > > > > [ 221.366237] ? copy_overflow+0x14/0x14 [videodev] > > > > > [ 221.366240] ? unmap_region+0xe0/0x10a > > > > > [ 221.366250] v4l2_ioctl+0x4d/0x58 [videodev] > > > > > [ 221.366253] vfs_ioctl+0x1e/0x2b > > > > > [ 221.366255] do_vfs_ioctl+0x531/0x559 > > > > > [ 221.366260] ksys_ioctl+0x50/0x70 > > > > > [ 221.366263] __x64_sys_ioctl+0x16/0x19 > > > > > [ 221.366266] do_syscall_64+0x53/0x60 > > > > > [ 221.366269] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > [ 221.366270] RIP: 0033:0x7fbe39f6af47 > > > > > [ 221.366272] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > > > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > > > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > > > > [ 221.366273] RSP: 002b:00007fff05638e68 EFLAGS: 00000246 ORIG_RAX: > > > > > 0000000000000010 > > > > > [ 221.366275] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: > > > > > 00007fbe39f6af47 > > > > > [ 221.366279] RDX: 00007fff05638f90 RSI: 00000000c0145608 RDI: > > > > > 0000000000000003 > > > > > [ 221.366283] RBP: 0000000000000004 R08: 0000000000000000 R09: > > > > > 0000000000000045 > > > > > [ 221.366287] R10: 0000000000000557 R11: 0000000000000246 R12: > > > > > 000055c83bd76750 > > > > > [ 221.366290] R13: 000055c83b6b26a0 R14: 0000000000000001 R15: > > > > > 00007fff0563a825 > > > > > > -- > > > Regards, > > > > > > Laurent Pinchart > > > > > > > > >
Hi Laurent, Tomasz, Jacopo, > -----Original Message----- > From: Jacopo Mondi [mailto:jacopo@jmondi.org] > Sent: Wednesday, January 09, 2019 8:41 AM > To: Tomasz Figa <tfiga@chromium.org> > Cc: Zhi, Yong <yong.zhi@intel.com>; Mani, Rajmohan > <rajmohan.mani@intel.com>; Qiu, Tian Shu <tian.shu.qiu@intel.com>; Cao, > Bingbu <bingbu.cao@intel.com>; Laurent Pinchart > <laurent.pinchart@ideasonboard.com>; Linux Media Mailing List <linux- > media@vger.kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Mauro > Carvalho Chehab <mchehab@kernel.org>; Hans Verkuil > <hans.verkuil@cisco.com>; Hu, Jerry W <jerry.w.hu@intel.com>; Toivonen, > Tuukka <tuukka.toivonen@intel.com> > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hello, > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > Hi Raj, Yong, Bingbu, Tianshu, > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa <tfiga@chromium.org> wrote: > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > Hellon > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > > > > Hello Yong, > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > A bit more information to help you debugging this. I've enabled > > > > KASAN in the kernel configuration, and get the following use-after-free > reports. > > I tested as well using the ipu-process.sh script shared by Laurent, with the > following command line: > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > and I got a very similar trace available at: > https://paste.debian.net/hidden/5855e15a/ > > Please note I have been able to process a set of images (with KASAN enabled > the machine does not freeze) but the kernel log gets flooded and it is not > possible to process any other frame after this. > > The issue is currently quite annoying and it's a blocker for libcamera > development on IPU3. Please let me know if I can support with more testing. > > Thanks > j > > > > > > > > > [ 166.332920] > > > > > ================================================================ > == > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by task > > > > yavta/1305 > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C 4.20.0- > rc6+ #3 > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ > > > > 166.332959] Call Trace: > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > print_address_description+0x65/0x227 > > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] > > > > __free_iova+0x23/0x31 [ 166.333011] ipu3_dmamap_free+0x118/0x1d6 > > > > [ipu3_imgu] > > > > > > Thanks Laurent, I think this is a very good hint. It looks like > > > we're basically freeing and already freed IOVA and corrupting some > > > allocator state? > > > > Did you have any luck in reproducing and fixing this double free issue? > > This issue is either hard to reproduce or comes with different signatures with the updated yavta (that now supports meta output) with the 4.4 kernel that I have been using. I am switching to 4.20-rc6 for better reproducibility. Enabling KASAN also results in storage space issues on my Chrome device. Will enable this just for ImgU to get ahead and get back with more updates. > > Best regards, > > Tomasz > > > > > > > > > [ 166.333022] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ > > > > 166.333032] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ > > > > 166.333043] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ > > > > 166.333056] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333067] > > > > ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.333079] > > > > ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333088] > > > > __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333096] ? > > > > __mutex_lock_interruptible_slowpath+0xf/0xf > > > > [ 166.333104] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ > > > > 166.333123] __video_do_ioctl+0x625/0x887 [videodev] [ > > > > 166.333142] ? copy_overflow+0x14/0x14 [videodev] [ 166.333147] > > > > ? slab_free_freelist_hook+0x46/0x94 [ 166.333151] ? > > > > kfree+0x107/0x1a0 [ 166.333169] video_usercopy+0x3a3/0x8ae > > > > [videodev] [ 166.333187] ? copy_overflow+0x14/0x14 [videodev] [ > > > > 166.333203] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.333207] ? > > > > __wake_up_common+0x342/0x342 [ 166.333215] ? > > > > atomic_long_add_return+0x15/0x24 [ 166.333219] ? > > > > ldsem_up_read+0x15/0x29 [ 166.333223] ? tty_write+0x4c6/0x4d8 [ > > > > 166.333227] ? n_tty_receive_char_special+0x1152/0x1152 > > > > [ 166.333244] ? video_usercopy+0x8ae/0x8ae [videodev] [ > > > > 166.333260] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333266] > > > > vfs_ioctl+0x76/0x89 [ 166.333271] do_vfs_ioctl+0xb33/0xb7e [ > > > > 166.333275] ? __switch_to_asm+0x40/0x70 [ 166.333279] ? > > > > __switch_to_asm+0x40/0x70 [ 166.333282] ? > > > > __switch_to_asm+0x34/0x70 [ 166.333286] ? > > > > __switch_to_asm+0x40/0x70 [ 166.333290] ? > > > > ioctl_preallocate+0x174/0x174 [ 166.333294] ? > > > > __switch_to+0x71c/0xb00 [ 166.333299] ? > > > > compat_start_thread+0x6b/0x6b [ 166.333302] ? > > > > __switch_to_asm+0x34/0x70 [ 166.333305] ? > > > > __switch_to_asm+0x40/0x70 [ 166.333309] ? mmdrop+0x12/0x23 [ > > > > 166.333313] ? finish_task_switch+0x34d/0x3de [ 166.333319] ? > > > > __schedule+0x1004/0x1045 [ 166.333325] ? > > > > firmware_map_remove+0x119/0x119 [ 166.333330] > > > > ksys_ioctl+0x50/0x70 [ 166.333335] __x64_sys_ioctl+0x82/0x89 [ > > > > 166.333340] do_syscall_64+0xa0/0xd2 [ 166.333345] > > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.333349] RIP: 0033:0x7f2481541f47 [ 166.333354] Code: 00 00 > > > > 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff > > > > c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 > > > > f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ > > > > 166.333357] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: > > > > 0000000000000010 [ 166.333362] RAX: ffffffffffffffda RBX: > > > > 0000000000000000 RCX: 00007f2481541f47 [ 166.333364] RDX: > > > > 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ > > > > 166.333367] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: > > > > 00007f2481c24700 [ 166.333369] R10: 0000000000000020 R11: > > > > 0000000000000246 R12: 0000555f1c494b06 [ 166.333372] R13: > > > > 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > > > > > [ 166.333383] Allocated by task 1305: > > > > [ 166.333389] kasan_kmalloc+0x8a/0x98 [ 166.333392] > > > > slab_post_alloc_hook+0x31/0x51 [ 166.333396] > > > > kmem_cache_alloc+0xd7/0x174 [ 166.333399] alloc_iova+0x24/0x2ea > > > > [ 166.333407] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] [ > > > > 166.333415] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] [ > > > > 166.333424] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] [ > > > > 166.333433] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] [ 166.333442] > > > > ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] [ 166.333449] > > > > vb2_start_streaming+0x164/0x33b [videobuf2_common] [ 166.333455] > > > > vb2_core_streamon+0x1a1/0x208 [videobuf2_common] [ 166.333471] > > > > __video_do_ioctl+0x625/0x887 [videodev] [ 166.333487] > > > > video_usercopy+0x3a3/0x8ae [videodev] [ 166.333501] > > > > v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333505] > > > > vfs_ioctl+0x76/0x89 [ 166.333508] do_vfs_ioctl+0xb33/0xb7e [ > > > > 166.333511] ksys_ioctl+0x50/0x70 [ 166.333514] > > > > __x64_sys_ioctl+0x82/0x89 [ 166.333518] do_syscall_64+0xa0/0xd2 > > > > [ 166.333521] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > > > [ 166.333526] Freed by task 1301: > > > > [ 166.333532] __kasan_slab_free+0xfa/0x11c [ 166.333535] > > > > slab_free_freelist_hook+0x46/0x94 [ 166.333538] > > > > kmem_cache_free+0x7b/0x172 [ 166.333542] __free_iova+0x23/0x31 [ > > > > 166.333550] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ > > > > 166.333557] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ > > > > 166.333566] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ > > > > 166.333574] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ > > > > 166.333584] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333593] > > > > ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333599] > > > > __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333606] > > > > vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.333621] > > > > __video_do_ioctl+0x625/0x887 [videodev] [ 166.333637] > > > > video_usercopy+0x3a3/0x8ae [videodev] [ 166.333652] > > > > v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333655] > > > > vfs_ioctl+0x76/0x89 [ 166.333658] do_vfs_ioctl+0xb33/0xb7e [ > > > > 166.333662] ksys_ioctl+0x50/0x70 [ 166.333665] > > > > __x64_sys_ioctl+0x82/0x89 [ 166.333668] do_syscall_64+0xa0/0xd2 > > > > [ 166.333671] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > > > [ 166.333678] The buggy address belongs to the object at > ffff888133823700 > > > > which belongs to the cache iommu_iova of size 40 [ > > > > 166.333685] The buggy address is located 24 bytes inside of > > > > 40-byte region [ffff888133823700, > > > > ffff888133823728) [ 166.333690] The buggy address belongs to the page: > > > > [ 166.333696] page:ffffea0004ce0880 count:1 mapcount:0 > > > > mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 [ > > > > 166.333703] flags: 0x200000000010200(slab|head) [ 166.333710] > > > > raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 > > > > ffff8881519e8640 [ 166.333717] raw: 0000000000000000 > > > > 0000000000120012 00000001ffffffff 0000000000000000 [ 166.333720] > > > > page dumped because: kasan: bad access detected > > > > > > > > [ 166.333726] Memory state around the buggy address: > > > > [ 166.333732] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc > > > > fc fc fc fc fc [ 166.333737] ffff888133823680: fc fc fc fc fc fc > > > > fc fc fc fc fc fc fc fc fc fc [ 166.333742] >ffff888133823700: fb fb fb fb fb fc fc > fc fc fc fc fc fc fc fc fc > > > > [ 166.333745] ^ > > > > [ 166.333750] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc > > > > fc fc fc fc fc [ 166.333755] ffff888133823800: fc fc fc fc fc fc > > > > fc fc fc fc fc fc fc fc fc fc [ 166.333759] > > > > > ================================================================ > == > > > > [ 166.333762] Disabling lock debugging due to kernel taint [ > > > > 166.333764] > > > > > ================================================================ > == > > > > [ 166.333770] BUG: KASAN: double-free or invalid-free in > > > > kmem_cache_free+0x7b/0x172 > > > > > > > > [ 166.333780] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0- > rc6+ #3 > > > > [ 166.333782] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ > > > > 166.333783] Call Trace: > > > > [ 166.333789] dump_stack+0x5b/0x81 [ 166.333795] > > > > print_address_description+0x65/0x227 > > > > [ 166.333799] ? kmem_cache_free+0x7b/0x172 [ 166.333803] > > > > kasan_report_invalid_free+0x67/0xa0 > > > > [ 166.333807] ? kmem_cache_free+0x7b/0x172 [ 166.333812] > > > > __kasan_slab_free+0x86/0x11c [ 166.333817] > > > > slab_free_freelist_hook+0x46/0x94 [ 166.333822] > > > > kmem_cache_free+0x7b/0x172 [ 166.333826] ? __free_iova+0x23/0x31 > > > > [ 166.333831] __free_iova+0x23/0x31 [ 166.333840] > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ 166.333851] > > > > ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.333861] > > > > ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.333872] > > > > ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.333885] > > > > imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333896] ? > > > > ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.333908] > > > > ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333917] > > > > __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333923] ? > > > > __mutex_lock_interruptible_slowpath+0xf/0xf > > > > [ 166.333932] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ > > > > 166.333950] __video_do_ioctl+0x625/0x887 [videodev] [ > > > > 166.333970] ? copy_overflow+0x14/0x14 [videodev] [ 166.333974] > > > > ? slab_free_freelist_hook+0x46/0x94 [ 166.333979] ? > > > > kfree+0x107/0x1a0 [ 166.333997] video_usercopy+0x3a3/0x8ae > > > > [videodev] [ 166.334015] ? copy_overflow+0x14/0x14 [videodev] [ > > > > 166.334031] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.334035] ? > > > > __wake_up_common+0x342/0x342 [ 166.334042] ? > > > > atomic_long_add_return+0x15/0x24 [ 166.334046] ? > > > > ldsem_up_read+0x15/0x29 [ 166.334050] ? tty_write+0x4c6/0x4d8 [ > > > > 166.334054] ? n_tty_receive_char_special+0x1152/0x1152 > > > > [ 166.334071] ? video_usercopy+0x8ae/0x8ae [videodev] [ > > > > 166.334087] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.334092] > > > > vfs_ioctl+0x76/0x89 [ 166.334097] do_vfs_ioctl+0xb33/0xb7e [ > > > > 166.334101] ? __switch_to_asm+0x40/0x70 [ 166.334105] ? > > > > __switch_to_asm+0x40/0x70 [ 166.334108] ? > > > > __switch_to_asm+0x34/0x70 [ 166.334111] ? > > > > __switch_to_asm+0x40/0x70 [ 166.334116] ? > > > > ioctl_preallocate+0x174/0x174 [ 166.334120] ? > > > > __switch_to+0x71c/0xb00 [ 166.334124] ? > > > > compat_start_thread+0x6b/0x6b [ 166.334127] ? > > > > __switch_to_asm+0x34/0x70 [ 166.334130] ? > > > > __switch_to_asm+0x40/0x70 [ 166.334134] ? mmdrop+0x12/0x23 [ > > > > 166.334137] ? finish_task_switch+0x34d/0x3de [ 166.334143] ? > > > > __schedule+0x1004/0x1045 [ 166.334148] ? > > > > firmware_map_remove+0x119/0x119 [ 166.334153] > > > > ksys_ioctl+0x50/0x70 [ 166.334158] __x64_sys_ioctl+0x82/0x89 [ > > > > 166.334163] do_syscall_64+0xa0/0xd2 [ 166.334167] > > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > [ 166.334171] RIP: 0033:0x7f2481541f47 [ 166.334175] Code: 00 00 > > > > 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff > > > > c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 > > > > f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ > > > > 166.334177] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: > > > > 0000000000000010 [ 166.334181] RAX: ffffffffffffffda RBX: > > > > 0000000000000000 RCX: 00007f2481541f47 [ 166.334184] RDX: > > > > 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ > > > > 166.334186] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: > > > > 00007f2481c24700 [ 166.334189] R10: 0000000000000020 R11: > > > > 0000000000000246 R12: 0000555f1c494b06 [ 166.334191] R13: > > > > 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > [snip]
Hello Raj, On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > Hi Laurent, Tomasz, Jacopo, > > > -----Original Message----- > > From: Jacopo Mondi [mailto:jacopo@jmondi.org] > > Sent: Wednesday, January 09, 2019 8:41 AM > > To: Tomasz Figa <tfiga@chromium.org> > > Cc: Zhi, Yong <yong.zhi@intel.com>; Mani, Rajmohan > > <rajmohan.mani@intel.com>; Qiu, Tian Shu <tian.shu.qiu@intel.com>; Cao, > > Bingbu <bingbu.cao@intel.com>; Laurent Pinchart > > <laurent.pinchart@ideasonboard.com>; Linux Media Mailing List <linux- > > media@vger.kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Mauro > > Carvalho Chehab <mchehab@kernel.org>; Hans Verkuil > > <hans.verkuil@cisco.com>; Hu, Jerry W <jerry.w.hu@intel.com>; Toivonen, > > Tuukka <tuukka.toivonen@intel.com> > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > Hello, > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa <tfiga@chromium.org> wrote: > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > Hellon > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > > > > > Hello Yong, > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > A bit more information to help you debugging this. I've enabled > > > > > KASAN in the kernel configuration, and get the following use-after-free > > reports. > > > > I tested as well using the ipu-process.sh script shared by Laurent, with the > > following command line: > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 frame-2592x1944.cio2 > > > > and I got a very similar trace available at: > > https://paste.debian.net/hidden/5855e15a/ > > > > Please note I have been able to process a set of images (with KASAN enabled > > the machine does not freeze) but the kernel log gets flooded and it is not > > possible to process any other frame after this. > > > > The issue is currently quite annoying and it's a blocker for libcamera > > development on IPU3. Please let me know if I can support with more testing. > > > > Thanks > > j > > > > > > > > > > > > [ 166.332920] > > > > > > > ================================================================ > > == > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by task > > > > > yavta/1305 > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C 4.20.0- > > rc6+ #3 > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ > > > > > 166.332959] Call Trace: > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > print_address_description+0x65/0x227 > > > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] > > > > > __free_iova+0x23/0x31 [ 166.333011] ipu3_dmamap_free+0x118/0x1d6 > > > > > [ipu3_imgu] > > > > > > > > Thanks Laurent, I think this is a very good hint. It looks like > > > > we're basically freeing and already freed IOVA and corrupting some > > > > allocator state? > > > > > > Did you have any luck in reproducing and fixing this double free issue? > > > > > This issue is either hard to reproduce or comes with different signatures with > the updated yavta (that now supports meta output) with the 4.4 kernel that > I have been using. > I am switching to 4.20-rc6 for better reproducibility. > Enabling KASAN also results in storage space issues on my Chrome device. > Will enable this just for ImgU to get ahead and get back with more updates. > Thanks for testing this. For your informations I'm using the following branch, from Sakari's tree: git://linuxtv.org/sailus/media_tree.git ipu3 Although it appears that the media tree master branch has everything that is there, with a few additional patches on top. I should move to use media tree master as well... I have here attached 2 configuration files for v4.20-rc5 I am using on Soraka, in case they might help you. One has KASAN enabled with an increased kernel log size, the other one is the one we use for daily development. Also, please make sure to use (the most) recent media-ctl and yavta utilities, as the ones provided by most distros are usually not recent enough to work with IPU3, but I'm sure you know that already ;) Thanks j > > > Best regards, > > > Tomasz > > > > > > > > > > > > [ 166.333022] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ > > > > > 166.333032] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ > > > > > 166.333043] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ > > > > > 166.333056] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333067] > > > > > ? ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.333079] > > > > > ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333088] > > > > > __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333096] ? > > > > > __mutex_lock_interruptible_slowpath+0xf/0xf > > > > > [ 166.333104] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ > > > > > 166.333123] __video_do_ioctl+0x625/0x887 [videodev] [ > > > > > 166.333142] ? copy_overflow+0x14/0x14 [videodev] [ 166.333147] > > > > > ? slab_free_freelist_hook+0x46/0x94 [ 166.333151] ? > > > > > kfree+0x107/0x1a0 [ 166.333169] video_usercopy+0x3a3/0x8ae > > > > > [videodev] [ 166.333187] ? copy_overflow+0x14/0x14 [videodev] [ > > > > > 166.333203] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.333207] ? > > > > > __wake_up_common+0x342/0x342 [ 166.333215] ? > > > > > atomic_long_add_return+0x15/0x24 [ 166.333219] ? > > > > > ldsem_up_read+0x15/0x29 [ 166.333223] ? tty_write+0x4c6/0x4d8 [ > > > > > 166.333227] ? n_tty_receive_char_special+0x1152/0x1152 > > > > > [ 166.333244] ? video_usercopy+0x8ae/0x8ae [videodev] [ > > > > > 166.333260] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333266] > > > > > vfs_ioctl+0x76/0x89 [ 166.333271] do_vfs_ioctl+0xb33/0xb7e [ > > > > > 166.333275] ? __switch_to_asm+0x40/0x70 [ 166.333279] ? > > > > > __switch_to_asm+0x40/0x70 [ 166.333282] ? > > > > > __switch_to_asm+0x34/0x70 [ 166.333286] ? > > > > > __switch_to_asm+0x40/0x70 [ 166.333290] ? > > > > > ioctl_preallocate+0x174/0x174 [ 166.333294] ? > > > > > __switch_to+0x71c/0xb00 [ 166.333299] ? > > > > > compat_start_thread+0x6b/0x6b [ 166.333302] ? > > > > > __switch_to_asm+0x34/0x70 [ 166.333305] ? > > > > > __switch_to_asm+0x40/0x70 [ 166.333309] ? mmdrop+0x12/0x23 [ > > > > > 166.333313] ? finish_task_switch+0x34d/0x3de [ 166.333319] ? > > > > > __schedule+0x1004/0x1045 [ 166.333325] ? > > > > > firmware_map_remove+0x119/0x119 [ 166.333330] > > > > > ksys_ioctl+0x50/0x70 [ 166.333335] __x64_sys_ioctl+0x82/0x89 [ > > > > > 166.333340] do_syscall_64+0xa0/0xd2 [ 166.333345] > > > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > [ 166.333349] RIP: 0033:0x7f2481541f47 [ 166.333354] Code: 00 00 > > > > > 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff > > > > > c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 > > > > > f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ > > > > > 166.333357] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: > > > > > 0000000000000010 [ 166.333362] RAX: ffffffffffffffda RBX: > > > > > 0000000000000000 RCX: 00007f2481541f47 [ 166.333364] RDX: > > > > > 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ > > > > > 166.333367] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: > > > > > 00007f2481c24700 [ 166.333369] R10: 0000000000000020 R11: > > > > > 0000000000000246 R12: 0000555f1c494b06 [ 166.333372] R13: > > > > > 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > > > > > > > [ 166.333383] Allocated by task 1305: > > > > > [ 166.333389] kasan_kmalloc+0x8a/0x98 [ 166.333392] > > > > > slab_post_alloc_hook+0x31/0x51 [ 166.333396] > > > > > kmem_cache_alloc+0xd7/0x174 [ 166.333399] alloc_iova+0x24/0x2ea > > > > > [ 166.333407] ipu3_dmamap_alloc+0x193/0x83f [ipu3_imgu] [ > > > > > 166.333415] ipu3_css_pool_init+0x80/0xdf [ipu3_imgu] [ > > > > > 166.333424] ipu3_css_start_streaming+0x58df/0x5ddc [ipu3_imgu] [ > > > > > 166.333433] imgu_s_stream+0x2dd/0x6c0 [ipu3_imgu] [ 166.333442] > > > > > ipu3_vb2_start_streaming+0x35f/0x3de [ipu3_imgu] [ 166.333449] > > > > > vb2_start_streaming+0x164/0x33b [videobuf2_common] [ 166.333455] > > > > > vb2_core_streamon+0x1a1/0x208 [videobuf2_common] [ 166.333471] > > > > > __video_do_ioctl+0x625/0x887 [videodev] [ 166.333487] > > > > > video_usercopy+0x3a3/0x8ae [videodev] [ 166.333501] > > > > > v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333505] > > > > > vfs_ioctl+0x76/0x89 [ 166.333508] do_vfs_ioctl+0xb33/0xb7e [ > > > > > 166.333511] ksys_ioctl+0x50/0x70 [ 166.333514] > > > > > __x64_sys_ioctl+0x82/0x89 [ 166.333518] do_syscall_64+0xa0/0xd2 > > > > > [ 166.333521] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > > > > > [ 166.333526] Freed by task 1301: > > > > > [ 166.333532] __kasan_slab_free+0xfa/0x11c [ 166.333535] > > > > > slab_free_freelist_hook+0x46/0x94 [ 166.333538] > > > > > kmem_cache_free+0x7b/0x172 [ 166.333542] __free_iova+0x23/0x31 [ > > > > > 166.333550] ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ > > > > > 166.333557] ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ > > > > > 166.333566] ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ > > > > > 166.333574] ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ > > > > > 166.333584] imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333593] > > > > > ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333599] > > > > > __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333606] > > > > > vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ 166.333621] > > > > > __video_do_ioctl+0x625/0x887 [videodev] [ 166.333637] > > > > > video_usercopy+0x3a3/0x8ae [videodev] [ 166.333652] > > > > > v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.333655] > > > > > vfs_ioctl+0x76/0x89 [ 166.333658] do_vfs_ioctl+0xb33/0xb7e [ > > > > > 166.333662] ksys_ioctl+0x50/0x70 [ 166.333665] > > > > > __x64_sys_ioctl+0x82/0x89 [ 166.333668] do_syscall_64+0xa0/0xd2 > > > > > [ 166.333671] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > > > > > > [ 166.333678] The buggy address belongs to the object at > > ffff888133823700 > > > > > which belongs to the cache iommu_iova of size 40 [ > > > > > 166.333685] The buggy address is located 24 bytes inside of > > > > > 40-byte region [ffff888133823700, > > > > > ffff888133823728) [ 166.333690] The buggy address belongs to the page: > > > > > [ 166.333696] page:ffffea0004ce0880 count:1 mapcount:0 > > > > > mapping:ffff8881519e8640 index:0x0 compound_mapcount: 0 [ > > > > > 166.333703] flags: 0x200000000010200(slab|head) [ 166.333710] > > > > > raw: 0200000000010200 ffffea0004dfc488 ffff88814bfbde70 > > > > > ffff8881519e8640 [ 166.333717] raw: 0000000000000000 > > > > > 0000000000120012 00000001ffffffff 0000000000000000 [ 166.333720] > > > > > page dumped because: kasan: bad access detected > > > > > > > > > > [ 166.333726] Memory state around the buggy address: > > > > > [ 166.333732] ffff888133823600: fc fc fc fc fc fc fc fc fc fc fc > > > > > fc fc fc fc fc [ 166.333737] ffff888133823680: fc fc fc fc fc fc > > > > > fc fc fc fc fc fc fc fc fc fc [ 166.333742] >ffff888133823700: fb fb fb fb fb fc fc > > fc fc fc fc fc fc fc fc fc > > > > > [ 166.333745] ^ > > > > > [ 166.333750] ffff888133823780: fc fc fc fc fc fc fc fc fc fc fc > > > > > fc fc fc fc fc [ 166.333755] ffff888133823800: fc fc fc fc fc fc > > > > > fc fc fc fc fc fc fc fc fc fc [ 166.333759] > > > > > > > ================================================================ > > == > > > > > [ 166.333762] Disabling lock debugging due to kernel taint [ > > > > > 166.333764] > > > > > > > ================================================================ > > == > > > > > [ 166.333770] BUG: KASAN: double-free or invalid-free in > > > > > kmem_cache_free+0x7b/0x172 > > > > > > > > > > [ 166.333780] CPU: 3 PID: 1305 Comm: yavta Tainted: G B C 4.20.0- > > rc6+ #3 > > > > > [ 166.333782] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 [ > > > > > 166.333783] Call Trace: > > > > > [ 166.333789] dump_stack+0x5b/0x81 [ 166.333795] > > > > > print_address_description+0x65/0x227 > > > > > [ 166.333799] ? kmem_cache_free+0x7b/0x172 [ 166.333803] > > > > > kasan_report_invalid_free+0x67/0xa0 > > > > > [ 166.333807] ? kmem_cache_free+0x7b/0x172 [ 166.333812] > > > > > __kasan_slab_free+0x86/0x11c [ 166.333817] > > > > > slab_free_freelist_hook+0x46/0x94 [ 166.333822] > > > > > kmem_cache_free+0x7b/0x172 [ 166.333826] ? __free_iova+0x23/0x31 > > > > > [ 166.333831] __free_iova+0x23/0x31 [ 166.333840] > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] [ 166.333851] > > > > > ipu3_css_pool_cleanup+0x25/0x2f [ipu3_imgu] [ 166.333861] > > > > > ipu3_css_pipeline_cleanup+0x79/0xcf [ipu3_imgu] [ 166.333872] > > > > > ipu3_css_stop_streaming+0x2fe/0x4dc [ipu3_imgu] [ 166.333885] > > > > > imgu_s_stream+0xc0/0x6c0 [ipu3_imgu] [ 166.333896] ? > > > > > ipu3_all_nodes_streaming+0x1ee/0x20d [ipu3_imgu] [ 166.333908] > > > > > ipu3_vb2_stop_streaming+0x27c/0x2d2 [ipu3_imgu] [ 166.333917] > > > > > __vb2_queue_cancel+0xa8/0x705 [videobuf2_common] [ 166.333923] ? > > > > > __mutex_lock_interruptible_slowpath+0xf/0xf > > > > > [ 166.333932] vb2_core_streamoff+0x68/0xf8 [videobuf2_common] [ > > > > > 166.333950] __video_do_ioctl+0x625/0x887 [videodev] [ > > > > > 166.333970] ? copy_overflow+0x14/0x14 [videodev] [ 166.333974] > > > > > ? slab_free_freelist_hook+0x46/0x94 [ 166.333979] ? > > > > > kfree+0x107/0x1a0 [ 166.333997] video_usercopy+0x3a3/0x8ae > > > > > [videodev] [ 166.334015] ? copy_overflow+0x14/0x14 [videodev] [ > > > > > 166.334031] ? v4l_enumstd+0x49/0x49 [videodev] [ 166.334035] ? > > > > > __wake_up_common+0x342/0x342 [ 166.334042] ? > > > > > atomic_long_add_return+0x15/0x24 [ 166.334046] ? > > > > > ldsem_up_read+0x15/0x29 [ 166.334050] ? tty_write+0x4c6/0x4d8 [ > > > > > 166.334054] ? n_tty_receive_char_special+0x1152/0x1152 > > > > > [ 166.334071] ? video_usercopy+0x8ae/0x8ae [videodev] [ > > > > > 166.334087] v4l2_ioctl+0xb7/0xc5 [videodev] [ 166.334092] > > > > > vfs_ioctl+0x76/0x89 [ 166.334097] do_vfs_ioctl+0xb33/0xb7e [ > > > > > 166.334101] ? __switch_to_asm+0x40/0x70 [ 166.334105] ? > > > > > __switch_to_asm+0x40/0x70 [ 166.334108] ? > > > > > __switch_to_asm+0x34/0x70 [ 166.334111] ? > > > > > __switch_to_asm+0x40/0x70 [ 166.334116] ? > > > > > ioctl_preallocate+0x174/0x174 [ 166.334120] ? > > > > > __switch_to+0x71c/0xb00 [ 166.334124] ? > > > > > compat_start_thread+0x6b/0x6b [ 166.334127] ? > > > > > __switch_to_asm+0x34/0x70 [ 166.334130] ? > > > > > __switch_to_asm+0x40/0x70 [ 166.334134] ? mmdrop+0x12/0x23 [ > > > > > 166.334137] ? finish_task_switch+0x34d/0x3de [ 166.334143] ? > > > > > __schedule+0x1004/0x1045 [ 166.334148] ? > > > > > firmware_map_remove+0x119/0x119 [ 166.334153] > > > > > ksys_ioctl+0x50/0x70 [ 166.334158] __x64_sys_ioctl+0x82/0x89 [ > > > > > 166.334163] do_syscall_64+0xa0/0xd2 [ 166.334167] > > > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > > > > [ 166.334171] RIP: 0033:0x7f2481541f47 [ 166.334175] Code: 00 00 > > > > > 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff > > > > > c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 > > > > > f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ > > > > > 166.334177] RSP: 002b:00007fffd6aff9b8 EFLAGS: 00000246 ORIG_RAX: > > > > > 0000000000000010 [ 166.334181] RAX: ffffffffffffffda RBX: > > > > > 0000000000000000 RCX: 00007f2481541f47 [ 166.334184] RDX: > > > > > 00007fffd6aff9c4 RSI: 0000000040045613 RDI: 0000000000000003 [ > > > > > 166.334186] RBP: 0000555f1c494af8 R08: 00007f247f34c000 R09: > > > > > 00007f2481c24700 [ 166.334189] R10: 0000000000000020 R11: > > > > > 0000000000000246 R12: 0000555f1c494b06 [ 166.334191] R13: > > > > > 0000000000000004 R14: 00007fffd6affb90 R15: 00007fffd6b00825 > > > > > > > [snip] # CONFIG_LOCALVERSION_AUTO is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y CONFIG_NO_HZ=y CONFIG_HIGH_RES_TIMERS=y CONFIG_PREEMPT=y CONFIG_BSD_PROCESS_ACCT=y CONFIG_BSD_PROCESS_ACCT_V3=y CONFIG_TASKSTATS=y CONFIG_TASK_DELAY_ACCT=y CONFIG_TASK_XACCT=y CONFIG_TASK_IO_ACCOUNTING=y CONFIG_LOG_BUF_SHIFT=18 CONFIG_CGROUPS=y CONFIG_MEMCG=y CONFIG_MEMCG_SWAP=y CONFIG_BLK_CGROUP=y CONFIG_CGROUP_SCHED=y CONFIG_CFS_BANDWIDTH=y CONFIG_RT_GROUP_SCHED=y CONFIG_CGROUP_PIDS=y CONFIG_CGROUP_FREEZER=y CONFIG_CGROUP_HUGETLB=y CONFIG_CPUSETS=y CONFIG_CGROUP_DEVICE=y CONFIG_CGROUP_CPUACCT=y CONFIG_CGROUP_PERF=y CONFIG_NAMESPACES=y CONFIG_USER_NS=y CONFIG_CHECKPOINT_RESTORE=y CONFIG_BLK_DEV_INITRD=y CONFIG_CC_OPTIMIZE_FOR_SIZE=y CONFIG_SYSCTL_SYSCALL=y # CONFIG_PCSPKR_PLATFORM is not set CONFIG_KALLSYMS_ALL=y CONFIG_BPF_SYSCALL=y CONFIG_EMBEDDED=y # CONFIG_COMPAT_BRK is not set CONFIG_PROFILING=y CONFIG_SMP=y CONFIG_X86_X2APIC=y CONFIG_X86_NUMACHIP=y CONFIG_X86_INTEL_LPSS=y CONFIG_IOSF_MBI_DEBUG=y CONFIG_HYPERVISOR_GUEST=y CONFIG_PARAVIRT=y CONFIG_PARAVIRT_SPINLOCKS=y CONFIG_KVM_DEBUG_FS=y CONFIG_PROCESSOR_SELECT=y CONFIG_GART_IOMMU=y CONFIG_CALGARY_IOMMU=y CONFIG_NR_CPUS=256 CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y CONFIG_I8K=m CONFIG_MICROCODE_AMD=y CONFIG_X86_MSR=m CONFIG_X86_CPUID=m CONFIG_NUMA=y CONFIG_ARCH_MEMORY_PROBE=y CONFIG_X86_PMEM_LEGACY=y CONFIG_X86_CHECK_BIOS_CORRUPTION=y CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1 CONFIG_EFI=y CONFIG_EFI_STUB=y CONFIG_EFI_MIXED=y CONFIG_HZ_1000=y CONFIG_KEXEC=y CONFIG_KEXEC_FILE=y CONFIG_KEXEC_VERIFY_SIG=y CONFIG_CRASH_DUMP=y CONFIG_KEXEC_JUMP=y CONFIG_PHYSICAL_ALIGN=0x1000000 CONFIG_LIVEPATCH=y CONFIG_HIBERNATION=y CONFIG_PM_WAKELOCKS=y CONFIG_PM_DEBUG=y CONFIG_PM_ADVANCED_DEBUG=y CONFIG_PM_TRACE_RTC=y CONFIG_WQ_POWER_EFFICIENT_DEFAULT=y CONFIG_ACPI_EC_DEBUGFS=m CONFIG_ACPI_DOCK=y CONFIG_ACPI_IPMI=m CONFIG_ACPI_PROCESSOR_AGGREGATOR=m CONFIG_ACPI_PCI_SLOT=y CONFIG_ACPI_HOTPLUG_MEMORY=y CONFIG_ACPI_SBS=m CONFIG_ACPI_BGRT=y CONFIG_ACPI_APEI=y CONFIG_ACPI_APEI_GHES=y CONFIG_ACPI_APEI_PCIEAER=y CONFIG_ACPI_APEI_MEMORY_FAILURE=y CONFIG_TPS68470_PMIC_OPREGION=y CONFIG_SFI=y CONFIG_CPU_FREQ_STAT=y CONFIG_CPU_FREQ_GOV_POWERSAVE=y CONFIG_CPU_FREQ_GOV_USERSPACE=y CONFIG_CPU_FREQ_GOV_ONDEMAND=y CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y CONFIG_X86_ACPI_CPUFREQ=y CONFIG_CPU_IDLE_GOV_LADDER=y CONFIG_INTEL_IDLE=y CONFIG_PCIEPORTBUS=y CONFIG_HOTPLUG_PCI_PCIE=y CONFIG_PCIEASPM_DEBUG=y CONFIG_PCI_MSI=y CONFIG_PCI_REALLOC_ENABLE_AUTO=y CONFIG_PCI_STUB=m CONFIG_PCI_IOV=y CONFIG_PCI_PRI=y CONFIG_PCI_PASID=y CONFIG_HOTPLUG_PCI=y CONFIG_HOTPLUG_PCI_ACPI=y CONFIG_HOTPLUG_PCI_ACPI_IBM=m CONFIG_HOTPLUG_PCI_CPCI=y CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m CONFIG_HOTPLUG_PCI_SHPC=y CONFIG_IA32_EMULATION=y CONFIG_X86_X32=y CONFIG_EDD=y CONFIG_EDD_OFF=y CONFIG_DMI_SYSFS=m CONFIG_ISCSI_IBFT_FIND=y CONFIG_EFI_VARS=y CONFIG_EFI_VARS_PSTORE=m # CONFIG_VIRTUALIZATION is not set CONFIG_OPROFILE=m CONFIG_KPROBES=y CONFIG_JUMP_LABEL=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_MODVERSIONS=y CONFIG_MODULE_SRCVERSION_ALL=y CONFIG_BLK_DEV_THROTTLING=y CONFIG_PARTITION_ADVANCED=y CONFIG_ACORN_PARTITION=y CONFIG_ACORN_PARTITION_CUMANA=y CONFIG_ACORN_PARTITION_EESOX=y CONFIG_ACORN_PARTITION_ICS=y CONFIG_ACORN_PARTITION_ADFS=y CONFIG_ACORN_PARTITION_POWERTEC=y CONFIG_ACORN_PARTITION_RISCIX=y CONFIG_AIX_PARTITION=y CONFIG_OSF_PARTITION=y CONFIG_AMIGA_PARTITION=y CONFIG_ATARI_PARTITION=y CONFIG_MAC_PARTITION=y CONFIG_BSD_DISKLABEL=y CONFIG_MINIX_SUBPARTITION=y CONFIG_SOLARIS_X86_PARTITION=y CONFIG_UNIXWARE_DISKLABEL=y CONFIG_LDM_PARTITION=y CONFIG_SGI_PARTITION=y CONFIG_ULTRIX_PARTITION=y CONFIG_SUN_PARTITION=y CONFIG_KARMA_PARTITION=y CONFIG_SYSV68_PARTITION=y CONFIG_CMDLINE_PARTITION=y CONFIG_IOSCHED_CFQ=m CONFIG_CFQ_GROUP_IOSCHED=y CONFIG_IOSCHED_BFQ=y CONFIG_BFQ_GROUP_IOSCHED=y CONFIG_BINFMT_MISC=m CONFIG_MEMORY_HOTPLUG=y CONFIG_MEMORY_HOTREMOVE=y CONFIG_KSM=y CONFIG_DEFAULT_MMAP_MIN_ADDR=65536 CONFIG_MEMORY_FAILURE=y CONFIG_HWPOISON_INJECT=m CONFIG_CLEANCACHE=y CONFIG_FRONTSWAP=y CONFIG_CMA=y CONFIG_MEM_SOFT_DIRTY=y CONFIG_ZSWAP=y CONFIG_ZBUD=y CONFIG_ZSMALLOC=y CONFIG_PGTABLE_MAPPING=y CONFIG_NET=y CONFIG_PACKET=y CONFIG_PACKET_DIAG=m CONFIG_UNIX=y CONFIG_UNIX_DIAG=m CONFIG_NET_KEY=m CONFIG_INET=y CONFIG_IP_MULTICAST=y CONFIG_IP_ADVANCED_ROUTER=y CONFIG_IP_FIB_TRIE_STATS=y CONFIG_IP_MULTIPLE_TABLES=y CONFIG_IP_ROUTE_MULTIPATH=y CONFIG_IP_ROUTE_VERBOSE=y CONFIG_IP_PNP=y CONFIG_IP_PNP_DHCP=y CONFIG_SYN_COOKIES=y # CONFIG_INET_XFRM_MODE_TRANSPORT is not set # CONFIG_INET_XFRM_MODE_TUNNEL is not set # CONFIG_INET_XFRM_MODE_BEET is not set CONFIG_TCP_CONG_ADVANCED=y # CONFIG_TCP_CONG_BIC is not set # CONFIG_TCP_CONG_WESTWOOD is not set # CONFIG_TCP_CONG_HTCP is not set CONFIG_TCP_MD5SIG=y CONFIG_IPV6_ROUTER_PREF=y CONFIG_IPV6_ROUTE_INFO=y # CONFIG_INET6_XFRM_MODE_TRANSPORT is not set # CONFIG_INET6_XFRM_MODE_TUNNEL is not set # CONFIG_INET6_XFRM_MODE_BEET is not set # CONFIG_IPV6_SIT is not set CONFIG_NETLABEL=y CONFIG_NETWORK_SECMARK=y CONFIG_NETFILTER=y CONFIG_NETFILTER_NETLINK_ACCT=m CONFIG_NETFILTER_NETLINK_QUEUE=m CONFIG_NETFILTER_NETLINK_LOG=m CONFIG_NETFILTER_NETLINK_OSF=m CONFIG_NF_TABLES=m CONFIG_NFT_COUNTER=m CONFIG_NFT_LOG=m CONFIG_NFT_LIMIT=m CONFIG_NFT_QUEUE=m CONFIG_NFT_REJECT=m CONFIG_NFT_COMPAT=m CONFIG_NFT_HASH=m CONFIG_NF_SOCKET_IPV4=m CONFIG_NF_TPROXY_IPV4=m CONFIG_NF_DUP_IPV4=m CONFIG_NF_LOG_ARP=m CONFIG_NF_LOG_IPV4=m CONFIG_NF_REJECT_IPV4=m CONFIG_IP_NF_IPTABLES=y CONFIG_IP_NF_FILTER=y CONFIG_NF_SOCKET_IPV6=m CONFIG_NF_TPROXY_IPV6=m CONFIG_NF_DUP_IPV6=m CONFIG_NF_LOG_IPV6=m CONFIG_IP6_NF_IPTABLES=m CONFIG_IP6_NF_TARGET_HL=m CONFIG_IP6_NF_FILTER=m CONFIG_IP6_NF_TARGET_REJECT=m CONFIG_IP6_NF_MANGLE=m CONFIG_IP6_NF_RAW=m CONFIG_IP6_NF_SECURITY=m CONFIG_DNS_RESOLVER=y CONFIG_CGROUP_NET_PRIO=y CONFIG_CGROUP_NET_CLASSID=y CONFIG_BPF_JIT=y CONFIG_CFG80211=m CONFIG_CFG80211_DEBUGFS=y CONFIG_CFG80211_WEXT=y CONFIG_MAC80211=m CONFIG_MAC80211_MESH=y CONFIG_MAC80211_DEBUGFS=y CONFIG_MAC80211_MESSAGE_TRACING=y CONFIG_RFKILL=y CONFIG_RFKILL_INPUT=y CONFIG_RFKILL_GPIO=m CONFIG_DEVTMPFS=y CONFIG_DEVTMPFS_MOUNT=y # CONFIG_STANDALONE is not set CONFIG_DEBUG_DEVRES=y CONFIG_CONNECTOR=y CONFIG_OF=y # CONFIG_PNP_DEBUG_MESSAGES is not set CONFIG_BLK_DEV_NULL_BLK=m CONFIG_BLK_DEV_FD=m CONFIG_BLK_DEV_PCIESSD_MTIP32XX=m CONFIG_ZRAM=m CONFIG_BLK_DEV_UMEM=m CONFIG_BLK_DEV_LOOP=y CONFIG_BLK_DEV_CRYPTOLOOP=m CONFIG_BLK_DEV_DRBD=m CONFIG_BLK_DEV_NBD=m CONFIG_BLK_DEV_SKD=m CONFIG_BLK_DEV_RAM=y CONFIG_BLK_DEV_RAM_SIZE=65536 CONFIG_BLK_DEV_NVME=m CONFIG_SRAM=y CONFIG_EEPROM_AT24=m CONFIG_EEPROM_93CX6=m CONFIG_CB710_CORE=m CONFIG_RAID_ATTRS=m CONFIG_SCSI=y # CONFIG_SCSI_MQ_DEFAULT is not set CONFIG_BLK_DEV_SD=y CONFIG_BLK_DEV_SR=m CONFIG_CHR_DEV_SG=y CONFIG_CHR_DEV_SCH=m CONFIG_SCSI_CONSTANTS=y CONFIG_SCSI_LOGGING=y CONFIG_SCSI_SCAN_ASYNC=y CONFIG_SCSI_SPI_ATTRS=m CONFIG_SCSI_FC_ATTRS=m CONFIG_SCSI_ISCSI_ATTRS=m CONFIG_SCSI_SAS_LIBSAS=m CONFIG_SCSI_SRP_ATTRS=m # CONFIG_SCSI_LOWLEVEL is not set CONFIG_SCSI_OSD_INITIATOR=m CONFIG_SCSI_OSD_ULD=m CONFIG_ATA=m CONFIG_SATA_AHCI=m CONFIG_SATA_AHCI_PLATFORM=m CONFIG_TARGET_CORE=m CONFIG_TCM_IBLOCK=m CONFIG_TCM_FILEIO=m CONFIG_TCM_PSCSI=m CONFIG_LOOPBACK_TARGET=m CONFIG_ISCSI_TARGET=m CONFIG_MACINTOSH_DRIVERS=y CONFIG_MAC_EMUMOUSEBTN=m CONFIG_NETDEVICES=y CONFIG_BONDING=m CONFIG_DUMMY=m CONFIG_EQUALIZER=m CONFIG_NET_TEAM=m CONFIG_NET_TEAM_MODE_BROADCAST=m CONFIG_NET_TEAM_MODE_ROUNDROBIN=m CONFIG_NET_TEAM_MODE_RANDOM=m CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m CONFIG_NET_TEAM_MODE_LOADBALANCE=m CONFIG_MACVLAN=m CONFIG_MACVTAP=m CONFIG_IPVLAN=m CONFIG_VXLAN=m CONFIG_GENEVE=m CONFIG_NETCONSOLE=m CONFIG_NETCONSOLE_DYNAMIC=y CONFIG_TUN=y CONFIG_VETH=m CONFIG_NLMON=m # CONFIG_NET_VENDOR_3COM is not set # CONFIG_NET_VENDOR_ADAPTEC is not set # CONFIG_NET_VENDOR_AGERE is not set # CONFIG_NET_VENDOR_ALACRITECH is not set # CONFIG_NET_VENDOR_ALTEON is not set # CONFIG_NET_VENDOR_AMAZON is not set # CONFIG_NET_VENDOR_AMD is not set # CONFIG_NET_VENDOR_AQUANTIA is not set # CONFIG_NET_VENDOR_ARC is not set # CONFIG_NET_VENDOR_ATHEROS is not set # CONFIG_NET_VENDOR_AURORA is not set # CONFIG_NET_VENDOR_BROADCOM is not set # CONFIG_NET_VENDOR_BROCADE is not set # CONFIG_NET_VENDOR_CADENCE is not set # CONFIG_NET_VENDOR_CAVIUM is not set # CONFIG_NET_VENDOR_CHELSIO is not set # CONFIG_NET_VENDOR_CISCO is not set # CONFIG_NET_VENDOR_CORTINA is not set # CONFIG_NET_VENDOR_DEC is not set # CONFIG_NET_VENDOR_DLINK is not set # CONFIG_NET_VENDOR_EMULEX is not set # CONFIG_NET_VENDOR_EZCHIP is not set # CONFIG_NET_VENDOR_HP is not set # CONFIG_NET_VENDOR_HUAWEI is not set # CONFIG_NET_VENDOR_I825XX is not set CONFIG_E1000=m CONFIG_IGBVF=m # CONFIG_NET_VENDOR_MARVELL is not set # CONFIG_NET_VENDOR_MELLANOX is not set # CONFIG_NET_VENDOR_MICREL is not set # CONFIG_NET_VENDOR_MICROCHIP is not set # CONFIG_NET_VENDOR_MICROSEMI is not set # CONFIG_NET_VENDOR_MYRI is not set # CONFIG_NET_VENDOR_NATSEMI is not set # CONFIG_NET_VENDOR_NETERION is not set # CONFIG_NET_VENDOR_NETRONOME is not set # CONFIG_NET_VENDOR_NI is not set # CONFIG_NET_VENDOR_NVIDIA is not set # CONFIG_NET_VENDOR_OKI is not set # CONFIG_NET_VENDOR_PACKET_ENGINES is not set # CONFIG_NET_VENDOR_QLOGIC is not set # CONFIG_NET_VENDOR_QUALCOMM is not set # CONFIG_NET_VENDOR_RDC is not set # CONFIG_NET_VENDOR_REALTEK is not set # CONFIG_NET_VENDOR_RENESAS is not set # CONFIG_NET_VENDOR_ROCKER is not set # CONFIG_NET_VENDOR_SAMSUNG is not set # CONFIG_NET_VENDOR_SEEQ is not set # CONFIG_NET_VENDOR_SOLARFLARE is not set # CONFIG_NET_VENDOR_SILAN is not set # CONFIG_NET_VENDOR_SIS is not set # CONFIG_NET_VENDOR_SMSC is not set # CONFIG_NET_VENDOR_SOCIONEXT is not set # CONFIG_NET_VENDOR_STMICRO is not set # CONFIG_NET_VENDOR_SUN is not set # CONFIG_NET_VENDOR_SYNOPSYS is not set # CONFIG_NET_VENDOR_TEHUTI is not set # CONFIG_NET_VENDOR_TI is not set # CONFIG_NET_VENDOR_VIA is not set # CONFIG_NET_VENDOR_WIZNET is not set CONFIG_PHYLIB=y CONFIG_PPP=y CONFIG_PPP_BSDCOMP=m CONFIG_PPP_DEFLATE=m CONFIG_PPP_FILTER=y CONFIG_PPP_MPPE=m CONFIG_PPP_MULTILINK=y CONFIG_PPPOE=m CONFIG_PPP_ASYNC=m CONFIG_PPP_SYNC_TTY=m CONFIG_USB_NET_DRIVERS=m CONFIG_USB_USBNET=m CONFIG_USB_NET_CDC_EEM=m CONFIG_USB_NET_CDC_MBIM=m CONFIG_USB_NET_SMSC95XX=m CONFIG_USB_NET_RNDIS_HOST=m # CONFIG_USB_ARMLINUX is not set # CONFIG_USB_NET_ZAURUS is not set # CONFIG_WLAN_VENDOR_ADMTEK is not set # CONFIG_WLAN_VENDOR_ATH is not set # CONFIG_WLAN_VENDOR_ATMEL is not set # CONFIG_WLAN_VENDOR_BROADCOM is not set # CONFIG_WLAN_VENDOR_CISCO is not set CONFIG_IWL4965=m CONFIG_IWL3945=m CONFIG_IWLWIFI=m CONFIG_IWLDVM=m CONFIG_IWLMVM=m CONFIG_IWLWIFI_DEBUGFS=y # CONFIG_WLAN_VENDOR_MARVELL is not set # CONFIG_WLAN_VENDOR_MEDIATEK is not set # CONFIG_WLAN_VENDOR_RALINK is not set # CONFIG_WLAN_VENDOR_REALTEK is not set # CONFIG_WLAN_VENDOR_RSI is not set # CONFIG_WLAN_VENDOR_ST is not set # CONFIG_WLAN_VENDOR_TI is not set # CONFIG_WLAN_VENDOR_ZYDAS is not set # CONFIG_WLAN_VENDOR_QUANTENNA is not set CONFIG_INPUT_FF_MEMLESS=m CONFIG_INPUT_POLLDEV=m CONFIG_INPUT_SPARSEKMAP=m CONFIG_INPUT_MOUSEDEV=y CONFIG_INPUT_MOUSEDEV_PSAUX=y CONFIG_INPUT_EVDEV=y CONFIG_INPUT_EVBUG=m CONFIG_KEYBOARD_CROS_EC=m CONFIG_MOUSE_PS2=m CONFIG_MOUSE_PS2_ELANTECH=y CONFIG_MOUSE_PS2_SENTELIC=y CONFIG_MOUSE_PS2_TOUCHKIT=y CONFIG_MOUSE_PS2_VMMOUSE=y CONFIG_INPUT_TOUCHSCREEN=y CONFIG_TOUCHSCREEN_ATMEL_MXT=y CONFIG_TOUCHSCREEN_ELAN=y CONFIG_TOUCHSCREEN_MELFAS_MIP4=y CONFIG_TOUCHSCREEN_WDT87XX_I2C=y CONFIG_TOUCHSCREEN_USB_COMPOSITE=m CONFIG_TOUCHSCREEN_RM_TS=y CONFIG_INPUT_MISC=y CONFIG_INPUT_UINPUT=m CONFIG_LEGACY_PTY_COUNT=0 CONFIG_SERIAL_NONSTANDARD=y CONFIG_SERIAL_8250=y # CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=48 CONFIG_SERIAL_8250_RUNTIME_UARTS=32 CONFIG_SERIAL_8250_EXTENDED=y CONFIG_SERIAL_8250_MANY_PORTS=y CONFIG_SERIAL_8250_SHARE_IRQ=y CONFIG_SERIAL_8250_RSA=y CONFIG_SERIAL_8250_DW=m # CONFIG_SERIAL_8250_MID is not set CONFIG_SERIAL_KGDB_NMI=y CONFIG_TTY_PRINTK=y CONFIG_IPMI_HANDLER=m CONFIG_IPMI_DEVICE_INTERFACE=m CONFIG_IPMI_SI=m CONFIG_IPMI_SSIF=m CONFIG_IPMI_WATCHDOG=m CONFIG_IPMI_POWEROFF=m CONFIG_HW_RANDOM=y CONFIG_HW_RANDOM_TIMERIOMEM=m CONFIG_HW_RANDOM_INTEL=m CONFIG_HW_RANDOM_AMD=m CONFIG_HW_RANDOM_VIA=m CONFIG_NVRAM=y CONFIG_R3964=m CONFIG_APPLICOM=m CONFIG_HPET=y CONFIG_HANGCHECK_TIMER=m CONFIG_TCG_TPM=y CONFIG_TCG_TIS=y CONFIG_TCG_TIS_I2C_ATMEL=m CONFIG_TCG_TIS_I2C_INFINEON=m CONFIG_TCG_TIS_I2C_NUVOTON=m CONFIG_TCG_NSC=m CONFIG_TCG_ATMEL=m CONFIG_TCG_INFINEON=m CONFIG_TELCLOCK=m CONFIG_I2C=y CONFIG_I2C_CHARDEV=y CONFIG_I2C_I801=y CONFIG_I2C_ISCH=m CONFIG_I2C_ISMT=m CONFIG_I2C_PIIX4=m CONFIG_I2C_SCMI=m CONFIG_I2C_DESIGNWARE_PCI=y CONFIG_I2C_DESIGNWARE_BAYTRAIL=y CONFIG_I2C_CROS_EC_TUNNEL=m CONFIG_I2C_SLAVE=y CONFIG_I2C_SLAVE_EEPROM=m CONFIG_SPI=y CONFIG_PINCTRL_BAYTRAIL=y CONFIG_PINCTRL_CHERRYVIEW=y CONFIG_PINCTRL_BROXTON=y CONFIG_PINCTRL_SUNRISEPOINT=y CONFIG_DEBUG_GPIO=y CONFIG_GPIO_SYSFS=y CONFIG_GPIO_GENERIC_PLATFORM=m CONFIG_GPIO_ICH=m CONFIG_GPIO_LYNXPOINT=m CONFIG_GPIO_SCH=m CONFIG_GPIO_TPS68470=y CONFIG_POWER_RESET=y CONFIG_PDA_POWER=m CONFIG_GENERIC_ADC_BATTERY=m CONFIG_BATTERY_SBS=m CONFIG_CHARGER_GPIO=m CONFIG_CHARGER_MANAGER=y CONFIG_SENSORS_I5500=m CONFIG_SENSORS_CORETEMP=m CONFIG_SENSORS_ACPI_POWER=m CONFIG_THERMAL_GOV_FAIR_SHARE=y CONFIG_THERMAL_GOV_BANG_BANG=y CONFIG_THERMAL_EMULATION=y CONFIG_INTEL_POWERCLAMP=m CONFIG_INTEL_SOC_DTS_THERMAL=m CONFIG_INT340X_THERMAL=m CONFIG_MFD_CROS_EC=m CONFIG_LPC_ICH=y CONFIG_LPC_SCH=y CONFIG_MFD_INTEL_LPSS_ACPI=y CONFIG_MFD_INTEL_LPSS_PCI=y CONFIG_MFD_TPS68470=y CONFIG_REGULATOR=y CONFIG_REGULATOR_FIXED_VOLTAGE=m CONFIG_REGULATOR_VIRTUAL_CONSUMER=m CONFIG_REGULATOR_USERSPACE_CONSUMER=m CONFIG_REGULATOR_GPIO=m CONFIG_REGULATOR_TPS51632=m CONFIG_REGULATOR_TPS62360=m CONFIG_REGULATOR_TPS65023=m CONFIG_REGULATOR_TPS6507X=m CONFIG_MEDIA_SUPPORT=m CONFIG_MEDIA_CAMERA_SUPPORT=y CONFIG_MEDIA_CONTROLLER=y CONFIG_VIDEO_V4L2_SUBDEV_API=y CONFIG_MEDIA_USB_SUPPORT=y CONFIG_USB_VIDEO_CLASS=m # CONFIG_USB_GSPCA is not set CONFIG_MEDIA_PCI_SUPPORT=y CONFIG_VIDEO_IPU3_CIO2=m CONFIG_V4L_PLATFORM_DRIVERS=y # CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set CONFIG_VIDEO_DW9714=m CONFIG_VIDEO_OV5670=m CONFIG_VIDEO_OV7670=m CONFIG_VIDEO_OV13858=m CONFIG_AGP=y CONFIG_AGP_AMD64=y CONFIG_AGP_INTEL=y CONFIG_VGA_SWITCHEROO=y CONFIG_DRM=m CONFIG_DRM_LOAD_EDID_FIRMWARE=y CONFIG_DRM_I915=m CONFIG_FB=y CONFIG_FIRMWARE_EDID=y CONFIG_FB_TILEBLITTING=y CONFIG_FB_INTEL=m CONFIG_FB_SIMPLE=y CONFIG_BACKLIGHT_CLASS_DEVICE=y CONFIG_BACKLIGHT_GENERIC=m CONFIG_FRAMEBUFFER_CONSOLE=y CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y CONFIG_HID=m CONFIG_HID_BATTERY_STRENGTH=y CONFIG_HIDRAW=y CONFIG_UHID=m CONFIG_HID_MULTITOUCH=m CONFIG_USB_HID=m CONFIG_HID_PID=y CONFIG_USB_HIDDEV=y CONFIG_USB_KBD=m CONFIG_USB_MOUSE=m CONFIG_I2C_HID=m CONFIG_USB=y CONFIG_USB_ANNOUNCE_NEW_DEVICES=y CONFIG_USB_DYNAMIC_MINORS=y CONFIG_USB_XHCI_HCD=y CONFIG_USB_EHCI_HCD=y CONFIG_USB_EHCI_ROOT_HUB_TT=y CONFIG_USB_EHCI_HCD_PLATFORM=y CONFIG_USB_OHCI_HCD=y CONFIG_USB_OHCI_HCD_PLATFORM=y CONFIG_USB_UHCI_HCD=y CONFIG_USB_STORAGE=m CONFIG_USB_UAS=m CONFIG_NOP_USB_XCEIV=y CONFIG_USB_GPIO_VBUS=m CONFIG_MMC=y CONFIG_MMC_BLOCK=m CONFIG_SDIO_UART=m CONFIG_MMC_SDHCI=m CONFIG_MMC_SDHCI_PCI=m # CONFIG_MMC_RICOH_MMC is not set CONFIG_MMC_SDHCI_ACPI=m CONFIG_MMC_SDHCI_PLTFM=m CONFIG_LEDS_CLASS=y CONFIG_RTC_CLASS=y CONFIG_RTC_DRV_HID_SENSOR_TIME=m CONFIG_DMADEVICES=y CONFIG_INTEL_IOATDMA=m CONFIG_DW_DMAC=y CONFIG_ASYNC_TX_DMA=y CONFIG_AUXDISPLAY=y # CONFIG_VIRTIO_MENU is not set CONFIG_STAGING=y CONFIG_STAGING_MEDIA=y CONFIG_VIDEO_IPU3_IMGU=m CONFIG_DCDBAS=m CONFIG_DELL_RBU=m CONFIG_CHROMEOS_LAPTOP=m CONFIG_CHROMEOS_PSTORE=m CONFIG_CROS_EC_LPC=m CONFIG_CROS_KBD_LED_BACKLIGHT=m CONFIG_INTEL_IOMMU=y # CONFIG_INTEL_IOMMU_DEFAULT_ON is not set CONFIG_IRQ_REMAP=y CONFIG_MEMORY=y CONFIG_IIO_BUFFER=y CONFIG_IIO_KFIFO_BUF=m CONFIG_RESET_CONTROLLER=y CONFIG_GENERIC_PHY=y CONFIG_POWERCAP=y CONFIG_INTEL_RAPL=m CONFIG_EXT4_FS=y CONFIG_EXT4_FS_POSIX_ACL=y CONFIG_EXT4_FS_SECURITY=y CONFIG_EXT4_ENCRYPTION=y CONFIG_FANOTIFY=y CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y CONFIG_AUTOFS4_FS=m CONFIG_FUSE_FS=y CONFIG_CUSE=m CONFIG_OVERLAY_FS=m CONFIG_FSCACHE=m CONFIG_FSCACHE_STATS=y CONFIG_CACHEFILES=m CONFIG_VFAT_FS=y CONFIG_PROC_KCORE=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_HUGETLBFS=y CONFIG_EFIVAR_FS=y CONFIG_PSTORE_RAM=m CONFIG_NFS_FS=m CONFIG_NFS_V3_ACL=y CONFIG_NFS_V4=m CONFIG_NFS_SWAP=y CONFIG_NFS_V4_1=y CONFIG_NFS_V4_2=y CONFIG_NFS_V4_1_MIGRATION=y CONFIG_NFS_FSCACHE=y CONFIG_SUNRPC_DEBUG=y CONFIG_NLS_DEFAULT="utf8" CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_UTF8=y CONFIG_PERSISTENT_KEYRINGS=y CONFIG_TRUSTED_KEYS=y CONFIG_ENCRYPTED_KEYS=y CONFIG_SECURITY=y CONFIG_SECURITY_NETWORK_XFRM=y CONFIG_INTEL_TXT=y CONFIG_SECURITY_APPARMOR=y # CONFIG_INTEGRITY is not set CONFIG_CRYPTO_ECDH=m CONFIG_CRYPTO_GF128MUL=y CONFIG_CRYPTO_AUTHENC=m CONFIG_CRYPTO_CRC32C_INTEL=y CONFIG_CRYPTO_SHA512=y CONFIG_CRYPTO_DES=m CONFIG_CRYPTO_DRBG_HASH=y CONFIG_CRYPTO_DRBG_CTR=y CONFIG_CRYPTO_DEV_PADLOCK=y CONFIG_CRYPTO_DEV_CCP=y # CONFIG_CRYPTO_DEV_CCP_DD is not set CONFIG_LIBCRC32C=y # CONFIG_XZ_DEC_POWERPC is not set # CONFIG_XZ_DEC_ARM is not set # CONFIG_XZ_DEC_ARMTHUMB is not set # CONFIG_XZ_DEC_SPARC is not set CONFIG_DDR=y CONFIG_PRINTK_TIME=y CONFIG_BOOT_PRINTK_DELAY=y CONFIG_DYNAMIC_DEBUG=y # CONFIG_ENABLE_MUST_CHECK is not set CONFIG_FRAME_WARN=1024 CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m CONFIG_HARDLOCKUP_DETECTOR=y CONFIG_SCHEDSTATS=y CONFIG_SCHED_STACK_END_CHECK=y CONFIG_RCU_CPU_STALL_TIMEOUT=60 CONFIG_NOTIFIER_ERROR_INJECTION=m CONFIG_SCHED_TRACER=y CONFIG_FTRACE_SYSCALLS=y CONFIG_STACK_TRACER=y CONFIG_BLK_DEV_IO_TRACE=y CONFIG_FUNCTION_PROFILER=y CONFIG_MMIOTRACE=y CONFIG_MEMTEST=y CONFIG_KGDB=y CONFIG_KGDB_LOW_LEVEL_TRAP=y CONFIG_KGDB_KDB=y CONFIG_KDB_KEYBOARD=y # CONFIG_X86_VERBOSE_BOOTUP is not set CONFIG_EARLY_PRINTK_DBGP=y CONFIG_EARLY_PRINTK_EFI=y CONFIG_IO_DELAY_0XED=y CONFIG_OPTIMIZE_INLINING=y # CONFIG_LOCALVERSION_AUTO is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y CONFIG_NO_HZ=y CONFIG_HIGH_RES_TIMERS=y CONFIG_PREEMPT=y CONFIG_BSD_PROCESS_ACCT=y CONFIG_BSD_PROCESS_ACCT_V3=y CONFIG_TASKSTATS=y CONFIG_TASK_DELAY_ACCT=y CONFIG_TASK_XACCT=y CONFIG_TASK_IO_ACCOUNTING=y CONFIG_LOG_BUF_SHIFT=20 CONFIG_LOG_CPU_MAX_BUF_SHIFT=17 CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=17 CONFIG_CGROUPS=y CONFIG_MEMCG=y CONFIG_MEMCG_SWAP=y CONFIG_BLK_CGROUP=y CONFIG_CGROUP_SCHED=y CONFIG_CFS_BANDWIDTH=y CONFIG_RT_GROUP_SCHED=y CONFIG_CGROUP_PIDS=y CONFIG_CGROUP_FREEZER=y CONFIG_CGROUP_HUGETLB=y CONFIG_CPUSETS=y CONFIG_CGROUP_DEVICE=y CONFIG_CGROUP_CPUACCT=y CONFIG_CGROUP_PERF=y CONFIG_NAMESPACES=y CONFIG_USER_NS=y CONFIG_CHECKPOINT_RESTORE=y CONFIG_BLK_DEV_INITRD=y CONFIG_CC_OPTIMIZE_FOR_SIZE=y CONFIG_SYSCTL_SYSCALL=y # CONFIG_PCSPKR_PLATFORM is not set CONFIG_KALLSYMS_ALL=y CONFIG_BPF_SYSCALL=y CONFIG_EMBEDDED=y # CONFIG_COMPAT_BRK is not set CONFIG_PROFILING=y CONFIG_SMP=y CONFIG_X86_X2APIC=y CONFIG_X86_NUMACHIP=y CONFIG_X86_INTEL_LPSS=y CONFIG_IOSF_MBI_DEBUG=y CONFIG_HYPERVISOR_GUEST=y CONFIG_PARAVIRT=y CONFIG_PARAVIRT_SPINLOCKS=y CONFIG_KVM_DEBUG_FS=y CONFIG_PROCESSOR_SELECT=y CONFIG_GART_IOMMU=y CONFIG_CALGARY_IOMMU=y CONFIG_NR_CPUS=256 CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y CONFIG_I8K=m CONFIG_MICROCODE_AMD=y CONFIG_X86_MSR=m CONFIG_X86_CPUID=m CONFIG_NUMA=y CONFIG_ARCH_MEMORY_PROBE=y CONFIG_X86_PMEM_LEGACY=y CONFIG_X86_CHECK_BIOS_CORRUPTION=y CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1 CONFIG_EFI=y CONFIG_EFI_STUB=y CONFIG_EFI_MIXED=y CONFIG_HZ_1000=y CONFIG_KEXEC=y CONFIG_KEXEC_FILE=y CONFIG_KEXEC_VERIFY_SIG=y CONFIG_CRASH_DUMP=y CONFIG_KEXEC_JUMP=y CONFIG_PHYSICAL_ALIGN=0x1000000 CONFIG_LIVEPATCH=y CONFIG_HIBERNATION=y CONFIG_PM_WAKELOCKS=y CONFIG_PM_DEBUG=y CONFIG_PM_ADVANCED_DEBUG=y CONFIG_PM_TRACE_RTC=y CONFIG_WQ_POWER_EFFICIENT_DEFAULT=y CONFIG_ACPI_EC_DEBUGFS=m CONFIG_ACPI_DOCK=y CONFIG_ACPI_IPMI=m CONFIG_ACPI_PROCESSOR_AGGREGATOR=m CONFIG_ACPI_PCI_SLOT=y CONFIG_ACPI_HOTPLUG_MEMORY=y CONFIG_ACPI_SBS=m CONFIG_ACPI_BGRT=y CONFIG_ACPI_APEI=y CONFIG_ACPI_APEI_GHES=y CONFIG_ACPI_APEI_PCIEAER=y CONFIG_ACPI_APEI_MEMORY_FAILURE=y CONFIG_TPS68470_PMIC_OPREGION=y CONFIG_SFI=y CONFIG_CPU_FREQ_STAT=y CONFIG_CPU_FREQ_GOV_POWERSAVE=y CONFIG_CPU_FREQ_GOV_USERSPACE=y CONFIG_CPU_FREQ_GOV_ONDEMAND=y CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y CONFIG_X86_ACPI_CPUFREQ=y CONFIG_CPU_IDLE_GOV_LADDER=y CONFIG_INTEL_IDLE=y CONFIG_PCIEPORTBUS=y CONFIG_HOTPLUG_PCI_PCIE=y CONFIG_PCIEASPM_DEBUG=y CONFIG_PCI_MSI=y CONFIG_PCI_REALLOC_ENABLE_AUTO=y CONFIG_PCI_STUB=m CONFIG_PCI_IOV=y CONFIG_PCI_PRI=y CONFIG_PCI_PASID=y CONFIG_HOTPLUG_PCI=y CONFIG_HOTPLUG_PCI_ACPI=y CONFIG_HOTPLUG_PCI_ACPI_IBM=m CONFIG_HOTPLUG_PCI_CPCI=y CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m CONFIG_HOTPLUG_PCI_SHPC=y CONFIG_IA32_EMULATION=y CONFIG_X86_X32=y CONFIG_EDD=y CONFIG_EDD_OFF=y CONFIG_DMI_SYSFS=m CONFIG_ISCSI_IBFT_FIND=y CONFIG_EFI_VARS=y CONFIG_EFI_VARS_PSTORE=m # CONFIG_VIRTUALIZATION is not set CONFIG_OPROFILE=m CONFIG_KPROBES=y CONFIG_JUMP_LABEL=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_MODVERSIONS=y CONFIG_MODULE_SRCVERSION_ALL=y CONFIG_BLK_DEV_THROTTLING=y CONFIG_PARTITION_ADVANCED=y CONFIG_ACORN_PARTITION=y CONFIG_ACORN_PARTITION_CUMANA=y CONFIG_ACORN_PARTITION_EESOX=y CONFIG_ACORN_PARTITION_ICS=y CONFIG_ACORN_PARTITION_ADFS=y CONFIG_ACORN_PARTITION_POWERTEC=y CONFIG_ACORN_PARTITION_RISCIX=y CONFIG_AIX_PARTITION=y CONFIG_OSF_PARTITION=y CONFIG_AMIGA_PARTITION=y CONFIG_ATARI_PARTITION=y CONFIG_MAC_PARTITION=y CONFIG_BSD_DISKLABEL=y CONFIG_MINIX_SUBPARTITION=y CONFIG_SOLARIS_X86_PARTITION=y CONFIG_UNIXWARE_DISKLABEL=y CONFIG_LDM_PARTITION=y CONFIG_SGI_PARTITION=y CONFIG_ULTRIX_PARTITION=y CONFIG_SUN_PARTITION=y CONFIG_KARMA_PARTITION=y CONFIG_SYSV68_PARTITION=y CONFIG_CMDLINE_PARTITION=y CONFIG_IOSCHED_CFQ=m CONFIG_CFQ_GROUP_IOSCHED=y CONFIG_IOSCHED_BFQ=y CONFIG_BFQ_GROUP_IOSCHED=y CONFIG_BINFMT_MISC=m CONFIG_MEMORY_HOTPLUG=y CONFIG_MEMORY_HOTREMOVE=y CONFIG_KSM=y CONFIG_DEFAULT_MMAP_MIN_ADDR=65536 CONFIG_MEMORY_FAILURE=y CONFIG_HWPOISON_INJECT=m CONFIG_CLEANCACHE=y CONFIG_FRONTSWAP=y CONFIG_CMA=y CONFIG_MEM_SOFT_DIRTY=y CONFIG_ZSWAP=y CONFIG_ZBUD=y CONFIG_ZSMALLOC=y CONFIG_PGTABLE_MAPPING=y CONFIG_NET=y CONFIG_PACKET=y CONFIG_PACKET_DIAG=m CONFIG_UNIX=y CONFIG_UNIX_DIAG=m CONFIG_NET_KEY=m CONFIG_INET=y CONFIG_IP_MULTICAST=y CONFIG_IP_ADVANCED_ROUTER=y CONFIG_IP_FIB_TRIE_STATS=y CONFIG_IP_MULTIPLE_TABLES=y CONFIG_IP_ROUTE_MULTIPATH=y CONFIG_IP_ROUTE_VERBOSE=y CONFIG_IP_PNP=y CONFIG_IP_PNP_DHCP=y CONFIG_SYN_COOKIES=y # CONFIG_INET_XFRM_MODE_TRANSPORT is not set # CONFIG_INET_XFRM_MODE_TUNNEL is not set # CONFIG_INET_XFRM_MODE_BEET is not set CONFIG_TCP_CONG_ADVANCED=y # CONFIG_TCP_CONG_BIC is not set # CONFIG_TCP_CONG_WESTWOOD is not set # CONFIG_TCP_CONG_HTCP is not set CONFIG_TCP_MD5SIG=y CONFIG_IPV6_ROUTER_PREF=y CONFIG_IPV6_ROUTE_INFO=y # CONFIG_INET6_XFRM_MODE_TRANSPORT is not set # CONFIG_INET6_XFRM_MODE_TUNNEL is not set # CONFIG_INET6_XFRM_MODE_BEET is not set # CONFIG_IPV6_SIT is not set CONFIG_NETLABEL=y CONFIG_NETWORK_SECMARK=y CONFIG_NETFILTER=y CONFIG_NETFILTER_NETLINK_ACCT=m CONFIG_NETFILTER_NETLINK_QUEUE=m CONFIG_NETFILTER_NETLINK_LOG=m CONFIG_NETFILTER_NETLINK_OSF=m CONFIG_NF_TABLES=m CONFIG_NFT_COUNTER=m CONFIG_NFT_LOG=m CONFIG_NFT_LIMIT=m CONFIG_NFT_QUEUE=m CONFIG_NFT_REJECT=m CONFIG_NFT_COMPAT=m CONFIG_NFT_HASH=m CONFIG_NF_SOCKET_IPV4=m CONFIG_NF_TPROXY_IPV4=m CONFIG_NF_DUP_IPV4=m CONFIG_NF_LOG_ARP=m CONFIG_NF_LOG_IPV4=m CONFIG_NF_REJECT_IPV4=m CONFIG_IP_NF_IPTABLES=y CONFIG_IP_NF_FILTER=y CONFIG_NF_SOCKET_IPV6=m CONFIG_NF_TPROXY_IPV6=m CONFIG_NF_DUP_IPV6=m CONFIG_NF_LOG_IPV6=m CONFIG_IP6_NF_IPTABLES=m CONFIG_IP6_NF_TARGET_HL=m CONFIG_IP6_NF_FILTER=m CONFIG_IP6_NF_TARGET_REJECT=m CONFIG_IP6_NF_MANGLE=m CONFIG_IP6_NF_RAW=m CONFIG_IP6_NF_SECURITY=m CONFIG_DNS_RESOLVER=y CONFIG_CGROUP_NET_PRIO=y CONFIG_CGROUP_NET_CLASSID=y CONFIG_BPF_JIT=y CONFIG_CFG80211=m CONFIG_CFG80211_DEBUGFS=y CONFIG_CFG80211_WEXT=y CONFIG_MAC80211=m CONFIG_MAC80211_MESH=y CONFIG_MAC80211_DEBUGFS=y CONFIG_MAC80211_MESSAGE_TRACING=y CONFIG_RFKILL=y CONFIG_RFKILL_INPUT=y CONFIG_RFKILL_GPIO=m CONFIG_DEVTMPFS=y CONFIG_DEVTMPFS_MOUNT=y # CONFIG_STANDALONE is not set CONFIG_DEBUG_DEVRES=y CONFIG_CONNECTOR=y CONFIG_OF=y # CONFIG_PNP_DEBUG_MESSAGES is not set CONFIG_BLK_DEV_NULL_BLK=m CONFIG_BLK_DEV_FD=m CONFIG_BLK_DEV_PCIESSD_MTIP32XX=m CONFIG_ZRAM=m CONFIG_BLK_DEV_UMEM=m CONFIG_BLK_DEV_LOOP=y CONFIG_BLK_DEV_CRYPTOLOOP=m CONFIG_BLK_DEV_DRBD=m CONFIG_BLK_DEV_NBD=m CONFIG_BLK_DEV_SKD=m CONFIG_BLK_DEV_RAM=y CONFIG_BLK_DEV_RAM_SIZE=65536 CONFIG_BLK_DEV_NVME=m CONFIG_SRAM=y CONFIG_EEPROM_AT24=m CONFIG_EEPROM_93CX6=m CONFIG_CB710_CORE=m CONFIG_RAID_ATTRS=m CONFIG_SCSI=y # CONFIG_SCSI_MQ_DEFAULT is not set CONFIG_BLK_DEV_SD=y CONFIG_BLK_DEV_SR=m CONFIG_CHR_DEV_SG=y CONFIG_CHR_DEV_SCH=m CONFIG_SCSI_CONSTANTS=y CONFIG_SCSI_LOGGING=y CONFIG_SCSI_SCAN_ASYNC=y CONFIG_SCSI_SPI_ATTRS=m CONFIG_SCSI_FC_ATTRS=m CONFIG_SCSI_ISCSI_ATTRS=m CONFIG_SCSI_SAS_LIBSAS=m CONFIG_SCSI_SRP_ATTRS=m # CONFIG_SCSI_LOWLEVEL is not set CONFIG_SCSI_OSD_INITIATOR=m CONFIG_SCSI_OSD_ULD=m CONFIG_ATA=m CONFIG_SATA_AHCI=m CONFIG_SATA_AHCI_PLATFORM=m CONFIG_TARGET_CORE=m CONFIG_TCM_IBLOCK=m CONFIG_TCM_FILEIO=m CONFIG_TCM_PSCSI=m CONFIG_LOOPBACK_TARGET=m CONFIG_ISCSI_TARGET=m CONFIG_MACINTOSH_DRIVERS=y CONFIG_MAC_EMUMOUSEBTN=m CONFIG_NETDEVICES=y CONFIG_BONDING=m CONFIG_DUMMY=m CONFIG_EQUALIZER=m CONFIG_NET_TEAM=m CONFIG_NET_TEAM_MODE_BROADCAST=m CONFIG_NET_TEAM_MODE_ROUNDROBIN=m CONFIG_NET_TEAM_MODE_RANDOM=m CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m CONFIG_NET_TEAM_MODE_LOADBALANCE=m CONFIG_MACVLAN=m CONFIG_MACVTAP=m CONFIG_IPVLAN=m CONFIG_VXLAN=m CONFIG_GENEVE=m CONFIG_NETCONSOLE=m CONFIG_NETCONSOLE_DYNAMIC=y CONFIG_TUN=y CONFIG_VETH=m CONFIG_NLMON=m # CONFIG_NET_VENDOR_3COM is not set # CONFIG_NET_VENDOR_ADAPTEC is not set # CONFIG_NET_VENDOR_AGERE is not set # CONFIG_NET_VENDOR_ALACRITECH is not set # CONFIG_NET_VENDOR_ALTEON is not set # CONFIG_NET_VENDOR_AMAZON is not set # CONFIG_NET_VENDOR_AMD is not set # CONFIG_NET_VENDOR_AQUANTIA is not set # CONFIG_NET_VENDOR_ARC is not set # CONFIG_NET_VENDOR_ATHEROS is not set # CONFIG_NET_VENDOR_AURORA is not set # CONFIG_NET_VENDOR_BROADCOM is not set # CONFIG_NET_VENDOR_BROCADE is not set # CONFIG_NET_VENDOR_CADENCE is not set # CONFIG_NET_VENDOR_CAVIUM is not set # CONFIG_NET_VENDOR_CHELSIO is not set # CONFIG_NET_VENDOR_CISCO is not set # CONFIG_NET_VENDOR_CORTINA is not set # CONFIG_NET_VENDOR_DEC is not set # CONFIG_NET_VENDOR_DLINK is not set # CONFIG_NET_VENDOR_EMULEX is not set # CONFIG_NET_VENDOR_EZCHIP is not set # CONFIG_NET_VENDOR_HP is not set # CONFIG_NET_VENDOR_HUAWEI is not set # CONFIG_NET_VENDOR_I825XX is not set CONFIG_E1000=m CONFIG_IGBVF=m # CONFIG_NET_VENDOR_MARVELL is not set # CONFIG_NET_VENDOR_MELLANOX is not set # CONFIG_NET_VENDOR_MICREL is not set # CONFIG_NET_VENDOR_MICROCHIP is not set # CONFIG_NET_VENDOR_MICROSEMI is not set # CONFIG_NET_VENDOR_MYRI is not set # CONFIG_NET_VENDOR_NATSEMI is not set # CONFIG_NET_VENDOR_NETERION is not set # CONFIG_NET_VENDOR_NETRONOME is not set # CONFIG_NET_VENDOR_NI is not set # CONFIG_NET_VENDOR_NVIDIA is not set # CONFIG_NET_VENDOR_OKI is not set # CONFIG_NET_VENDOR_PACKET_ENGINES is not set # CONFIG_NET_VENDOR_QLOGIC is not set # CONFIG_NET_VENDOR_QUALCOMM is not set # CONFIG_NET_VENDOR_RDC is not set # CONFIG_NET_VENDOR_REALTEK is not set # CONFIG_NET_VENDOR_RENESAS is not set # CONFIG_NET_VENDOR_ROCKER is not set # CONFIG_NET_VENDOR_SAMSUNG is not set # CONFIG_NET_VENDOR_SEEQ is not set # CONFIG_NET_VENDOR_SOLARFLARE is not set # CONFIG_NET_VENDOR_SILAN is not set # CONFIG_NET_VENDOR_SIS is not set # CONFIG_NET_VENDOR_SMSC is not set # CONFIG_NET_VENDOR_SOCIONEXT is not set # CONFIG_NET_VENDOR_STMICRO is not set # CONFIG_NET_VENDOR_SUN is not set # CONFIG_NET_VENDOR_SYNOPSYS is not set # CONFIG_NET_VENDOR_TEHUTI is not set # CONFIG_NET_VENDOR_TI is not set # CONFIG_NET_VENDOR_VIA is not set # CONFIG_NET_VENDOR_WIZNET is not set CONFIG_PHYLIB=y CONFIG_PPP=y CONFIG_PPP_BSDCOMP=m CONFIG_PPP_DEFLATE=m CONFIG_PPP_FILTER=y CONFIG_PPP_MPPE=m CONFIG_PPP_MULTILINK=y CONFIG_PPPOE=m CONFIG_PPP_ASYNC=m CONFIG_PPP_SYNC_TTY=m CONFIG_USB_NET_DRIVERS=m CONFIG_USB_USBNET=m CONFIG_USB_NET_CDC_EEM=m CONFIG_USB_NET_CDC_MBIM=m CONFIG_USB_NET_SMSC95XX=m CONFIG_USB_NET_RNDIS_HOST=m # CONFIG_USB_ARMLINUX is not set # CONFIG_USB_NET_ZAURUS is not set # CONFIG_WLAN_VENDOR_ADMTEK is not set # CONFIG_WLAN_VENDOR_ATH is not set # CONFIG_WLAN_VENDOR_ATMEL is not set # CONFIG_WLAN_VENDOR_BROADCOM is not set # CONFIG_WLAN_VENDOR_CISCO is not set CONFIG_IWL4965=m CONFIG_IWL3945=m CONFIG_IWLWIFI=m CONFIG_IWLDVM=m CONFIG_IWLMVM=m CONFIG_IWLWIFI_DEBUGFS=y # CONFIG_WLAN_VENDOR_MARVELL is not set # CONFIG_WLAN_VENDOR_MEDIATEK is not set # CONFIG_WLAN_VENDOR_RALINK is not set # CONFIG_WLAN_VENDOR_REALTEK is not set # CONFIG_WLAN_VENDOR_RSI is not set # CONFIG_WLAN_VENDOR_ST is not set # CONFIG_WLAN_VENDOR_TI is not set # CONFIG_WLAN_VENDOR_ZYDAS is not set # CONFIG_WLAN_VENDOR_QUANTENNA is not set CONFIG_INPUT_FF_MEMLESS=m CONFIG_INPUT_POLLDEV=m CONFIG_INPUT_SPARSEKMAP=m CONFIG_INPUT_MOUSEDEV=y CONFIG_INPUT_MOUSEDEV_PSAUX=y CONFIG_INPUT_EVDEV=y CONFIG_INPUT_EVBUG=m CONFIG_KEYBOARD_CROS_EC=m CONFIG_MOUSE_PS2=m CONFIG_MOUSE_PS2_ELANTECH=y CONFIG_MOUSE_PS2_SENTELIC=y CONFIG_MOUSE_PS2_TOUCHKIT=y CONFIG_MOUSE_PS2_VMMOUSE=y CONFIG_INPUT_TOUCHSCREEN=y CONFIG_TOUCHSCREEN_ATMEL_MXT=y CONFIG_TOUCHSCREEN_ELAN=y CONFIG_TOUCHSCREEN_MELFAS_MIP4=y CONFIG_TOUCHSCREEN_WDT87XX_I2C=y CONFIG_TOUCHSCREEN_USB_COMPOSITE=m CONFIG_TOUCHSCREEN_RM_TS=y CONFIG_INPUT_MISC=y CONFIG_INPUT_UINPUT=m CONFIG_LEGACY_PTY_COUNT=0 CONFIG_SERIAL_NONSTANDARD=y CONFIG_SERIAL_8250=y # CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=48 CONFIG_SERIAL_8250_RUNTIME_UARTS=32 CONFIG_SERIAL_8250_EXTENDED=y CONFIG_SERIAL_8250_MANY_PORTS=y CONFIG_SERIAL_8250_SHARE_IRQ=y CONFIG_SERIAL_8250_RSA=y CONFIG_SERIAL_8250_DW=m # CONFIG_SERIAL_8250_MID is not set CONFIG_SERIAL_KGDB_NMI=y CONFIG_TTY_PRINTK=y CONFIG_IPMI_HANDLER=m CONFIG_IPMI_DEVICE_INTERFACE=m CONFIG_IPMI_SI=m CONFIG_IPMI_SSIF=m CONFIG_IPMI_WATCHDOG=m CONFIG_IPMI_POWEROFF=m CONFIG_HW_RANDOM=y CONFIG_HW_RANDOM_TIMERIOMEM=m CONFIG_HW_RANDOM_INTEL=m CONFIG_HW_RANDOM_AMD=m CONFIG_HW_RANDOM_VIA=m CONFIG_NVRAM=y CONFIG_R3964=m CONFIG_APPLICOM=m CONFIG_HPET=y CONFIG_HANGCHECK_TIMER=m CONFIG_TCG_TPM=y CONFIG_TCG_TIS=y CONFIG_TCG_TIS_I2C_ATMEL=m CONFIG_TCG_TIS_I2C_INFINEON=m CONFIG_TCG_TIS_I2C_NUVOTON=m CONFIG_TCG_NSC=m CONFIG_TCG_ATMEL=m CONFIG_TCG_INFINEON=m CONFIG_TELCLOCK=m CONFIG_I2C=y CONFIG_I2C_CHARDEV=y CONFIG_I2C_I801=y CONFIG_I2C_ISCH=m CONFIG_I2C_ISMT=m CONFIG_I2C_PIIX4=m CONFIG_I2C_SCMI=m CONFIG_I2C_DESIGNWARE_PCI=y CONFIG_I2C_DESIGNWARE_BAYTRAIL=y CONFIG_I2C_CROS_EC_TUNNEL=m CONFIG_I2C_SLAVE=y CONFIG_I2C_SLAVE_EEPROM=m CONFIG_SPI=y CONFIG_PINCTRL_BAYTRAIL=y CONFIG_PINCTRL_CHERRYVIEW=y CONFIG_PINCTRL_BROXTON=y CONFIG_PINCTRL_SUNRISEPOINT=y CONFIG_DEBUG_GPIO=y CONFIG_GPIO_SYSFS=y CONFIG_GPIO_GENERIC_PLATFORM=m CONFIG_GPIO_ICH=m CONFIG_GPIO_LYNXPOINT=m CONFIG_GPIO_SCH=m CONFIG_GPIO_TPS68470=y CONFIG_POWER_RESET=y CONFIG_PDA_POWER=m CONFIG_GENERIC_ADC_BATTERY=m CONFIG_BATTERY_SBS=m CONFIG_CHARGER_GPIO=m CONFIG_CHARGER_MANAGER=y CONFIG_SENSORS_I5500=m CONFIG_SENSORS_CORETEMP=m CONFIG_SENSORS_ACPI_POWER=m CONFIG_THERMAL_GOV_FAIR_SHARE=y CONFIG_THERMAL_GOV_BANG_BANG=y CONFIG_THERMAL_EMULATION=y CONFIG_INTEL_POWERCLAMP=m CONFIG_INTEL_SOC_DTS_THERMAL=m CONFIG_INT340X_THERMAL=m CONFIG_MFD_CROS_EC=m CONFIG_LPC_ICH=y CONFIG_LPC_SCH=y CONFIG_MFD_INTEL_LPSS_ACPI=y CONFIG_MFD_INTEL_LPSS_PCI=y CONFIG_MFD_TPS68470=y CONFIG_REGULATOR=y CONFIG_REGULATOR_FIXED_VOLTAGE=m CONFIG_REGULATOR_VIRTUAL_CONSUMER=m CONFIG_REGULATOR_USERSPACE_CONSUMER=m CONFIG_REGULATOR_GPIO=m CONFIG_REGULATOR_TPS51632=m CONFIG_REGULATOR_TPS62360=m CONFIG_REGULATOR_TPS65023=m CONFIG_REGULATOR_TPS6507X=m CONFIG_MEDIA_SUPPORT=m CONFIG_MEDIA_CAMERA_SUPPORT=y CONFIG_MEDIA_CONTROLLER=y CONFIG_VIDEO_V4L2_SUBDEV_API=y CONFIG_MEDIA_USB_SUPPORT=y CONFIG_USB_VIDEO_CLASS=m # CONFIG_USB_GSPCA is not set CONFIG_MEDIA_PCI_SUPPORT=y CONFIG_VIDEO_IPU3_CIO2=m CONFIG_V4L_PLATFORM_DRIVERS=y # CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set CONFIG_VIDEO_DW9714=m CONFIG_VIDEO_OV5670=m CONFIG_VIDEO_OV7670=m CONFIG_VIDEO_OV13858=m CONFIG_AGP=y CONFIG_AGP_AMD64=y CONFIG_AGP_INTEL=y CONFIG_VGA_SWITCHEROO=y CONFIG_DRM=m CONFIG_DRM_LOAD_EDID_FIRMWARE=y CONFIG_DRM_I915=m CONFIG_FB=y CONFIG_FIRMWARE_EDID=y CONFIG_FB_TILEBLITTING=y CONFIG_FB_INTEL=m CONFIG_FB_SIMPLE=y CONFIG_BACKLIGHT_CLASS_DEVICE=y CONFIG_BACKLIGHT_GENERIC=m CONFIG_FRAMEBUFFER_CONSOLE=y CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y CONFIG_HID=m CONFIG_HID_BATTERY_STRENGTH=y CONFIG_HIDRAW=y CONFIG_UHID=m CONFIG_HID_MULTITOUCH=m CONFIG_USB_HID=m CONFIG_HID_PID=y CONFIG_USB_HIDDEV=y CONFIG_USB_KBD=m CONFIG_USB_MOUSE=m CONFIG_I2C_HID=m CONFIG_USB=y CONFIG_USB_ANNOUNCE_NEW_DEVICES=y CONFIG_USB_DYNAMIC_MINORS=y CONFIG_USB_XHCI_HCD=y CONFIG_USB_EHCI_HCD=y CONFIG_USB_EHCI_ROOT_HUB_TT=y CONFIG_USB_EHCI_HCD_PLATFORM=y CONFIG_USB_OHCI_HCD=y CONFIG_USB_OHCI_HCD_PLATFORM=y CONFIG_USB_UHCI_HCD=y CONFIG_USB_STORAGE=m CONFIG_USB_UAS=m CONFIG_NOP_USB_XCEIV=y CONFIG_USB_GPIO_VBUS=m CONFIG_MMC=y CONFIG_MMC_BLOCK=m CONFIG_SDIO_UART=m CONFIG_MMC_SDHCI=m CONFIG_MMC_SDHCI_PCI=m # CONFIG_MMC_RICOH_MMC is not set CONFIG_MMC_SDHCI_ACPI=m CONFIG_MMC_SDHCI_PLTFM=m CONFIG_LEDS_CLASS=y CONFIG_RTC_CLASS=y CONFIG_RTC_DRV_HID_SENSOR_TIME=m CONFIG_DMADEVICES=y CONFIG_INTEL_IOATDMA=m CONFIG_DW_DMAC=y CONFIG_ASYNC_TX_DMA=y CONFIG_AUXDISPLAY=y # CONFIG_VIRTIO_MENU is not set CONFIG_STAGING=y CONFIG_STAGING_MEDIA=y CONFIG_VIDEO_IPU3_IMGU=m CONFIG_DCDBAS=m CONFIG_DELL_RBU=m CONFIG_CHROMEOS_LAPTOP=m CONFIG_CHROMEOS_PSTORE=m CONFIG_CROS_EC_LPC=m CONFIG_CROS_KBD_LED_BACKLIGHT=m CONFIG_INTEL_IOMMU=y # CONFIG_INTEL_IOMMU_DEFAULT_ON is not set CONFIG_IRQ_REMAP=y CONFIG_MEMORY=y CONFIG_IIO_BUFFER=y CONFIG_IIO_KFIFO_BUF=m CONFIG_RESET_CONTROLLER=y CONFIG_GENERIC_PHY=y CONFIG_POWERCAP=y CONFIG_INTEL_RAPL=m CONFIG_EXT4_FS=y CONFIG_EXT4_FS_POSIX_ACL=y CONFIG_EXT4_FS_SECURITY=y CONFIG_EXT4_ENCRYPTION=y CONFIG_FANOTIFY=y CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y CONFIG_AUTOFS4_FS=m CONFIG_FUSE_FS=y CONFIG_CUSE=m CONFIG_OVERLAY_FS=m CONFIG_FSCACHE=m CONFIG_FSCACHE_STATS=y CONFIG_CACHEFILES=m CONFIG_VFAT_FS=y CONFIG_PROC_KCORE=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_HUGETLBFS=y CONFIG_EFIVAR_FS=y CONFIG_PSTORE_RAM=m CONFIG_NFS_FS=m CONFIG_NFS_V3_ACL=y CONFIG_NFS_V4=m CONFIG_NFS_SWAP=y CONFIG_NFS_V4_1=y CONFIG_NFS_V4_2=y CONFIG_NFS_V4_1_MIGRATION=y CONFIG_NFS_FSCACHE=y CONFIG_SUNRPC_DEBUG=y CONFIG_NLS_DEFAULT="utf8" CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_UTF8=y CONFIG_PERSISTENT_KEYRINGS=y CONFIG_TRUSTED_KEYS=y CONFIG_ENCRYPTED_KEYS=y CONFIG_SECURITY=y CONFIG_SECURITY_NETWORK_XFRM=y CONFIG_INTEL_TXT=y CONFIG_SECURITY_APPARMOR=y # CONFIG_INTEGRITY is not set CONFIG_CRYPTO_ECDH=m CONFIG_CRYPTO_GF128MUL=y CONFIG_CRYPTO_AUTHENC=m CONFIG_CRYPTO_CRC32C_INTEL=y CONFIG_CRYPTO_SHA512=y CONFIG_CRYPTO_DES=m CONFIG_CRYPTO_DRBG_HASH=y CONFIG_CRYPTO_DRBG_CTR=y CONFIG_CRYPTO_DEV_PADLOCK=y CONFIG_CRYPTO_DEV_CCP=y # CONFIG_CRYPTO_DEV_CCP_DD is not set CONFIG_LIBCRC32C=y # CONFIG_XZ_DEC_POWERPC is not set # CONFIG_XZ_DEC_ARM is not set # CONFIG_XZ_DEC_ARMTHUMB is not set # CONFIG_XZ_DEC_SPARC is not set CONFIG_DDR=y CONFIG_PRINTK_TIME=y CONFIG_BOOT_PRINTK_DELAY=y CONFIG_DYNAMIC_DEBUG=y # CONFIG_ENABLE_MUST_CHECK is not set CONFIG_FRAME_WARN=1024 CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m CONFIG_KASAN=y CONFIG_KASAN_EXTRA=y CONFIG_KASAN_INLINE=y CONFIG_HARDLOCKUP_DETECTOR=y CONFIG_SCHEDSTATS=y CONFIG_SCHED_STACK_END_CHECK=y CONFIG_RCU_CPU_STALL_TIMEOUT=60 CONFIG_NOTIFIER_ERROR_INJECTION=m CONFIG_SCHED_TRACER=y CONFIG_FTRACE_SYSCALLS=y CONFIG_STACK_TRACER=y CONFIG_BLK_DEV_IO_TRACE=y CONFIG_FUNCTION_PROFILER=y CONFIG_MMIOTRACE=y CONFIG_MEMTEST=y CONFIG_KGDB=y CONFIG_KGDB_LOW_LEVEL_TRAP=y CONFIG_KGDB_KDB=y CONFIG_KDB_KEYBOARD=y # CONFIG_X86_VERBOSE_BOOTUP is not set CONFIG_EARLY_PRINTK_DBGP=y CONFIG_EARLY_PRINTK_EFI=y CONFIG_IO_DELAY_0XED=y CONFIG_OPTIMIZE_INLINING=y
Hi Jacopo, > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hello Raj, > > On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > > Hi Laurent, Tomasz, Jacopo, > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > Hello, > > > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa <tfiga@chromium.org> > wrote: > > > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > > > Hellon > > > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > > > > > > Hello Yong, > > > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > > > A bit more information to help you debugging this. I've > > > > > > enabled KASAN in the kernel configuration, and get the > > > > > > following use-after-free > > > reports. > > > > > > I tested as well using the ipu-process.sh script shared by Laurent, > > > with the following command line: > > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 > > > frame-2592x1944.cio2 > > > > > > and I got a very similar trace available at: > > > https://paste.debian.net/hidden/5855e15a/ > > > > > > Please note I have been able to process a set of images (with KASAN > > > enabled the machine does not freeze) but the kernel log gets flooded > > > and it is not possible to process any other frame after this. > > > > > > The issue is currently quite annoying and it's a blocker for > > > libcamera development on IPU3. Please let me know if I can support with > more testing. > > > > > > Thanks > > > j > > > > > > > > > > > > > > > [ 166.332920] > > > > > > > > > > ================================================================ > > > == > > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by task > > > > > > yavta/1305 > > > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C > 4.20.0- > > > rc6+ #3 > > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS > > > > > > 08/30/2018 [ 166.332959] Call Trace: > > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > > print_address_description+0x65/0x227 > > > > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] > > > > > > __free_iova+0x23/0x31 [ 166.333011] > > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > > > > > > > Thanks Laurent, I think this is a very good hint. It looks like > > > > > we're basically freeing and already freed IOVA and corrupting > > > > > some allocator state? > > > > > > > > Did you have any luck in reproducing and fixing this double free issue? > > > > > > > > This issue is either hard to reproduce or comes with different > > signatures with the updated yavta (that now supports meta output) with > > the 4.4 kernel that I have been using. > > I am switching to 4.20-rc6 for better reproducibility. > > Enabling KASAN also results in storage space issues on my Chrome device. > > Will enable this just for ImgU to get ahead and get back with more updates. > > > > Thanks for testing this. > > For your informations I'm using the following branch, from Sakari's > tree: git://linuxtv.org/sailus/media_tree.git ipu3 > > Although it appears that the media tree master branch has everything that is > there, with a few additional patches on top. I should move to use media tree > master as well... > > I have here attached 2 configuration files for v4.20-rc5 I am using on Soraka, in > case they might help you. One has KASAN enabled with an increased kernel > log size, the other one is the one we use for daily development. I think I am missing a trick here to override the default chrome os kernel config with the one that you supplied. In particular I am looking for steps to build the upstream kernel within chrome os build environment using your config, so I can update my Soraka device. > > Also, please make sure to use (the most) recent media-ctl and yavta utilities, as > the ones provided by most distros are usually not recent enough to work with > IPU3, but I'm sure you know that already ;) Ack > > Thanks > j > [snip]
Hi Raj, On Wed, Jan 09, 2019 at 06:01:39PM +0000, Mani, Rajmohan wrote: > Hi Jacopo, > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > Hello Raj, > > > > On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > > > Hi Laurent, Tomasz, Jacopo, > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > Hello, > > > > > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa <tfiga@chromium.org> > > wrote: > > > > > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > > > > > Hellon > > > > > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart wrote: > > > > > > > > Hello Yong, > > > > > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > > > > > A bit more information to help you debugging this. I've > > > > > > > enabled KASAN in the kernel configuration, and get the > > > > > > > following use-after-free > > > > reports. > > > > > > > > I tested as well using the ipu-process.sh script shared by Laurent, > > > > with the following command line: > > > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 > > > > frame-2592x1944.cio2 > > > > > > > > and I got a very similar trace available at: > > > > https://paste.debian.net/hidden/5855e15a/ > > > > > > > > Please note I have been able to process a set of images (with KASAN > > > > enabled the machine does not freeze) but the kernel log gets flooded > > > > and it is not possible to process any other frame after this. > > > > > > > > The issue is currently quite annoying and it's a blocker for > > > > libcamera development on IPU3. Please let me know if I can support with > > more testing. > > > > > > > > Thanks > > > > j > > > > > > > > > > > > > > > > > > [ 166.332920] > > > > > > > > > > > > > ================================================================ > > > > == > > > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by task > > > > > > > yavta/1305 > > > > > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C > > 4.20.0- > > > > rc6+ #3 > > > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS > > > > > > > 08/30/2018 [ 166.332959] Call Trace: > > > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > > > print_address_description+0x65/0x227 > > > > > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] > > > > > > > __free_iova+0x23/0x31 [ 166.333011] > > > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > > > > > > > > > Thanks Laurent, I think this is a very good hint. It looks like > > > > > > we're basically freeing and already freed IOVA and corrupting > > > > > > some allocator state? > > > > > > > > > > Did you have any luck in reproducing and fixing this double free issue? > > > > > > > > > > > This issue is either hard to reproduce or comes with different > > > signatures with the updated yavta (that now supports meta output) with > > > the 4.4 kernel that I have been using. > > > I am switching to 4.20-rc6 for better reproducibility. > > > Enabling KASAN also results in storage space issues on my Chrome device. > > > Will enable this just for ImgU to get ahead and get back with more updates. > > > > > > > Thanks for testing this. > > > > For your informations I'm using the following branch, from Sakari's > > tree: git://linuxtv.org/sailus/media_tree.git ipu3 > > > > Although it appears that the media tree master branch has everything that is > > there, with a few additional patches on top. I should move to use media tree > > master as well... > > > > I have here attached 2 configuration files for v4.20-rc5 I am using on Soraka, in > > case they might help you. One has KASAN enabled with an increased kernel > > log size, the other one is the one we use for daily development. > > I think I am missing a trick here to override the default chrome os kernel > config with the one that you supplied. > > In particular I am looking for steps to build the upstream kernel within chrome os > build environment using your config, so I can update my Soraka device. I'm sorry I can not help much building 'withing chrome os build environment'. Care to explain what you mean? What I usually do, provided you're running a debian-based Linux distro on your Soraka device, is compile the kernel on host with 'make bindeb-pkg' and then upload and install the resulting .deb package on the Soraka chromebook. If that might work for you, we can share more details on how to do so (tomorrow maybe :p ) Thanks j > > > > > Also, please make sure to use (the most) recent media-ctl and yavta utilities, as > > the ones provided by most distros are usually not recent enough to work with > > IPU3, but I'm sure you know that already ;) > > Ack > > > > > Thanks > > j > > > > [snip]
Hi Jacopo, > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hi Raj, > > On Wed, Jan 09, 2019 at 06:01:39PM +0000, Mani, Rajmohan wrote: > > Hi Jacopo, > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > Hello Raj, > > > > > > On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > > > > Hi Laurent, Tomasz, Jacopo, > > > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > > > Hello, > > > > > > > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa > > > > > > <tfiga@chromium.org> > > > wrote: > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > > > > > > > Hellon > > > > > > > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart > wrote: > > > > > > > > > Hello Yong, > > > > > > > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > > > > > > > A bit more information to help you debugging this. I've > > > > > > > > enabled KASAN in the kernel configuration, and get the > > > > > > > > following use-after-free > > > > > reports. > > > > > > > > > > I tested as well using the ipu-process.sh script shared by > > > > > Laurent, with the following command line: > > > > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 > > > > > frame-2592x1944.cio2 > > > > > > > > > > and I got a very similar trace available at: > > > > > https://paste.debian.net/hidden/5855e15a/ > > > > > > > > > > Please note I have been able to process a set of images (with > > > > > KASAN enabled the machine does not freeze) but the kernel log > > > > > gets flooded and it is not possible to process any other frame after this. > > > > > > > > > > The issue is currently quite annoying and it's a blocker for > > > > > libcamera development on IPU3. Please let me know if I can > > > > > support with > > > more testing. > > > > > > > > > > Thanks > > > > > j > > > > > > > > > > > > > > > > > > > > > [ 166.332920] > > > > > > > > > > > > > > > > > ================================================================ > > > > > == > > > > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by > > > > > > > > task > > > > > > > > yavta/1305 > > > > > > > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C > > > 4.20.0- > > > > > rc6+ #3 > > > > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS > > > > > > > > 08/30/2018 [ 166.332959] Call Trace: > > > > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > > > > print_address_description+0x65/0x227 > > > > > > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] > > > > > > > > __free_iova+0x23/0x31 [ 166.333011] > > > > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > > > > > > > > > > > Thanks Laurent, I think this is a very good hint. It looks > > > > > > > like we're basically freeing and already freed IOVA and > > > > > > > corrupting some allocator state? > > > > > > > > > > > > Did you have any luck in reproducing and fixing this double free > issue? > > > > > > > > > > > > > > This issue is either hard to reproduce or comes with different > > > > signatures with the updated yavta (that now supports meta output) > > > > with the 4.4 kernel that I have been using. > > > > I am switching to 4.20-rc6 for better reproducibility. > > > > Enabling KASAN also results in storage space issues on my Chrome > device. > > > > Will enable this just for ImgU to get ahead and get back with more > updates. > > > > > > > > > > Thanks for testing this. > > > > > > For your informations I'm using the following branch, from Sakari's > > > tree: git://linuxtv.org/sailus/media_tree.git ipu3 > > > > > > Although it appears that the media tree master branch has everything > > > that is there, with a few additional patches on top. I should move > > > to use media tree master as well... > > > > > > I have here attached 2 configuration files for v4.20-rc5 I am using > > > on Soraka, in case they might help you. One has KASAN enabled with > > > an increased kernel log size, the other one is the one we use for daily > development. > > > > I think I am missing a trick here to override the default chrome os > > kernel config with the one that you supplied. > > > > In particular I am looking for steps to build the upstream kernel > > within chrome os build environment using your config, so I can update my > Soraka device. > > I'm sorry I can not help much building 'withing chrome os build environment'. > Care to explain what you mean? > This is part of the Chromium OS build environment and development workflow. https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq.md No worries. I will sync up with Tomasz, as he managed to get this working with 4.20 kernel. > What I usually do, provided you're running a debian-based Linux distro on > your Soraka device, is compile the kernel on host with 'make bindeb-pkg' > and then upload and install the resulting .deb package on the Soraka > chromebook. > > If that might work for you, we can share more details on how to do so > (tomorrow maybe :p ) > The above info would be very helpful for us to optimize use of upstream kernel in Chrome OS devices. Please share when your time permits. > Thanks > j > > > > > > > > > Also, please make sure to use (the most) recent media-ctl and yavta > > > utilities, as the ones provided by most distros are usually not > > > recent enough to work with IPU3, but I'm sure you know that already > > > ;) > > > > Ack > > > > > > > > Thanks > > > j > > > > > > > [snip]
Hi Raj, On Wed, Jan 09, 2019 at 06:36:02PM +0000, Mani, Rajmohan wrote: > Hi Jacopo, > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > Hi Raj, > > > > On Wed, Jan 09, 2019 at 06:01:39PM +0000, Mani, Rajmohan wrote: > > > Hi Jacopo, > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > Hello Raj, > > > > > > > > On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > > > > > Hi Laurent, Tomasz, Jacopo, > > > > > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > > > > > Hello, > > > > > > > > > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > > > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa > > > > > > > <tfiga@chromium.org> > > > > wrote: > > > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > > > > > > > > > Hellon > > > > > > > > > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent Pinchart > > wrote: > > > > > > > > > > Hello Yong, > > > > > > > > > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > > > > > > > > > A bit more information to help you debugging this. I've > > > > > > > > > enabled KASAN in the kernel configuration, and get the > > > > > > > > > following use-after-free > > > > > > reports. > > > > > > > > > > > > I tested as well using the ipu-process.sh script shared by > > > > > > Laurent, with the following command line: > > > > > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 > > > > > > frame-2592x1944.cio2 > > > > > > > > > > > > and I got a very similar trace available at: > > > > > > https://paste.debian.net/hidden/5855e15a/ > > > > > > > > > > > > Please note I have been able to process a set of images (with > > > > > > KASAN enabled the machine does not freeze) but the kernel log > > > > > > gets flooded and it is not possible to process any other frame after this. > > > > > > > > > > > > The issue is currently quite annoying and it's a blocker for > > > > > > libcamera development on IPU3. Please let me know if I can > > > > > > support with > > > > more testing. > > > > > > > > > > > > Thanks > > > > > > j > > > > > > > > > > > > > > > > > > > > > > > > [ 166.332920] > > > > > > > > > > > > > > > > > > > > > ================================================================ > > > > > > == > > > > > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 by > > > > > > > > > task > > > > > > > > > yavta/1305 > > > > > > > > > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C > > > > 4.20.0- > > > > > > rc6+ #3 > > > > > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS > > > > > > > > > 08/30/2018 [ 166.332959] Call Trace: > > > > > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > > > > > print_address_description+0x65/0x227 > > > > > > > > > [ 166.332979] ? __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ 166.332999] > > > > > > > > > __free_iova+0x23/0x31 [ 166.333011] > > > > > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > > > > > > > > > > > > > Thanks Laurent, I think this is a very good hint. It looks > > > > > > > > like we're basically freeing and already freed IOVA and > > > > > > > > corrupting some allocator state? > > > > > > > > > > > > > > Did you have any luck in reproducing and fixing this double free > > issue? > > > > > > > > > > > > > > > > > This issue is either hard to reproduce or comes with different > > > > > signatures with the updated yavta (that now supports meta output) > > > > > with the 4.4 kernel that I have been using. > > > > > I am switching to 4.20-rc6 for better reproducibility. > > > > > Enabling KASAN also results in storage space issues on my Chrome > > device. > > > > > Will enable this just for ImgU to get ahead and get back with more > > updates. > > > > > > > > > > > > > Thanks for testing this. > > > > > > > > For your informations I'm using the following branch, from Sakari's > > > > tree: git://linuxtv.org/sailus/media_tree.git ipu3 > > > > > > > > Although it appears that the media tree master branch has everything > > > > that is there, with a few additional patches on top. I should move > > > > to use media tree master as well... > > > > > > > > I have here attached 2 configuration files for v4.20-rc5 I am using > > > > on Soraka, in case they might help you. One has KASAN enabled with > > > > an increased kernel log size, the other one is the one we use for daily > > development. > > > > > > I think I am missing a trick here to override the default chrome os > > > kernel config with the one that you supplied. > > > > > > In particular I am looking for steps to build the upstream kernel > > > within chrome os build environment using your config, so I can update my > > Soraka device. > > > > I'm sorry I can not help much building 'withing chrome os build environment'. > > Care to explain what you mean? > > > > This is part of the Chromium OS build environment and development workflow. > https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq.md > > No worries. > I will sync up with Tomasz, as he managed to get this working with 4.20 kernel. > I'm sorry I can't help much here. I suggest to work with mainline (or better, media master) and install a GNU/Linux distro on the chromebook so you can easily update your kernel. I personally used https://chrx.org/ that makes installing gallium (or else) very easy. Once you get that running, I find easy enough to update the kernel installing a .deb package.
Hi Jacopo, > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hi Raj, > > On Wed, Jan 09, 2019 at 06:36:02PM +0000, Mani, Rajmohan wrote: > > Hi Jacopo, > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > Hi Raj, > > > > > > On Wed, Jan 09, 2019 at 06:01:39PM +0000, Mani, Rajmohan wrote: > > > > Hi Jacopo, > > > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > > > Hello Raj, > > > > > > > > > > On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > > > > > > Hi Laurent, Tomasz, Jacopo, > > > > > > > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > > > > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa > > > > > > > > <tfiga@chromium.org> > > > > > wrote: > > > > > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > > > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > > > > > > > > > > > Hellon > > > > > > > > > > > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent > > > > > > > > > > Pinchart > > > wrote: > > > > > > > > > > > Hello Yong, > > > > > > > > > > > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > > > > > > > > > > > A bit more information to help you debugging this. > > > > > > > > > > I've enabled KASAN in the kernel configuration, and > > > > > > > > > > get the following use-after-free > > > > > > > reports. > > > > > > > > > > > > > > I tested as well using the ipu-process.sh script shared by > > > > > > > Laurent, with the following command line: > > > > > > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 > > > > > > > frame-2592x1944.cio2 > > > > > > > > > > > > > > and I got a very similar trace available at: > > > > > > > https://paste.debian.net/hidden/5855e15a/ > > > > > > > > > > > > > > Please note I have been able to process a set of images > > > > > > > (with KASAN enabled the machine does not freeze) but the > > > > > > > kernel log gets flooded and it is not possible to process any other > frame after this. > > > > > > > > > > > > > > The issue is currently quite annoying and it's a blocker for > > > > > > > libcamera development on IPU3. Please let me know if I can > > > > > > > support with > > > > > more testing. > > > > > > > > > > > > > > Thanks > > > > > > > j > > > > > > > > > > > > > > > > > > > > > > > > > > > [ 166.332920] > > > > > > > > > > > > > > > > > > > > > > > > > > ================================================================ > > > > > > > == > > > > > > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 > > > > > > > > > > by task > > > > > > > > > > yavta/1305 > > > > > > > > > > > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C > > > > > 4.20.0- > > > > > > > rc6+ #3 > > > > > > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS > > > > > > > > > > 08/30/2018 [ 166.332959] Call Trace: > > > > > > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > > > > > > print_address_description+0x65/0x227 > > > > > > > > > > [ 166.332979] ? > > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > > [ 166.332983] kasan_report+0x247/0x285 [ > > > > > > > > > > 166.332989] > > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ > > > > > > > > > > 166.332999] > > > > > > > > > > __free_iova+0x23/0x31 [ 166.333011] > > > > > > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > > > > > > > > > > > > > > > Thanks Laurent, I think this is a very good hint. It > > > > > > > > > looks like we're basically freeing and already freed > > > > > > > > > IOVA and corrupting some allocator state? > > > > > > > > > > > > > > > > Did you have any luck in reproducing and fixing this > > > > > > > > double free > > > issue? > > > > > > > > > > > > > > > > > > > > This issue is either hard to reproduce or comes with different > > > > > > signatures with the updated yavta (that now supports meta > > > > > > output) with the 4.4 kernel that I have been using. > > > > > > I am switching to 4.20-rc6 for better reproducibility. > > > > > > Enabling KASAN also results in storage space issues on my > > > > > > Chrome > > > device. > > > > > > Will enable this just for ImgU to get ahead and get back with > > > > > > more > > > updates. > > > > > > > > > > > > > > > > Thanks for testing this. > > > > > > > > > > For your informations I'm using the following branch, from > > > > > Sakari's > > > > > tree: git://linuxtv.org/sailus/media_tree.git ipu3 > > > > > > > > > > Although it appears that the media tree master branch has > > > > > everything that is there, with a few additional patches on top. > > > > > I should move to use media tree master as well... > > > > > > > > > > I have here attached 2 configuration files for v4.20-rc5 I am > > > > > using on Soraka, in case they might help you. One has KASAN > > > > > enabled with an increased kernel log size, the other one is the > > > > > one we use for daily > > > development. > > > > > > > > I think I am missing a trick here to override the default chrome > > > > os kernel config with the one that you supplied. > > > > > > > > In particular I am looking for steps to build the upstream kernel > > > > within chrome os build environment using your config, so I can > > > > update my > > > Soraka device. > > > > > > I'm sorry I can not help much building 'withing chrome os build > environment'. > > > Care to explain what you mean? > > > > > > > This is part of the Chromium OS build environment and development > workflow. > > > https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq. > > md > > > > No worries. > > I will sync up with Tomasz, as he managed to get this working with 4.20 > kernel. > > > > I'm sorry I can't help much here. I suggest to work with mainline (or better, > media master) and install a GNU/Linux distro on the chromebook so you can > easily update your kernel. > > I personally used https://chrx.org/ that makes installing gallium (or > else) very easy. Once you get that running, I find easy enough to update the > kernel installing a .deb package. Thanks for the pointers. I will have a look at this workflow.
Hi Laurent et al, > Subject: RE: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hi Jacopo, > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > Hi Raj, > > > > On Wed, Jan 09, 2019 at 06:01:39PM +0000, Mani, Rajmohan wrote: > > > Hi Jacopo, > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > Hello Raj, > > > > > > > > On Wed, Jan 09, 2019 at 05:00:21PM +0000, Mani, Rajmohan wrote: > > > > > Hi Laurent, Tomasz, Jacopo, > > > > > > > > > > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > > > > > > > > > Hello, > > > > > > > > > > > > On Tue, Jan 08, 2019 at 03:54:34PM +0900, Tomasz Figa wrote: > > > > > > > Hi Raj, Yong, Bingbu, Tianshu, > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 12:04 PM Tomasz Figa > > > > > > > <tfiga@chromium.org> > > > > wrote: > > > > > > > > > > > > > > > > On Fri, Dec 21, 2018 at 7:24 AM Laurent Pinchart > > > > > > > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > > > > > > > > > > > > > Hellon > > > > > > > > > > > > > > > > > > On Sunday, 16 December 2018 09:26:18 EET Laurent > > > > > > > > > Pinchart > > wrote: > > > > > > > > > > Hello Yong, > > > > > > > > > > > > > > > > > > > > Could you please have a look at the crash reported below ? > > > > > > > > > > > > > > > > > > A bit more information to help you debugging this. I've > > > > > > > > > enabled KASAN in the kernel configuration, and get the > > > > > > > > > following use-after-free > > > > > > reports. > > > > > > > > > > > > I tested as well using the ipu-process.sh script shared by > > > > > > Laurent, with the following command line: > > > > > > ./ipu3-process.sh --out 2560x1920 --vf 1920x1080 > > > > > > frame-2592x1944.cio2 > > > > > > > > > > > > and I got a very similar trace available at: > > > > > > https://paste.debian.net/hidden/5855e15a/ > > > > > > > > > > > > Please note I have been able to process a set of images (with > > > > > > KASAN enabled the machine does not freeze) but the kernel log > > > > > > gets flooded and it is not possible to process any other frame after > this. > > > > > > > > > > > > The issue is currently quite annoying and it's a blocker for > > > > > > libcamera development on IPU3. Please let me know if I can > > > > > > support with > > > > more testing. > > > > > > > > > > > > Thanks > > > > > > j > > > > > > > > > > > > > > > > > > > > > > > > [ 166.332920] > > > > > > > > > > > > > > > > > > > > > > ================================================================ > > > > > > == > > > > > > > > > [ 166.332937] BUG: KASAN: use-after-free in > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > [ 166.332944] Read of size 8 at addr ffff888133823718 > > > > > > > > > by task > > > > > > > > > yavta/1305 > > > > > > > > > > > > > > > > > > [ 166.332955] CPU: 3 PID: 1305 Comm: yavta Tainted: G C > > > > 4.20.0- > > > > > > rc6+ #3 > > > > > > > > > [ 166.332958] Hardware name: HP Soraka/Soraka, BIOS > > > > > > > > > 08/30/2018 [ 166.332959] Call Trace: > > > > > > > > > [ 166.332967] dump_stack+0x5b/0x81 [ 166.332974] > > > > > > > > > print_address_description+0x65/0x227 > > > > > > > > > [ 166.332979] ? > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > [ 166.332983] kasan_report+0x247/0x285 [ 166.332989] > > > > > > > > > __cached_rbnode_delete_update+0x36/0x202 > > > > > > > > > [ 166.332995] private_free_iova+0x57/0x6d [ > > > > > > > > > 166.332999] > > > > > > > > > __free_iova+0x23/0x31 [ 166.333011] > > > > > > > > > ipu3_dmamap_free+0x118/0x1d6 [ipu3_imgu] > > > > > > > > > > > > > > > > Thanks Laurent, I think this is a very good hint. It looks > > > > > > > > like we're basically freeing and already freed IOVA and > > > > > > > > corrupting some allocator state? > > > > > > > > > > > > > > Did you have any luck in reproducing and fixing this double > > > > > > > free > > issue? > > > > > > > > > > > > > > > > > This issue is either hard to reproduce or comes with different > > > > > signatures with the updated yavta (that now supports meta > > > > > output) with the 4.4 kernel that I have been using. > > > > > I am switching to 4.20-rc6 for better reproducibility. > > > > > Enabling KASAN also results in storage space issues on my Chrome > > device. > > > > > Will enable this just for ImgU to get ahead and get back with > > > > > more > > updates. > > > > > > > > > > > > > Thanks for testing this. > > > > > > > > For your informations I'm using the following branch, from > > > > Sakari's > > > > tree: git://linuxtv.org/sailus/media_tree.git ipu3 > > > > > > > > Although it appears that the media tree master branch has > > > > everything that is there, with a few additional patches on top. I > > > > should move to use media tree master as well... > > > > > > > > I have here attached 2 configuration files for v4.20-rc5 I am > > > > using on Soraka, in case they might help you. One has KASAN > > > > enabled with an increased kernel log size, the other one is the > > > > one we use for daily > > development. > > > > > > I think I am missing a trick here to override the default chrome os > > > kernel config with the one that you supplied. > > > > > > In particular I am looking for steps to build the upstream kernel > > > within chrome os build environment using your config, so I can > > > update my > > Soraka device. > > > > I'm sorry I can not help much building 'withing chrome os build > environment'. > > Care to explain what you mean? > > > > This is part of the Chromium OS build environment and development > workflow. > https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq. > md > > No worries. > I will sync up with Tomasz, as he managed to get this working with 4.20 kernel. > I finally managed to reproduce the issue with 4.20-rc6, with KASAN enabled and with CONFIG_SLUB_DEBUG_ON with SLAB_STORE_USER. The following line indicates the crash happens when yavta PID 10289 tries to free the memory. [ 452.437844] BUG: KASAN: use-after-free in ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.446123] Read of size 8 at addr ffff8881503481a0 by task yavta/10289 The above looks to be normal, since it's the same task that allocated this memory. [ 452.685731] Allocated by task 10289: Before the above happened, yavta/10187 came in and freed this memory per KASAN. [ 452.787656] Freed by task 10187: Is this (one instance of yavta freeing the memory allocated by another instance of yavta) expected? Or does it indicate that mmap giving the same address across these 2 instances of yavta? I need to debug / confirm the latter case. With the help of local application that operates these pipes in a serial fashion, I do not see this issue. I have pasted the relevant parts of the dmesg. [ 452.038082] WARNING: CPU: 1 PID: 10289 at /mnt/host/source/src/third_party/kernel/v4.4/drivers/staging/media/ipu3/ipu3-dmamap.c:172 ipu3_dmamap_unmap+0xf6/0x107 [ipu3_imgu] [ 452.055293] Modules linked in: cmac rfcomm uinput snd_soc_kbl_rt5663_max98927 snd_soc_skl_ssp_clk snd_soc_hdac_hdmi snd_soc_dmic btusb btrtl btbcm asix usbnet btintel bluetooth snd_soc_skl snd_soc_skl_ipc ecdh_generic snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_hda_core ipu3_imgu(C) ipu3_cio2 iova videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 videobuf2_common snd_soc_rt5663 snd_soc_max98927 at24 snd_soc_rl6231 ov13858 ov5670 v4l2_fwnode dw9714 bridge stp llc acpi_als kfifo_buf industrialio ipt_MASQUERADE lzo lzo_compress zram xt_mark fuse snd_seq_dummy snd_seq snd_seq_device cfg80211 ip6table_filter r8152 mii joydev [ 452.117513] CPU: 1 PID: 10289 Comm: yavta Tainted: G WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 452.128705] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 452.137476] RIP: 0010:ipu3_dmamap_unmap+0xf6/0x107 [ipu3_imgu] [ 452.144007] Code: e1 48 d3 e2 48 8b 7d c8 48 89 de e8 7b f5 ff ff 48 8b 7d d0 4c 89 fe 48 83 c4 20 5b 41 5c 41 5d 41 5e 41 5f 5d e9 b2 f5 ee ff <0f> 0b 48 83 c4 20 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 44 00 00 [ 452.165007] RSP: 0018:ffff88814ef67a20 EFLAGS: 00010246 [ 452.170857] RAX: 0000000000000000 RBX: 00000000000e527e RCX: 0000000000000001 [ 452.178842] RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff8881179076b8 [ 452.186828] RBP: ffff88814ef67a68 R08: 0000000000000000 R09: ffffed1022f20ed8 [ 452.194812] R10: ffff8881179076bb R11: dffffc0000000000 R12: ffff8881179076e8 [ 452.202799] R13: ffff888117900028 R14: ffff8881179076b8 R15: 0000000000000000 [ 452.210784] FS: 00007a5d6524a700(0000) GS:ffff88815b680000(0000) knlGS:0000000000000000 [ 452.219837] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 452.226271] CR2: 00005ba49b474078 CR3: 0000000129e1e002 CR4: 00000000003606e0 [ 452.234251] Call Trace: [ 452.237003] ipu3_dmamap_free+0x41/0x9c [ipu3_imgu] [ 452.242473] ipu3_css_pool_cleanup+0x24/0x37 [ipu3_imgu] [ 452.248431] ipu3_css_pipeline_cleanup+0x61/0xb9 [ipu3_imgu] [ 452.254772] ipu3_css_stop_streaming+0x1f2/0x321 [ipu3_imgu] [ 452.261119] imgu_s_stream+0x94/0x443 [ipu3_imgu] [ 452.266392] ? ipu3_vb2_buf_queue+0x280/0x280 [ipu3_imgu] [ 452.272438] ? vb2_dma_sg_unmap_dmabuf+0x16/0x6f [videobuf2_dma_sg] [ 452.279456] ? vb2_buffer_in_use+0x36/0x58 [videobuf2_common] [ 452.285894] ipu3_vb2_stop_streaming+0xf9/0x135 [ipu3_imgu] [ 452.292137] __vb2_queue_cancel+0x35/0x215 [videobuf2_common] [ 452.298576] vb2_core_streamoff+0x19/0x73 [videobuf2_common] [ 452.304920] __video_do_ioctl+0x34e/0x450 [ 452.309414] video_usercopy+0x25e/0x597 [ 452.313718] ? video_ioctl2+0x16/0x16 [ 452.317823] ? __switch_to_asm+0x34/0x70 [ 452.322215] v4l2_ioctl+0x45/0x49 [ 452.325932] vfs_ioctl+0x1b/0x30 [ 452.329551] do_vfs_ioctl+0x479/0x6d0 [ 452.333660] ksys_ioctl+0x53/0x79 [ 452.337375] __se_sys_ioctl+0xe/0x12 [ 452.341379] do_syscall_64+0x52/0x60 [ 452.345384] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 452.351035] RIP: 0033:0x7a5d64b73967 [ 452.355042] Code: 8a 66 90 48 8b 05 29 55 2b 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f9 54 2b 00 f7 d8 64 89 01 48 [ 452.376041] RSP: 002b:00007fff3483aca8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 452.384515] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007a5d64b73967 [ 452.392501] RDX: 00007fff3483acb4 RSI: 0000000040045613 RDI: 0000000000000003 [ 452.400484] RBP: 0000000000404c48 R08: fffffffffed7c030 R09: fffffffffed7c020 [ 452.408467] R10: fffffffffed7c010 R11: 0000000000000246 R12: 0000000000404c56 [ 452.416450] R13: 0000000000000001 R14: 00007fff3483c75c R15: 000000000062b800 [ 452.424432] ---[ end trace ed0895d0744ba932 ]--- [ 452.429752] ================================================================== [ 452.437844] BUG: KASAN: use-after-free in ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.446123] Read of size 8 at addr ffff8881503481a0 by task yavta/10289 [ 452.455191] CPU: 1 PID: 10289 Comm: yavta Tainted: G WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 452.466380] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 452.475133] Call Trace: [ 452.477880] dump_stack+0x6a/0xb1 [ 452.481600] print_address_description+0x8e/0x279 [ 452.486873] ? ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.492530] kasan_report+0x260/0x28a [ 452.496637] ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.502103] ipu3_css_pool_cleanup+0x24/0x37 [ipu3_imgu] [ 452.508056] ipu3_css_pipeline_cleanup+0x61/0xb9 [ipu3_imgu] [ 452.514395] ipu3_css_stop_streaming+0x1f2/0x321 [ipu3_imgu] [ 452.520737] imgu_s_stream+0x94/0x443 [ipu3_imgu] [ 452.526010] ? ipu3_vb2_buf_queue+0x280/0x280 [ipu3_imgu] [ 452.532058] ? vb2_dma_sg_unmap_dmabuf+0x16/0x6f [videobuf2_dma_sg] [ 452.539076] ? vb2_buffer_in_use+0x36/0x58 [videobuf2_common] [ 452.545513] ipu3_vb2_stop_streaming+0xf9/0x135 [ipu3_imgu] [ 452.551762] __vb2_queue_cancel+0x35/0x215 [videobuf2_common] [ 452.558203] vb2_core_streamoff+0x19/0x73 [videobuf2_common] [ 452.564542] __video_do_ioctl+0x34e/0x450 [ 452.569039] video_usercopy+0x25e/0x597 [ 452.573341] ? video_ioctl2+0x16/0x16 [ 452.577443] ? __switch_to_asm+0x34/0x70 [ 452.581838] v4l2_ioctl+0x45/0x49 [ 452.585559] vfs_ioctl+0x1b/0x30 [ 452.589178] do_vfs_ioctl+0x479/0x6d0 [ 452.593277] ksys_ioctl+0x53/0x79 [ 452.596991] __se_sys_ioctl+0xe/0x12 [ 452.601000] do_syscall_64+0x52/0x60 [ 452.605010] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 452.610668] RIP: 0033:0x7a5d64b73967 [ 452.614677] Code: 8a 66 90 48 8b 05 29 55 2b 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f9 54 2b 00 f7 d8 64 89 01 48 [ 452.635676] RSP: 002b:00007fff3483aca8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 452.644149] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007a5d64b73967 [ 452.652125] RDX: 00007fff3483acb4 RSI: 0000000040045613 RDI: 0000000000000003 [ 452.660109] RBP: 0000000000404c48 R08: fffffffffed7c030 R09: fffffffffed7c020 [ 452.668092] R10: fffffffffed7c010 R11: 0000000000000246 R12: 0000000000404c56 [ 452.676076] R13: 0000000000000001 R14: 00007fff3483c75c R15: 000000000062b800 [ 452.685731] Allocated by task 10289: [ 452.689736] set_track+0x64/0xfb [ 452.693354] __kmalloc+0x94/0x1af [ 452.697066] __get_vm_area_node+0x9e/0x103 [ 452.701654] __get_vm_area+0x26/0x29 [ 452.705653] ipu3_dmamap_alloc+0x333/0x503 [ipu3_imgu] [ 452.711408] ipu3_css_pool_init+0x43/0x99 [ipu3_imgu] [ 452.717070] ipu3_css_start_streaming+0x25cf/0x29a7 [ipu3_imgu] [ 452.723697] imgu_s_stream+0x133/0x443 [ipu3_imgu] [ 452.729055] ipu3_vb2_start_streaming+0x1a3/0x1f1 [ipu3_imgu] [ 452.735492] vb2_start_streaming+0x71/0x11c [videobuf2_common] [ 452.742027] vb2_core_streamon+0xf8/0x118 [videobuf2_common] [ 452.748371] __video_do_ioctl+0x34e/0x450 [ 452.752857] video_usercopy+0x25e/0x597 [ 452.757155] v4l2_ioctl+0x45/0x49 [ 452.760869] vfs_ioctl+0x1b/0x30 [ 452.764497] do_vfs_ioctl+0x479/0x6d0 [ 452.768602] ksys_ioctl+0x53/0x79 [ 452.772318] __se_sys_ioctl+0xe/0x12 [ 452.776322] do_syscall_64+0x52/0x60 [ 452.780330] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 452.787656] Freed by task 10187: [ 452.791274] set_track+0x64/0xfb [ 452.794897] __kasan_slab_free+0xde/0x101 [ 452.799393] slab_free_freelist_hook+0x4d/0x9e [ 452.804373] kfree+0x8b/0x4d7 [ 452.807703] ipu3_dmamap_free+0x7e/0x9c [ipu3_imgu] [ 452.813167] ipu3_css_pool_cleanup+0x24/0x37 [ipu3_imgu] [ 452.819117] ipu3_css_pipeline_cleanup+0x61/0xb9 [ipu3_imgu] [ 452.825454] ipu3_css_stop_streaming+0x1f2/0x321 [ipu3_imgu] [ 452.831796] imgu_s_stream+0x94/0x443 [ipu3_imgu] [ 452.837066] ipu3_vb2_stop_streaming+0xf9/0x135 [ipu3_imgu] [ 452.843308] __vb2_queue_cancel+0x35/0x215 [videobuf2_common] [ 452.849747] vb2_core_streamoff+0x19/0x73 [videobuf2_common] [ 452.856086] __video_do_ioctl+0x34e/0x450 [ 452.860610] video_usercopy+0x25e/0x597 [ 452.864913] v4l2_ioctl+0x45/0x49 [ 452.868630] vfs_ioctl+0x1b/0x30 [ 452.872245] do_vfs_ioctl+0x479/0x6d0 [ 452.876349] ksys_ioctl+0x53/0x79 [ 452.880059] __se_sys_ioctl+0xe/0x12 [ 452.884068] do_syscall_64+0x52/0x60 [ 452.888075] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 452.895396] The buggy address belongs to the object at ffff888150348180 which belongs to the cache kmalloc-64 of size 64 [ 452.909203] The buggy address is located 32 bytes inside of 64-byte region [ffff888150348180, ffff8881503481c0) [ 452.922142] The buggy address belongs to the page: [ 452.927506] page:ffffea000540d200 count:1 mapcount:0 mapping:ffff88815ac0f840 index:0x0 compound_mapcount: 0 [ 452.938503] flags: 0x8000000000010200(slab|head) [ 452.943675] raw: 8000000000010200 ffffea00055d1d08 ffffea0005345e08 ffff88815ac0f840 [ 452.952342] raw: 0000000000000000 0000000000150015 00000001ffffffff 0000000000000000 [ 452.961008] page dumped because: kasan: bad access detected [ 452.968915] Memory state around the buggy address: [ 452.974277] ffff888150348080: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 452.982361] ffff888150348100: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 452.990435] >ffff888150348180: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc [ 452.998518] ^ [ 453.003291] ffff888150348200: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 453.011376] ffff888150348280: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 453.019457] ================================================================== [ 453.027537] Disabling lock debugging due to kernel taint [ 453.034645] ================================================================== [ 453.042736] BUG: KASAN: double-free or invalid-free in kfree+0x8b/0x4d7 [ 453.051817] CPU: 1 PID: 10289 Comm: yavta Tainted: G B WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 453.063006] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 453.071767] Call Trace: [ 453.074513] dump_stack+0x6a/0xb1 [ 453.078233] ? kfree+0x8b/0x4d7 [ 453.081757] ? kfree+0x8b/0x4d7 [ 453.085284] print_address_description+0x8e/0x279 [ 453.090556] ? kfree+0x8b/0x4d7 [ 453.094082] ? kfree+0x8b/0x4d7 [ 453.097608] kasan_report_invalid_free+0x58/0x95 [ 453.102787] __kasan_slab_free+0x9f/0x101 [ 453.107286] slab_free_freelist_hook+0x4d/0x9e [ 453.112266] ? ipu3_dmamap_free+0x6d/0x9c [ipu3_imgu] [ 453.117930] kfree+0x8b/0x4d7 [ 453.121262] ? __free_pages+0x2f/0x71 [ 453.125368] ipu3_dmamap_free+0x6d/0x9c [ipu3_imgu] [ 453.130833] ipu3_css_pool_cleanup+0x24/0x37 [ipu3_imgu] [ 453.136786] ipu3_css_pipeline_cleanup+0x61/0xb9 [ipu3_imgu] [ 453.143131] ipu3_css_stop_streaming+0x1f2/0x321 [ipu3_imgu] [ 453.149476] imgu_s_stream+0x94/0x443 [ipu3_imgu] [ 453.154750] ? ipu3_vb2_buf_queue+0x280/0x280 [ipu3_imgu] [ 453.160798] ? vb2_dma_sg_unmap_dmabuf+0x16/0x6f [videobuf2_dma_sg] [ 453.167821] ? vb2_buffer_in_use+0x36/0x58 [videobuf2_common] [ 453.174263] ipu3_vb2_stop_streaming+0xf9/0x135 [ipu3_imgu] [ 453.180500] __vb2_queue_cancel+0x35/0x215 [videobuf2_common] [ 453.186938] vb2_core_streamoff+0x19/0x73 [videobuf2_common] [ 453.193276] __video_do_ioctl+0x34e/0x450 [ 453.197789] video_usercopy+0x25e/0x597 [ 453.202088] ? video_ioctl2+0x16/0x16 [ 453.206193] ? __switch_to_asm+0x34/0x70 [ 453.210587] v4l2_ioctl+0x45/0x49 [ 453.214302] vfs_ioctl+0x1b/0x30 [ 453.217923] do_vfs_ioctl+0x479/0x6d0 [ 453.222033] ksys_ioctl+0x53/0x79 [ 453.225750] __se_sys_ioctl+0xe/0x12 [ 453.229757] do_syscall_64+0x52/0x60 [ 453.233765] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 453.239423] RIP: 0033:0x7a5d64b73967 [ 453.243429] Code: 8a 66 90 48 8b 05 29 55 2b 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f9 54 2b 00 f7 d8 64 89 01 48 [ 453.264428] RSP: 002b:00007fff3483aca8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 453.272904] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007a5d64b73967 [ 453.280892] RDX: 00007fff3483acb4 RSI: 0000000040045613 RDI: 0000000000000003 [ 453.288877] RBP: 0000000000404c48 R08: fffffffffed7c030 R09: fffffffffed7c020 [ 453.296861] R10: fffffffffed7c010 R11: 0000000000000246 R12: 0000000000404c56 [ 453.304842] R13: 0000000000000001 R14: 00007fff3483c75c R15: 000000000062b800 [ 453.314500] Allocated by task 10289: [ 453.318507] set_track+0x64/0xfb [ 453.322120] __kmalloc+0x94/0x1af [ 453.325832] kvmalloc_node+0x4e/0x84 [ 453.329839] ipu3_dmamap_alloc+0xec/0x503 [ipu3_imgu] [ 453.335488] ipu3_css_pool_init+0x43/0x99 [ipu3_imgu] [ 453.341144] ipu3_css_start_streaming+0x25cf/0x29a7 [ipu3_imgu] [ 453.347775] imgu_s_stream+0x133/0x443 [ipu3_imgu] [ 453.353144] ipu3_vb2_start_streaming+0x1a3/0x1f1 [ipu3_imgu] [ 453.359579] vb2_start_streaming+0x71/0x11c [videobuf2_common] [ 453.366111] vb2_core_streamon+0xf8/0x118 [videobuf2_common] [ 453.372449] __video_do_ioctl+0x34e/0x450 [ 453.376935] video_usercopy+0x25e/0x597 [ 453.381230] v4l2_ioctl+0x45/0x49 [ 453.384945] vfs_ioctl+0x1b/0x30 [ 453.388568] do_vfs_ioctl+0x479/0x6d0 [ 453.392672] ksys_ioctl+0x53/0x79 [ 453.396387] __se_sys_ioctl+0xe/0x12 [ 453.400393] do_syscall_64+0x52/0x60 [ 453.404395] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 453.411716] Freed by task 10187: [ 453.415340] set_track+0x64/0xfb [ 453.418960] __kasan_slab_free+0xde/0x101 [ 453.423452] slab_free_freelist_hook+0x4d/0x9e [ 453.428430] kfree+0x8b/0x4d7 [ 453.431766] ipu3_dmamap_free+0x6d/0x9c [ipu3_imgu] [ 453.437231] ipu3_css_pool_cleanup+0x24/0x37 [ipu3_imgu] [ 453.443183] ipu3_css_pipeline_cleanup+0x61/0xb9 [ipu3_imgu] [ 453.449522] ipu3_css_stop_streaming+0x1f2/0x321 [ipu3_imgu] [ 453.455864] imgu_s_stream+0x94/0x443 [ipu3_imgu] [ 453.461133] ipu3_vb2_stop_streaming+0xf9/0x135 [ipu3_imgu] [ 453.467375] __vb2_queue_cancel+0x35/0x215 [videobuf2_common] [ 453.473810] vb2_core_streamoff+0x19/0x73 [videobuf2_common] [ 453.480154] __video_do_ioctl+0x34e/0x450 [ 453.484650] video_usercopy+0x25e/0x597 [ 453.488946] v4l2_ioctl+0x45/0x49 [ 453.492652] vfs_ioctl+0x1b/0x30 [ 453.496271] do_vfs_ioctl+0x479/0x6d0 [ 453.500376] ksys_ioctl+0x53/0x79 [ 453.504091] __se_sys_ioctl+0xe/0x12 [ 453.508094] do_syscall_64+0x52/0x60 [ 453.512099] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 453.519424] The buggy address belongs to the object at ffff888153eaf380 which belongs to the cache kmalloc-1k of size 1024 [ 453.533431] The buggy address is located 0 bytes inside of 1024-byte region [ffff888153eaf380, ffff888153eaf780) [ 453.546466] The buggy address belongs to the page: [ 453.551828] page:ffffea00054faa00 count:1 mapcount:0 mapping:ffff88815ac0f180 index:0x0 compound_mapcount: 0 [ 453.562829] flags: 0x8000000000010200(slab|head) [ 453.568002] raw: 8000000000010200 ffffea0004b88a08 ffffea000565e808 ffff88815ac0f180 [ 453.576670] raw: 0000000000000000 0000000000180018 00000001ffffffff 0000000000000000 [ 453.585337] page dumped because: kasan: bad access detected [ 453.593239] Memory state around the buggy address: [ 453.598596] ffff888153eaf280: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 453.606680] ffff888153eaf300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [ 453.614765] >ffff888153eaf380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 453.622846] ^ [ 453.626467] ffff888153eaf400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 453.634550] ffff888153eaf480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 453.642624] ================================================================== [ 453.653315] ------------[ cut here ]------------ [ 453.658485] kernel BUG at /mnt/host/source/src/third_party/kernel/v4.4/mm/slub.c:3940! [ 453.667369] invalid opcode: 0000 [#1] PREEMPT SMP KASAN PTI [ 453.673604] CPU: 0 PID: 9 Comm: ksoftirqd/0 Tainted: G B WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 453.684990] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 453.693762] RIP: 0010:kfree+0x4d3/0x4d7 [ 453.698049] Code: 7d b0 48 8b 75 a0 e8 38 e6 6e 00 4c 89 ff 4c 89 f6 e8 3f a9 ff ff e9 22 fc ff ff 4c 89 ff 4c 89 f6 e8 1d b6 ff ff eb d6 0f 0b <0f> 0b 0f 0b 0f 1f 44 00 00 55 48 89 e5 48 8b 07 48 8b 4f 08 48 89 [ 453.719051] RSP: 0018:ffff88815af17d20 EFLAGS: 00010246 [ 453.724904] RAX: ffffea0001d9a288 RBX: ffff88807644d860 RCX: ffffea0001d91300 [ 453.732891] RDX: ffffea0001d91340 RSI: 0000000000000004 RDI: 0000000001d91361 [ 453.740866] RBP: ffff88815af17da8 R08: 0000000000000000 R09: fffffbfff6521ca7 [ 453.748852] R10: ffffffffb290e533 R11: dffffc0000000000 R12: ffffffffb16dec02 [ 453.751044] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout [ 453.756835] R13: ffff88815b62ab70 R14: ffffea0001d91300 R15: ffff88815af08000 [ 453.756839] FS: 0000000000000000(0000) GS:ffff88815b600000(0000) knlGS:0000000000000000 [ 453.756841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 453.756843] CR2: 00007a8ac018f000 CR3: 00000000744ba006 CR4: 00000000003606f0 [ 453.756845] Call Trace: [ 453.756855] rcu_process_callbacks+0x20a/0x437 [ 453.769422] BUG: Bad page state in process yavta pfn:74756 [ 453.771362] __do_softirq+0x16c/0x33e [ 453.780406] page:ffffea0001d1d580 count:0 mapcount:0 mapping:ffff88815ae4c6c0 index:0x0 [ 453.786839] run_ksoftirqd+0x1d/0x34 [ 453.794809] flags: 0x4000000000000000() [ 453.797551] smpboot_thread_fn+0x1bb/0x291 [ 453.802503] raw: 4000000000000000 dead000000000100 dead000000000200 ffff88815ae4c6c0 [ 453.808736] ? cpu_report_death+0x84/0x84 [ 453.812829] raw: 0000000000000000 0000000000100010 00000000ffffffff 0000000000000000 [ 453.821781] kthread+0xfd/0x10d [ 453.825773] page dumped because: non-NULL mapping [ 453.830072] ? cpu_report_death+0x84/0x84 [ 453.834649] Modules linked in: cmac rfcomm uinput snd_soc_kbl_rt5663_max98927 snd_soc_skl_ssp_clk snd_soc_hdac_hdmi snd_soc_dmic btusb btrtl btbcm asix usbnet btintel bluetooth snd_soc_skl snd_soc_skl_ipc ecdh_generic snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_hda_core ipu3_imgu(C) ipu3_cio2 iova videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 videobuf2_common snd_soc_rt5663 snd_soc_max98927 at24 snd_soc_rl6231 ov13858 ov5670 v4l2_fwnode dw9714 bridge stp llc acpi_als kfifo_buf industrialio ipt_MASQUERADE lzo lzo_compress zram xt_mark fuse snd_seq_dummy snd_seq snd_seq_device cfg80211 ip6table_filter r8152 mii joydev [ 453.843306] ? kthread_destroy_worker+0x49/0x49 [ 453.847795] CPU: 1 PID: 10289 Comm: yavta Tainted: G B WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 453.856449] ret_from_fork+0x35/0x40 [ 453.859956] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 453.865208] Modules linked in: cmac rfcomm uinput snd_soc_kbl_rt5663_max98927 snd_soc_skl_ssp_clk snd_soc_hdac_hdmi snd_soc_dmic btusb btrtl btbcm asix usbnet btintel bluetooth snd_soc_skl snd_soc_skl_ipc ecdh_generic snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_hda_core ipu3_imgu(C) ipu3_cio2 iova videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 videobuf2_common snd_soc_rt5663 snd_soc_max98927 at24 snd_soc_rl6231 ov13858 ov5670 v4l2_fwnode dw9714 bridge stp llc acpi_als kfifo_buf industrialio ipt_MASQUERADE lzo lzo_compress zram xt_mark fuse snd_seq_dummy snd_seq snd_seq_device cfg80211 ip6table_filter r8152 mii joydev [ 453.869692] Call Trace: [ 453.933872] gsmi: Log Shutdown Reason 0x03 [ 453.938939] dump_stack+0x6a/0xb1 [ 453.950359] ---[ end trace ed0895d0744ba933 ]--- [ 453.954123] bad_page+0x140/0x14a [ 453.954128] free_pages_check+0x87/0x95 [ 453.954132] free_pcppages_bulk+0xbd/0x218 [ 453.954137] free_unref_page+0x49/0x6e [ 453.954142] __free_pages+0x4a/0x71 [ 453.962932] RIP: 0010:kfree+0x4d3/0x4d7 [ 454.025068] vb2_dma_sg_put+0x8f/0xec [videobuf2_dma_sg] [ 454.025074] __vb2_buf_mem_free+0x39/0x75 [videobuf2_common] [ 454.025079] __vb2_queue_free+0xb3/0x19f [videobuf2_common] [ 454.025084] vb2_core_reqbufs+0x12a/0x312 [videobuf2_common] [ 454.025090] vb2_ioctl_reqbufs+0x81/0xa8 [videobuf2_v4l2] [ 454.025098] __video_do_ioctl+0x34e/0x450 [ 454.025105] video_usercopy+0x25e/0x597 [ 454.025109] ? video_ioctl2+0x16/0x16 [ 454.025116] v4l2_ioctl+0x45/0x49 [ 454.025121] vfs_ioctl+0x1b/0x30 [ 454.025125] do_vfs_ioctl+0x479/0x6d0 [ 454.025131] ksys_ioctl+0x53/0x79 [ 454.025136] __se_sys_ioctl+0xe/0x12 [ 454.027896] Code: 7d b0 48 8b 75 a0 e8 38 e6 6e 00 4c 89 ff 4c 89 f6 e8 3f a9 ff ff e9 22 fc ff ff 4c 89 ff 4c 89 f6 e8 1d b6 ff ff eb d6 0f 0b <0f> 0b 0f 0b 0f 1f 44 00 00 55 48 89 e5 48 8b 07 48 8b 4f 08 48 89 [ 454.032459] do_syscall_64+0x52/0x60 [ 454.032465] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 454.032469] RIP: 0033:0x7a5d64b73967 [ 454.032473] Code: 8a 66 90 48 8b 05 29 55 2b 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f9 54 2b 00 f7 d8 64 89 01 48 [ 454.032475] RSP: 002b:00007fff3483acd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 454.032479] RAX: ffffffffffffffda RBX: 00000000023d97a0 RCX: 00007a5d64b73967 [ 454.036194] RSP: 0018:ffff88815af17d20 EFLAGS: 00010246 [ 454.041352] RDX: 00007fff3483ade0 RSI: 00000000c0145608 RDI: 0000000000000003 [ 454.041355] RBP: 0000000000000007 R08: 00007a5d64e287c0 R09: 0000000000000045 [ 454.041356] R10: fffffffffffff88f R11: 0000000000000246 R12: 0000000000000001 [ 454.041358] R13: 00000000023d9778 R14: 00000000023d9750 R15: 000000000062b800 [ 454.041364] BUG: Bad page state in process yavta pfn:7671e [ 454.041368] page:ffffea0001d9c780 count:0 mapcount:0 mapping:ffff88815ae4c6c0 index:0x0 [ 454.041370] flags: 0x4000000000000000() [ 454.041375] raw: 4000000000000000 dead000000000100 dead000000000200 ffff88815ae4c6c0 [ 454.041378] raw: 0000000000000000 0000000000100010 00000000ffffffff 0000000000000000 [ 454.041379] page dumped because: non-NULL mapping [ 454.041380] Modules linked in: cmac rfcomm uinput snd_soc_kbl_rt5663_max98927 snd_soc_skl_ssp_clk snd_soc_hdac_hdmi snd_soc_dmic btusb btrtl btbcm asix usbnet btintel bluetooth snd_soc_skl snd_soc_skl_ipc ecdh_generic snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_hda_core ipu3_imgu(C) ipu3_cio2 iova videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 videobuf2_common snd_soc_rt5663 snd_soc_max98927 at24 snd_soc_rl6231 ov13858 ov5670 v4l2_fwnode dw9714 bridge stp llc acpi_als kfifo_buf industrialio ipt_MASQUERADE lzo lzo_compress zram xt_mark fuse snd_seq_dummy snd_seq snd_seq_device cfg80211 ip6table_filter r8152 mii joydev [ 454.045094] RAX: ffffea0001d9a288 RBX: ffff88807644d860 RCX: ffffea0001d91300 [ 454.049378] CPU: 1 PID: 10289 Comm: yavta Tainted: G B D WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 454.049380] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 454.049381] Call Trace: [ 454.049389] dump_stack+0x6a/0xb1 [ 454.049395] bad_page+0x140/0x14a [ 454.049399] free_pages_check+0x87/0x95 [ 454.049403] free_pcppages_bulk+0xbd/0x218 [ 454.049408] free_unref_page+0x49/0x6e [ 454.049412] __free_pages+0x4a/0x71 [ 454.049420] vb2_dma_sg_put+0x8f/0xec [videobuf2_dma_sg] [ 454.049433] __vb2_buf_mem_free+0x39/0x75 [videobuf2_common] [ 454.049438] __vb2_queue_free+0xb3/0x19f [videobuf2_common] [ 454.049444] vb2_core_reqbufs+0x12a/0x312 [videobuf2_common] [ 454.049450] vb2_ioctl_reqbufs+0x81/0xa8 [videobuf2_v4l2] [ 454.049455] __video_do_ioctl+0x34e/0x450 [ 454.054060] RDX: ffffea0001d91340 RSI: 0000000000000004 RDI: 0000000001d91361 [ 454.058234] video_usercopy+0x25e/0x597 [ 454.058238] ? video_ioctl2+0x16/0x16 [ 454.058243] v4l2_ioctl+0x45/0x49 [ 454.062148] RBP: ffff88815af17da8 R08: 0000000000000000 R09: fffffbfff6521ca7 [ 454.066440] vfs_ioctl+0x1b/0x30 [ 454.066444] do_vfs_ioctl+0x479/0x6d0 [ 454.066448] ksys_ioctl+0x53/0x79 [ 454.066452] __se_sys_ioctl+0xe/0x12 [ 454.066456] do_syscall_64+0x52/0x60 [ 454.066461] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 454.066464] RIP: 0033:0x7a5d64b73967 [ 454.066468] Code: 8a 66 90 48 8b 05 29 55 2b 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d f9 54 2b 00 f7 d8 64 89 01 48 [ 454.066469] RSP: 002b:00007fff3483acd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 454.066476] RAX: ffffffffffffffda RBX: 00000000023d97a0 RCX: 00007a5d64b73967 [ 454.072423] R10: ffffffffb290e533 R11: dffffc0000000000 R12: ffffffffb16dec02 [ 454.078745] RDX: 00007fff3483ade0 RSI: 00000000c0145608 RDI: 0000000000000003 [ 454.078747] RBP: 0000000000000007 R08: 00007a5d64e287c0 R09: 0000000000000045 [ 454.078749] R10: fffffffffffff88f R11: 0000000000000246 R12: 0000000000000001 [ 454.078750] R13: 00000000023d9778 R14: 00000000023d9750 R15: 000000000062b800 [ 454.078758] BUG: Bad page state in process yavta pfn:746a6 [ 454.085002] R13: ffff88815b62ab70 R14: ffffea0001d91300 R15: ffff88815af08000 [ 454.091322] page:ffffea0001d1a980 count:0 mapcount:0 mapping:ffff88815ae4c6c0 index:0x0 [ 454.091325] flags: 0x4000000000000000() [ 454.091329] raw: 4000000000000000 dead000000000100 dead000000000200 ffff88815ae4c6c0 [ 454.091331] raw: 0000000000000000 0000000000100010 00000000ffffffff 0000000000000000 [ 454.097378] FS: 0000000000000000(0000) GS:ffff88815b600000(0000) knlGS:0000000000000000 [ 454.101841] page dumped because: non-NULL mapping [ 454.101843] Modules linked in: cmac rfcomm uinput snd_soc_kbl_rt5663_max98927 snd_soc_skl_ssp_clk snd_soc_hdac_hdmi snd_soc_dmic btusb btrtl btbcm asix usbnet btintel bluetooth snd_soc_skl snd_soc_skl_ipc ecdh_generic snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_hda_core ipu3_imgu(C) ipu3_cio2 iova videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 videobuf2_common snd_soc_rt5663 snd_soc_max98927 at24 snd_soc_rl6231 ov13858 ov5670 v4l2_fwnode dw9714 bridge stp llc acpi_als kfifo_buf industrialio ipt_MASQUERADE lzo lzo_compress zram xt_mark fuse snd_seq_dummy snd_seq snd_seq_device cfg80211 ip6table_filter r8152 mii joydev [ 454.106153] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 454.110240] CPU: 1 PID: 10289 Comm: yavta Tainted: G B D WC 4.20.0-rc6-00031-g3b32400169db-dirty #37 [ 454.110242] Hardware name: HP Soraka/Soraka, BIOS Google_Soraka.10431.17.0 03/22/2018 [ 454.110243] Call Trace: [ 454.110250] dump_stack+0x6a/0xb1 [ 454.110265] bad_page+0x140/0x14a [ 454.113968] CR2: 00007a8ac018f000 CR3: 00000000744ba006 CR4: 00000000003606f0 [ 454.117574] free_pages_check+0x87/0x95 [ 454.117579] free_pcppages_bulk+0xbd/0x218 [ 454.117583] free_unref_page+0x49/0x6e [ 454.121691] Kernel panic - not syncing: Fatal exception in interrupt [ 454.125400] __free_pages+0x4a/0x71 [snip]
Hello Raj, On Saturday, 12 January 2019 04:30:49 EET Mani, Rajmohan wrote: [snip] > I finally managed to reproduce the issue with 4.20-rc6, with KASAN enabled > and with CONFIG_SLUB_DEBUG_ON with SLAB_STORE_USER. Nice ! Thank you for your work. > The following line indicates the crash happens when yavta PID 10289 tries to > free the memory. > > [ 452.437844] BUG: KASAN: use-after-free in ipu3_dmamap_free+0x50/0x9c > [ipu3_imgu] > [ 452.446123] Read of size 8 at addr ffff8881503481a0 by task yavta/10289 > > The above looks to be normal, since it's the same task that allocated this > memory. > [ 452.685731] Allocated by task 10289: > > Before the above happened, yavta/10187 came in and freed this memory per > KASAN. > [ 452.787656] Freed by task 10187: > > Is this (one instance of yavta freeing the memory allocated by another > instance of yavta) expected? Or does it indicate that mmap giving the same > address across these 2 instances of yavta? I need to debug / confirm the > latter case. KASAN prints the task name (and process ID) to help you debugging the problem, but this doesn't mean that yavta is freeing the memory. yavta exercises the V4L2 API exposed by the driver, and internally, down the call stack, ipu3_dmamap_free() is called by the driver. According to the backtraces you posted, this is in response to a VIDIOC_STREAMOFF call from yavta. I would expect VIDIOC_STREAMOFF to free DMA mappings created for the buffers on the corresponding video nodes, and thus allocated by the same task. The fact that memory is allocated in one task and freed in another seems weird to me in this case. My guess is that when using multiple instances of yavta the calls to VIDIOC_STREAMOFF on the different video nodes are asynchronous and happen in a way that the driver does not expect. Regardless of how the API is exercised by applications, in a good or bad way, the IPU3 driver must not crash. It needs to be prepared for all V4L2 ioctls to be called at any time, and an application could call VIDIOC_STREAMOFF on any video node while the IPU3 is busy processing images. > With the help of local application that operates these pipes in a serial > fashion, I do not see this issue. [snip]
Hi Raj, On Wed, Jan 16, 2019 at 11:16 AM Mani, Rajmohan <rajmohan.mani@intel.com> wrote: > > Hi Laurent, > > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > > Hello Raj, > > > > On Saturday, 12 January 2019 04:30:49 EET Mani, Rajmohan wrote: > > > > [snip] > > > > > I finally managed to reproduce the issue with 4.20-rc6, with KASAN > > > enabled and with CONFIG_SLUB_DEBUG_ON with SLAB_STORE_USER. > > > > Nice ! Thank you for your work. > > > > > The following line indicates the crash happens when yavta PID 10289 > > > tries to free the memory. > > > > > > [ 452.437844] BUG: KASAN: use-after-free in > > > ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.446123] Read of size 8 > > > at addr ffff8881503481a0 by task yavta/10289 > > > > > > The above looks to be normal, since it's the same task that allocated > > > this memory. > > > [ 452.685731] Allocated by task 10289: > > > > > > Before the above happened, yavta/10187 came in and freed this memory > > > per KASAN. > > > [ 452.787656] Freed by task 10187: > > > > > > Is this (one instance of yavta freeing the memory allocated by another > > > instance of yavta) expected? Or does it indicate that mmap giving the > > > same address across these 2 instances of yavta? I need to debug / > > > confirm the latter case. > > > > KASAN prints the task name (and process ID) to help you debugging the > > problem, but this doesn't mean that yavta is freeing the memory. yavta > > exercises the > > V4L2 API exposed by the driver, and internally, down the call stack, > > ipu3_dmamap_free() is called by the driver. According to the backtraces you > > posted, this is in response to a VIDIOC_STREAMOFF call from yavta. I would > > expect VIDIOC_STREAMOFF to free DMA mappings created for the buffers on > > the corresponding video nodes, and thus allocated by the same task. > > Ack. > > > The fact > > that memory is allocated in one task and freed in another seems weird to me > > in this case. > > > > I have instrumented the code around ipu3 dma map code, with a change to skip > dma free operations, if the current->pid is not the same as the pid that originally > did the dma alloc. > > There are no crashes in this case, as expected. > > I also confirmed that STREAM_ON/OFF is the one that results in this crash. > I need to spend more time on the alloc / free operations done by the yavta > Instances to see where the problem could be. > > This below line doesn't make sense, as the free call for pid 12986 occurs first, > before the alloc calls. Yavta application logs indicate the dma alloc has been > done for pid 12986, although I don't see corresponding dma alloc calls from pid 12986. I wonder if that doesn't mean that for some reason some V4L2 ioctls done from a context other than the owner (the one that first allocated vb2 buffers) end up triggering some buffer freeing/re-allocation. For VB2 buffers that's normally prevented by the core, but possibly we do some internal buffer management in non-buffer related V4L2 ioctls in the driver? > > [ 1604.194264] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > > [ 1603.804102] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530d000 @ VA 00000000a90fcad9 pid: 13281 comm: yavta > [ 1603.816015] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530c000 @ VA 00000000a2315b8c pid: 13281 comm: yavta > [ 1603.827932] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530b000 @ VA 0000000068fcc232 pid: 13281 comm: yavta > [ 1603.839818] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530a000 @ VA 00000000bd8c0fc7 pid: 13281 comm: yavta > [ 1603.851904] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e52c4000 @ VA 00000000b19ebc35 pid: 13281 comm: yavta > [ 1603.864093] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e527e000 @ VA 00000000d890dde9 pid: 13281 comm: yavta > [ 1603.876335] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e5238000 @ VA 0000000032cb057a pid: 13281 comm: yavta > [ 1603.888533] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e51f2000 @ VA 000000004fdbe7b7 pid: 13281 comm: yavta > [ 1603.900747] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e51b9000 @ VA 000000001f7481bb pid: 13281 comm: yavta > [ 1603.912924] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e5180000 @ VA 000000005488930b pid: 13281 comm: yavta > [ 1603.925079] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e5147000 @ VA 00000000a1ef0f70 pid: 13281 comm: yavta > [ 1603.937276] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e510e000 @ VA 000000008f127f52 pid: 13281 comm: yavta > [ 1603.949461] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e50c4000 @ VA 000000002f4ec9a5 pid: 13281 comm: yavta > [ 1603.961689] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e507a000 @ VA 0000000003233f40 pid: 13281 comm: yavta > [ 1603.973868] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e5030000 @ VA 0000000069e1621c pid: 13281 comm: yavta > [ 1603.986152] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e4fe6000 @ VA 00000000b39f1cf0 pid: 13281 comm: yavta > [ 1603.998265] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe5000 @ VA 000000002bd48bfe pid: 13281 comm: yavta > [ 1604.010163] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe4000 @ VA 00000000261436cd pid: 13281 comm: yavta > [ 1604.022056] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe3000 @ VA 00000000375b1a2a pid: 13281 comm: yavta > [ 1604.033989] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe2000 @ VA 00000000a10eb873 pid: 13281 comm: yavta > [ 1604.045873] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe1000 @ VA 00000000377717e8 pid: 13281 comm: yavta > [ 1604.057767] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe0000 @ VA 000000004274cd53 pid: 13281 comm: yavta > [ 1604.069648] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fdf000 @ VA 000000008442a829 pid: 13281 comm: yavta > [ 1604.081537] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fde000 @ VA 000000007bd91d8f pid: 13281 comm: yavta > [ 1604.093973] ipu3-imgu 0000:00:05.0: dma buf resized from 3112960 to 7372800 > [ 1604.101777] SKIPPING ipu3_dmamap_free map pid: 1453 this pid 13281... > [ 1604.112144] ipu3_dmamap_alloc: allocated 7372800 @ IOVA 0x00000000e48d6000 @ VA 000000008f3f13db pid: 13281 comm: yavta > [ 1604.187741] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.189093] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530d000 @ VA 00000000a90fcad9 pid: 13281 comm: yavta > [ 1604.194264] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194267] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194268] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194270] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194271] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194273] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194275] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194277] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194279] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194280] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194282] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194283] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194285] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194286] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194288] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194289] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194291] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194293] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194294] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194296] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194645] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194647] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194649] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194650] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194652] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194654] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194655] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194657] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > [ 1604.194659] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194661] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194663] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194665] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194667] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194669] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194671] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194672] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194674] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194676] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194678] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194680] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194681] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194683] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194685] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194686] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194688] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194690] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194691] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194693] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194695] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194696] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194698] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194700] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194702] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194703] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194705] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.194707] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > [ 1604.195044] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195046] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195048] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195049] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195051] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195053] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195054] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195056] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195058] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195060] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195062] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195063] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195065] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195066] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195067] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195069] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195070] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1604.195071] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1604.195072] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1604.195073] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1604.195075] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195077] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195078] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.195086] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.196725] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196727] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196728] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196730] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196731] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196732] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196733] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196734] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196735] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196736] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196738] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196739] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196740] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196741] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196742] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196744] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196745] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196746] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196747] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196748] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196749] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196751] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196752] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196753] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196754] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196755] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196756] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > [ 1604.196757] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196759] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196760] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196761] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196762] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196763] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196764] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196765] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196767] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196768] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196769] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196770] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196771] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196772] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196773] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196775] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196776] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196777] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196778] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196779] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196780] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196781] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196782] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196784] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196785] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196786] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196787] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.196788] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > [ 1604.206728] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530c000 @ VA 00000000a2315b8c pid: 13281 comm: yavta > [ 1604.217497] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.221514] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530b000 @ VA 0000000068fcc232 pid: 13281 comm: yavta > [ 1604.230381] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.236134] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530a000 @ VA 00000000bd8c0fc7 pid: 13281 comm: yavta > [ 1604.236171] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e52c4000 @ VA 00000000b19ebc35 pid: 13281 comm: yavta > [ 1604.236253] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e527e000 @ VA 00000000d890dde9 pid: 13281 comm: yavta > [ 1604.236336] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e5238000 @ VA 0000000032cb057a pid: 13281 comm: yavta > [ 1604.236421] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e51f2000 @ VA 000000004fdbe7b7 pid: 13281 comm: yavta > [ 1604.244291] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.251102] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e51b9000 @ VA 000000001f7481bb pid: 13281 comm: yavta > [ 1604.251190] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e5180000 @ VA 000000005488930b pid: 13281 comm: yavta > [ 1604.251279] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e5147000 @ VA 00000000a1ef0f70 pid: 13281 comm: yavta > [ 1604.251354] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e510e000 @ VA 000000008f127f52 pid: 13281 comm: yavta > [ 1604.251430] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e50c4000 @ VA 000000002f4ec9a5 pid: 13281 comm: yavta > [ 1604.251529] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e507a000 @ VA 0000000003233f40 pid: 13281 comm: yavta > [ 1604.251623] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e5030000 @ VA 0000000069e1621c pid: 13281 comm: yavta > [ 1604.251716] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e4fe6000 @ VA 00000000b39f1cf0 pid: 13281 comm: yavta > [ 1604.251821] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.259683] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > [ 1604.266371] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266374] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266376] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266381] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe5000 @ VA 000000002bd48bfe pid: 13281 comm: yavta > [ 1604.266438] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe4000 @ VA 00000000261436cd pid: 13281 comm: yavta > [ 1604.266477] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe3000 @ VA 00000000375b1a2a pid: 13281 comm: yavta > [ 1604.266514] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe2000 @ VA 00000000a10eb873 pid: 13281 comm: yavta > [ 1604.266550] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe1000 @ VA 00000000377717e8 pid: 13281 comm: yavta > [ 1604.266586] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe0000 @ VA 000000004274cd53 pid: 13281 comm: yavta > [ 1604.266623] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fdf000 @ VA 000000008442a829 pid: 13281 comm: yavta > [ 1604.266659] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fde000 @ VA 000000007bd91d8f pid: 13281 comm: yavta > [ 1604.266694] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266695] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266697] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266699] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266701] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266702] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266704] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266706] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266707] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266709] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266711] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266712] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266714] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266716] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266717] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266719] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266721] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266722] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266724] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266726] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266728] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266729] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266731] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266733] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266735] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266736] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266738] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.266740] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > [ 1604.296912] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > [ 1604.298946] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1604.366931] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > [ 1604.371182] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.722685] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.729641] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.737607] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.744549] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.751506] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.758465] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.765394] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.772447] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.779387] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.787021] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.801912] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.808976] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.816089] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.823311] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.830260] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.837218] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.844203] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.851192] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.858148] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.865073] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.872038] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.878971] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.885905] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.892876] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.899815] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1605.906829] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > [ 1606.013925] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > > > My guess is that when using multiple instances of yavta the calls to > > VIDIOC_STREAMOFF on the different video nodes are asynchronous and > > happen in a way that the driver does not expect. Regardless of how the API is > > exercised by applications, in a good or bad way, the IPU3 driver must not > > crash. It needs to be prepared for all V4L2 ioctls to be called at any time, and > > an application could call VIDIOC_STREAMOFF on any video node while the > > IPU3 is busy processing images. > > > > > With the help of local application that operates these pipes in a > > > serial fashion, I do not see this issue. > > > > [snip] > > > > -- > > Regards, > > > > Laurent Pinchart > > > > >
Hello Raj, On Mon, Jan 21, 2019 at 02:41:03PM +0900, Tomasz Figa wrote: > On Wed, Jan 16, 2019 at 11:16 AM Mani, Rajmohan <rajmohan.mani@intel.com> wrote: > >> Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > >> On Saturday, 12 January 2019 04:30:49 EET Mani, Rajmohan wrote: > >> > >> [snip] > >> > >>> I finally managed to reproduce the issue with 4.20-rc6, with KASAN > >>> enabled and with CONFIG_SLUB_DEBUG_ON with SLAB_STORE_USER. > >> > >> Nice ! Thank you for your work. > >> > >>> The following line indicates the crash happens when yavta PID 10289 > >>> tries to free the memory. > >>> > >>> [ 452.437844] BUG: KASAN: use-after-free in > >>> ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.446123] Read of size 8 > >>> at addr ffff8881503481a0 by task yavta/10289 > >>> > >>> The above looks to be normal, since it's the same task that allocated > >>> this memory. > >>> [ 452.685731] Allocated by task 10289: > >>> > >>> Before the above happened, yavta/10187 came in and freed this memory > >>> per KASAN. > >>> [ 452.787656] Freed by task 10187: > >>> > >>> Is this (one instance of yavta freeing the memory allocated by another > >>> instance of yavta) expected? Or does it indicate that mmap giving the > >>> same address across these 2 instances of yavta? I need to debug / > >>> confirm the latter case. > >> > >> KASAN prints the task name (and process ID) to help you debugging the > >> problem, but this doesn't mean that yavta is freeing the memory. yavta > >> exercises the V4L2 API exposed by the driver, and internally, down the > >> call stack, ipu3_dmamap_free() is called by the driver. According to the > >> backtraces you posted, this is in response to a VIDIOC_STREAMOFF call > >> from yavta. I would expect VIDIOC_STREAMOFF to free DMA mappings created > >> for the buffers on the corresponding video nodes, and thus allocated by > >> the same task. > > > > Ack. > > > >> The fact > >> that memory is allocated in one task and freed in another seems weird to me > >> in this case. > >> > > > > I have instrumented the code around ipu3 dma map code, with a change to skip > > dma free operations, if the current->pid is not the same as the pid that originally > > did the dma alloc. > > > > There are no crashes in this case, as expected. > > > > I also confirmed that STREAM_ON/OFF is the one that results in this crash. > > I need to spend more time on the alloc / free operations done by the yavta > > Instances to see where the problem could be. > > > > This below line doesn't make sense, as the free call for pid 12986 occurs first, > > before the alloc calls. Yavta application logs indicate the dma alloc has been > > done for pid 12986, although I don't see corresponding dma alloc calls from pid 12986. > > I wonder if that doesn't mean that for some reason some V4L2 ioctls > done from a context other than the owner (the one that first allocated > vb2 buffers) end up triggering some buffer freeing/re-allocation. For > VB2 buffers that's normally prevented by the core, but possibly we do > some internal buffer management in non-buffer related V4L2 ioctls in > the driver? I had a quick look at the driver, and found the following code in the VIDIOC_STREAMOFF handler ipu3_vb2_stop_streaming(): /* Was this the first node with streaming disabled? */ if (imgu->streaming && ipu3_all_nodes_streaming(imgu, node)) { /* Yes, really stop streaming now */ dev_dbg(dev, "IMGU streaming is ready to stop"); r = imgu_s_stream(imgu, false); if (!r) imgu->streaming = false; } The queue is initialized in ipu3_v4l2_node_setup() with vbq->lock = &node->lock; which means that concurrent VIDIOC_STREAMOFF operations on different nodes can race each other. Could you enable dynamic debugging to get the "IMGU streaming is ready to stop" message printed to the kernel log, and see if this could explain the double-free problem ? In any case this race condition should be handled by proper locking. Both the imgu->streaming and the ipu3_all_nodes_streaming() tests are very racy, and can lead to many different problems (failure at processing start also comes to mind). > > > > [ 1604.194264] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > > > > > [ 1603.804102] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530d000 @ VA 00000000a90fcad9 pid: 13281 comm: yavta > > [ 1603.816015] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530c000 @ VA 00000000a2315b8c pid: 13281 comm: yavta > > [ 1603.827932] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530b000 @ VA 0000000068fcc232 pid: 13281 comm: yavta > > [ 1603.839818] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e530a000 @ VA 00000000bd8c0fc7 pid: 13281 comm: yavta > > [ 1603.851904] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e52c4000 @ VA 00000000b19ebc35 pid: 13281 comm: yavta > > [ 1603.864093] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e527e000 @ VA 00000000d890dde9 pid: 13281 comm: yavta > > [ 1603.876335] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e5238000 @ VA 0000000032cb057a pid: 13281 comm: yavta > > [ 1603.888533] ipu3_dmamap_alloc: allocated 286720 @ IOVA 0x00000000e51f2000 @ VA 000000004fdbe7b7 pid: 13281 comm: yavta > > [ 1603.900747] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e51b9000 @ VA 000000001f7481bb pid: 13281 comm: yavta > > [ 1603.912924] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e5180000 @ VA 000000005488930b pid: 13281 comm: yavta > > [ 1603.925079] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e5147000 @ VA 00000000a1ef0f70 pid: 13281 comm: yavta > > [ 1603.937276] ipu3_dmamap_alloc: allocated 233472 @ IOVA 0x00000000e510e000 @ VA 000000008f127f52 pid: 13281 comm: yavta > > [ 1603.949461] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e50c4000 @ VA 000000002f4ec9a5 pid: 13281 comm: yavta > > [ 1603.961689] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e507a000 @ VA 0000000003233f40 pid: 13281 comm: yavta > > [ 1603.973868] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e5030000 @ VA 0000000069e1621c pid: 13281 comm: yavta > > [ 1603.986152] ipu3_dmamap_alloc: allocated 303104 @ IOVA 0x00000000e4fe6000 @ VA 00000000b39f1cf0 pid: 13281 comm: yavta > > [ 1603.998265] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe5000 @ VA 000000002bd48bfe pid: 13281 comm: yavta > > [ 1604.010163] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe4000 @ VA 00000000261436cd pid: 13281 comm: yavta > > [ 1604.022056] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe3000 @ VA 00000000375b1a2a pid: 13281 comm: yavta > > [ 1604.033989] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe2000 @ VA 00000000a10eb873 pid: 13281 comm: yavta > > [ 1604.045873] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe1000 @ VA 00000000377717e8 pid: 13281 comm: yavta > > [ 1604.057767] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fe0000 @ VA 000000004274cd53 pid: 13281 comm: yavta > > [ 1604.069648] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fdf000 @ VA 000000008442a829 pid: 13281 comm: yavta > > [ 1604.081537] ipu3_dmamap_alloc: allocated 4096 @ IOVA 0x00000000e4fde000 @ VA 000000007bd91d8f pid: 13281 comm: yavta > > [ 1604.093973] ipu3-imgu 0000:00:05.0: dma buf resized from 3112960 to 7372800 > > [ 1604.101777] SKIPPING ipu3_dmamap_free map pid: 1453 this pid 13281... > > [ 1604.112144] ipu3_dmamap_alloc: allocated 7372800 @ IOVA 0x00000000e48d6000 @ VA 000000008f3f13db pid: 13281 comm: yavta > > [ 1604.187741] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.189093] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530d000 @ VA 00000000a90fcad9 pid: 13281 comm: yavta > > [ 1604.194264] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194267] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194268] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194270] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194271] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194273] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194275] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194277] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194279] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194280] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194282] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194283] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194285] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194286] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194288] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194289] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194291] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194293] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194294] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194296] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194645] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194647] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194649] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194650] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194652] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194654] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194655] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194657] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 12986... > > [ 1604.194659] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194661] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194663] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194665] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194667] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194669] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194671] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194672] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194674] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194676] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194678] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194680] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194681] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194683] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194685] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194686] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194688] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194690] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194691] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194693] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194695] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194696] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194698] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194700] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194702] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194703] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194705] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.194707] SKIPPING ipu3_dmamap_free map pid: 0 this pid 12986... > > [ 1604.195044] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195046] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195048] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195049] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195051] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195053] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195054] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195056] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195058] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195060] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195062] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195063] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195065] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195066] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195067] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195069] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195070] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1604.195071] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1604.195072] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1604.195073] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1604.195075] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195077] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195078] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.195086] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.196725] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196727] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196728] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196730] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196731] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196732] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196733] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196734] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196735] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196736] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196738] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196739] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196740] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196741] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196742] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196744] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196745] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196746] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196747] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196748] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196749] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196751] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196752] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196753] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196754] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196755] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196756] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13182... > > [ 1604.196757] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196759] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196760] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196761] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196762] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196763] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196764] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196765] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196767] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196768] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196769] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196770] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196771] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196772] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196773] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196775] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196776] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196777] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196778] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196779] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196780] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196781] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196782] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196784] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196785] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196786] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196787] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.196788] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13182... > > [ 1604.206728] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530c000 @ VA 00000000a2315b8c pid: 13281 comm: yavta > > [ 1604.217497] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.221514] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530b000 @ VA 0000000068fcc232 pid: 13281 comm: yavta > > [ 1604.230381] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.236134] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e530a000 @ VA 00000000bd8c0fc7 pid: 13281 comm: yavta > > [ 1604.236171] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e52c4000 @ VA 00000000b19ebc35 pid: 13281 comm: yavta > > [ 1604.236253] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e527e000 @ VA 00000000d890dde9 pid: 13281 comm: yavta > > [ 1604.236336] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e5238000 @ VA 0000000032cb057a pid: 13281 comm: yavta > > [ 1604.236421] ipu3_dmamap_free: freeing 286720 @ IOVA 0x00000000e51f2000 @ VA 000000004fdbe7b7 pid: 13281 comm: yavta > > [ 1604.244291] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.251102] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e51b9000 @ VA 000000001f7481bb pid: 13281 comm: yavta > > [ 1604.251190] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e5180000 @ VA 000000005488930b pid: 13281 comm: yavta > > [ 1604.251279] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e5147000 @ VA 00000000a1ef0f70 pid: 13281 comm: yavta > > [ 1604.251354] ipu3_dmamap_free: freeing 233472 @ IOVA 0x00000000e510e000 @ VA 000000008f127f52 pid: 13281 comm: yavta > > [ 1604.251430] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e50c4000 @ VA 000000002f4ec9a5 pid: 13281 comm: yavta > > [ 1604.251529] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e507a000 @ VA 0000000003233f40 pid: 13281 comm: yavta > > [ 1604.251623] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e5030000 @ VA 0000000069e1621c pid: 13281 comm: yavta > > [ 1604.251716] ipu3_dmamap_free: freeing 303104 @ IOVA 0x00000000e4fe6000 @ VA 00000000b39f1cf0 pid: 13281 comm: yavta > > [ 1604.251821] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.259683] SKIPPING ipu3_dmamap_free map pid: 13281 this pid 13084... > > [ 1604.266371] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266374] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266376] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266381] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe5000 @ VA 000000002bd48bfe pid: 13281 comm: yavta > > [ 1604.266438] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe4000 @ VA 00000000261436cd pid: 13281 comm: yavta > > [ 1604.266477] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe3000 @ VA 00000000375b1a2a pid: 13281 comm: yavta > > [ 1604.266514] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe2000 @ VA 00000000a10eb873 pid: 13281 comm: yavta > > [ 1604.266550] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe1000 @ VA 00000000377717e8 pid: 13281 comm: yavta > > [ 1604.266586] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fe0000 @ VA 000000004274cd53 pid: 13281 comm: yavta > > [ 1604.266623] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fdf000 @ VA 000000008442a829 pid: 13281 comm: yavta > > [ 1604.266659] ipu3_dmamap_free: freeing 4096 @ IOVA 0x00000000e4fde000 @ VA 000000007bd91d8f pid: 13281 comm: yavta > > [ 1604.266694] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266695] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266697] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266699] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266701] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266702] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266704] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266706] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266707] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266709] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266711] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266712] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266714] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266716] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266717] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266719] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266721] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266722] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266724] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266726] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266728] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266729] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266731] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266733] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266735] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266736] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266738] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.266740] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13281... > > [ 1604.296912] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > [ 1604.298946] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1604.366931] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > [ 1604.371182] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.722685] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.729641] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.737607] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.744549] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.751506] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.758465] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.765394] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.772447] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.779387] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.787021] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.801912] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.808976] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.816089] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.823311] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.830260] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.837218] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.844203] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.851192] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.858148] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.865073] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.872038] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.878971] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.885905] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.892876] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.899815] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1605.906829] SKIPPING ipu3_dmamap_free map pid: 0 this pid 13084... > > [ 1606.013925] ipu3-imgu 0000:00:05.0: wait cio gate idle timeout > > > > > >> My guess is that when using multiple instances of yavta the calls to > >> VIDIOC_STREAMOFF on the different video nodes are asynchronous and > >> happen in a way that the driver does not expect. Regardless of how the API is > >> exercised by applications, in a good or bad way, the IPU3 driver must not > >> crash. It needs to be prepared for all V4L2 ioctls to be called at any time, and > >> an application could call VIDIOC_STREAMOFF on any video node while the > >> IPU3 is busy processing images. > >> > >>> With the help of local application that operates these pipes in a > >>> serial fashion, I do not see this issue. > >> > >> [snip]
Hi Tomasz, Laurent, > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > Hello Raj, > > On Mon, Jan 21, 2019 at 02:41:03PM +0900, Tomasz Figa wrote: > > On Wed, Jan 16, 2019 at 11:16 AM Mani, Rajmohan > <rajmohan.mani@intel.com> wrote: > > >> Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset On Saturday, > > >> 12 January 2019 04:30:49 EET Mani, Rajmohan wrote: > > >> > > >> [snip] > > >> > > >>> I finally managed to reproduce the issue with 4.20-rc6, with KASAN > > >>> enabled and with CONFIG_SLUB_DEBUG_ON with SLAB_STORE_USER. > > >> > > >> Nice ! Thank you for your work. > > >> > > >>> The following line indicates the crash happens when yavta PID > > >>> 10289 tries to free the memory. > > >>> > > >>> [ 452.437844] BUG: KASAN: use-after-free in > > >>> ipu3_dmamap_free+0x50/0x9c [ipu3_imgu] [ 452.446123] Read of size > > >>> 8 at addr ffff8881503481a0 by task yavta/10289 > > >>> > > >>> The above looks to be normal, since it's the same task that > > >>> allocated this memory. > > >>> [ 452.685731] Allocated by task 10289: > > >>> > > >>> Before the above happened, yavta/10187 came in and freed this > > >>> memory per KASAN. > > >>> [ 452.787656] Freed by task 10187: > > >>> > > >>> Is this (one instance of yavta freeing the memory allocated by > > >>> another instance of yavta) expected? Or does it indicate that mmap > > >>> giving the same address across these 2 instances of yavta? I need > > >>> to debug / confirm the latter case. > > >> > > >> KASAN prints the task name (and process ID) to help you debugging > > >> the problem, but this doesn't mean that yavta is freeing the > > >> memory. yavta exercises the V4L2 API exposed by the driver, and > > >> internally, down the call stack, ipu3_dmamap_free() is called by > > >> the driver. According to the backtraces you posted, this is in > > >> response to a VIDIOC_STREAMOFF call from yavta. I would expect > > >> VIDIOC_STREAMOFF to free DMA mappings created for the buffers on > > >> the corresponding video nodes, and thus allocated by the same task. > > > > > > Ack. > > > > > >> The fact > > >> that memory is allocated in one task and freed in another seems > > >> weird to me in this case. > > >> > > > > > > I have instrumented the code around ipu3 dma map code, with a change > > > to skip dma free operations, if the current->pid is not the same as > > > the pid that originally did the dma alloc. > > > > > > There are no crashes in this case, as expected. > > > > > > I also confirmed that STREAM_ON/OFF is the one that results in this crash. > > > I need to spend more time on the alloc / free operations done by the > > > yavta Instances to see where the problem could be. > > > > > > This below line doesn't make sense, as the free call for pid 12986 > > > occurs first, before the alloc calls. Yavta application logs > > > indicate the dma alloc has been done for pid 12986, although I don't see > corresponding dma alloc calls from pid 12986. > > > > I wonder if that doesn't mean that for some reason some V4L2 ioctls > > done from a context other than the owner (the one that first allocated > > vb2 buffers) end up triggering some buffer freeing/re-allocation. For > > VB2 buffers that's normally prevented by the core, but possibly we do > > some internal buffer management in non-buffer related V4L2 ioctls in > > the driver? > > I had a quick look at the driver, and found the following code in the > VIDIOC_STREAMOFF handler ipu3_vb2_stop_streaming(): > > /* Was this the first node with streaming disabled? */ > if (imgu->streaming && ipu3_all_nodes_streaming(imgu, node)) { > /* Yes, really stop streaming now */ > dev_dbg(dev, "IMGU streaming is ready to stop"); > r = imgu_s_stream(imgu, false); > if (!r) > imgu->streaming = false; > } > > The queue is initialized in ipu3_v4l2_node_setup() with > > vbq->lock = &node->lock; > > which means that concurrent VIDIOC_STREAMOFF operations on different > nodes can race each other. Could you enable dynamic debugging to get the > "IMGU streaming is ready to stop" message printed to the kernel log, and see if > this could explain the double-free problem ? > > In any case this race condition should be handled by proper locking. > Both the imgu->streaming and the ipu3_all_nodes_streaming() tests are very > racy, and can lead to many different problems (failure at processing start also > comes to mind). > Thanks for your inputs. I have confirmed so far that ipu3_vb2_stop_streaming() is called for all 4 instances of yavta exit, while ipu3_vb2_start_streaming() is called for just one instance. This causes 4 free operations for a single alloc, resulting in this failure. ipu3_vb2_stop_streaming() should be done just once, just like ipu3_vb2_start_streaming(). ipu3_vb2_stop_streaming() should also be improved a bit to handle multiple applications handling all the nodes. Laurent's findings point to the same. Let me get back soon with more details. [snip]
Hi Sakari, everyone.. On Wed, Jan 02, 2019 at 10:26:56PM +0200, Sakari Ailus wrote: > Hi Laurent, > > On Wed, Jan 02, 2019 at 10:20:13AM +0200, Laurent Pinchart wrote: > > Hello Bingbu, > > > > On Wednesday, 2 January 2019 04:38:33 EET Bingbu Cao wrote: > > > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > > > >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > > > >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > > > >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > > > >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > >>>>>> > > > >>>>>> [snip] > > > >>>>>> > > > >>>>>>> I can see a couple of steps missing in the script below. > > > >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November > > > >>>>>>> /000040.html) > > > >>>>>>> > > > >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for > > > >>>>>>> image processing"... > > > >>>>>>> > > > >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > >>>>>>> > > > >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the > > > >>>>>>> control id 0x009819a1 as below. > > > >>>>>>> > > > >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > >>>>>> > > > >>>>>> I assume the control takes a valid default value ? It's better to set > > > >>>>>> it explicitly anyway, so I'll do so. > > > >>>> > > > >>>> The video mode is set by default. If you want to set to still mode or > > > >>>> change mode, you need set the subdev control. > > > >>>> > > > >>>>>>> 2. ImgU pipeline needs to be configured for image processing as > > > >>>>>>> below. > > > >>>>>>> > > > >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > >>>>>>> have the processed image output to the DDR memory. > > > >>>>>>> > > > >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > >>>>>>> Geometric Distortion Correction (GDC) -> DDR > > > >>>>>>> > > > >>>>>>> The ImgU V4L2 subdev has to be configured with the supported > > > >>>>>>> resolutions in all the above HW blocks, for a given input > > > >>>>>>> resolution. > > > >>>>>>> > > > >>>>>>> For a given supported resolution for an input frame, the Input > > > >>>>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured with > > > >>>>>>> the supported resolutions. This information can be obtained by > > > >>>>>>> looking at the following IPU3 ISP configuration table for ov5670 > > > >>>>>>> sensor. > > > >>>>>>> > > > >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays > > > >>>>>>> /+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > > > >>>>>>> files/gcss/graph_settings_ov5670.xml > > > >>>>>>> > > > >>>>>>> For the ov5670 example, for an input frame with a resolution of > > > >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > > > >>>>>>> corresponding resolutions for input feeder, BDS and GDC are > > > >>>>>>> 2592x1944, 2592x1944 and 2560x1920 respectively. > > > >>>>>> > > > >>>>>> How is the GDC output resolution computed from the input resolution ? > > > >>>>>> Does the GDC always consume 32 columns and 22 lines ? > > > >>>> > > > >>>> All the intermediate resolutions in the pipeline are determined by the > > > >>>> actual use case, in other word determined by the IMGU input > > > >>>> resolution(sensor output) and the final output and viewfinder > > > >>>> resolution. BDS mainly do Bayer downscaling, it has limitation that the > > > >>>> downscaling factor must be a value a integer multiple of 1/32. > > > >>>> GDC output depends on the input and width should be x8 and height x4 > > > >>>> alignment. > > > >>> > > > >>> Thank you for the information. This will need to be captured in the > > > >>> documentation, along with information related to how each block in the > > > >>> hardware pipeline interacts with the image size. It should be possible > > > >>> for a developer to compute the output and viewfinder resolutions based > > > >>> on the parameters of the image processing algorithms just with the > > > >>> information contained in the driver documentation. > > > >>> > > > >>>>>>> The following steps prepare the ImgU ISP pipeline for the image > > > >>>>>>> processing. > > > >>>>>>> > > > >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the > > > >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > > > >>>>>>> obtained > > > >>>>>>> above. > > > >>>>>> > > > >>>>>> If I understand things correctly, the GDC resolution is the pipeline > > > >>>>>> output resolution. Why is it configured on pad 0 ? > > > >>>> > > > >>>> We see the GDC output resolution as the input of output system, the > > > >>>> sink pad format is used for output and viewfinder resolutions. > > > >>> > > > >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > > > >>> the ImgU input, the format configured there should correspond to the > > > >>> format on the connected video node, and should thus be the sensor > > > >>> format. You can then use the crop and compose rectangles on pad 0, along > > > >>> with the format, crop and compose rectangles on the output and > > > >>> viewfinder pads, to configure the device. This should be fixed in the > > > >>> driver, and the documentation should then be updated accordingly. > > > >> > > > >> Hi, Laurent, > > > >> > > > >> Thanks for your review. > > > >> > > > >> I think it make sense for me that using Pad 0 as the ImgU input(IF). > > > >> However, I prefer using the 2 source pads for output and viewfinder. > > > >> It makes more sense because the output and viewfinder are independent > > > >> output. > > > >> > > > >> The whole pipeline in ImgU looks like: > > > >> IF --> BDS --> GDC ---> OUTPUT > > > >> |-----> VF > > > >> > > > >> The BDS is used to do Bayer downscaling and GDC can do cropping. > > > > > > > > Does this mean that the main output and the viewfinder output share the > > > > same scaler, and that the only difference in size between the two outputs > > > > is solely due to cropping ? > > > > > > Laurent, > > > No, output only can do crop and viewfinder support crop and scaling, they > > > share same input. > > > > Then you can't support this with a single subdev for the ImgU, you need at > > least two subdevs. I can offer more guidance, but I'll need more information > > about the GDC. > > While the current documentation only defines the functionality of the > compose target for sink pads, there are a few sensor drivers supporting it > on source pads already. Some drivers such as the OMAP3 ISP also use the > format on source pads to configure scaling. > > The current API certainly allows exposing the compose rectangle also on the > source pads, but to make that generic we'd need to amend the API to tell in > which order these steps take place. In the meantime the behaviour remains > device specific. > My understanding is that what is currently missing is the support for viewfinder's ability to scale, as the scaler should get programmed by configuring a composing rectangle on a source pad which is not supported by the V4L2 APIs at the moment. Is my understanding correct? As the composing rectangle is set for both 'output' and 'viewfinder' through the image format sizes configured on the first sink pad (*), the viewfinder output is obtained by cropping-only to the image format sizes configured on source pad number 3 (though SUBDEV_S_FMT not through SUBDEV_S_SELECTION, as SUBDEV_S_SELECTION is only allowed on sink pad 0 in the driver: see "ipu3_subdev_set_selection()"). As you mentioned "device specific behaviour", what is the intended one for the ipu3? I assumed the viewfinder scaling/cropping was configured on the 'viewfinder' video device node, through the VIDIOC_S_SELECTION ioctl, but looking at the code, that doesn't seem to be listed as supported in "ipu3_v4l2_ioctl_ops". How am I supposed to configure scaling on the viewfinder output? Would adding support for crop/compose to the 'output' and 'viewfinder' video devices be supported by the V4L2 APIs? That would work with the single subdevice model that is currently implemented in this patches... Thanks j (*) and this should also be changed, as the image format sizes should reflect what the imgu receives as input, and the crop() and compose() rectangles should instead be used to configure BDS and GDC respectively. > > > > > >> My understanding is that scaled size is configured on the CROP rectangle > > > >> by COMPOSE selection target, the order seems like not aligned with the > > > >> actual processing in ImgU if we set the crop/compose on sink pad. > > > >> > > > >> Is there some rules for the order of the configuration in the subdev API? > > > >> Could I use crop selection based on the scaled size? > > > > > > > > Please see figure 4.6 in > > > > https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html. > > > > Scaling is configured on the sink pad through the crop and compose > > > > rectangles, while the source crop rectangle is used to perform cropping > > > > on the output. If you have a single scaler in the hardware pipeline you > > > > can thus configure it on the sink pad, with output and viewfinder > > > > separate cropping configure on the source pad. > > > > > > > >>>>>>> 2. The ImgU V4L2 subdev cropping should be set by using the > > > >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the > > > >>>>>>> target, using the input feeder height and width. > > > >>>>>>> > > > >>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the > > > >>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > > >>>>>>> target, using the BDS height and width. > > > >>>>>>> > > > >>>>>>> Once these 2 steps are done, the raw bayer frames can be input to > > > >>>>>>> the ImgU V4L2 subdev for processing. > > > >>>>>> > > > >>>>>> Do I need to capture from both the output and viewfinder nodes ? How > > > >>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > > > >>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in > > > >>>>>> that picture ? > > > >>>> > > > >>>> The output capture should be set, the viewfinder can be disabled. > > > >>>> The IF and BDS are seen as crop and compose of the imgu input video > > > >>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > > > >>>> pads. > > > >>> > > > >>> The GDC is the last block in the pipeline according to the information > > > >>> provided above. How can it be seen as the subdev sink pad ? That doesn't > > > >>> make sense to me. I'm not asking for the MC graph to expose all internal > > > >>> blocks of the ImgU, but if you want to retain a single subdev model, the > > > >>> format on the sink pad needs to correspond to what is provided to the > > > >>> ImgU. Please see figure 4.6 of > > > >>> https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/dev-subdev.html for > > > >>> more information regarding how you can use the sink crop, sink compose > > > >>> and source crop rectangles. > > > > > > > > [snip] > > > > -- > > Regards, > > > > Laurent Pinchart > > > > > > > > -- > Regards, > > Sakari Ailus > sakari.ailus@linux.intel.com
On Mon, Jan 28, 2019 at 7:08 PM Jacopo Mondi <jacopo@jmondi.org> wrote: > > Hi Sakari, everyone.. > > On Wed, Jan 02, 2019 at 10:26:56PM +0200, Sakari Ailus wrote: > > Hi Laurent, > > > > On Wed, Jan 02, 2019 at 10:20:13AM +0200, Laurent Pinchart wrote: > > > Hello Bingbu, > > > > > > On Wednesday, 2 January 2019 04:38:33 EET Bingbu Cao wrote: > > > > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > > > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > > > > >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > > > > >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > > > > >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > > > > >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > > >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > > >>>>>> > > > > >>>>>> [snip] > > > > >>>>>> > > > > >>>>>>> I can see a couple of steps missing in the script below. > > > > >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November > > > > >>>>>>> /000040.html) > > > > >>>>>>> > > > > >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > > >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for > > > > >>>>>>> image processing"... > > > > >>>>>>> > > > > >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > >>>>>>> > > > > >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > > >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the > > > > >>>>>>> control id 0x009819a1 as below. > > > > >>>>>>> > > > > >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > > >>>>>> > > > > >>>>>> I assume the control takes a valid default value ? It's better to set > > > > >>>>>> it explicitly anyway, so I'll do so. > > > > >>>> > > > > >>>> The video mode is set by default. If you want to set to still mode or > > > > >>>> change mode, you need set the subdev control. > > > > >>>> > > > > >>>>>>> 2. ImgU pipeline needs to be configured for image processing as > > > > >>>>>>> below. > > > > >>>>>>> > > > > >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > > >>>>>>> have the processed image output to the DDR memory. > > > > >>>>>>> > > > > >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > > >>>>>>> Geometric Distortion Correction (GDC) -> DDR > > > > >>>>>>> > > > > >>>>>>> The ImgU V4L2 subdev has to be configured with the supported > > > > >>>>>>> resolutions in all the above HW blocks, for a given input > > > > >>>>>>> resolution. > > > > >>>>>>> > > > > >>>>>>> For a given supported resolution for an input frame, the Input > > > > >>>>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured with > > > > >>>>>>> the supported resolutions. This information can be obtained by > > > > >>>>>>> looking at the following IPU3 ISP configuration table for ov5670 > > > > >>>>>>> sensor. > > > > >>>>>>> > > > > >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays > > > > >>>>>>> /+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > > > > >>>>>>> files/gcss/graph_settings_ov5670.xml > > > > >>>>>>> > > > > >>>>>>> For the ov5670 example, for an input frame with a resolution of > > > > >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > > > > >>>>>>> corresponding resolutions for input feeder, BDS and GDC are > > > > >>>>>>> 2592x1944, 2592x1944 and 2560x1920 respectively. > > > > >>>>>> > > > > >>>>>> How is the GDC output resolution computed from the input resolution ? > > > > >>>>>> Does the GDC always consume 32 columns and 22 lines ? > > > > >>>> > > > > >>>> All the intermediate resolutions in the pipeline are determined by the > > > > >>>> actual use case, in other word determined by the IMGU input > > > > >>>> resolution(sensor output) and the final output and viewfinder > > > > >>>> resolution. BDS mainly do Bayer downscaling, it has limitation that the > > > > >>>> downscaling factor must be a value a integer multiple of 1/32. > > > > >>>> GDC output depends on the input and width should be x8 and height x4 > > > > >>>> alignment. > > > > >>> > > > > >>> Thank you for the information. This will need to be captured in the > > > > >>> documentation, along with information related to how each block in the > > > > >>> hardware pipeline interacts with the image size. It should be possible > > > > >>> for a developer to compute the output and viewfinder resolutions based > > > > >>> on the parameters of the image processing algorithms just with the > > > > >>> information contained in the driver documentation. > > > > >>> > > > > >>>>>>> The following steps prepare the ImgU ISP pipeline for the image > > > > >>>>>>> processing. > > > > >>>>>>> > > > > >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the > > > > >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > > > > >>>>>>> obtained > > > > >>>>>>> above. > > > > >>>>>> > > > > >>>>>> If I understand things correctly, the GDC resolution is the pipeline > > > > >>>>>> output resolution. Why is it configured on pad 0 ? > > > > >>>> > > > > >>>> We see the GDC output resolution as the input of output system, the > > > > >>>> sink pad format is used for output and viewfinder resolutions. > > > > >>> > > > > >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > > > > >>> the ImgU input, the format configured there should correspond to the > > > > >>> format on the connected video node, and should thus be the sensor > > > > >>> format. You can then use the crop and compose rectangles on pad 0, along > > > > >>> with the format, crop and compose rectangles on the output and > > > > >>> viewfinder pads, to configure the device. This should be fixed in the > > > > >>> driver, and the documentation should then be updated accordingly. > > > > >> > > > > >> Hi, Laurent, > > > > >> > > > > >> Thanks for your review. > > > > >> > > > > >> I think it make sense for me that using Pad 0 as the ImgU input(IF). > > > > >> However, I prefer using the 2 source pads for output and viewfinder. > > > > >> It makes more sense because the output and viewfinder are independent > > > > >> output. > > > > >> > > > > >> The whole pipeline in ImgU looks like: > > > > >> IF --> BDS --> GDC ---> OUTPUT > > > > >> |-----> VF > > > > >> > > > > >> The BDS is used to do Bayer downscaling and GDC can do cropping. > > > > > > > > > > Does this mean that the main output and the viewfinder output share the > > > > > same scaler, and that the only difference in size between the two outputs > > > > > is solely due to cropping ? > > > > > > > > Laurent, > > > > No, output only can do crop and viewfinder support crop and scaling, they > > > > share same input. > > > > > > Then you can't support this with a single subdev for the ImgU, you need at > > > least two subdevs. I can offer more guidance, but I'll need more information > > > about the GDC. > > > > While the current documentation only defines the functionality of the > > compose target for sink pads, there are a few sensor drivers supporting it > > on source pads already. Some drivers such as the OMAP3 ISP also use the > > format on source pads to configure scaling. > > > > The current API certainly allows exposing the compose rectangle also on the > > source pads, but to make that generic we'd need to amend the API to tell in > > which order these steps take place. In the meantime the behaviour remains > > device specific. > > > > My understanding is that what is currently missing is the support > for viewfinder's ability to scale, as the scaler should get > programmed by configuring a composing rectangle on a source pad which > is not supported by the V4L2 APIs at the moment. Is my understanding correct? > > As the composing rectangle is set for both 'output' and 'viewfinder' > through the image format sizes configured on the first sink pad (*), > the viewfinder output is obtained by cropping-only to the image format > sizes configured on source pad number 3 (though SUBDEV_S_FMT not through > SUBDEV_S_SELECTION, as SUBDEV_S_SELECTION is only allowed on sink pad > 0 in the driver: see "ipu3_subdev_set_selection()"). > > As you mentioned "device specific behaviour", what is the intended one > for the ipu3? I assumed the viewfinder scaling/cropping was configured > on the 'viewfinder' video device node, through the VIDIOC_S_SELECTION > ioctl, but looking at the code, that doesn't seem to be listed as > supported in "ipu3_v4l2_ioctl_ops". > > How am I supposed to configure scaling on the viewfinder output? Would > adding support for crop/compose to the 'output' and 'viewfinder' video > devices be supported by the V4L2 APIs? That would work with the single > subdevice model that is currently implemented in this patches... Isn't this what the driver actually implements? My understanding was that the format on the VF video node determined the scaling settings, based on the source pad. That was my understanding on how we should make things work, i.e. try to put as much as possible into core V4L2 interfaces and only defer to subdevices or media controller if that's impossible. Laurent didn't seem to agree with that when we talked last time, though, and I can see some reasons why actually moving simplifying the video devices as much as possible and moving as much as possible into subdevices could make sense (simpler interfaces of endpoints, finer granularity for feature description, less redundance - only subdevices do processing, video nodes are only DMAs, etc.). Best regards, Tomasz
Hi Tomasz, On Tue, Jan 29, 2019 at 05:56:35PM +0900, Tomasz Figa wrote: > On Mon, Jan 28, 2019 at 7:08 PM Jacopo Mondi <jacopo@jmondi.org> wrote: > > > > Hi Sakari, everyone.. > > > > On Wed, Jan 02, 2019 at 10:26:56PM +0200, Sakari Ailus wrote: > > > Hi Laurent, > > > > > > On Wed, Jan 02, 2019 at 10:20:13AM +0200, Laurent Pinchart wrote: > > > > Hello Bingbu, > > > > > > > > On Wednesday, 2 January 2019 04:38:33 EET Bingbu Cao wrote: > > > > > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > > > > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > > > > > >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > > > > > >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > > > > > >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > > > > > >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > > > >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > > > >>>>>> > > > > > >>>>>> [snip] > > > > > >>>>>> > > > > > >>>>>>> I can see a couple of steps missing in the script below. > > > > > >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November > > > > > >>>>>>> /000040.html) > > > > > >>>>>>> > > > > > >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > > > >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for > > > > > >>>>>>> image processing"... > > > > > >>>>>>> > > > > > >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > > >>>>>>> > > > > > >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > > > >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the > > > > > >>>>>>> control id 0x009819a1 as below. > > > > > >>>>>>> > > > > > >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > > > >>>>>> > > > > > >>>>>> I assume the control takes a valid default value ? It's better to set > > > > > >>>>>> it explicitly anyway, so I'll do so. > > > > > >>>> > > > > > >>>> The video mode is set by default. If you want to set to still mode or > > > > > >>>> change mode, you need set the subdev control. > > > > > >>>> > > > > > >>>>>>> 2. ImgU pipeline needs to be configured for image processing as > > > > > >>>>>>> below. > > > > > >>>>>>> > > > > > >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > > > >>>>>>> have the processed image output to the DDR memory. > > > > > >>>>>>> > > > > > >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > > > >>>>>>> Geometric Distortion Correction (GDC) -> DDR > > > > > >>>>>>> > > > > > >>>>>>> The ImgU V4L2 subdev has to be configured with the supported > > > > > >>>>>>> resolutions in all the above HW blocks, for a given input > > > > > >>>>>>> resolution. > > > > > >>>>>>> > > > > > >>>>>>> For a given supported resolution for an input frame, the Input > > > > > >>>>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured with > > > > > >>>>>>> the supported resolutions. This information can be obtained by > > > > > >>>>>>> looking at the following IPU3 ISP configuration table for ov5670 > > > > > >>>>>>> sensor. > > > > > >>>>>>> > > > > > >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays > > > > > >>>>>>> /+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > > > > > >>>>>>> files/gcss/graph_settings_ov5670.xml > > > > > >>>>>>> > > > > > >>>>>>> For the ov5670 example, for an input frame with a resolution of > > > > > >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > > > > > >>>>>>> corresponding resolutions for input feeder, BDS and GDC are > > > > > >>>>>>> 2592x1944, 2592x1944 and 2560x1920 respectively. > > > > > >>>>>> > > > > > >>>>>> How is the GDC output resolution computed from the input resolution ? > > > > > >>>>>> Does the GDC always consume 32 columns and 22 lines ? > > > > > >>>> > > > > > >>>> All the intermediate resolutions in the pipeline are determined by the > > > > > >>>> actual use case, in other word determined by the IMGU input > > > > > >>>> resolution(sensor output) and the final output and viewfinder > > > > > >>>> resolution. BDS mainly do Bayer downscaling, it has limitation that the > > > > > >>>> downscaling factor must be a value a integer multiple of 1/32. > > > > > >>>> GDC output depends on the input and width should be x8 and height x4 > > > > > >>>> alignment. > > > > > >>> > > > > > >>> Thank you for the information. This will need to be captured in the > > > > > >>> documentation, along with information related to how each block in the > > > > > >>> hardware pipeline interacts with the image size. It should be possible > > > > > >>> for a developer to compute the output and viewfinder resolutions based > > > > > >>> on the parameters of the image processing algorithms just with the > > > > > >>> information contained in the driver documentation. > > > > > >>> > > > > > >>>>>>> The following steps prepare the ImgU ISP pipeline for the image > > > > > >>>>>>> processing. > > > > > >>>>>>> > > > > > >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the > > > > > >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > > > > > >>>>>>> obtained > > > > > >>>>>>> above. > > > > > >>>>>> > > > > > >>>>>> If I understand things correctly, the GDC resolution is the pipeline > > > > > >>>>>> output resolution. Why is it configured on pad 0 ? > > > > > >>>> > > > > > >>>> We see the GDC output resolution as the input of output system, the > > > > > >>>> sink pad format is used for output and viewfinder resolutions. > > > > > >>> > > > > > >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > > > > > >>> the ImgU input, the format configured there should correspond to the > > > > > >>> format on the connected video node, and should thus be the sensor > > > > > >>> format. You can then use the crop and compose rectangles on pad 0, along > > > > > >>> with the format, crop and compose rectangles on the output and > > > > > >>> viewfinder pads, to configure the device. This should be fixed in the > > > > > >>> driver, and the documentation should then be updated accordingly. > > > > > >> > > > > > >> Hi, Laurent, > > > > > >> > > > > > >> Thanks for your review. > > > > > >> > > > > > >> I think it make sense for me that using Pad 0 as the ImgU input(IF). > > > > > >> However, I prefer using the 2 source pads for output and viewfinder. > > > > > >> It makes more sense because the output and viewfinder are independent > > > > > >> output. > > > > > >> > > > > > >> The whole pipeline in ImgU looks like: > > > > > >> IF --> BDS --> GDC ---> OUTPUT > > > > > >> |-----> VF > > > > > >> > > > > > >> The BDS is used to do Bayer downscaling and GDC can do cropping. > > > > > > > > > > > > Does this mean that the main output and the viewfinder output share the > > > > > > same scaler, and that the only difference in size between the two outputs > > > > > > is solely due to cropping ? > > > > > > > > > > Laurent, > > > > > No, output only can do crop and viewfinder support crop and scaling, they > > > > > share same input. > > > > > > > > Then you can't support this with a single subdev for the ImgU, you need at > > > > least two subdevs. I can offer more guidance, but I'll need more information > > > > about the GDC. > > > > > > While the current documentation only defines the functionality of the > > > compose target for sink pads, there are a few sensor drivers supporting it > > > on source pads already. Some drivers such as the OMAP3 ISP also use the > > > format on source pads to configure scaling. > > > > > > The current API certainly allows exposing the compose rectangle also on the > > > source pads, but to make that generic we'd need to amend the API to tell in > > > which order these steps take place. In the meantime the behaviour remains > > > device specific. > > > > > > > My understanding is that what is currently missing is the support > > for viewfinder's ability to scale, as the scaler should get > > programmed by configuring a composing rectangle on a source pad which > > is not supported by the V4L2 APIs at the moment. Is my understanding correct? > > > > As the composing rectangle is set for both 'output' and 'viewfinder' > > through the image format sizes configured on the first sink pad (*), > > the viewfinder output is obtained by cropping-only to the image format > > sizes configured on source pad number 3 (though SUBDEV_S_FMT not through > > SUBDEV_S_SELECTION, as SUBDEV_S_SELECTION is only allowed on sink pad > > 0 in the driver: see "ipu3_subdev_set_selection()"). > > > > As you mentioned "device specific behaviour", what is the intended one > > for the ipu3? I assumed the viewfinder scaling/cropping was configured > > on the 'viewfinder' video device node, through the VIDIOC_S_SELECTION > > ioctl, but looking at the code, that doesn't seem to be listed as > > supported in "ipu3_v4l2_ioctl_ops". > > > > How am I supposed to configure scaling on the viewfinder output? Would > > adding support for crop/compose to the 'output' and 'viewfinder' video > > devices be supported by the V4L2 APIs? That would work with the single > > subdevice model that is currently implemented in this patches... > > Isn't this what the driver actually implements? My understanding was > that the format on the VF video node determined the scaling settings, > based on the source pad. > I might surely be wrong, but VIDIOC_S_SELECTION seems not to be supported in the "ipu3_v4l2_ioctl_ops" list. So we can just confiure cropping sizes on the source pad number 3 of the 'imgu' entity, and set the same sizes on the video device node that represents the VF output. Inspecting the images, and reading again the long mail thread, it seems to me that the imgu perform composing internally, based on the GDC output dimensions and the required VF ones, to maintain the field of view. I cannot find any link to post here to Bing Bu's reply to Sakari on this, but seems like this discussion happened already, and my understanding is that composing on VF cannot be set from user, but calculated by the imgu internally. Sakari, Bing Bu, is that the current state? Thanks j ------------------------------------------------------------------------------ Pasting down here the relevan bits from Bing Bu's and Sakari's exchange from "Wed, 14 Nov 2018 15:02:37 +0800" which I cannot find a suitable link to to paste here: >>>>>>>> pad3: Source >>>>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] >>>>>>>> -> "ipu3-imgu 0 viewfinder":0 [] >>>>>>> Are there other differences between output and viewfinder? >>>>>> output and viewfinder are the main and secondary output of output system. >>>>>> 'main' output is not allowed to be scaled, only support crop. secondary >>>>>> output 'viewfinder' >>>>>> can support both cropping and scaling. User can select different nodes >>>>>> to use >>>>>> as preview and capture flexibly based on the actual use cases. >>>>> If there's scaling to be configured, I'd expect to see the COMPOSE target >>>>> supported. >>>> Actually the viewfinder is the result of scaling, that means you can not >>>> do more scaling. >>> How do you configure the scaling of the viewfinder currently? >> We consider that the viewfinder as a secondary output, and set the format by >> subdev set_fmt() directly and all pads formats will be used to find >> binary and >> build pipeline. > Ok. > > Could you instead use the compose target to configure the scaling? Setting > the format on the source pad would have no effect. Hi, Sakari, For the secondary output (viewfinder), it support both cropping and scaling, in order to keep the aspect ratio, system will do [crop --> compose], sounds like it should have 2 selection targets, but firmware/driver did not provide clear interfaces to allow user to configure the cropping, driver calculate the scaler and cropping parameters based on the output-system input and output resolution to keep aspect ratio and field of view. I think it is more succinct to set the actual output by set format instead of using selection targets. ------------------------------------------------------------------------------
Hi Jacopo, On Fri, Feb 1, 2019 at 7:04 PM Jacopo Mondi <jacopo@jmondi.org> wrote: > > Hi Tomasz, > > On Tue, Jan 29, 2019 at 05:56:35PM +0900, Tomasz Figa wrote: > > On Mon, Jan 28, 2019 at 7:08 PM Jacopo Mondi <jacopo@jmondi.org> wrote: > > > > > > Hi Sakari, everyone.. > > > > > > On Wed, Jan 02, 2019 at 10:26:56PM +0200, Sakari Ailus wrote: > > > > Hi Laurent, > > > > > > > > On Wed, Jan 02, 2019 at 10:20:13AM +0200, Laurent Pinchart wrote: > > > > > Hello Bingbu, > > > > > > > > > > On Wednesday, 2 January 2019 04:38:33 EET Bingbu Cao wrote: > > > > > > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > > > > > > On Monday, 17 December 2018 05:14:44 EET Bingbu Cao wrote: > > > > > > >> On 12/14/2018 06:24 AM, Laurent Pinchart wrote: > > > > > > >>> On Wednesday, 12 December 2018 06:55:53 EET Bingbu Cao wrote: > > > > > > >>>> On 12/11/2018 09:43 PM, Laurent Pinchart wrote: > > > > > > >>>>> On Tuesday, 11 December 2018 15:34:49 EET Laurent Pinchart wrote: > > > > > > >>>>>> On Wednesday, 5 December 2018 02:30:46 EET Mani, Rajmohan wrote: > > > > > > >>>>>> > > > > > > >>>>>> [snip] > > > > > > >>>>>> > > > > > > >>>>>>> I can see a couple of steps missing in the script below. > > > > > > >>>>>>> (https://lists.libcamera.org/pipermail/libcamera-devel/2018-November > > > > > > >>>>>>> /000040.html) > > > > > > >>>>>>> > > > > > > >>>>>>> From patch 02 of this v7 series "doc-rst: Add Intel IPU3 > > > > > > >>>>>>> documentation", under section "Configuring ImgU V4L2 subdev for > > > > > > >>>>>>> image processing"... > > > > > > >>>>>>> > > > > > > >>>>>>> 1. The pipe mode needs to be configured for the V4L2 subdev. > > > > > > >>>>>>> > > > > > > >>>>>>> Also the pipe mode of the corresponding V4L2 subdev should be set as > > > > > > >>>>>>> desired (e.g 0 for video mode or 1 for still mode) through the > > > > > > >>>>>>> control id 0x009819a1 as below. > > > > > > >>>>>>> > > > > > > >>>>>>> e.g v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 > > > > > > >>>>>> > > > > > > >>>>>> I assume the control takes a valid default value ? It's better to set > > > > > > >>>>>> it explicitly anyway, so I'll do so. > > > > > > >>>> > > > > > > >>>> The video mode is set by default. If you want to set to still mode or > > > > > > >>>> change mode, you need set the subdev control. > > > > > > >>>> > > > > > > >>>>>>> 2. ImgU pipeline needs to be configured for image processing as > > > > > > >>>>>>> below. > > > > > > >>>>>>> > > > > > > >>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > > > > > > >>>>>>> have the processed image output to the DDR memory. > > > > > > >>>>>>> > > > > > > >>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > > > > >>>>>>> Geometric Distortion Correction (GDC) -> DDR > > > > > > >>>>>>> > > > > > > >>>>>>> The ImgU V4L2 subdev has to be configured with the supported > > > > > > >>>>>>> resolutions in all the above HW blocks, for a given input > > > > > > >>>>>>> resolution. > > > > > > >>>>>>> > > > > > > >>>>>>> For a given supported resolution for an input frame, the Input > > > > > > >>>>>>> Feeder, Bayer Down Scaling and GDC blocks should be configured with > > > > > > >>>>>>> the supported resolutions. This information can be obtained by > > > > > > >>>>>>> looking at the following IPU3 ISP configuration table for ov5670 > > > > > > >>>>>>> sensor. > > > > > > >>>>>>> > > > > > > >>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays > > > > > > >>>>>>> /+/master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > > > > > > >>>>>>> files/gcss/graph_settings_ov5670.xml > > > > > > >>>>>>> > > > > > > >>>>>>> For the ov5670 example, for an input frame with a resolution of > > > > > > >>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the > > > > > > >>>>>>> corresponding resolutions for input feeder, BDS and GDC are > > > > > > >>>>>>> 2592x1944, 2592x1944 and 2560x1920 respectively. > > > > > > >>>>>> > > > > > > >>>>>> How is the GDC output resolution computed from the input resolution ? > > > > > > >>>>>> Does the GDC always consume 32 columns and 22 lines ? > > > > > > >>>> > > > > > > >>>> All the intermediate resolutions in the pipeline are determined by the > > > > > > >>>> actual use case, in other word determined by the IMGU input > > > > > > >>>> resolution(sensor output) and the final output and viewfinder > > > > > > >>>> resolution. BDS mainly do Bayer downscaling, it has limitation that the > > > > > > >>>> downscaling factor must be a value a integer multiple of 1/32. > > > > > > >>>> GDC output depends on the input and width should be x8 and height x4 > > > > > > >>>> alignment. > > > > > > >>> > > > > > > >>> Thank you for the information. This will need to be captured in the > > > > > > >>> documentation, along with information related to how each block in the > > > > > > >>> hardware pipeline interacts with the image size. It should be possible > > > > > > >>> for a developer to compute the output and viewfinder resolutions based > > > > > > >>> on the parameters of the image processing algorithms just with the > > > > > > >>> information contained in the driver documentation. > > > > > > >>> > > > > > > >>>>>>> The following steps prepare the ImgU ISP pipeline for the image > > > > > > >>>>>>> processing. > > > > > > >>>>>>> > > > > > > >>>>>>> 1. The ImgU V4L2 subdev data format should be set by using the > > > > > > >>>>>>> VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height > > > > > > >>>>>>> obtained > > > > > > >>>>>>> above. > > > > > > >>>>>> > > > > > > >>>>>> If I understand things correctly, the GDC resolution is the pipeline > > > > > > >>>>>> output resolution. Why is it configured on pad 0 ? > > > > > > >>>> > > > > > > >>>> We see the GDC output resolution as the input of output system, the > > > > > > >>>> sink pad format is used for output and viewfinder resolutions. > > > > > > >>> > > > > > > >>> The ImgU subdev is supposed to represent the ImgU. Pad 0 should thus be > > > > > > >>> the ImgU input, the format configured there should correspond to the > > > > > > >>> format on the connected video node, and should thus be the sensor > > > > > > >>> format. You can then use the crop and compose rectangles on pad 0, along > > > > > > >>> with the format, crop and compose rectangles on the output and > > > > > > >>> viewfinder pads, to configure the device. This should be fixed in the > > > > > > >>> driver, and the documentation should then be updated accordingly. > > > > > > >> > > > > > > >> Hi, Laurent, > > > > > > >> > > > > > > >> Thanks for your review. > > > > > > >> > > > > > > >> I think it make sense for me that using Pad 0 as the ImgU input(IF). > > > > > > >> However, I prefer using the 2 source pads for output and viewfinder. > > > > > > >> It makes more sense because the output and viewfinder are independent > > > > > > >> output. > > > > > > >> > > > > > > >> The whole pipeline in ImgU looks like: > > > > > > >> IF --> BDS --> GDC ---> OUTPUT > > > > > > >> |-----> VF > > > > > > >> > > > > > > >> The BDS is used to do Bayer downscaling and GDC can do cropping. > > > > > > > > > > > > > > Does this mean that the main output and the viewfinder output share the > > > > > > > same scaler, and that the only difference in size between the two outputs > > > > > > > is solely due to cropping ? > > > > > > > > > > > > Laurent, > > > > > > No, output only can do crop and viewfinder support crop and scaling, they > > > > > > share same input. > > > > > > > > > > Then you can't support this with a single subdev for the ImgU, you need at > > > > > least two subdevs. I can offer more guidance, but I'll need more information > > > > > about the GDC. > > > > > > > > While the current documentation only defines the functionality of the > > > > compose target for sink pads, there are a few sensor drivers supporting it > > > > on source pads already. Some drivers such as the OMAP3 ISP also use the > > > > format on source pads to configure scaling. > > > > > > > > The current API certainly allows exposing the compose rectangle also on the > > > > source pads, but to make that generic we'd need to amend the API to tell in > > > > which order these steps take place. In the meantime the behaviour remains > > > > device specific. > > > > > > > > > > My understanding is that what is currently missing is the support > > > for viewfinder's ability to scale, as the scaler should get > > > programmed by configuring a composing rectangle on a source pad which > > > is not supported by the V4L2 APIs at the moment. Is my understanding correct? > > > > > > As the composing rectangle is set for both 'output' and 'viewfinder' > > > through the image format sizes configured on the first sink pad (*), > > > the viewfinder output is obtained by cropping-only to the image format > > > sizes configured on source pad number 3 (though SUBDEV_S_FMT not through > > > SUBDEV_S_SELECTION, as SUBDEV_S_SELECTION is only allowed on sink pad > > > 0 in the driver: see "ipu3_subdev_set_selection()"). > > > > > > As you mentioned "device specific behaviour", what is the intended one > > > for the ipu3? I assumed the viewfinder scaling/cropping was configured > > > on the 'viewfinder' video device node, through the VIDIOC_S_SELECTION > > > ioctl, but looking at the code, that doesn't seem to be listed as > > > supported in "ipu3_v4l2_ioctl_ops". > > > > > > How am I supposed to configure scaling on the viewfinder output? Would > > > adding support for crop/compose to the 'output' and 'viewfinder' video > > > devices be supported by the V4L2 APIs? That would work with the single > > > subdevice model that is currently implemented in this patches... > > > > Isn't this what the driver actually implements? My understanding was > > that the format on the VF video node determined the scaling settings, > > based on the source pad. > > > > I might surely be wrong, but VIDIOC_S_SELECTION seems not to be > supported in the "ipu3_v4l2_ioctl_ops" list. > VIDIOC_S_SELECTION is not implemented, but VIDIOC_S_FMT is. I might be missing something, but it doesn't seem to consider pad formats and anything that comes from the userspace and meets alignment and min/max constraints seems to be accepted. What happens if you set a resolution smaller than the pad on the VF video node? (On OUT note it shouldn't be possible to do so, though...) > So we can just confiure cropping sizes on the source pad number 3 of > the 'imgu' entity, and set the same sizes on the video device node > that represents the VF output. > > Inspecting the images, and reading again the long mail thread, it > seems to me that the imgu perform composing internally, based on the > GDC output dimensions and the required VF ones, to maintain the field > of view. > Yeah, I think we may be missing the separate cropping control between VF and OUT, but I'm not sure if there is any real use case which would need it, since one would normally want to have the preview match the full frame, but possibly scaled down. > I cannot find any link to post here to Bing Bu's reply to Sakari on > this, but seems like this discussion happened already, and my > understanding is that composing on VF cannot be set from user, but > calculated by the imgu internally. Sakari, Bing Bu, is that the > current state? > > Thanks > j > > ------------------------------------------------------------------------------ > Pasting down here the relevan bits from Bing Bu's and Sakari's > exchange from "Wed, 14 Nov 2018 15:02:37 +0800" which I cannot find a > suitable link to to paste here: > > >>>>>>>> pad3: Source > >>>>>>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] > >>>>>>>> -> "ipu3-imgu 0 viewfinder":0 [] > >>>>>>> Are there other differences between output and viewfinder? > >>>>>> output and viewfinder are the main and secondary output of output system. > >>>>>> 'main' output is not allowed to be scaled, only support crop. secondary > >>>>>> output 'viewfinder' > >>>>>> can support both cropping and scaling. User can select different nodes > >>>>>> to use > >>>>>> as preview and capture flexibly based on the actual use cases. > >>>>> If there's scaling to be configured, I'd expect to see the COMPOSE target > >>>>> supported. > >>>> Actually the viewfinder is the result of scaling, that means you can not > >>>> do more scaling. > >>> How do you configure the scaling of the viewfinder currently? > >> We consider that the viewfinder as a secondary output, and set the format by > >> subdev set_fmt() directly and all pads formats will be used to find > >> binary and > >> build pipeline. > > Ok. > > > > Could you instead use the compose target to configure the scaling? Setting > > the format on the source pad would have no effect. > Hi, Sakari, > > For the secondary output (viewfinder), it support both cropping and > scaling, in order > to keep the aspect ratio, system will do [crop --> compose], sounds like > it should > have 2 selection targets, but firmware/driver did not provide clear > interfaces to allow > user to configure the cropping, driver calculate the scaler and cropping > parameters > based on the output-system input and output resolution to keep aspect ratio > and field of view. I think it is more succinct to set the actual output > by set format > instead of using selection targets. > ------------------------------------------------------------------------------
Hello, sorry for resurrecting the thread. The development of libcamera has IPU3 devices as first target, and we're now at the point where some clarifications are required. I'll re-use this email, as some of points were already stated here and, as they've become pressing for libcamera development, I would like to have them clarified here on the list. On Wed, Jan 02, 2019 at 10:38:33AM +0800, Bingbu Cao wrote: > > On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > > Hello Bingbu, > > [snip] > > > > > > > > 2. ImgU pipeline needs to be configured for image processing as below. > > > > > > > > > > > > > > > > RAW bayer frames go through the following ISP pipeline HW blocks to > > > > > > > > have the processed image output to the DDR memory. > > > > > > > > > > > > > > > > RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > > > > > > > > Geometric Distortion Correction (GDC) -> DDR > > > > > > > > > > > > > > > > The ImgU V4L2 subdev has to be configured with the supported > > > > > > > > resolutions in all the above HW blocks, for a given input resolution. > > > > > > > > > > > > > > > > For a given supported resolution for an input frame, the Input Feeder, > > > > > > > > Bayer Down Scaling and GDC blocks should be configured with the > > > > > > > > supported resolutions. This information can be obtained by looking at > > > > > > > > the following IPU3 ISP configuration table for ov5670 sensor. > > > > > > > > > > > > > > > > https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > > > > > > > > /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > > > > > > > > files/gcss/graph_settings_ov5670.xml > > > > > > > > > > > > > > > > For the ov5670 example, for an input frame with a resolution of > > > > > > > > 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > > > > > > > > resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > > > > > > > > 2560x1920 respectively. > > > > > > > How is the GDC output resolution computed from the input resolution ? > > > > > > > Does the GDC always consume 32 columns and 22 lines ? > > > > > All the intermediate resolutions in the pipeline are determined by the > > > > > actual use case, in other word determined by the IMGU input > > > > > resolution(sensor output) and the final output and viewfinder resolution. > > > > > BDS mainly do Bayer downscaling, it has limitation that the downscaling > > > > > factor must be a value a integer multiple of 1/32. > > > > > GDC output depends on the input and width should be x8 and height x4 > > > > > alignment. > > > > Thank you for the information. This will need to be captured in the > > > > documentation, along with information related to how each block in the > > > > hardware pipeline interacts with the image size. It should be possible for > > > > a developer to compute the output and viewfinder resolutions based on the > > > > parameters of the image processing algorithms just with the information > > > > contained in the driver documentation. In libcamera development we're now at the point of having to calculate the sizes to apply to all intermediate pipeline stages based on the following informations: 1) Main output resolution 2) Secondary output resolution (optional) 3) Image sensor's available resolutions Right now that informations are captured in the xml file you linked here above, but we need a programmatic way to do the calculation, without going through an XML file, that refers to two specific sensors only. As Laurent said here, this should come as part of the documentation for driver users and would unblock libcamera IPU3 support development. Could you provide documentation on how to calculate each intermediate step resolutions? > > > > [snip] > > > > > > > > 3. The ImgU V4L2 subdev composing should be set by using the > > > > > > > > VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > > > > > > > target, using the BDS height and width. > > > > > > > > > > > > > > > > Once these 2 steps are done, the raw bayer frames can be input to the > > > > > > > > ImgU V4L2 subdev for processing. > > > > > > > Do I need to capture from both the output and viewfinder nodes ? How > > > > > > > are they related to the IF -> BDS -> GDC pipeline, are they both fed > > > > > > > from the GDC output ? If so, how does the viewfinder scaler fit in that > > > > > > > picture ? > > > > > The output capture should be set, the viewfinder can be disabled. > > > > > The IF and BDS are seen as crop and compose of the imgu input video > > > > > device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > > > > > pads. This is another point that we would like to have clarified: 1) which outputs are mandatory and which one are not 2) which operations are mandatory on un-used outputs 3) does the 'ipu_pipe_mode' control impact this As you mentioned here, "output" seems to be mandatory, while "viewfinder" and "stat" are optional. We have tried using the "output" video node only but the system hangs to an un-recoverable state. What I have noticed is instead that the viewfinder and stat nodes needs to be: 1) Linked to the respective "ImgU" subdevice pads 2) Format configured 3) Memory reserved 4) video device nodes started It it not required to queue/dequeue buffers from viewfinder and stat, but steps 1-4 have to be performed. Can you confirm this is intended? Could you please list all the steps that have to be applied to the ImgU's capture video nodes, and which ones are mandatory and which ones are optional, for the following use cases: 1) Main output capture only 2) Main + secondary output capture 3) Secondary capture only. Thanks j
On 3/23/19 9:02 PM, Jacopo Mondi wrote: > Hello, > sorry for resurrecting the thread. > > The development of libcamera has IPU3 devices as first target, and > we're now at the point where some clarifications are required. > I'll re-use this email, as some of points were already stated here > and, as they've become pressing for libcamera development, I would like > to have them clarified here on the list. > > On Wed, Jan 02, 2019 at 10:38:33AM +0800, Bingbu Cao wrote: >> >> On 12/26/2018 07:03 PM, Laurent Pinchart wrote: >>> Hello Bingbu, >>> > > [snip] > >>>>>>>>> 2. ImgU pipeline needs to be configured for image processing as below. >>>>>>>>> >>>>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to >>>>>>>>> have the processed image output to the DDR memory. >>>>>>>>> >>>>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> >>>>>>>>> Geometric Distortion Correction (GDC) -> DDR >>>>>>>>> >>>>>>>>> The ImgU V4L2 subdev has to be configured with the supported >>>>>>>>> resolutions in all the above HW blocks, for a given input resolution. >>>>>>>>> >>>>>>>>> For a given supported resolution for an input frame, the Input Feeder, >>>>>>>>> Bayer Down Scaling and GDC blocks should be configured with the >>>>>>>>> supported resolutions. This information can be obtained by looking at >>>>>>>>> the following IPU3 ISP configuration table for ov5670 sensor. >>>>>>>>> >>>>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ >>>>>>>>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ >>>>>>>>> files/gcss/graph_settings_ov5670.xml >>>>>>>>> >>>>>>>>> For the ov5670 example, for an input frame with a resolution of >>>>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding >>>>>>>>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and >>>>>>>>> 2560x1920 respectively. >>>>>>>> How is the GDC output resolution computed from the input resolution ? >>>>>>>> Does the GDC always consume 32 columns and 22 lines ? >>>>>> All the intermediate resolutions in the pipeline are determined by the >>>>>> actual use case, in other word determined by the IMGU input >>>>>> resolution(sensor output) and the final output and viewfinder resolution. >>>>>> BDS mainly do Bayer downscaling, it has limitation that the downscaling >>>>>> factor must be a value a integer multiple of 1/32. >>>>>> GDC output depends on the input and width should be x8 and height x4 >>>>>> alignment. >>>>> Thank you for the information. This will need to be captured in the >>>>> documentation, along with information related to how each block in the >>>>> hardware pipeline interacts with the image size. It should be possible for >>>>> a developer to compute the output and viewfinder resolutions based on the >>>>> parameters of the image processing algorithms just with the information >>>>> contained in the driver documentation. > > In libcamera development we're now at the point of having to calculate > the sizes to apply to all intermediate pipeline stages based on the > following informations: > > 1) Main output resolution > 2) Secondary output resolution (optional) > 3) Image sensor's available resolutions > > Right now that informations are captured in the xml file you linked > here above, but we need a programmatic way to do the calculation, > without going through an XML file, that refers to two specific sensors > only. > > As Laurent said here, this should come as part of the documentation > for driver users and would unblock libcamera IPU3 support > development. > > Could you provide documentation on how to calculate each > intermediate step resolutions? All the intermediate step resolutions are generated by the specific tool with sensor input and outputs resolutions. The tool try to keep maximum fov and has the knowledge of all the limitations of each intermediate hardware components(mainly BDS and GDC). Currently, there is not a very simple calculation to get the intermediate resolutions. Let's take some effort to try find a programmatic way to do calculation instead of the tool. > >>>>> > > [snip] > >>>>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the >>>>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>>>>>>>> target, using the BDS height and width. >>>>>>>>> >>>>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the >>>>>>>>> ImgU V4L2 subdev for processing. >>>>>>>> Do I need to capture from both the output and viewfinder nodes ? How >>>>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed >>>>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that >>>>>>>> picture ? >>>>>> The output capture should be set, the viewfinder can be disabled. >>>>>> The IF and BDS are seen as crop and compose of the imgu input video >>>>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source >>>>>> pads. > > This is another point that we would like to have clarified: > 1) which outputs are mandatory and which one are not > 2) which operations are mandatory on un-used outputs > 3) does the 'ipu_pipe_mode' control impact this > > As you mentioned here, "output" seems to be mandatory, while > "viewfinder" and "stat" are optional. We have tried using the "output" > video node only but the system hangs to an un-recoverable state. Yes, main output is mandatory, 'vf' and 'stat' are optional. > > What I have noticed is instead that the viewfinder and stat nodes > needs to be: > 1) Linked to the respective "ImgU" subdevice pads > 2) Format configured > 3) Memory reserved > 4) video device nodes started > > It it not required to queue/dequeue buffers from viewfinder and stat, > but steps 1-4 have to be performed. > > Can you confirm this is intended? viewfinder and stats are enabled when the link for respective subdev pads enabled, and then driver can use these input conditions to find the binary to run. > Could you please list all the steps that have to be applied to the > ImgU's capture video nodes, and which ones are mandatory and which ones > are optional, for the following use cases: > 1) Main output capture only > 2) Main + secondary output capture > 3) Secondary capture only. I think the 3) is not supported. The steps are: 1). link necessary the respective subdevices input --> imgu -->output | -->vf | -->3a stats 2). set all the formats for input, output and intermediate resolutions. 3). start stream The ipu pipe_mode will not impact the whole pipe behavior. It just ask firmware to run different processing to generate same format outputs. > > Thanks > j >
Hi Bingbu, On Mon, Mar 25, 2019 at 11:45:58AM +0800, Bingbu Cao wrote: > On 3/23/19 9:02 PM, Jacopo Mondi wrote: > > Hello, > > sorry for resurrecting the thread. > > > > The development of libcamera has IPU3 devices as first target, and > > we're now at the point where some clarifications are required. > > I'll re-use this email, as some of points were already stated here > > and, as they've become pressing for libcamera development, I would like > > to have them clarified here on the list. > > > > On Wed, Jan 02, 2019 at 10:38:33AM +0800, Bingbu Cao wrote: > >> On 12/26/2018 07:03 PM, Laurent Pinchart wrote: > >>> Hello Bingbu, > >>> > > > > [snip] > > > >>>>>>>>> 2. ImgU pipeline needs to be configured for image processing as below. > >>>>>>>>> > >>>>>>>>> RAW bayer frames go through the following ISP pipeline HW blocks to > >>>>>>>>> have the processed image output to the DDR memory. > >>>>>>>>> > >>>>>>>>> RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> > >>>>>>>>> Geometric Distortion Correction (GDC) -> DDR > >>>>>>>>> > >>>>>>>>> The ImgU V4L2 subdev has to be configured with the supported > >>>>>>>>> resolutions in all the above HW blocks, for a given input resolution. > >>>>>>>>> > >>>>>>>>> For a given supported resolution for an input frame, the Input Feeder, > >>>>>>>>> Bayer Down Scaling and GDC blocks should be configured with the > >>>>>>>>> supported resolutions. This information can be obtained by looking at > >>>>>>>>> the following IPU3 ISP configuration table for ov5670 sensor. > >>>>>>>>> > >>>>>>>>> https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+ > >>>>>>>>> /master/baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/ > >>>>>>>>> files/gcss/graph_settings_ov5670.xml > >>>>>>>>> > >>>>>>>>> For the ov5670 example, for an input frame with a resolution of > >>>>>>>>> 2592x1944 (which is input to the ImgU subdev pad 0), the corresponding > >>>>>>>>> resolutions for input feeder, BDS and GDC are 2592x1944, 2592x1944 and > >>>>>>>>> 2560x1920 respectively. > >>>>>>>> > >>>>>>>> How is the GDC output resolution computed from the input resolution ? > >>>>>>>> Does the GDC always consume 32 columns and 22 lines ? > >>>>>> > >>>>>> All the intermediate resolutions in the pipeline are determined by the > >>>>>> actual use case, in other word determined by the IMGU input > >>>>>> resolution(sensor output) and the final output and viewfinder resolution. > >>>>>> BDS mainly do Bayer downscaling, it has limitation that the downscaling > >>>>>> factor must be a value a integer multiple of 1/32. > >>>>>> GDC output depends on the input and width should be x8 and height x4 > >>>>>> alignment. > >>>>> > >>>>> Thank you for the information. This will need to be captured in the > >>>>> documentation, along with information related to how each block in the > >>>>> hardware pipeline interacts with the image size. It should be possible for > >>>>> a developer to compute the output and viewfinder resolutions based on the > >>>>> parameters of the image processing algorithms just with the information > >>>>> contained in the driver documentation. > > > > In libcamera development we're now at the point of having to calculate > > the sizes to apply to all intermediate pipeline stages based on the > > following informations: > > > > 1) Main output resolution > > 2) Secondary output resolution (optional) > > 3) Image sensor's available resolutions > > > > Right now that informations are captured in the xml file you linked > > here above, but we need a programmatic way to do the calculation, > > without going through an XML file, that refers to two specific sensors > > only. > > > > As Laurent said here, this should come as part of the documentation > > for driver users and would unblock libcamera IPU3 support > > development. > > > > Could you provide documentation on how to calculate each > > intermediate step resolutions? > > All the intermediate step resolutions are generated by the specific tool > with sensor input and outputs resolutions. > > The tool try to keep maximum fov and has the knowledge of all the > limitations of each intermediate hardware components(mainly BDS and GDC). That's exactly what we want to do in software in libcamera :-) And that's why we need more infirmation about the limitations of each intermediate hardware component. Eventually those limitations should be documented in the IPU3 driver documentation in the kernel sources, but for now we can move forward if they're just communicated by e-mail (if time permits we may be able to submit a kernel patch to integrate that in the documentation). > Currently, there is not a very simple calculation to get the > intermediate resolutions. > Let's take some effort to try find a programmatic way to do calculation > instead of the tool. > > > [snip] > > > >>>>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the > >>>>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > >>>>>>>>> target, using the BDS height and width. > >>>>>>>>> > >>>>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the > >>>>>>>>> ImgU V4L2 subdev for processing. > >>>>>>>> Do I need to capture from both the output and viewfinder nodes ? How > >>>>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > >>>>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that > >>>>>>>> picture ? > >>>>>> The output capture should be set, the viewfinder can be disabled. > >>>>>> The IF and BDS are seen as crop and compose of the imgu input video > >>>>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > >>>>>> pads. > > > > This is another point that we would like to have clarified: > > 1) which outputs are mandatory and which one are not > > 2) which operations are mandatory on un-used outputs > > 3) does the 'ipu_pipe_mode' control impact this > > > > As you mentioned here, "output" seems to be mandatory, while > > "viewfinder" and "stat" are optional. We have tried using the "output" > > video node only but the system hangs to an un-recoverable state. > > Yes, main output is mandatory, 'vf' and 'stat' are optional. I will let Jacopo confirm this, but unless I'm mistaken, when he tried to use the main output only (with the links between the ImgU subdev and the vf and stat video nodes disabled), the driver would hang without processing any frame. I believe this was a complete system hang, requiring a hard reboot to recover. > > What I have noticed is instead that the viewfinder and stat nodes > > needs to be: > > 1) Linked to the respective "ImgU" subdevice pads > > 2) Format configured > > 3) Memory reserved > > 4) video device nodes started > > > > It it not required to queue/dequeue buffers from viewfinder and stat, > > but steps 1-4 have to be performed. > > > > Can you confirm this is intended? > > viewfinder and stats are enabled when the link for respective subdev > pads enabled, and then driver can use these input conditions to find the > binary to run. > > > Could you please list all the steps that have to be applied to the > > ImgU's capture video nodes, and which ones are mandatory and which ones > > are optional, for the following use cases: > > 1) Main output capture only > > 2) Main + secondary output capture > > 3) Secondary capture only. > > I think the 3) is not supported. > > The steps are: > 1). link necessary the respective subdevices > input --> imgu -->output > | -->vf > | -->3a stats > > 2). set all the formats for input, output and intermediate resolutions. > 3). start stream > > The ipu pipe_mode will not impact the whole pipe behavior. It just ask > firmware to run different processing to generate same format outputs.
Hi Laurent, Bingbu On Mon, Mar 25, 2019 at 06:06:30AM +0200, Laurent Pinchart wrote: > Hi Bingbu, > [snip] > > >>>>> > > >>>>> Thank you for the information. This will need to be captured in the > > >>>>> documentation, along with information related to how each block in the > > >>>>> hardware pipeline interacts with the image size. It should be possible for > > >>>>> a developer to compute the output and viewfinder resolutions based on the > > >>>>> parameters of the image processing algorithms just with the information > > >>>>> contained in the driver documentation. > > > > > > In libcamera development we're now at the point of having to calculate > > > the sizes to apply to all intermediate pipeline stages based on the > > > following informations: > > > > > > 1) Main output resolution > > > 2) Secondary output resolution (optional) > > > 3) Image sensor's available resolutions > > > > > > Right now that informations are captured in the xml file you linked > > > here above, but we need a programmatic way to do the calculation, > > > without going through an XML file, that refers to two specific sensors > > > only. > > > > > > As Laurent said here, this should come as part of the documentation > > > for driver users and would unblock libcamera IPU3 support > > > development. > > > > > > Could you provide documentation on how to calculate each > > > intermediate step resolutions? > > > > All the intermediate step resolutions are generated by the specific tool > > with sensor input and outputs resolutions. > > > > The tool try to keep maximum fov and has the knowledge of all the > > limitations of each intermediate hardware components(mainly BDS and GDC). > > That's exactly what we want to do in software in libcamera :-) And > that's why we need more infirmation about the limitations of each > intermediate hardware component. Eventually those limitations should be > documented in the IPU3 driver documentation in the kernel sources, but > for now we can move forward if they're just communicated by e-mail (if > time permits we may be able to submit a kernel patch to integrate that > in the documentation). > > > Currently, there is not a very simple calculation to get the > > intermediate resolutions. > > Let's take some effort to try find a programmatic way to do calculation > > instead of the tool. Thank you for your effort. > > > > > [snip] > > > > > >>>>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the > > >>>>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > > >>>>>>>>> target, using the BDS height and width. > > >>>>>>>>> > > >>>>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the > > >>>>>>>>> ImgU V4L2 subdev for processing. > > >>>>>>>> Do I need to capture from both the output and viewfinder nodes ? How > > >>>>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > > >>>>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that > > >>>>>>>> picture ? > > >>>>>> The output capture should be set, the viewfinder can be disabled. > > >>>>>> The IF and BDS are seen as crop and compose of the imgu input video > > >>>>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > > >>>>>> pads. > > > > > > This is another point that we would like to have clarified: > > > 1) which outputs are mandatory and which one are not > > > 2) which operations are mandatory on un-used outputs > > > 3) does the 'ipu_pipe_mode' control impact this > > > > > > As you mentioned here, "output" seems to be mandatory, while > > > "viewfinder" and "stat" are optional. We have tried using the "output" > > > video node only but the system hangs to an un-recoverable state. > > > > Yes, main output is mandatory, 'vf' and 'stat' are optional. > > I will let Jacopo confirm this, but unless I'm mistaken, when he tried > to use the main output only (with the links between the ImgU subdev and > the vf and stat video nodes disabled), the driver would hang without > processing any frame. I believe this was a complete system hang, > requiring a hard reboot to recover. > Yes, that's what I have noticed. On the other hand, if I link, configure, prepare buffers and start the 'vf' and 'stat' nodes, but never queue buffers there, I can capture from output only. > > > What I have noticed is instead that the viewfinder and stat nodes > > > needs to be: > > > 1) Linked to the respective "ImgU" subdevice pads > > > 2) Format configured > > > 3) Memory reserved > > > 4) video device nodes started > > > > > > It it not required to queue/dequeue buffers from viewfinder and stat, > > > but steps 1-4 have to be performed. > > > > > > Can you confirm this is intended? > > > > viewfinder and stats are enabled when the link for respective subdev > > pads enabled, and then driver can use these input conditions to find the > > binary to run. > > As Laurent reported above, if I leave the the 'vf' and 'stat' links disabled, the system hangs. > > > Could you please list all the steps that have to be applied to the > > > ImgU's capture video nodes, and which ones are mandatory and which ones > > > are optional, for the following use cases: > > > 1) Main output capture only > > > 2) Main + secondary output capture > > > 3) Secondary capture only. > > > > I think the 3) is not supported. > > > > The steps are: > > 1). link necessary the respective subdevices > > input --> imgu -->output > > | -->vf > > | -->3a stats For which use case, in the above reported list? 1) Main output capture only Does 'vf' and 'stat' links needs to be enabled? 2) Main + secondary output capture Does 'stat' link need to be enabled? 3) Secondary capture only. not supported > > > > 2). set all the formats for input, output and intermediate resolutions. > > 3). start stream > > > > The ipu pipe_mode will not impact the whole pipe behavior. It just ask > > firmware to run different processing to generate same format outputs. > I would apreciate to have a better description of the pipe_mode control, in order to better understand when and if the library has to modify its value and which mode to use (0=video, 1=still_capture). Thanks j > -- > Regards, > > Laurent Pinchart
On 3/25/19 4:11 PM, Jacopo Mondi wrote: > Hi Laurent, Bingbu > > On Mon, Mar 25, 2019 at 06:06:30AM +0200, Laurent Pinchart wrote: >> Hi Bingbu, >> > [snip] > >>>>>>>> >>>>>>>> Thank you for the information. This will need to be captured in the >>>>>>>> documentation, along with information related to how each block in the >>>>>>>> hardware pipeline interacts with the image size. It should be possible for >>>>>>>> a developer to compute the output and viewfinder resolutions based on the >>>>>>>> parameters of the image processing algorithms just with the information >>>>>>>> contained in the driver documentation. >>>> >>>> In libcamera development we're now at the point of having to calculate >>>> the sizes to apply to all intermediate pipeline stages based on the >>>> following informations: >>>> >>>> 1) Main output resolution >>>> 2) Secondary output resolution (optional) >>>> 3) Image sensor's available resolutions >>>> >>>> Right now that informations are captured in the xml file you linked >>>> here above, but we need a programmatic way to do the calculation, >>>> without going through an XML file, that refers to two specific sensors >>>> only. >>>> >>>> As Laurent said here, this should come as part of the documentation >>>> for driver users and would unblock libcamera IPU3 support >>>> development. >>>> >>>> Could you provide documentation on how to calculate each >>>> intermediate step resolutions? >>> >>> All the intermediate step resolutions are generated by the specific tool >>> with sensor input and outputs resolutions. >>> >>> The tool try to keep maximum fov and has the knowledge of all the >>> limitations of each intermediate hardware components(mainly BDS and GDC). >> >> That's exactly what we want to do in software in libcamera :-) And >> that's why we need more infirmation about the limitations of each >> intermediate hardware component. Eventually those limitations should be >> documented in the IPU3 driver documentation in the kernel sources, but >> for now we can move forward if they're just communicated by e-mail (if >> time permits we may be able to submit a kernel patch to integrate that >> in the documentation). >> >>> Currently, there is not a very simple calculation to get the >>> intermediate resolutions. >>> Let's take some effort to try find a programmatic way to do calculation >>> instead of the tool. > > Thank you for your effort. > >>> >>>> [snip] >>>> >>>>>>>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the >>>>>>>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>>>>>>>>>>> target, using the BDS height and width. >>>>>>>>>>>> >>>>>>>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the >>>>>>>>>>>> ImgU V4L2 subdev for processing. >>>>>>>>>>> Do I need to capture from both the output and viewfinder nodes ? How >>>>>>>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed >>>>>>>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that >>>>>>>>>>> picture ? >>>>>>>>> The output capture should be set, the viewfinder can be disabled. >>>>>>>>> The IF and BDS are seen as crop and compose of the imgu input video >>>>>>>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source >>>>>>>>> pads. >>>> >>>> This is another point that we would like to have clarified: >>>> 1) which outputs are mandatory and which one are not >>>> 2) which operations are mandatory on un-used outputs >>>> 3) does the 'ipu_pipe_mode' control impact this >>>> >>>> As you mentioned here, "output" seems to be mandatory, while >>>> "viewfinder" and "stat" are optional. We have tried using the "output" >>>> video node only but the system hangs to an un-recoverable state. >>> >>> Yes, main output is mandatory, 'vf' and 'stat' are optional. >> >> I will let Jacopo confirm this, but unless I'm mistaken, when he tried >> to use the main output only (with the links between the ImgU subdev and >> the vf and stat video nodes disabled), the driver would hang without >> processing any frame. I believe this was a complete system hang, >> requiring a hard reboot to recover. >> > > Yes, that's what I have noticed. > > On the other hand, if I link, configure, prepare buffers and start the > 'vf' and 'stat' nodes, but never queue buffers there, I can capture > from output only. > >>>> What I have noticed is instead that the viewfinder and stat nodes >>>> needs to be: >>>> 1) Linked to the respective "ImgU" subdevice pads >>>> 2) Format configured >>>> 3) Memory reserved >>>> 4) video device nodes started >>>> >>>> It it not required to queue/dequeue buffers from viewfinder and stat, >>>> but steps 1-4 have to be performed. >>>> >>>> Can you confirm this is intended? >>> >>> viewfinder and stats are enabled when the link for respective subdev >>> pads enabled, and then driver can use these input conditions to find the >>> binary to run. >>> > > As Laurent reported above, if I leave the the 'vf' and 'stat' links > disabled, the system hangs. > >>>> Could you please list all the steps that have to be applied to the >>>> ImgU's capture video nodes, and which ones are mandatory and which ones >>>> are optional, for the following use cases: >>>> 1) Main output capture only >>>> 2) Main + secondary output capture >>>> 3) Secondary capture only. >>> >>> I think the 3) is not supported. >>> >>> The steps are: >>> 1). link necessary the respective subdevices >>> input --> imgu -->output >>> | -->vf >>> | -->3a stats > > For which use case, in the above reported list? > > 1) Main output capture only > Does 'vf' and 'stat' links needs to be enabled? > > 2) Main + secondary output capture > Does 'stat' link need to be enabled? > > 3) Secondary capture only. > not supported The list above is a typical use, all outputs enabled, you can setup link for main output only. > >>> >>> 2). set all the formats for input, output and intermediate resolutions. >>> 3). start stream >>> >>> The ipu pipe_mode will not impact the whole pipe behavior. It just ask >>> firmware to run different processing to generate same format outputs. >> > > I would apreciate to have a better description of the pipe_mode > control, in order to better understand when and if the library has to > modify its value and which mode to use (0=video, 1=still_capture). In some application, it will request continuous viewfinder, that means you must keep preview continuous when take capture. That means you can not switch out pipeline (preview and still mode, back and force), so the driver need create 2 pipelines to satisfy this usage, 1 video mode pipe and another is still mode. Both 2 pipes are created at first, run the pipe as you command. It also can support still during video usage. > > Thanks > j > >> -- >> Regards, >> >> Laurent Pinchart
Hi Bingbu, On Mon, Mar 25, 2019 at 06:07:58PM +0800, Bingbu Cao wrote: > > > On 3/25/19 4:11 PM, Jacopo Mondi wrote: > > Hi Laurent, Bingbu > > > > On Mon, Mar 25, 2019 at 06:06:30AM +0200, Laurent Pinchart wrote: > >> Hi Bingbu, > >> > > [snip] > > > >>>>>>>> > >>>>>>>> Thank you for the information. This will need to be captured in the > >>>>>>>> documentation, along with information related to how each block in the > >>>>>>>> hardware pipeline interacts with the image size. It should be possible for > >>>>>>>> a developer to compute the output and viewfinder resolutions based on the > >>>>>>>> parameters of the image processing algorithms just with the information > >>>>>>>> contained in the driver documentation. > >>>> > >>>> In libcamera development we're now at the point of having to calculate > >>>> the sizes to apply to all intermediate pipeline stages based on the > >>>> following informations: > >>>> > >>>> 1) Main output resolution > >>>> 2) Secondary output resolution (optional) > >>>> 3) Image sensor's available resolutions > >>>> > >>>> Right now that informations are captured in the xml file you linked > >>>> here above, but we need a programmatic way to do the calculation, > >>>> without going through an XML file, that refers to two specific sensors > >>>> only. > >>>> > >>>> As Laurent said here, this should come as part of the documentation > >>>> for driver users and would unblock libcamera IPU3 support > >>>> development. > >>>> > >>>> Could you provide documentation on how to calculate each > >>>> intermediate step resolutions? > >>> > >>> All the intermediate step resolutions are generated by the specific tool > >>> with sensor input and outputs resolutions. > >>> > >>> The tool try to keep maximum fov and has the knowledge of all the > >>> limitations of each intermediate hardware components(mainly BDS and GDC). > >> > >> That's exactly what we want to do in software in libcamera :-) And > >> that's why we need more infirmation about the limitations of each > >> intermediate hardware component. Eventually those limitations should be > >> documented in the IPU3 driver documentation in the kernel sources, but > >> for now we can move forward if they're just communicated by e-mail (if > >> time permits we may be able to submit a kernel patch to integrate that > >> in the documentation). > >> > >>> Currently, there is not a very simple calculation to get the > >>> intermediate resolutions. > >>> Let's take some effort to try find a programmatic way to do calculation > >>> instead of the tool. > > > > Thank you for your effort. > > > >>> > >>>> [snip] > >>>> > >>>>>>>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the > >>>>>>>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > >>>>>>>>>>>> target, using the BDS height and width. > >>>>>>>>>>>> > >>>>>>>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the > >>>>>>>>>>>> ImgU V4L2 subdev for processing. > >>>>>>>>>>> Do I need to capture from both the output and viewfinder nodes ? How > >>>>>>>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed > >>>>>>>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that > >>>>>>>>>>> picture ? > >>>>>>>>> The output capture should be set, the viewfinder can be disabled. > >>>>>>>>> The IF and BDS are seen as crop and compose of the imgu input video > >>>>>>>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source > >>>>>>>>> pads. > >>>> > >>>> This is another point that we would like to have clarified: > >>>> 1) which outputs are mandatory and which one are not > >>>> 2) which operations are mandatory on un-used outputs > >>>> 3) does the 'ipu_pipe_mode' control impact this > >>>> > >>>> As you mentioned here, "output" seems to be mandatory, while > >>>> "viewfinder" and "stat" are optional. We have tried using the "output" > >>>> video node only but the system hangs to an un-recoverable state. > >>> > >>> Yes, main output is mandatory, 'vf' and 'stat' are optional. > >> > >> I will let Jacopo confirm this, but unless I'm mistaken, when he tried > >> to use the main output only (with the links between the ImgU subdev and > >> the vf and stat video nodes disabled), the driver would hang without > >> processing any frame. I believe this was a complete system hang, > >> requiring a hard reboot to recover. > >> > > > > Yes, that's what I have noticed. > > > > On the other hand, if I link, configure, prepare buffers and start the > > 'vf' and 'stat' nodes, but never queue buffers there, I can capture > > from output only. > > > >>>> What I have noticed is instead that the viewfinder and stat nodes > >>>> needs to be: > >>>> 1) Linked to the respective "ImgU" subdevice pads > >>>> 2) Format configured > >>>> 3) Memory reserved > >>>> 4) video device nodes started > >>>> > >>>> It it not required to queue/dequeue buffers from viewfinder and stat, > >>>> but steps 1-4 have to be performed. > >>>> > >>>> Can you confirm this is intended? > >>> > >>> viewfinder and stats are enabled when the link for respective subdev > >>> pads enabled, and then driver can use these input conditions to find the > >>> binary to run. > >>> > > > > As Laurent reported above, if I leave the the 'vf' and 'stat' links > > disabled, the system hangs. > > > >>>> Could you please list all the steps that have to be applied to the > >>>> ImgU's capture video nodes, and which ones are mandatory and which ones > >>>> are optional, for the following use cases: > >>>> 1) Main output capture only > >>>> 2) Main + secondary output capture > >>>> 3) Secondary capture only. > >>> > >>> I think the 3) is not supported. > >>> > >>> The steps are: > >>> 1). link necessary the respective subdevices > >>> input --> imgu -->output > >>> | -->vf > >>> | -->3a stats > > > > For which use case, in the above reported list? > > > > 1) Main output capture only > > Does 'vf' and 'stat' links needs to be enabled? > > > > 2) Main + secondary output capture > > Does 'stat' link need to be enabled? > > > > 3) Secondary capture only. > > not supported > > The list above is a typical use, all outputs enabled, you can setup link > for main output only. I think we should clarify better what you mean by 'enabled'. From my testing what I see is that in order to operate the main output I have to: - link the stat and vf nodes (as well as input and output of course) - reserve memory buffers on stat and vf video nodes, even if not used - set format on all device ndoes - start the all video devices If one of these steps is not performed, the ImgU processing stalls and I need to hard reboot the device to have it operational again. On the other hand, I see that there is no need to queue any buffer to any of the output capture devices to have frames processed by the ImgU. IF links are setup as explained above, format configured and all the video device node started, I can queue all the frames I want to the ImgU input and never queue anything on its outputs, and they will get processed and returned to userspace nicely from the ImgU input device node. As soon as I queue a buffer to the ImgU main output video device, I see it returned filled with the processed data. This is good, as it doesen't stall the pipeline if there are no capture buffers to queue to the ImgU output, but now I wonder what you meant with "main output is mandatory" > > > >>> > >>> 2). set all the formats for input, output and intermediate resolutions. > >>> 3). start stream > >>> > >>> The ipu pipe_mode will not impact the whole pipe behavior. It just ask > >>> firmware to run different processing to generate same format outputs. > >> > > > > I would apreciate to have a better description of the pipe_mode > > control, in order to better understand when and if the library has to > > modify its value and which mode to use (0=video, 1=still_capture). > > In some application, it will request continuous viewfinder, that means > you must keep preview continuous when take capture. That means you can > not switch out pipeline (preview and still mode, back and force), so the > driver need create 2 pipelines to satisfy this usage, 1 video mode pipe > and another is still mode. Both 2 pipes are created at first, run the > pipe as you command. It also can support still during video usage. > Thanks for the explanation, but it is still vague to me and it worries me a bit as in this typical usage scenario (viewfinder + sporadic capture) -both- ImgU pipes have to be used, preventing usage of two cameras at the same time (one camera assigned to one ImgU pipe instance). Why would you use both the ImgU pipes in this case? Shouldn't you always capture from viewfinder (discarding main output frames if not required), and when requested by the application capture from both the main and secondary output from the same ImgU pipe? Why would I need to change the pipe_mode for doing this? Thank for your patience to answer all this questions :) > > > > Thanks > > j > > > >> -- > >> Regards, > >> > >> Laurent Pinchart
On 3/26/19 7:16 PM, Jacopo Mondi wrote: > Hi Bingbu, > > On Mon, Mar 25, 2019 at 06:07:58PM +0800, Bingbu Cao wrote: >> >> >> On 3/25/19 4:11 PM, Jacopo Mondi wrote: >>> Hi Laurent, Bingbu >>> >>> On Mon, Mar 25, 2019 at 06:06:30AM +0200, Laurent Pinchart wrote: >>>> Hi Bingbu, >>>> >>> [snip] >>> >>>>>>>>>> >>>>>>>>>> Thank you for the information. This will need to be captured in the >>>>>>>>>> documentation, along with information related to how each block in the >>>>>>>>>> hardware pipeline interacts with the image size. It should be possible for >>>>>>>>>> a developer to compute the output and viewfinder resolutions based on the >>>>>>>>>> parameters of the image processing algorithms just with the information >>>>>>>>>> contained in the driver documentation. >>>>>> >>>>>> In libcamera development we're now at the point of having to calculate >>>>>> the sizes to apply to all intermediate pipeline stages based on the >>>>>> following informations: >>>>>> >>>>>> 1) Main output resolution >>>>>> 2) Secondary output resolution (optional) >>>>>> 3) Image sensor's available resolutions >>>>>> >>>>>> Right now that informations are captured in the xml file you linked >>>>>> here above, but we need a programmatic way to do the calculation, >>>>>> without going through an XML file, that refers to two specific sensors >>>>>> only. >>>>>> >>>>>> As Laurent said here, this should come as part of the documentation >>>>>> for driver users and would unblock libcamera IPU3 support >>>>>> development. >>>>>> >>>>>> Could you provide documentation on how to calculate each >>>>>> intermediate step resolutions? >>>>> >>>>> All the intermediate step resolutions are generated by the specific tool >>>>> with sensor input and outputs resolutions. >>>>> >>>>> The tool try to keep maximum fov and has the knowledge of all the >>>>> limitations of each intermediate hardware components(mainly BDS and GDC). >>>> >>>> That's exactly what we want to do in software in libcamera :-) And >>>> that's why we need more infirmation about the limitations of each >>>> intermediate hardware component. Eventually those limitations should be >>>> documented in the IPU3 driver documentation in the kernel sources, but >>>> for now we can move forward if they're just communicated by e-mail (if >>>> time permits we may be able to submit a kernel patch to integrate that >>>> in the documentation). >>>> >>>>> Currently, there is not a very simple calculation to get the >>>>> intermediate resolutions. >>>>> Let's take some effort to try find a programmatic way to do calculation >>>>> instead of the tool. >>> >>> Thank you for your effort. >>> >>>>> >>>>>> [snip] >>>>>> >>>>>>>>>>>>>> 3. The ImgU V4L2 subdev composing should be set by using the >>>>>>>>>>>>>> VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the >>>>>>>>>>>>>> target, using the BDS height and width. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Once these 2 steps are done, the raw bayer frames can be input to the >>>>>>>>>>>>>> ImgU V4L2 subdev for processing. >>>>>>>>>>>>> Do I need to capture from both the output and viewfinder nodes ? How >>>>>>>>>>>>> are they related to the IF -> BDS -> GDC pipeline, are they both fed >>>>>>>>>>>>> from the GDC output ? If so, how does the viewfinder scaler fit in that >>>>>>>>>>>>> picture ? >>>>>>>>>>> The output capture should be set, the viewfinder can be disabled. >>>>>>>>>>> The IF and BDS are seen as crop and compose of the imgu input video >>>>>>>>>>> device. The GDC is seen as the subdev sink pad and OUTPUT/VF are source >>>>>>>>>>> pads. >>>>>> >>>>>> This is another point that we would like to have clarified: >>>>>> 1) which outputs are mandatory and which one are not >>>>>> 2) which operations are mandatory on un-used outputs >>>>>> 3) does the 'ipu_pipe_mode' control impact this >>>>>> >>>>>> As you mentioned here, "output" seems to be mandatory, while >>>>>> "viewfinder" and "stat" are optional. We have tried using the "output" >>>>>> video node only but the system hangs to an un-recoverable state. >>>>> >>>>> Yes, main output is mandatory, 'vf' and 'stat' are optional. >>>> >>>> I will let Jacopo confirm this, but unless I'm mistaken, when he tried >>>> to use the main output only (with the links between the ImgU subdev and >>>> the vf and stat video nodes disabled), the driver would hang without >>>> processing any frame. I believe this was a complete system hang, >>>> requiring a hard reboot to recover. >>>> >>> >>> Yes, that's what I have noticed. >>> >>> On the other hand, if I link, configure, prepare buffers and start the >>> 'vf' and 'stat' nodes, but never queue buffers there, I can capture >>> from output only. >>> >>>>>> What I have noticed is instead that the viewfinder and stat nodes >>>>>> needs to be: >>>>>> 1) Linked to the respective "ImgU" subdevice pads >>>>>> 2) Format configured >>>>>> 3) Memory reserved >>>>>> 4) video device nodes started >>>>>> >>>>>> It it not required to queue/dequeue buffers from viewfinder and stat, >>>>>> but steps 1-4 have to be performed. >>>>>> >>>>>> Can you confirm this is intended? >>>>> >>>>> viewfinder and stats are enabled when the link for respective subdev >>>>> pads enabled, and then driver can use these input conditions to find the >>>>> binary to run. >>>>> >>> >>> As Laurent reported above, if I leave the the 'vf' and 'stat' links >>> disabled, the system hangs. >>> >>>>>> Could you please list all the steps that have to be applied to the >>>>>> ImgU's capture video nodes, and which ones are mandatory and which ones >>>>>> are optional, for the following use cases: >>>>>> 1) Main output capture only >>>>>> 2) Main + secondary output capture >>>>>> 3) Secondary capture only. >>>>> >>>>> I think the 3) is not supported. >>>>> >>>>> The steps are: >>>>> 1). link necessary the respective subdevices >>>>> input --> imgu -->output >>>>> | -->vf >>>>> | -->3a stats >>> >>> For which use case, in the above reported list? >>> >>> 1) Main output capture only >>> Does 'vf' and 'stat' links needs to be enabled? >>> >>> 2) Main + secondary output capture >>> Does 'stat' link need to be enabled? >>> >>> 3) Secondary capture only. >>> not supported >> >> The list above is a typical use, all outputs enabled, you can setup link >> for main output only. > > I think we should clarify better what you mean by 'enabled'. > > From my testing what I see is that in order to operate the main > output I have to: > - link the stat and vf nodes (as well as input and output of course) > - reserve memory buffers on stat and vf video nodes, even if not used > - set format on all device ndoes > - start the all video devices > > If one of these steps is not performed, the ImgU processing stalls and > I need to hard reboot the device to have it operational again. > > On the other hand, I see that there is no need to queue any buffer to > any of the output capture devices to have frames processed by the > ImgU. > > IF links are setup as explained above, format configured and all the video > device node started, I can queue all the frames I want to the ImgU input > and never queue anything on its outputs, and they will get processed and > returned to userspace nicely from the ImgU input device node. As soon > as I queue a buffer to the ImgU main output video device, I see it > returned filled with the processed data. This is good, as it doesen't > stall the pipeline if there are no capture buffers to queue to the > ImgU output, but now I wonder what you meant with "main output is > mandatory" Hi, Mondi Sorry for late response. I think you do not need setup link for vf and stat if you did not need them. If you try setup link (enable) some queues (not input) and not queue the buffers, driver will queue some dummy buffers to make the pipe run. It needs time for me to check all the cases and try to answer you. I am a little busy on other work items recently, so forgive me late response for your questions. > >>> >>>>> >>>>> 2). set all the formats for input, output and intermediate resolutions. >>>>> 3). start stream >>>>> >>>>> The ipu pipe_mode will not impact the whole pipe behavior. It just ask >>>>> firmware to run different processing to generate same format outputs. >>>> >>> >>> I would apreciate to have a better description of the pipe_mode >>> control, in order to better understand when and if the library has to >>> modify its value and which mode to use (0=video, 1=still_capture). >> >> In some application, it will request continuous viewfinder, that means >> you must keep preview continuous when take capture. That means you can >> not switch out pipeline (preview and still mode, back and force), so the >> driver need create 2 pipelines to satisfy this usage, 1 video mode pipe >> and another is still mode. Both 2 pipes are created at first, run the >> pipe as you command. It also can support still during video usage. >> > > Thanks for the explanation, but it is still vague to me and it worries > me a bit as in this typical usage scenario (viewfinder + sporadic > capture) -both- ImgU pipes have to be used, preventing usage of two > cameras at the same time (one camera assigned to one ImgU pipe > instance). > > Why would you use both the ImgU pipes in this case? Shouldn't you > always capture from viewfinder (discarding main output frames > if not required), and when requested by the application capture from > both the main and secondary output from the same ImgU pipe? Why would I > need to change the pipe_mode for doing this? Each pipe has its own pipe_mode and outputs. So you can run 2 parallel pipes with different pipe mode, different pipe modes mean to run different pipe in firmware, typically, 1 is still pipe which used to capture and 1 is video pipe which used for video or preview. You can only enable the main output for still pipe, enable viewfinder and main for video pipe, viewfinder for video preview and main for video recording. It depends on the real use case and they are not fixed. > > Thank for your patience to answer all this questions :) > > >>> >>> Thanks >>> j >>> >>>> -- >>>> Regards, >>>> >>>> Laurent Pinchart
Hi, This series adds support for the Intel IPU3 (Image Processing Unit) ImgU which is essentially a modern memory-to-memory ISP. It implements raw Bayer to YUV image format conversion as well as a large number of other pixel processing algorithms for improving the image quality. Meta data formats are defined for image statistics (3A, i.e. automatic white balance, exposure and focus, histogram and local area contrast enhancement) as well as for the pixel processing algorithm parameters. The documentation for these formats is currently not included in the patchset but will be added in a future version of this set. The algorithm parameters need to be considered specific to a given frame and typically a large number of these parameters change on frame to frame basis. Additionally, the parameters are highly structured (and not a flat space of independent configuration primitives). They also reflect the data structures used by the firmware and the hardware. On top of that, the algorithms require highly specialized user space to make meaningful use of them. For these reasons it has been chosen video buffers to pass the parameters to the device. On individual patches: The heart of ImgU is the CSS, or Camera Subsystem, which contains the image processors and HW accelerators. The 3A statistics and other firmware parameter computation related functions are implemented in patch 11. All IPU3 pipeline default settings can be found in patch 10. To access DDR via ImgU's own memory space, IPU3 is also equipped with its own MMU unit, the driver is implemented in patch 6. Patch 7 uses above driver for DMA mapping operation. The communication between IPU3 firmware and driver is implemented with circular queues in patch 8. Patch 9 provide some utility functions and manage IPU3 fw download and install. The firmware which is called ipu3-fw.bin can be downloaded from: git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) Firmware ABI is defined in patches 4 and 5. Patches 12 and 13 are of the same file, the former contains all h/w programming related code, the latter implements interface functions for access fw & hw capabilities. Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: <URL:https://patchwork.kernel.org/patch/9976295/> Patch 15 represents the top level that glues all of the other components together, passing arguments between the components. Patch 16 is a recent effort to extend v6 for advanced camera features like Continuous View Finder (CVF) and Snapshot During Video(SDV) support. Link to user space implementation: git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera ImgU media topology print: # media-ctl -d /dev/media0 -p Media controller API version 4.19.0 Media device information ------------------------ driver ipu3-imgu model ipu3-imgu serial bus info PCI:0000:00:05.0 hw revision 0x80862015 driver version 4.19.0 Device topology - entity 1: ipu3-imgu 0 (5 pads, 5 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown crop:(0,0)/1920x1080 compose:(0,0)/1920x1080] <- "ipu3-imgu 0 input":0 [] pad1: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] <- "ipu3-imgu 0 parameters":0 [] pad2: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 0 output":0 [] pad3: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 0 viewfinder":0 [] pad4: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 0 3a stat":0 [] - entity 7: ipu3-imgu 1 (5 pads, 5 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown crop:(0,0)/1920x1080 compose:(0,0)/1920x1080] <- "ipu3-imgu 1 input":0 [] pad1: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] <- "ipu3-imgu 1 parameters":0 [] pad2: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 1 output":0 [] pad3: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 1 viewfinder":0 [] pad4: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 1 3a stat":0 [] - entity 17: ipu3-imgu 0 input (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Source -> "ipu3-imgu 0":0 [] - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video1 pad0: Source -> "ipu3-imgu 0":1 [] - entity 29: ipu3-imgu 0 output (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video2 pad0: Sink <- "ipu3-imgu 0":2 [] - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video3 pad0: Sink <- "ipu3-imgu 0":3 [] - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video4 pad0: Sink <- "ipu3-imgu 0":4 [] - entity 47: ipu3-imgu 1 input (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video5 pad0: Source -> "ipu3-imgu 1":0 [] - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video6 pad0: Source -> "ipu3-imgu 1":1 [] - entity 59: ipu3-imgu 1 output (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video7 pad0: Sink <- "ipu3-imgu 1":2 [] - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video8 pad0: Sink <- "ipu3-imgu 1":3 [] - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video9 pad0: Sink <- "ipu3-imgu 1":4 [] v4l2-compliance utility is built with Sakari's patches for meta data output support(rebased): <URL:https://patchwork.linuxtv.org/patch/43370/> <URL:https://patchwork.linuxtv.org/patch/43369/> The test (v4l2-compliance -m 0) passes without error, outputs are appended at the end of revision history. Note: 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be enabled prior to the test. 2. Stream tests are not performed since it requires pre-configuration for each case. =========== = history = =========== v7 update: 1. Add driver and uAPI documentation. Update based on v1 review from Tomasz, Hans, Sokari and Mauro: https://patchwork.kernel.org/patch/10465663/ https://patchwork.kernel.org/patch/10465665/ 2. Add dual pipe support which includes: - Extend current IMGU device to contain 2 subdevs and two groups of video nodes. - Add a v4l2 ctrl to allow user to specify the mode(video or still) of the pipe. 3. Kconfig - Restrict build for X86 arch to fix build error for ia64/sparc. (fatal error: asm/set_memory.h: No such file or directory) 4. ipu3-abi.h - Change __u32 to u32. - Use generic __attribute__((aligned(x))) format. (Mauro/Hans) - Split abi to 2 patches, one for register defines, enums, the other for structs. (Tomasz) 5. ipu3-mmu.c - Fix ipu3-mmu/dmamap exit functions. (Tomasz) (Port from https://chromium-review.googlesource.com/1084522) - Use free_page instead of kfree. (Tomasz) - document struct ipu3_mmu_info. - Fix copyright information. 6. ipu3-dmamap.c (Tomasz) - Update APIs based on v6 review. - Replace sizeof(struct page *) with sizeof(*pages). - Remove un-needed (WARN_ON(!dev)) inside void *ipu3_dmamap_alloc(). 7. ipu3.c (Tomasz) - imgu_video_nodes_init() Fix the missing call to ipu3_v4l2_unregister() in the error path of imgu_dummybufs_preallocate(). - imgu_queue_buffers() Evaluate loop condition explicitly for code clarity and simplicity. FW requires all output buffers to be queued at start, so adjust the order of buffer queuing accordingly. (bufix by Tianshu) - imgu_isr_threaded() Fix interrupt handler return value. (Port from https://chromium-review.googlesource.com/1088539) - Add back the buf_drain_wq from ("avoid sleep in wait_event condition")' (Port from https://chromium-review.googlesource.com/875420) 8. ipu3-v4l2.c - ipu3_v4l2_register(). (Tomasz) Split media initialization and registration, also change media device register/un-register order. - Fix v4l2-compliance fail on sub-devicef for VIDIOC_CREATE_BUFS and VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT. 9. ipu3-css.c, ipu3-css.h, ipu3-css-fw.h, ipu3-abi.h - Convert macros in structs to enums. (Tomasz) 10. ipu3-css-pool.c, ipu3-css-pool.h, ipu3.c - Document the structs. (Hans/Maruo) 11. ipu3-css-params.c - Fixup for noise reduction parameters processing. (bug fixing) version 6: - intel-ipu3.h uAPI Move out the definitions not used by user space. (suggested by Sakari) - ipu3-abi.h, ipu3-css-fw.h Clean up the header files. Remove enum type from ABI structs. - ipu3-css.h and ipu3-css.c Disable DVS support and remove related code. - ipu3-v4l2.c Fixes of v4l2_compliance test fails on ImgU sub-dev. - ipu3-css-params.c Refactor awb/awb_fr/af_ops_calc() functions. (Sakari) - Build mmu and dmamap driver as part of ImgU ko module; (Sakari) - Add "ipu3-imgu" prefix to media entity names; (Sakari) - Fix indentation and white space; (Sakari) - Rebase to kernel v4.16; - Use SPDX license identifiers in all drivers; (Sakari) - Internal fix and performance improvements such as: Stop fw gracefully during stream off. Enable irq only after start streaming to avoid unexpected interrupt. Use spinlock to protect IPU3_CSS_QUEUES access. Return NULL when dequeuing buffer before streaming. TODOs: - Documentation on ImgU driver programming interface to configure and enable ISP HW, which will include details on complete V4L2 Kernel driver interface and IO-Control parameters, except for the ISP internal algorithm and its parameters (which is Intel proprietary IP). version 5: - ipu3-css-pool.c/ipu3_css_pool_check(). add handling of the framenum wrap around case in ipu3_css_pool_check(). - ipu3.c, ipu3-v4l2.c, ipu3.h merge struct ipu3_mem2mem2_device into imgu_device and update the code accordingly. (Suggested by Sakari) - ipu3-mmu.c driver: use __get_free_page() for page-aligned allocations (Tomasz). optimize tlb invalidation by calling them at the end of map/unmap. (Tomasz). remove dependency on iommu. (Sakari) introduce few new functions from iommu.c. - ipu3-dmamap.c driver call mmu directly without IOMMU_SUPPORT (Sakari) update dmamap APIs. (Suggested by Tomasz) - ipu3_v4l2.c move g/s_selection callback to V4l2 sub-device (Sakari) remove colon from ImgU sub-device name. (Sakari) - ipu3-css-params.c fix indentation, 0-day scan warnings etc. - ipu3-css.c fix warning about NULL comparison. (Sakari) - intel-ipu3.h: remove redundant IPU3_ALIGN attribute (Sakari). fix up un-needed fields in struct ipu3_uapi_params (Sakari) re-order this to be 2nd in the patch set. - Makefile: remove Copyright header. (Sakari) - Internal fix: optimize shot-to-shot performance. update default white balance gains defined in ipu3-tables.c TODOs: - Documentation on ImgU driver programming interface to configure and enable ISP HW, which will include details on complete V4L2 Kernel driver interface and IO-Control parameters, except for the ISP internal algorithm and its parameters (which is Intel proprietary IP). - Review ipu3_css_pool_* group APIs usage. version 4: - Used V4L2_BUF_TYPE_META_OUTPUT for: - V4L2_META_FMT_IPU3_STAT_PARAMS - Used V4L2_BUF_TYPE_META_CAPTURE for: - V4L2_META_FMT_IPU3_STAT_3A - V4L2_META_FMT_IPU3_STAT_DVS - V4L2_META_FMT_IPU3_STAT_LACE - Supported v4l2 MPLANE format on video nodes. - ipu3-dmamap.c: Removed dma ops and dependencies on IOMMU_DMA lib. - ipu3-mmu.c: Restructured the driver. - intel-ipu3.h: Added __padding qualifier for uapi definitions. - Internal fix: power and performance related issues. - Fixed v4l2-compliance test. - Fixed build failure for x86 with 32bit config. version 3: - ipu3-mmu.c and ipu3-dmamap.c: Tomasz Figa reworked both drivers and updated related files. - ipu2-abi.h: update imgu_abi_binary_info ABI to support latest ipu3-fw.bin. use __packed qualifier on structs suggested by Sakari Ailus. - ipu3-css-fw.c/ipu3-css-fw.h: following fix were suggested by Tomasz Figa: remove pointer type in firmware blob structs. fix binary_header array in struct imgu_fw_header. fix calling ipu3_css_fw_show_binary() before proper checking. fix logic error for valid length checking of blob name. - ipu3-css-params.c/ipu3_css_scaler_get_exp(): use lib helper suggested by Andy Shevchenko. - ipu3-v4l2.c/ipu3_videoc_querycap(): fill device_caps fix suggested by Hans Verkuil. add VB2_DMABUF suggested by Tomasz Figa. - ipu3-css.c: increase IMGU freq from 300MHZ to 450MHZ (internal fix) - ipu3.c: use vb2_dma_sg_memop for the time being(internal fix). version 2: This version cherry-picked firmware ABI change and other fix in order to bring the code up-to-date with our internal release. I will go over the review comments in v1 and address them in v3 and future update. version 1: - Initial submission -------------------------------------------------------------------------------- v4l2-compliance test output: ./v4l2-compliance -m 0 v4l2-compliance SHA: 7aa151889ffe89b1cd94a8198b0caba1a8c70398, 64 bits Compliance test for device /dev/media0: Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Required ioctls: test MEDIA_IOC_DEVICE_INFO: OK Allow for multiple opens: test second /dev/media0 open: OK test MEDIA_IOC_DEVICE_INFO: OK test for unlimited opens: OK Media Controller ioctls: test MEDIA_IOC_G_TOPOLOGY: OK Entities: 12 Interfaces: 12 Pads: 20 Links: 22 test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK test MEDIA_IOC_SETUP_LINK: OK -------------------------------------------------------------------------------- Compliance test for device /dev/v4l-subdev0: Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x0300000d Type : V4L Sub-Device Entity Info: ID : 0x00000001 (1) Name : ipu3-imgu 0 Function : Video Statistics Pad 0x01000002 : 0: Sink Link 0x02000015: from remote pad 0x1000012 of entity 'ipu3-imgu 0 input': Data, Enabled Pad 0x01000003 : 1: Sink Link 0x0200001b: from remote pad 0x1000018 of entity 'ipu3-imgu 0 parameters': Data Pad 0x01000004 : 2: Source Link 0x02000021: to remote pad 0x100001e of entity 'ipu3-imgu 0 output': Data, Enabled Pad 0x01000005 : 3: Source Link 0x02000027: to remote pad 0x1000024 of entity 'ipu3-imgu 0 viewfinder': Data, Enabled Pad 0x01000006 : 4: Source Link 0x0200002d: to remote pad 0x100002a of entity 'ipu3-imgu 0 3a stat': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK Allow for multiple opens: test second /dev/v4l-subdev0 open: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Sub-Device ioctls (Sink Pad 0): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Sink Pad 1): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Source Pad 2): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Source Pad 3): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Source Pad 4): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK test VIDIOC_G/S_CTRL: OK test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 1 Private Controls: 1 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported) test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK (Not Supported) test VIDIOC_TRY_FMT: OK (Not Supported) test VIDIOC_S_FMT: OK (Not Supported) test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported) test VIDIOC_EXPBUF: OK (Not Supported) -------------------------------------------------------------------------------- Compliance test for device /dev/v4l-subdev1: Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x0300000f Type : V4L Sub-Device Entity Info: ID : 0x00000007 (7) Name : ipu3-imgu 1 Function : Video Statistics Pad 0x01000008 : 0: Sink Link 0x02000033: from remote pad 0x1000030 of entity 'ipu3-imgu 1 input': Data, Enabled Pad 0x01000009 : 1: Sink Link 0x02000039: from remote pad 0x1000036 of entity 'ipu3-imgu 1 parameters': Data Pad 0x0100000a : 2: Source Link 0x0200003f: to remote pad 0x100003c of entity 'ipu3-imgu 1 output': Data, Enabled Pad 0x0100000b : 3: Source Link 0x02000045: to remote pad 0x1000042 of entity 'ipu3-imgu 1 viewfinder': Data, Enabled Pad 0x0100000c : 4: Source Link 0x0200004b: to remote pad 0x1000048 of entity 'ipu3-imgu 1 3a stat': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK Allow for multiple opens: test second /dev/v4l-subdev1 open: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Sub-Device ioctls (Sink Pad 0): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Sink Pad 1): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Source Pad 2): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Source Pad 3): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Sub-Device ioctls (Source Pad 4): test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Try VIDIOC_SUBDEV_G/S_FMT: OK test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK (Not Supported) test Active VIDIOC_SUBDEV_G/S_FMT: OK test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK test VIDIOC_G/S_CTRL: OK test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 1 Private Controls: 1 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported) test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK (Not Supported) test VIDIOC_TRY_FMT: OK (Not Supported) test VIDIOC_S_FMT: OK (Not Supported) test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported) test VIDIOC_EXPBUF: OK (Not Supported) -------------------------------------------------------------------------------- Compliance test for device /dev/video0: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:input Driver version : 4.19.0 Capabilities : 0x84202000 Video Output Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04202000 Video Output Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000013 Type : V4L Video Entity Info: ID : 0x00000011 (17) Name : ipu3-imgu 0 input Function : V4L2 I/O Pad 0x01000012 : 0: Source Link 0x02000015: to remote pad 0x1000002 of entity 'ipu3-imgu 0': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video0 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 1 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls (Output 0): test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls (Output 0): test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK Codec ioctls (Output 0): test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls (Output 0): test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video1: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:parameters Driver version : 4.19.0 Capabilities : 0x8c200000 Metadata Output Streaming Extended Pix Format Device Capabilities Device Caps : 0x0c200000 Metadata Output Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000019 Type : V4L Video Entity Info: ID : 0x00000017 (23) Name : ipu3-imgu 0 parameters Function : V4L2 I/O Pad 0x01000018 : 0: Source Link 0x0200001b: to remote pad 0x1000003 of entity 'ipu3-imgu 0': Data Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video1 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video2: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:output Driver version : 4.19.0 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04201000 Video Capture Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x0300001f Type : V4L Video Entity Info: ID : 0x0000001d (29) Name : ipu3-imgu 0 output Function : V4L2 I/O Pad 0x0100001e : 0: Sink Link 0x02000021: from remote pad 0x1000004 of entity 'ipu3-imgu 0': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video2 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 1 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls (Input 0): test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls (Input 0): test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK Codec ioctls (Input 0): test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls (Input 0): test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video3: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:viewfinder Driver version : 4.19.0 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04201000 Video Capture Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000025 Type : V4L Video Entity Info: ID : 0x00000023 (35) Name : ipu3-imgu 0 viewfinder Function : V4L2 I/O Pad 0x01000024 : 0: Sink Link 0x02000027: from remote pad 0x1000005 of entity 'ipu3-imgu 0': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video3 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 1 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls (Input 0): test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls (Input 0): test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK Codec ioctls (Input 0): test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls (Input 0): test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video4: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:3a stat Driver version : 4.19.0 Capabilities : 0x84a00000 Metadata Capture Streaming Extended Pix Format Device Capabilities Device Caps : 0x04a00000 Metadata Capture Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x0300002b Type : V4L Video Entity Info: ID : 0x00000029 (41) Name : ipu3-imgu 0 3a stat Function : V4L2 I/O Pad 0x0100002a : 0: Sink Link 0x0200002d: from remote pad 0x1000006 of entity 'ipu3-imgu 0': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video4 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video5: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:input Driver version : 4.19.0 Capabilities : 0x84202000 Video Output Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04202000 Video Output Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000031 Type : V4L Video Entity Info: ID : 0x0000002f (47) Name : ipu3-imgu 1 input Function : V4L2 I/O Pad 0x01000030 : 0: Source Link 0x02000033: to remote pad 0x1000008 of entity 'ipu3-imgu 1': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video5 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 1 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls (Output 0): test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls (Output 0): test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK Codec ioctls (Output 0): test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls (Output 0): test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video6: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:parameters Driver version : 4.19.0 Capabilities : 0x8c200000 Metadata Output Streaming Extended Pix Format Device Capabilities Device Caps : 0x0c200000 Metadata Output Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000037 Type : V4L Video Entity Info: ID : 0x00000035 (53) Name : ipu3-imgu 1 parameters Function : V4L2 I/O Pad 0x01000036 : 0: Source Link 0x02000039: to remote pad 0x1000009 of entity 'ipu3-imgu 1': Data Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video6 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video7: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:output Driver version : 4.19.0 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04201000 Video Capture Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x0300003d Type : V4L Video Entity Info: ID : 0x0000003b (59) Name : ipu3-imgu 1 output Function : V4L2 I/O Pad 0x0100003c : 0: Sink Link 0x0200003f: from remote pad 0x100000a of entity 'ipu3-imgu 1': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video7 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 1 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls (Input 0): test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls (Input 0): test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK Codec ioctls (Input 0): test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls (Input 0): test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video8: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:viewfinder Driver version : 4.19.0 Capabilities : 0x84201000 Video Capture Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04201000 Video Capture Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000043 Type : V4L Video Entity Info: ID : 0x00000041 (65) Name : ipu3-imgu 1 viewfinder Function : V4L2 I/O Pad 0x01000042 : 0: Sink Link 0x02000045: from remote pad 0x100000b of entity 'ipu3-imgu 1': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video8 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 1 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls (Input 0): test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls (Input 0): test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK Codec ioctls (Input 0): test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls (Input 0): test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK -------------------------------------------------------------------------------- Compliance test for device /dev/video9: Driver Info: Driver name : ipu3-imgu Card type : ipu3-imgu Bus info : PCI:3a stat Driver version : 4.19.0 Capabilities : 0x84a00000 Metadata Capture Streaming Extended Pix Format Device Capabilities Device Caps : 0x04a00000 Metadata Capture Streaming Extended Pix Format Media Driver Info: Driver name : ipu3-imgu Model : ipu3-imgu Serial : Bus info : PCI:0000:00:05.0 Media version : 4.19.0 Hardware revision: 0x80862015 (2156273685) Driver version : 4.19.0 Interface Info: ID : 0x03000049 Type : V4L Video Entity Info: ID : 0x00000047 (71) Name : ipu3-imgu 1 3a stat Function : V4L2 I/O Pad 0x01000048 : 0: Sink Link 0x0200004b: from remote pad 0x100000c of entity 'ipu3-imgu 1': Data, Enabled Required ioctls: test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video9 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK Total: 597, Succeeded: 597, Failed: 0, Warnings: 0 -------------------------------------------------------------------------------- Thanks, Yong Cao,Bing Bu (1): intel-ipu3: add dual pipe support Rajmohan Mani (1): doc-rst: Add Intel IPU3 documentation Tomasz Figa (2): intel-ipu3: mmu: Implement driver intel-ipu3: Implement DMA mapping functions Yong Zhi (12): v4l: Add Intel IPU3 meta buffer formats v4l: Add Intel IPU3 meta data uAPI intel-ipu3: abi: Add register definitions and enum intel-ipu3: abi: Add structs intel-ipu3: css: Add dma buff pool utility functions intel-ipu3: css: Add support for firmware management intel-ipu3: css: Add static settings for image pipeline intel-ipu3: css: Compute and program ccs intel-ipu3: css: Initialize css hardware intel-ipu3: Add css pipeline programming intel-ipu3: Add v4l2 driver based on media framework intel-ipu3: Add imgu top level pci device driver Documentation/media/uapi/v4l/meta-formats.rst | 1 + .../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 181 + Documentation/media/v4l-drivers/index.rst | 1 + Documentation/media/v4l-drivers/ipu3.rst | 326 + drivers/media/pci/intel/ipu3/Kconfig | 16 + drivers/media/pci/intel/ipu3/Makefile | 12 + drivers/media/pci/intel/ipu3/ipu3-abi.h | 2011 ++++ drivers/media/pci/intel/ipu3/ipu3-css-fw.c | 265 + drivers/media/pci/intel/ipu3/ipu3-css-fw.h | 188 + drivers/media/pci/intel/ipu3/ipu3-css-params.c | 2936 ++++++ drivers/media/pci/intel/ipu3/ipu3-css-params.h | 28 + drivers/media/pci/intel/ipu3/ipu3-css-pool.c | 136 + drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 56 + drivers/media/pci/intel/ipu3/ipu3-css.c | 2407 +++++ drivers/media/pci/intel/ipu3/ipu3-css.h | 215 + drivers/media/pci/intel/ipu3/ipu3-dmamap.c | 270 + drivers/media/pci/intel/ipu3/ipu3-dmamap.h | 22 + drivers/media/pci/intel/ipu3/ipu3-mmu.c | 560 ++ drivers/media/pci/intel/ipu3/ipu3-mmu.h | 35 + drivers/media/pci/intel/ipu3/ipu3-tables.c | 9609 ++++++++++++++++++++ drivers/media/pci/intel/ipu3/ipu3-tables.h | 66 + drivers/media/pci/intel/ipu3/ipu3-v4l2.c | 1436 +++ drivers/media/pci/intel/ipu3/ipu3.c | 830 ++ drivers/media/pci/intel/ipu3/ipu3.h | 169 + drivers/media/v4l2-core/v4l2-ioctl.c | 2 + include/uapi/linux/intel-ipu3.h | 2827 ++++++ include/uapi/linux/videodev2.h | 4 + 27 files changed, 24609 insertions(+) create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst create mode 100644 Documentation/media/v4l-drivers/ipu3.rst create mode 100644 drivers/media/pci/intel/ipu3/ipu3-abi.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-tables.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-tables.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-v4l2.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h create mode 100644 include/uapi/linux/intel-ipu3.h