| Message ID | 20210215042741.28850-1-laurent.pinchart@ideasonboard.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | media: imx: Miscellaneous fixes and cleanups for i.MX7 | expand |
Hi Laurent, On 15.02.21 05:26, Laurent Pinchart wrote: > Hello, > > This large patch series is a collection of miscellaneous fixes, cleanups > and enhancements for the i.MX7 camera support. Most notably, it > implements support for the Media Controller API in the driver. > > Compared to v1, review comments have been taken into account, and the > patches have been rebased on top of the DT bindings and latest imx > changes as present in the linux-media tree. Patches 38/77, 39/77, 60/77 > and 61/77 are new. For additional information, please see individual > patches. > > I have successfully tested the code on an i.MX6ULL board (with an > MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an i.MX8MM > board (with an OV5640 sensor, and additional patches for i.MX8MM > support). First of all, thanks for the great work! I'm currently trying to get a setup with CSI + MIPI + ADV7280 working on an i.MX8MM system. Would you mind sharing the additional patches I need for this? I guess you are referring to patches for the MIPI CSIS PHY and the devicetree. Anything else, that I'm missing? Thanks Frieder
Hi Frieder, On Wed, Mar 17, 2021 at 07:04:19PM +0100, Frieder Schrempf wrote: > On 15.02.21 05:26, Laurent Pinchart wrote: > > Hello, > > > > This large patch series is a collection of miscellaneous fixes, cleanups > > and enhancements for the i.MX7 camera support. Most notably, it > > implements support for the Media Controller API in the driver. > > > > Compared to v1, review comments have been taken into account, and the > > patches have been rebased on top of the DT bindings and latest imx > > changes as present in the linux-media tree. Patches 38/77, 39/77, 60/77 > > and 61/77 are new. For additional information, please see individual > > patches. > > > > I have successfully tested the code on an i.MX6ULL board (with an > > MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an i.MX8MM > > board (with an OV5640 sensor, and additional patches for i.MX8MM > > support). > > First of all, thanks for the great work! You're welcome. > I'm currently trying to get a setup with CSI + MIPI + ADV7280 working on > an i.MX8MM system. Would you mind sharing the additional patches I need > for this? > > I guess you are referring to patches for the MIPI CSIS PHY and the > devicetree. Anything else, that I'm missing? I've pushed my work in progress patches to git://linuxtv.org/pinchartl/media.git imx/next I haven't tested this on mainline though, as the i.MX8MM board I'm using currently requires an NXP BSP. For the DT integration, here's what I have in imx8mm.dtsi: csi1_bridge: csi1_bridge@32e20000 { compatible = "fsl,imx8mm-csi", "fsl,imx7-csi"; reg = <0x32e20000 0x1000>; interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>; clocks = <&clk IMX8MM_CLK_DISP_AXI_ROOT>, <&clk IMX8MM_CLK_CSI1_ROOT>, <&clk IMX8MM_CLK_DISP_APB_ROOT>; clock-names = "axi", "mclk", "dcic"; power-domains = <&dispmix_pd>; status = "disabled"; port { csi_in: endpoint { remote-endpoint = <&mipi_csi_out>; }; }; }; mipi_csi: mipi_csi@32e30000 { compatible = "fsl,imx8mm-mipi-csi2", "fsl,imx7-mipi-csi2"; reg = <0x32e30000 0x1000>; interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>; clock-frequency = <333000000>; clocks = <&clk IMX8MM_CLK_DISP_APB_ROOT>, <&clk IMX8MM_CLK_CSI1_ROOT>, <&clk IMX8MM_CLK_CSI1_PHY_REF>, <&clk IMX8MM_CLK_DISP_AXI_ROOT>; clock-names = "pclk", "wrap", "phy", "axi"; power-domains = <&mipi_pd>; status = "disabled"; ports { #address-cells = <1>; #size-cells = <0>; port@0 { reg = <0>; }; port@1 { reg = <1>; mipi_csi_out: endpoint { remote-endpoint = <&csi_in>; }; }; }; }; Mainline seems to be missing the power domains, so you'll likely have to sort this out.
Hi Laurent, On 19.03.21 02:27, Laurent Pinchart wrote: > Hi Frieder, > > On Wed, Mar 17, 2021 at 07:04:19PM +0100, Frieder Schrempf wrote: >> On 15.02.21 05:26, Laurent Pinchart wrote: >>> Hello, >>> >>> This large patch series is a collection of miscellaneous fixes, cleanups >>> and enhancements for the i.MX7 camera support. Most notably, it >>> implements support for the Media Controller API in the driver. >>> >>> Compared to v1, review comments have been taken into account, and the >>> patches have been rebased on top of the DT bindings and latest imx >>> changes as present in the linux-media tree. Patches 38/77, 39/77, 60/77 >>> and 61/77 are new. For additional information, please see individual >>> patches. >>> >>> I have successfully tested the code on an i.MX6ULL board (with an >>> MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an i.MX8MM >>> board (with an OV5640 sensor, and additional patches for i.MX8MM >>> support). >> >> First of all, thanks for the great work! > > You're welcome. > >> I'm currently trying to get a setup with CSI + MIPI + ADV7280 working on >> an i.MX8MM system. Would you mind sharing the additional patches I need >> for this? >> >> I guess you are referring to patches for the MIPI CSIS PHY and the >> devicetree. Anything else, that I'm missing? > > I've pushed my work in progress patches to > > git://linuxtv.org/pinchartl/media.git imx/next > > I haven't tested this on mainline though, as the i.MX8MM board I'm using > currently requires an NXP BSP. > > For the DT integration, here's what I have in imx8mm.dtsi: > [...] > > Mainline seems to be missing the power domains, so you'll likely have to > sort this out. > Thanks a lot for the patches. I finally found some time to test this with our hardware. I'm working on v5.10 mainline and applied pending support for the power-domains from Lucas [1]. It doesn't look bad and I can bring up the media devices, etc. but up to now I couldn't get any image from the ADV7280A-M. This is the first time I'm working with the media/v4l subsystem, so I might be missing some trivial things. Also I'm not quite sure if the adv7180 driver is currently fully compatible. Anyway here are some things I noticed. Maybe you could have a quick look and see if you can gather something from that, which helps me to get this running? After enabling the link to the adv7180 with: media-ctl -l "'adv7180 1-0021':0 -> 'imx7-mipi-csis.0':0[1]" all the links look ok, but not all the parameters seem to be propagated up to the subdevs (see [2]). But I don't even know if this should happen automatically. After setting up the format like this, the subdev setup looks ok as far as I can judge (see [3]): media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x240 field:alternate]" media-ctl -V "'imx7-mipi-csis.0':1 [fmt:UYVY2X8/720x240 field:alternate]" Also there is one small issue with the adv7180 not implementing neither V4L2_CID_LINK_FREQ nor V4L2_CID_PIXEL_RATE. So v4l2_get_link_rate() fails, but as this seems to be only used to calculate hs_settle, I just hardcoded the hs_settle value for now. Finally I don't get any image from the analog converter. The stream seems to be setup correctly, but I don't receive any frames. Though I can see that there is data on the MIPI CSI lanes. I'm also not quite sure which value to use for the CSI clock-frequency. On the clock lane of the CSI interface I can measure something around 106 MHz coming from the ADV7280. Maybe you got some pointers for me to look at? Thanks Frieder [1] https://patchwork.kernel.org/project/linux-arm-kernel/cover/20201105174434.1817539-1-l.stach@pengutronix.de/ [2] https://paste.ee/p/HuIfu [3] https://paste.ee/p/os6dU
On 15.04.21 11:00, Frieder Schrempf wrote: > Hi Laurent, > > On 19.03.21 02:27, Laurent Pinchart wrote: >> Hi Frieder, >> >> On Wed, Mar 17, 2021 at 07:04:19PM +0100, Frieder Schrempf wrote: >>> On 15.02.21 05:26, Laurent Pinchart wrote: >>>> Hello, >>>> >>>> This large patch series is a collection of miscellaneous fixes, >>>> cleanups >>>> and enhancements for the i.MX7 camera support. Most notably, it >>>> implements support for the Media Controller API in the driver. >>>> >>>> Compared to v1, review comments have been taken into account, and the >>>> patches have been rebased on top of the DT bindings and latest imx >>>> changes as present in the linux-media tree. Patches 38/77, 39/77, 60/77 >>>> and 61/77 are new. For additional information, please see individual >>>> patches. >>>> >>>> I have successfully tested the code on an i.MX6ULL board (with an >>>> MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an >>>> i.MX8MM >>>> board (with an OV5640 sensor, and additional patches for i.MX8MM >>>> support). >>> >>> First of all, thanks for the great work! >> >> You're welcome. >> >>> I'm currently trying to get a setup with CSI + MIPI + ADV7280 working on >>> an i.MX8MM system. Would you mind sharing the additional patches I need >>> for this? >>> >>> I guess you are referring to patches for the MIPI CSIS PHY and the >>> devicetree. Anything else, that I'm missing? >> >> I've pushed my work in progress patches to >> >> git://linuxtv.org/pinchartl/media.git imx/next >> >> I haven't tested this on mainline though, as the i.MX8MM board I'm using >> currently requires an NXP BSP. >> >> For the DT integration, here's what I have in imx8mm.dtsi: >> > [...] >> >> Mainline seems to be missing the power domains, so you'll likely have to >> sort this out. >> > > Thanks a lot for the patches. I finally found some time to test this > with our hardware. I'm working on v5.10 mainline and applied pending > support for the power-domains from Lucas [1]. > > It doesn't look bad and I can bring up the media devices, etc. but up to > now I couldn't get any image from the ADV7280A-M. > > This is the first time I'm working with the media/v4l subsystem, so I > might be missing some trivial things. Also I'm not quite sure if the > adv7180 driver is currently fully compatible. > > Anyway here are some things I noticed. Maybe you could have a quick look > and see if you can gather something from that, which helps me to get > this running? > > After enabling the link to the adv7180 with: > > media-ctl -l "'adv7180 1-0021':0 -> 'imx7-mipi-csis.0':0[1]" > > all the links look ok, but not all the parameters seem to be propagated > up to the subdevs (see [2]). But I don't even know if this should happen > automatically. > > After setting up the format like this, the subdev setup looks ok as far > as I can judge (see [3]): > > media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x240 field:alternate]" > media-ctl -V "'imx7-mipi-csis.0':1 [fmt:UYVY2X8/720x240 field:alternate]" > > Also there is one small issue with the adv7180 not implementing neither > V4L2_CID_LINK_FREQ nor V4L2_CID_PIXEL_RATE. So v4l2_get_link_rate() > fails, but as this seems to be only used to calculate hs_settle, I just > hardcoded the hs_settle value for now. > > Finally I don't get any image from the analog converter. The stream > seems to be setup correctly, but I don't receive any frames. Though I > can see that there is data on the MIPI CSI lanes. I discovered, that when the stream is running and I toggle the CSITX_PWRDN bit in the ADV7280 (disable and the enable again the MIPI CSI TX), I receive a single frame from the camera, but nothing more until I restart the streaming and toggle the CSITX_PWRDN again. Also I'm not able to get any frame data if I try to get progressive frames from the ADV (setting field = none and fmt = UYVY2X8/720x480), or if I try to use PAL instead of NTSC. So at least I know now that my hardware seems to work in general. But I still hope to get this setup fully working... > > I'm also not quite sure which value to use for the CSI clock-frequency. > On the clock lane of the CSI interface I can measure something around > 106 MHz coming from the ADV7280. > > Maybe you got some pointers for me to look at? > > Thanks > Frieder > > [1] > https://patchwork.kernel.org/project/linux-arm-kernel/cover/20201105174434.1817539-1-l.stach@pengutronix.de/ > > [2] https://paste.ee/p/HuIfu > [3] https://paste.ee/p/os6dU
On 15.04.21 18:04, Frieder Schrempf wrote: > On 15.04.21 11:00, Frieder Schrempf wrote: >> Hi Laurent, >> >> On 19.03.21 02:27, Laurent Pinchart wrote: >>> Hi Frieder, >>> >>> On Wed, Mar 17, 2021 at 07:04:19PM +0100, Frieder Schrempf wrote: >>>> On 15.02.21 05:26, Laurent Pinchart wrote: >>>>> Hello, >>>>> >>>>> This large patch series is a collection of miscellaneous fixes, >>>>> cleanups >>>>> and enhancements for the i.MX7 camera support. Most notably, it >>>>> implements support for the Media Controller API in the driver. >>>>> >>>>> Compared to v1, review comments have been taken into account, and the >>>>> patches have been rebased on top of the DT bindings and latest imx >>>>> changes as present in the linux-media tree. Patches 38/77, 39/77, >>>>> 60/77 >>>>> and 61/77 are new. For additional information, please see individual >>>>> patches. >>>>> >>>>> I have successfully tested the code on an i.MX6ULL board (with an >>>>> MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an >>>>> i.MX8MM >>>>> board (with an OV5640 sensor, and additional patches for i.MX8MM >>>>> support). >>>> >>>> First of all, thanks for the great work! >>> >>> You're welcome. >>> >>>> I'm currently trying to get a setup with CSI + MIPI + ADV7280 >>>> working on >>>> an i.MX8MM system. Would you mind sharing the additional patches I need >>>> for this? >>>> >>>> I guess you are referring to patches for the MIPI CSIS PHY and the >>>> devicetree. Anything else, that I'm missing? >>> >>> I've pushed my work in progress patches to >>> >>> git://linuxtv.org/pinchartl/media.git imx/next >>> >>> I haven't tested this on mainline though, as the i.MX8MM board I'm using >>> currently requires an NXP BSP. >>> >>> For the DT integration, here's what I have in imx8mm.dtsi: >>> >> [...] >>> >>> Mainline seems to be missing the power domains, so you'll likely have to >>> sort this out. >>> >> >> Thanks a lot for the patches. I finally found some time to test this >> with our hardware. I'm working on v5.10 mainline and applied pending >> support for the power-domains from Lucas [1]. >> >> It doesn't look bad and I can bring up the media devices, etc. but up >> to now I couldn't get any image from the ADV7280A-M. >> >> This is the first time I'm working with the media/v4l subsystem, so I >> might be missing some trivial things. Also I'm not quite sure if the >> adv7180 driver is currently fully compatible. >> >> Anyway here are some things I noticed. Maybe you could have a quick >> look and see if you can gather something from that, which helps me to >> get this running? >> >> After enabling the link to the adv7180 with: >> >> media-ctl -l "'adv7180 1-0021':0 -> 'imx7-mipi-csis.0':0[1]" >> >> all the links look ok, but not all the parameters seem to be >> propagated up to the subdevs (see [2]). But I don't even know if this >> should happen automatically. >> >> After setting up the format like this, the subdev setup looks ok as >> far as I can judge (see [3]): >> >> media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x240 field:alternate]" >> media-ctl -V "'imx7-mipi-csis.0':1 [fmt:UYVY2X8/720x240 field:alternate]" >> >> Also there is one small issue with the adv7180 not implementing >> neither V4L2_CID_LINK_FREQ nor V4L2_CID_PIXEL_RATE. So >> v4l2_get_link_rate() fails, but as this seems to be only used to >> calculate hs_settle, I just hardcoded the hs_settle value for now. >> >> Finally I don't get any image from the analog converter. The stream >> seems to be setup correctly, but I don't receive any frames. Though I >> can see that there is data on the MIPI CSI lanes. > > I discovered, that when the stream is running and I toggle the > CSITX_PWRDN bit in the ADV7280 (disable and the enable again the MIPI > CSI TX), I receive a single frame from the camera, but nothing more > until I restart the streaming and toggle the CSITX_PWRDN again. > > Also I'm not able to get any frame data if I try to get progressive > frames from the ADV (setting field = none and fmt = UYVY2X8/720x480), or > if I try to use PAL instead of NTSC. > > So at least I know now that my hardware seems to work in general. But I > still hope to get this setup fully working... > Just to wrap this up: It seems like for YUV422 you need some "special" settings. First of all, just as with the RGB formats I needed to set MIPI_CSIS_ISPCFG_PIXEL_MODE_DUAL in the MIPI CSIS driver and BIT_TWO_8BIT_SENSOR in the CR3 register in the CSI driver. Without this the MIPI CSIS hardware will always run into an overflow condition as soon as some data has been received. Further I needed to set BIT_MIPI_DOUBLE_CMPNT in CR18 to get the colors right. No idea what it really does, but the RM mentions YUV422 in the description for this bit. With this setup (and some changes on the adv7180 driver) I can get proper images from the camera. Though there's one problem remaining related to synchronization. It looks like the ADV outputs some invalid data at the beginning of the stream which results in an arbitrary horizontal and vertical offset of the image. I guess that this could maybe be solved by implementing g_skip_frames() for the CSI driver, but I haven't tried that. If anyone is interested, here is my WIP branch: https://git.kontron-electronics.de/linux/linux/-/commits/v5.10-mx8mm-csi.
Hello, This large patch series is a collection of miscellaneous fixes, cleanups and enhancements for the i.MX7 camera support. Most notably, it implements support for the Media Controller API in the driver. Compared to v1, review comments have been taken into account, and the patches have been rebased on top of the DT bindings and latest imx changes as present in the linux-media tree. Patches 38/77, 39/77, 60/77 and 61/77 are new. For additional information, please see individual patches. I have successfully tested the code on an i.MX6ULL board (with an MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an i.MX8MM board (with an OV5640 sensor, and additional patches for i.MX8MM support). Laurent Pinchart (77): media: imx: Drop dependency on I2C media: imx: Move dependency on VIDEO_DEV to common Kconfig symbol media: imx: Drop manual dependency on VIDEO_IMX_MEDIA media: imx: Compile imx6-media-objs only for CONFIG_VIDEO_IMX_CSI media: imx: Set default sizes through macros in all drivers media: imx: utils: Add ability to filter pixel formats by mbus code media: imx: capture: Use dev_* instead of v4l2_* to log messages media: imx: capture: Use device name to construct bus_info media: imx: capture: Remove forward declaration of capture_qops media: imx: capture: Handle errors from v4l2_fh_open() media: imx: capture: Clean up capture_priv structure media: imx: capture: Remove capture_priv stop field media: imx: capture: Move queue and ctrl handler init to init function media: imx: capture: Initialize video_device programmatically media: imx: capture: Register the video device after completing init media: imx: capture: Store v4l2_pix_format in imx_media_video_dev media: imx: capture: Move default format init to a separate function media: imx: capture: Rename querycap handler to capture_querycap media: imx: capture: Rename ioctl operations with legacy prefix media: imx: capture: Add a mechanism to disable control inheritance media: imx: capture: Remove unneeded variable in __capture_legacy_try_fmt media: imx: capture: Pass v4l2_pix_format to __capture_legacy_try_fmt() media: imx: capture: Return -EPIPE from __capture_legacy_try_fmt() media: imx: capture: Extract format lookup from __capture_legacy_try_fmt media: imx: capture: Simplify capture_validate_fmt() implementation media: imx: capture: Simplify __capture_legacy_try_fmt() media: imx: capture: Decouple video node from source with MC-centric API media: imx: capture: Expose V4L2_CAP_IO_MC for the MC-centric API media: imx: imx7-media-csi: Disable legacy video node API media: imx: capture: Support creating immutable link to capture device media: imx: imx7-media-csi: Remove control handler media: imx: imx7-media-csi: Move (de)init from link setup to .s_stream() media: imx: imx7-media-csi: Create immutable link to capture device media: imx: imx7-media-csi: Replace CSICR*_RESET_VAL with values media: imx: imx7-media-csi: Tidy up register fields macros media: imx: imx7-media-csi: Reorganize code in sections media: imx: imx7-media-csi: Validate capture format in .link_validate() media: imx: imx7-media-csi: Fix source type identification media: imx: imx7-media-csi: Don't lock access to is_csi2 media: imx: imx7-media-csi: Rename imx7_csi_dma_start() to *_setup() media: imx: imx7-media-csi: Split imx7_csi_dma_stop() media: imx: imx7-media-csi: Move CSI configuration before source start media: imx: imx7-media-csi: Merge streaming_start() with csi_enable() media: imx: imx7-media-csi: Merge hw_reset() with init_interface() media: imx: imx7-media-csi: Set the MIPI data type based on the bus code media: imx: imx7-media-csi: Don't set the buffer stride when disabling media: imx: imx7-media-csi: Merge all config in imx7_csi_configure() media: imx: imx7-media-csi: Clear all configurable CSICR18 fields media: imx: imx7-media-csi: Set RFF burst type in imx7_csi_configure() media: imx: imx7-media-csi: Simplify imx7_csi_rx_fifo_clear() media: imx: imx7-media-csi: Don't double-enable the CSI media: imx: imx7-media-csi: Don't double-enable the RxFIFO media: imx: imx7-media-csi: Remove double reflash of DMA controller media: imx: imx7-media-csi: Don't enable SOF and EOF interrupts media: imx: imx7_media-csi: Add support for additional Bayer patterns media: v4l2-mc: Add link flags to v4l2_create_fwnode_links_to_pad() media: imx: imx7_media-csi: Create immutable link to source device dt-bindings: media: nxp,imx7-mipi-csi2: Drop the reset-names property dt-bindings: media: nxp,imx7-mipi-csi2: Drop fsl,csis-hs-settle property dt-bindings: media: nxp,imx7-mipi-csi2: Indent example with 4 spaces dt-bindings: media: nxp,imx7-mipi-csi2: Expand descriptions media: imx: imx7_mipi_csis: Acquire reset control without naming it media: imx: imx7_mipi_csis: Fix input size alignment media: imx: imx7_mipi_csis: Make source .s_power() optional media: imx: imx7_mipi_csis: Avoid double get of wrap clock media: imx: imx7_mipi_csis: Drop 10-bit YUV support media: imx: imx7_mipi_csis: Fix UYVY8 media bus format media: imx: imx7_mipi_csis: Inline mipi_csis_set_hsync_settle() media: imx: imx7_mipi_csis: Move link setup check out of locked section media: imx: imx7_mipi_csis: Calculate Ths_settle from source lane rate media: imx: imx7_mipi_csis: Turn register access macros into functions media: imx: imx7_mipi_csis: Fully initialize MIPI_CSIS_DPHYCTRL register media: imx: imx7_mipi_csis: Define macros for DPHY_BCTRL_L fields media: imx: imx7_mipi_csis: Make ISP registers macros take channel ID media: imx: imx7_mipi_csis: Rename register macros to match datasheet media: imx: imx7_mipi_csis: Use register macros in mipi_csis_dump_regs() media: imx: imx7_mipi_csis: Print shadow registers in mipi_csis_dump_regs() .../bindings/media/nxp,imx7-mipi-csi2.yaml | 94 +- drivers/media/v4l2-core/v4l2-mc.c | 6 +- drivers/staging/media/imx/Kconfig | 9 +- drivers/staging/media/imx/Makefile | 6 +- drivers/staging/media/imx/TODO | 9 +- drivers/staging/media/imx/imx-ic-prp.c | 4 +- drivers/staging/media/imx/imx-ic-prpencvf.c | 24 +- drivers/staging/media/imx/imx-media-capture.c | 685 +++++++---- .../staging/media/imx/imx-media-csc-scaler.c | 2 +- drivers/staging/media/imx/imx-media-csi.c | 33 +- .../staging/media/imx/imx-media-dev-common.c | 4 + drivers/staging/media/imx/imx-media-utils.c | 23 +- drivers/staging/media/imx/imx-media-vdic.c | 7 +- drivers/staging/media/imx/imx-media.h | 12 +- drivers/staging/media/imx/imx6-mipi-csi2.c | 6 +- drivers/staging/media/imx/imx7-media-csi.c | 1042 ++++++++--------- drivers/staging/media/imx/imx7-mipi-csis.c | 399 ++++--- include/media/v4l2-mc.h | 8 +- 18 files changed, 1275 insertions(+), 1098 deletions(-)