From patchwork Tue Jun 14 22:49:25 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Longerbeam X-Patchwork-Id: 9177099 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C7C856021C for ; Tue, 14 Jun 2016 22:51:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A76E828307 for ; Tue, 14 Jun 2016 22:51:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 676F22833B; Tue, 14 Jun 2016 22:51:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 426A0282F9 for ; Tue, 14 Jun 2016 22:51:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932254AbcFNWvf (ORCPT ); Tue, 14 Jun 2016 18:51:35 -0400 Received: from mail-pf0-f194.google.com ([209.85.192.194]:33535 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752986AbcFNWv3 (ORCPT ); Tue, 14 Jun 2016 18:51:29 -0400 Received: by mail-pf0-f194.google.com with SMTP id c74so307006pfb.0 for ; Tue, 14 Jun 2016 15:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HZtMFAl3pKOSz776tk5SrfSIMkPhd54x/dNcy5UJLbI=; b=Zp6nu6zLqthwQO82YZK5c2RDjhISpVxhW0rxBq5oWkTvbRYK6SAbPXyue/l5weiT6e AS7nPyedFm0VJHZRVeMnboQ/VI1AWzBIWZvK8DGDZG3bsQS1PAgUHY+sC04rHspzHR+z 86qP5OQQXneMNB/Fz0QjCw0XMl07oHp7ICU2firaOf0cQok8nmgtbE/Y0wIYYP86xo/A RfiinOKJRGeQTkpd4fbXHZSuee9q3Kaz0oUAS/6IV4OcRm2Q9Tq3mKkcqgb76H+UlbOo NlGVOM8I5KxXdyTsm6xcZT5BAavZ+TvD5R6nmc//W8qqe7stXk9nZByFHgmvmNAphsPN PvvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HZtMFAl3pKOSz776tk5SrfSIMkPhd54x/dNcy5UJLbI=; b=JKRMox+fG5FQpg/Jhnjzor5HjWFVO4qEGjIqS9JLJYaeTQ74uNs+QHXNmkPfxpNuEe SGBEtOaCBpPEaUHlWaLUkgjgma9Rswvxo0KBKjClTKuukdQUr6nVzrTSDVQ/pEGn0b8O WEmtbYDP8DLUhHDltgLGqRyZf0NGf1n0YLvezYWnn51V6gIRyn1zqEeertCju0iKqCbK kHSSHxxjH6USM3jyz9Kq7D4NX/9pbyuFXia6y2m6IR7JfUati2tWxgrWZMjMkglzb8xE EV/ZXOMH/o4uVu9M/mInKVVxElEbcL8GERsuDuJYJl+t65Kv7emlPxKJlAu4stz5Q7Fw yvUA== X-Gm-Message-State: ALyK8tLSqMMPxfZybQcJQT7Hf397tFu9+D8uhC6W2fj01NDRxWnKd8sXNu8+Lnbo478scA== X-Received: by 10.66.126.11 with SMTP id mu11mr104373pab.18.1465944686082; Tue, 14 Jun 2016 15:51:26 -0700 (PDT) Received: from mothership.sklembedded.com (c-73-241-2-21.hsd1.ca.comcast.net. [73.241.2.21]) by smtp.googlemail.com with ESMTPSA id v184sm47829999pfb.24.2016.06.14.15.51.24 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 14 Jun 2016 15:51:25 -0700 (PDT) From: Steve Longerbeam X-Google-Original-From: Steve Longerbeam To: linux-media@vger.kernel.org Cc: Steve Longerbeam , Dmitry Eremin-Solenikov , Jiada Wang , Vladimir Zapolskiy Subject: [PATCH 29/38] media: Add camera interface driver for i.MX5/6 Date: Tue, 14 Jun 2016 15:49:25 -0700 Message-Id: <1465944574-15745-30-git-send-email-steve_longerbeam@mentor.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1465944574-15745-1-git-send-email-steve_longerbeam@mentor.com> References: <1465944574-15745-1-git-send-email-steve_longerbeam@mentor.com> Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is a V4L2 camera interface driver for i.MX5/6. See Documentation/video4linux/imx_camera.txt and device tree binding documentation at Documentation/devicetree/bindings/media/imx.txt. Signed-off-by: Steve Longerbeam Signed-off-by: Dmitry Eremin-Solenikov Signed-off-by: Jiada Wang Signed-off-by: Vladimir Zapolskiy --- Documentation/devicetree/bindings/media/imx.txt | 426 ++++ Documentation/video4linux/imx_camera.txt | 243 ++ drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/imx/Kconfig | 23 + drivers/staging/media/imx/Makefile | 1 + drivers/staging/media/imx/capture/Kconfig | 3 + drivers/staging/media/imx/capture/Makefile | 5 + drivers/staging/media/imx/capture/imx-camif.c | 2496 +++++++++++++++++++++ drivers/staging/media/imx/capture/imx-camif.h | 281 +++ drivers/staging/media/imx/capture/imx-csi.c | 195 ++ drivers/staging/media/imx/capture/imx-ic-prpenc.c | 660 ++++++ drivers/staging/media/imx/capture/imx-of.c | 354 +++ drivers/staging/media/imx/capture/imx-of.h | 18 + drivers/staging/media/imx/capture/imx-smfc.c | 505 +++++ drivers/staging/media/imx/capture/imx-vdic.c | 994 ++++++++ include/media/imx.h | 15 + include/uapi/Kbuild | 1 + include/uapi/linux/v4l2-controls.h | 4 + include/uapi/media/Kbuild | 2 + include/uapi/media/imx.h | 22 + 21 files changed, 6251 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/imx.txt create mode 100644 Documentation/video4linux/imx_camera.txt create mode 100644 drivers/staging/media/imx/Kconfig create mode 100644 drivers/staging/media/imx/Makefile create mode 100644 drivers/staging/media/imx/capture/Kconfig create mode 100644 drivers/staging/media/imx/capture/Makefile create mode 100644 drivers/staging/media/imx/capture/imx-camif.c create mode 100644 drivers/staging/media/imx/capture/imx-camif.h create mode 100644 drivers/staging/media/imx/capture/imx-csi.c create mode 100644 drivers/staging/media/imx/capture/imx-ic-prpenc.c create mode 100644 drivers/staging/media/imx/capture/imx-of.c create mode 100644 drivers/staging/media/imx/capture/imx-of.h create mode 100644 drivers/staging/media/imx/capture/imx-smfc.c create mode 100644 drivers/staging/media/imx/capture/imx-vdic.c create mode 100644 include/media/imx.h create mode 100644 include/uapi/media/Kbuild create mode 100644 include/uapi/media/imx.h diff --git a/Documentation/devicetree/bindings/media/imx.txt b/Documentation/devicetree/bindings/media/imx.txt new file mode 100644 index 0000000..1f2ca4b --- /dev/null +++ b/Documentation/devicetree/bindings/media/imx.txt @@ -0,0 +1,426 @@ +Freescale i.MX Video Capture + +Video Capture node +------------------ + +This is the imx video capture host interface node. The host node is an IPU +client and uses the register-level primitives of the IPU, so it does +not require reg or interrupt properties. Only a compatible property +and a list of IPU CSI port phandles is required. + +Required properties: +- compatible : "fsl,imx-video-capture"; +- ports : a list of CSI port phandles this device will control + +Optional properties: +- fim : child node that sets boot-time behavior of the + Frame Interval Monitor; + +fim child node +-------------- + +This is an optional child node of the video capture node. It can +be used to modify the default control values for the video capture +Frame Interval Monitor. Refer to Documentation/video4linux/imx_camera.txt +for more info on the Frame Interval Monitor. + +Optional properties: +- enable : enable (1) or disable (0) the FIM; +- num-avg : how many frame intervals the FIM will average; +- num-skip : how many frames the FIM will skip after a video + capture restart before beginning to sample frame + intervals; +- tolerance-range : a range of tolerances for the averaged frame + interval error, specified as , in usec. + The FIM will signal a frame interval error if + min < error < max. If the max is <= min, then + tolerance range is disabled (interval error if + error > min). +- input-capture-channel: an input capture channel and channel flags, + specified as . The channel number + must be 0 or 1. The flags can be + IRQ_TYPE_EDGE_RISING, IRQ_TYPE_EDGE_FALLING, or + IRQ_TYPE_EDGE_BOTH, and specify which input + capture signal edge will trigger the event. If + an input capture channel is specified, the FIM + will use this method to measure frame intervals + instead of via the EOF interrupt. The input capture + method is much preferred over EOF as it is not + subject to interrupt latency errors. However it + requires routing the VSYNC or FIELD output + signals of the camera sensor to one of the + i.MX input capture pads (SD1_DAT0, SD1_DAT1), + which also gives up support for SD1. + + +mipi_csi2 node +-------------- + +This is the device node for the MIPI CSI-2 Receiver, required for MIPI +CSI-2 sensors. + +Required properties: +- compatible : "fsl,imx-mipi-csi2"; +- reg : physical base address and length of the register set; +- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx + (the DPHY clock), video_27m, and eim_sel; +- clock-names : must contain "dphy_clk", "cfg_clk", "pix_clk"; + +Optional properties: +- interrupts : must contain two level-triggered interrupts, + in order: 100 and 101; + + +Device tree nodes of the image sensors' controlled directly by the imx +camera host interface driver must be child nodes of their corresponding +I2C bus controller node. The data link of these image sensors must be +specified using the common video interfaces bindings, defined in +video-interfaces.txt. + +Video capture is supported with the following imx-based reference +platforms: + + +SabreLite with OV5642 +--------------------- + +The OV5642 module is connected to the parallel bus input on the internal +video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2, so the ov5642 +sensor node must be a child of i2c2. + +OV5642 Required properties: +- compatible : "ovti,ov5642"; +- clocks : the OV5642 system clock (cko2, 200); +- clock-names : must be "xclk"; +- reg : must be 0x3c; +- xclk : the system clock frequency, must be 24000000; +- reset-gpios : must be <&gpio1 8 0>; +- pwdn-gpios : must be <&gpio1 6 0>; + +OV5642 Endpoint Required properties: +- remote-endpoint : must connect to parallel sensor interface input endpoint + on ipu1_csi0 video mux (ipu1_csi0_mux_from_parallel_sensor). +- bus-width : must be 8; +- hsync-active : must be 1; +- vsync-active : must be 1; + +The following is an example devicetree video capture configuration for +SabreLite: + +/ { + ipucap0: ipucap@0 { + compatible = "fsl,imx-video-capture"; + #address-cells = <1>; + #size-cells = <0>; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ipu1_csi0>; + ports = <&ipu1_csi0>; + status = "okay"; + }; +}; + +&ipu1_csi0_from_ipu1_csi0_mux { + bus-width = <8>; + data-shift = <12>; /* Lines 19:12 used */ + hsync-active = <1>; + vync-active = <1>; +}; + +&ipu1_csi0_mux_from_parallel_sensor { + remote-endpoint = <&ov5642_to_ipu1_csi0_mux>; +}; + +&ipu1_csi0_mux { + status = "okay"; +}; + +&i2c2 { + camera: ov5642@3c { + compatible = "ovti,ov5642"; + clocks = <&clks 200>; + clock-names = "xclk"; + reg = <0x3c>; + xclk = <24000000>; + reset-gpios = <&gpio1 8 0>; + pwdn-gpios = <&gpio1 6 0>; + gp-gpios = <&gpio1 16 0>; + + port { + ov5642_to_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>; + bus-width = <8>; + hsync-active = <1>; + vsync-active = <1>; + }; + }; + }; +}; + + +SabreAuto with ADV7180 +---------------------- + +On the SabreAuto, an on-board ADV7180 SD decoder is connected to the +parallel bus input on the internal video mux to IPU1 CSI0. + +Two analog video inputs are routed to the ADV7180 on the SabreAuto, +composite on Ain1, and composite on Ain3. Those inputs are defined +via inputs and input-names properties under the ipu1_csi0_mux parallel +sensor input endpoint (ipu1_csi0_mux_from_parallel_sensor). + +Regulators and port expanders are required for the ADV7180 (power pin +is via port expander gpio on i2c3). The reset pin to the port expander +chip (MAX7310) is controlled by a gpio, so a reset-gpios property must +be defined under the port expander node to control it. + +The sabreauto uses a steering pin to select between the SDA signal on +i2c3 bus, and a data-in pin for an SPI NOR chip. i2cmux can be used to +control this steering pin. Idle state of the i2cmux selects SPI NOR. +This is not classic way to use i2cmux, since one side of the mux selects +something other than an i2c bus, but it works and is probably the cleanest +solution. Note that if one thread is attempting to access SPI NOR while +another thread is accessing i2c3, the SPI NOR access will fail since the +i2cmux has selected the SDA pin rather than SPI NOR data-in. This couldn't +be avoided in any case, the board is not designed to allow concurrent +i2c3 and SPI NOR functions (and the default device-tree does not enable +SPI NOR anyway). + +Endpoint ipu1_csi0_mux_from_parallel_sensor Optional Properties: +- inputs : list of input mux values, must be 0x00 followed by + 0x02 on SabreAuto; +- input-names : names of the inputs; + +ADV7180 Required properties: +- compatible : "adi,adv7180"; +- reg : must be 0x21; + +ADV7180 Optional properties: +- DOVDD-supply : DOVDD regulator supply; +- AVDD-supply : AVDD regulator supply; +- DVDD-supply : DVDD regulator supply; +- PVDD-supply : PVDD regulator supply; +- pwdn-gpio : gpio to control ADV7180 power pin, must be + <&port_exp_b 2 0> on SabreAuto; +- interrupts : interrupt from ADV7180, must be <27 0x8> on SabreAuto; +- interrupt-parent : must be <&gpio1> on SabreAuto; + +ADV7180 Endpoint Required properties: +- remote-endpoint : must connect to parallel sensor interface input endpoint + on ipu1_csi0 video mux (ipu1_csi0_mux_from_parallel_sensor). +- bus-width : must be 8; + + +The following is an example devicetree video capture configuration for +SabreAuto: + +/ { + i2cmux { + i2c@1 { + #address-cells = <1>; + #size-cells = <0>; + reg = <1>; + + camera: adv7180@21 { + compatible = "adi,adv7180"; + reg = <0x21>; + pwdn-gpio = <&port_exp_b 2 0>; + interrupt-parent = <&gpio1>; + interrupts = <27 0x8>; + + port { + adv7180_to_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>; + bus-width = <8>; + }; + }; + }; + + port_exp_b: gpio_pca953x@32 { + compatible = "maxim,max7310"; + gpio-controller; + #gpio-cells = <2>; + reg = <0x32>; + reset-gpios = <&gpio1 15 GPIO_ACTIVE_LOW>; + }; + + }; + }; + + ipucap0: ipucap@0 { + compatible = "fsl,imx-video-capture"; + #address-cells = <1>; + #size-cells = <0>; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ipu1_csi0>; + ports = <&ipu1_csi0>; + status = "okay"; + + fim { + enable = <1>; + tolerance-range = <20 0>; + num-avg = <1>; + input-capture-channel = <0 IRQ_TYPE_EDGE_RISING>; + }; + }; +}; + +&ipu1_csi0_from_ipu1_csi0_mux { + bus-width = <8>; +}; + +&ipu1_csi0_mux_from_parallel_sensor { + remote-endpoint = <&adv7180_to_ipu1_csi0_mux>; + inputs = <0x00 0x02>; + input-names = "ADV7180 Composite on Ain1", "ADV7180 Composite on Ain3"; +}; + +&ipu1_csi0_mux { + status = "okay"; +}; + +/* input capture requires the input capture pin */ +&gpt { + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_gpt_input_capture0>; +}; + +/* enabling input capture requires disabling SDHC1 */ +&usdhc1 { + status = "disabled"; +}; + + + +SabreSD Quad with OV5642 and MIPI CSI-2 OV5640 +---------------------------------------------- + +On the imx6q SabreSD, two camera sensors are supported: a parallel interface +OV5642 on IPU1 CSI0, and a MIPI CSI-2 OV5640 on IPU1 CSI1 on MIPI virtual +channel 1. The OV5642 connects to i2c bus 1 (i2c1) and the OV5640 to i2c +bus 2 (i2c2). + +The mipi_csi2 receiver node must be enabled and its input endpoint connected +via remote-endpoint to the OV5640 MIPI CSI-2 endpoint. + +OV5642 properties are as described above on SabreLite. + +OV5640 Required properties: +- compatible : "ovti,ov5640_mipi"; +- clocks : the OV5640 system clock (cko, 201); +- clock-names : must be "xclk"; +- reg : must be 0x3c; +- xclk : the system clock frequency, must be 24000000; +- reset-gpios : must be <&gpio1 20 1>; +- pwdn-gpios : must be <&gpio1 19 0>; + +OV5640 Optional properties: +- DOVDD-supply : DOVDD regulator supply; +- AVDD-supply : AVDD regulator supply; +- DVDD-supply : DVDD regulator supply; + +OV5640 MIPI CSI-2 Endpoint Required properties: +- remote-endpoint : must connect to mipi_csi receiver input endpoint + (mipi_csi_from_mipi_sensor). +- reg : must be 1; /* virtual channel 1 */ +- data-lanes : must be <0 1>; +- clock-lanes : must be <2>; + + +The following is an example devicetree video capture configuration for +SabreSD: + +/ { + ipucap0: ipucap@0 { + compatible = "fsl,imx-video-capture"; + #address-cells = <1>; + #size-cells = <0>; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ipu1_csi0>; + ports = <&ipu1_csi0>, <&ipu1_csi1>; + status = "okay"; + }; +}; + +&i2c1 { + camera: ov5642@3c { + compatible = "ovti,ov5642"; + clocks = <&clks 201>; + clock-names = "xclk"; + reg = <0x3c>; + xclk = <24000000>; + DOVDD-supply = <&vgen4_reg>; /* 1.8v */ + AVDD-supply = <&vgen5_reg>; /* 2.8v, rev C board is VGEN3 + rev B board is VGEN5 */ + DVDD-supply = <&vgen2_reg>; /* 1.5v*/ + pwdn-gpios = <&gpio1 16 GPIO_ACTIVE_LOW>; /* SD1_DAT0 */ + reset-gpios = <&gpio1 17 GPIO_ACTIVE_HIGH>; /* SD1_DAT1 */ + + port { + ov5642_to_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>; + bus-width = <8>; + hsync-active = <1>; + vsync-active = <1>; + }; + }; + }; +}; + +&i2c2 { + mipi_camera: ov5640@3c { + compatible = "ovti,ov5640_mipi"; + reg = <0x3c>; + clocks = <&clks 201>; + clock-names = "xclk"; + xclk = <24000000>; + DOVDD-supply = <&vgen4_reg>; /* 1.8v */ + AVDD-supply = <&vgen5_reg>; /* 2.8v, rev C board is VGEN3 + rev B board is VGEN5 */ + DVDD-supply = <&vgen2_reg>; /* 1.5v*/ + pwdn-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>; /* SD1_DAT2 */ + reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>; /* SD1_CLK */ + + port { + #address-cells = <1>; + #size-cells = <0>; + + ov5640_to_mipi_csi: endpoint@1 { + reg = <1>; /* virtual channel 1 */ + remote-endpoint = <&mipi_csi_from_mipi_sensor>; + data-lanes = <0 1>; + clock-lanes = <2>; + }; + }; + }; +}; + +ipu1_csi0_from_ipu1_csi0_mux { + bus-width = <8>; + data-shift = <12>; /* Lines 19:12 used */ + hsync-active = <1>; + vsync-active = <1>; +}; + +&ipu1_csi0_mux_from_parallel_sensor { + remote-endpoint = <&ov5642_to_ipu1_csi0_mux>; +}; + +&ipu1_csi0_mux { + status = "okay"; +}; + +&mipi_csi { + status = "okay"; +}; + +/* Incoming port from sensor */ +&mipi_csi_from_mipi_sensor { + remote-endpoint = <&ov5640_to_mipi_csi>; + data-lanes = <0 1>; + clock-lanes = <2>; +}; + +&ipu1_csi1_from_mipi_vc1 { + data-lanes = <0 1>; + clock-lanes = <2>; +}; diff --git a/Documentation/video4linux/imx_camera.txt b/Documentation/video4linux/imx_camera.txt new file mode 100644 index 0000000..1d391c2 --- /dev/null +++ b/Documentation/video4linux/imx_camera.txt @@ -0,0 +1,243 @@ + i.MX Video Capture Driver + ========================== + +Introduction +------------ + +The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which +handles the flow of image frames to and from capture devices and +display devices. + +For image capture, the IPU contains the following subunits: + +- Image DMA Controller (IDMAC) +- Camera Serial Interface (CSI) +- Image Converter (IC) +- Sensor Multi-FIFO Controller (SMFC) +- Image Rotator (IRT) +- Video De-Interlace Controller (VDIC) + +The IDMAC is the DMA controller for transfer of image frames to and from +memory. Various dedicated DMA channels exist for both video capture and +display paths. + +The CSI is the frontend capture unit that interfaces directly with +capture devices over Parallel, BT.656, and MIPI CSI-2 busses. + +The IC handles color-space conversion, resizing, and rotation +operations. + +The SMFC is used to send image frames directly to memory, bypassing the +IC. The SMFC is used when no color-space conversion or resizing is +required, i.e. the requested V4L2 formats and color-space are identical +to raw frames from the capture device. + +The IRT carries out 90 and 270 degree image rotation operations. + +Finally, the VDIC handles the conversion of interlaced video to +progressive, with support for different motion compensation modes (low +and high). + +For more info, refer to the latest versions of the i.MX5/6 reference +manuals listed under References. + + +Features +-------- + +Some of the features of this driver include: + +- Supports parallel, BT.565, and MIPI CSI-2 interfaces. + +- Multiple subdev sensors can be registered and controlled by a single + interface driver instance. Input enumeration will list every registered + sensor's inputs and input names, and setting an input will switch to + a different sensor if the input index is handled by a different sensor. + +- Simultaneous streaming from two separate sensors is possible with two + interface driver instances, each instance controlling a different + sensor. This is currently possible with the SabreSD reference board + with OV5642 and MIPI CSI-2 OV5640 sensors. + +- Scaling, color-space conversion, and image rotation. + +- Many pixel formats supported (RGB, packed and planar YUV, partial + planar YUV). + +- Full device-tree support using OF graph bindings. + +- Analog decoder input video source hot-swap support (during streaming) + via decoder status change subdev notification. + +- MMAP, USERPTR, and DMABUF importer/exporter buffers supported. + +- Motion compensated de-interlacing using the VDIC, with three + motion compensation modes: low, medium, and high motion. The mode is + specified with a custom control. + +- Includes a Frame Interval Monitor (FIM) that can correct vertical sync + problems with the ADV718x video decoders. See below for a description + of the FIM. + + +Usage Notes +----------- + +The i.MX capture driver is a standardized driver that supports the +following community V4L2 tools: + +- v4l2-ctl +- v4l2-cap +- v4l2src gstreamer plugin + + +The following platforms have been tested: + + +SabreLite with parallel-interface OV5642 +---------------------------------------- + +This platform requires the OmniVision OV5642 module with a parallel +camera interface from Boundary Devices for the SabreLite +(http://boundarydevices.com/products/nit6x_5mp/). + +There is a pin conflict between OV5642 and ethernet devices on this +platform, so by default video capture is disabled in the device tree. To +enable video capture, edit arch/arm/boot/dts/imx6qdl-sabrelite.dtsi and +uncomment the macro __OV5642_CAPTURE__. + + +SabreAuto with ADV7180 decoder +------------------------------ + +This platform accepts Composite Video analog inputs on Ain1 (connector +J42) and Ain3 (connector J43). + +To switch to Ain1: + +# v4l2-ctl -i0 + +To switch to Ain3: + +# v4l2-ctl -i1 + + +Frame Interval Monitor +---------------------- + +The adv718x decoders can occasionally send corrupt fields during +NTSC/PAL signal re-sync (too little or too many video lines). When +this happens, the IPU triggers a mechanism to re-establish vertical +sync by adding 1 dummy line every frame, which causes a rolling effect +from image to image, and can last a long time before a stable image is +recovered. Or sometimes the mechanism doesn't work at all, causing a +permanent split image (one frame contains lines from two consecutive +captured images). + +From experiment it was found that during image rolling, the frame +intervals (elapsed time between two EOF's) drop below the nominal +value for the current standard, by about one frame time (60 usec), +and remain at that value until rolling stops. + +While the reason for this observation isn't known (the IPU dummy +line mechanism should show an increase in the intervals by 1 line +time every frame, not a fixed value), we can use it to detect the +corrupt fields using a frame interval monitor. If the FIM detects a +bad frame interval, the camera interface driver restarts IPU capture +which corrects the rolling/split image. + +Custom controls exist to tweak some dials for FIM. If one of these +controls is changed during streaming, the FIM will be reset and will +continue at the new settings. + +- V4L2_CID_IMX_FIM_ENABLE + +Enable/disable the FIM. + +- V4L2_CID_IMX_FIM_NUM + +How many frame interval errors to average before comparing against the nominal +frame interval reported by the sensor. This can reduce noise from interrupt +latency. + +- V4L2_CID_IMX_FIM_TOLERANCE_MIN + +If the averaged intervals fall outside nominal by this amount, in +microseconds, streaming will be restarted. + +- V4L2_CID_IMX_FIM_TOLERANCE_MAX + +If any interval errors are higher than this value, those error samples +are discarded and do not enter into the average. This can be used to +discard really high interval errors that might be due to very high +system load, causing excessive interrupt latencies. + +- V4L2_CID_IMX_FIM_NUM_SKIP + +How many frames to skip after a FIM reset or stream restart before +FIM begins to average intervals. It has been found that there are +always a few bad frame intervals after stream restart, so this is +used to skip those frames to prevent endless restarts. + +Finally, all the defaults for these controls can be modified via a +device tree child node of the capture node, see +Documentation/devicetree/bindings/media/imx.txt. + + +SabreSD with MIPI CSI-2 OV5640 +------------------------------ + +The default device tree for SabreSD includes endpoints for both the +parallel OV5642 and the MIPI CSI-2 OV5640, but as of this writing only +the MIPI CSI-2 OV5640 has been tested. The OV5640 module connects to +MIPI connector J5 (sorry I don't have the compatible module part number +or URL). + +Inputs are registered for both the OV5642 and OV5640, and by default the +OV5642 is selected. To switch to the OV5640: + +# v4l2-ctl -i1 + + +Known Issues +------------ + +1. When using 90 or 270 degree rotation control at capture resolutions + near the IC resizer limit of 1024x1024, and combined with planar + pixel formats (YUV420, YUV422p), frame capture will often fail with + no end-of-frame interrupts from the IDMAC channel. To work around + this, use lower resolution and/or packed formats (YUYV, RGB3, etc.) + when 90 or 270 rotations are needed. + +2. Simple IDMAC interleaving using the ILO field in the IDMAC cpmem + doesn't work when combined with the 16-bit planar pixel formats + (YUV422P and NV16). This looks like a silicon bug, and there is + no satisfactory replies to queries about it from Freescale. So + the driver works around the issue by forcing the format to the + 12-bit planar versions (YUV420 and NV12) when simple interleaving + is used and the sensor sends interlaced fields (ADV718x). Another + option to workaround the issue is to use motion compensation when + combined with YUV422P or NV16. + +File list +--------- + +drivers/staging/media/imx/capture/ +include/media/imx.h +include/uapi/media/imx.h + +References +---------- + +[1] "i.MX 6Dual/6Quad Applications Processor Reference Manual" +[2] "i.MX 6Solo/6DualLite Applications Processor Reference Manual" + + +Authors +------- +Steve Longerbeam +Dmitry Eremin-Solenikov +Jiada Wang +Vladimir Zapolskiy + +Copyright (C) 2012-2016 Mentor Graphics Inc. diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig index ee91868..1ebcd1d 100644 --- a/drivers/staging/media/Kconfig +++ b/drivers/staging/media/Kconfig @@ -25,6 +25,8 @@ source "drivers/staging/media/cxd2099/Kconfig" source "drivers/staging/media/davinci_vpfe/Kconfig" +source "drivers/staging/media/imx/Kconfig" + source "drivers/staging/media/omap4iss/Kconfig" source "drivers/staging/media/tw686x-kh/Kconfig" diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile index 8c05d0a..9f00aea 100644 --- a/drivers/staging/media/Makefile +++ b/drivers/staging/media/Makefile @@ -1,5 +1,6 @@ obj-$(CONFIG_I2C_BCM2048) += bcm2048/ obj-$(CONFIG_DVB_CXD2099) += cxd2099/ +obj-$(CONFIG_VIDEO_IMX) += imx/ obj-$(CONFIG_LIRC_STAGING) += lirc/ obj-$(CONFIG_VIDEO_DM365_VPFE) += davinci_vpfe/ obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/ diff --git a/drivers/staging/media/imx/Kconfig b/drivers/staging/media/imx/Kconfig new file mode 100644 index 0000000..65e1645 --- /dev/null +++ b/drivers/staging/media/imx/Kconfig @@ -0,0 +1,23 @@ +config VIDEO_IMX + tristate "i.MX5/6 V4L2 devices" + depends on VIDEO_V4L2 && ARCH_MXC && IMX_IPUV3_CORE + default y + ---help--- + Say yes here to enable support for video4linux drivers for + the i.MX5/6 SOC. + +config VIDEO_IMX_CAMERA + tristate "i.MX5/6 Camera Interface driver" + depends on VIDEO_IMX && VIDEO_DEV && I2C + select VIDEOBUF2_DMA_CONTIG + default y + ---help--- + A video4linux capture driver for i.MX5/6 SOC. Some of the + features of this driver include MIPI CSI-2 sensor support, + hardware scaling, colorspace conversion, and rotation, + simultaneous capture from separate sensors, dmabuf + importer/exporter, and full devicetree support. + +if VIDEO_IMX_CAMERA +source "drivers/staging/media/imx/capture/Kconfig" +endif diff --git a/drivers/staging/media/imx/Makefile b/drivers/staging/media/imx/Makefile new file mode 100644 index 0000000..7c97629 --- /dev/null +++ b/drivers/staging/media/imx/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_VIDEO_IMX_CAMERA) += capture/ diff --git a/drivers/staging/media/imx/capture/Kconfig b/drivers/staging/media/imx/capture/Kconfig new file mode 100644 index 0000000..ee2cbab --- /dev/null +++ b/drivers/staging/media/imx/capture/Kconfig @@ -0,0 +1,3 @@ +menu "i.MX5/6 Camera Sub devices" + +endmenu diff --git a/drivers/staging/media/imx/capture/Makefile b/drivers/staging/media/imx/capture/Makefile new file mode 100644 index 0000000..5c965f9 --- /dev/null +++ b/drivers/staging/media/imx/capture/Makefile @@ -0,0 +1,5 @@ +imx-camera-objs := imx-camif.o imx-ic-prpenc.o imx-of.o \ + imx-smfc.o imx-vdic.o + +obj-$(CONFIG_VIDEO_IMX_CAMERA) += imx-camera.o +obj-$(CONFIG_VIDEO_IMX_CAMERA) += imx-csi.o diff --git a/drivers/staging/media/imx/capture/imx-camif.c b/drivers/staging/media/imx/capture/imx-camif.c new file mode 100644 index 0000000..0276426 --- /dev/null +++ b/drivers/staging/media/imx/capture/imx-camif.c @@ -0,0 +1,2496 @@ +/* + * Video Camera Capture driver for Freescale i.MX5/6 SOC + * + * Copyright (c) 2012-2016 Mentor Graphics Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include