From patchwork Tue Apr 16 20:11:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 13632535 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 090D213A41B for ; Tue, 16 Apr 2024 20:11:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713298316; cv=none; b=rT/SnwFyjs5xJtepbIGS28LIMlqlDZeoljgka+Q7OfikHkVJQcIqMD7xW1xbNJg2J7dYuZX+KHxaI4ycoNxq5f1D4ruYK8Y0xJA23wiurAUY9wSo5iWF9kH8/wAAoR03qmWtMNkn2JX7WzrnhSXo4NFNTCOsz5pJ4YMFF03w/D4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713298316; c=relaxed/simple; bh=Lm+FcoaXCc0TSjrR8EiUYPIToTmBGf1IZSzkw7ShqeQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CEi83nbNdAFIsPwil5uKpG1xjVfdHx5KYmk9PpDeiHLqBuh/Dbc1jKtTox/W1K7HnM67MH09fC8F4x/RMpPQXmXt8VlnLYNNhDnArcdF6qGw4Vwg4y9MHZbQQf/pnI+nOVg/BN4URt38ELNsfmv665bmbbtG6cNinaVqvlegQ8I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=IeF6sboe; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="IeF6sboe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1713298315; x=1744834315; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Lm+FcoaXCc0TSjrR8EiUYPIToTmBGf1IZSzkw7ShqeQ=; b=IeF6sboeU9W/sz1D9/4vDhqO5qIiMQUqufqzXP0QDCWpDuMm6ZnnqGed jmhc3rAZ3rUTYawYefOcBth9vkYit0s3iZCCmVKMntDkc73reYnpFax9C TO40AeREx+MM+xxEFg8BFd1DoBP0pJ4k+3By8rj7wVbBsjFfwAf3aZZSr pQQh1ahQwy38BxYuOMgd+hSY7QDGizLnxOHV/y7gnahwIEmFl+V22/yyY askyF5Mt4BCVCpoJSztgEcTz9ae2J2PlQ2XJHblQdPV2K5VEmDsEYKphv x5q/s7Lyyac8HCADH6qmNn3+QVpjcqpc+vB+R1WYZTN3e0ymZ5DLAXsig A==; X-CSE-ConnectionGUID: FfkKhrP9RMGlF6l49LnSbg== X-CSE-MsgGUID: 5CEmrvACSmyZXB4hoLGxLw== X-IronPort-AV: E=McAfee;i="6600,9927,11046"; a="8877624" X-IronPort-AV: E=Sophos;i="6.07,207,1708416000"; d="scan'208";a="8877624" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Apr 2024 13:11:54 -0700 X-CSE-ConnectionGUID: WiraDkxvTjKkJFFfNIBtgQ== X-CSE-MsgGUID: evwr1Q7XREeBgmTcuxnE2Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,207,1708416000"; d="scan'208";a="22447939" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Apr 2024 13:11:49 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 296D31204F6; Tue, 16 Apr 2024 23:11:45 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: bingbu.cao@intel.com, laurent.pinchart@ideasonboard.com, andriy.shevchenko@linux.intel.com, hdegoede@redhat.com, ilpo.jarvinen@linux.intel.com, claus.stovgaard@gmail.com, tomi.valkeinen@ideasonboard.com, tfiga@chromium.org, senozhatsky@chromium.org, andreaskleist@gmail.com, bingbu.cao@linux.intel.com, tian.shu.qiu@intel.com, hongju.wang@intel.com Subject: [PATCH v4 18/19] Documentation: add Intel IPU6 ISYS driver admin-guide doc Date: Tue, 16 Apr 2024 23:11:04 +0300 Message-Id: <20240416201105.781496-19-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240416201105.781496-1-sakari.ailus@linux.intel.com> References: <20240416201105.781496-1-sakari.ailus@linux.intel.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Bingbu Cao This document mainly describe the functionality of IPU6 and IPU6 isys driver, and gives an example that how user can do imaging capture with tools. Signed-off-by: Bingbu Cao Signed-off-by: Sakari Ailus --- Documentation/admin-guide/media/ipu6-isys.rst | 161 +++++ .../admin-guide/media/ipu6_isys_graph.svg | 548 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 710 insertions(+) create mode 100644 Documentation/admin-guide/media/ipu6-isys.rst create mode 100644 Documentation/admin-guide/media/ipu6_isys_graph.svg diff --git a/Documentation/admin-guide/media/ipu6-isys.rst b/Documentation/admin-guide/media/ipu6-isys.rst new file mode 100644 index 000000000000..0721e920b5e6 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6-isys.rst @@ -0,0 +1,161 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. include:: + +======================================================== +Intel Image Processing Unit 6 (IPU6) Input System driver +======================================================== + +Copyright |copy| 2023--2024 Intel Corporation + +Introduction +============ + +This file documents the Intel IPU6 (6th generation Image Processing Unit) +Input System (MIPI CSI2 receiver) drivers located under +drivers/media/pci/intel/ipu6. + +The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs: + +* Tiger Lake +* Jasper Lake +* Alder Lake +* Raptor Lake +* Meteor Lake + +Intel IPU6 is made up of two components - Input System (ISYS) and Processing +System (PSYS). + +The Input System mainly works as MIPI CSI-2 receiver which receives and +processes the image data from the sensors and outputs the frames to memory. + +There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an +IPU6 common driver which does PCI configuration, firmware loading and parsing, +firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) +configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 +sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected +to the IPU6 ISYS through V4L2 sub-device sensor drivers. + +.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more + information about the IPU6 hardware. + +Input system driver +=================== + +The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware +stream configuration, sends commands to firmware, gets response from hardware +and firmware and then returns buffers to user. The ISYS is represented as +several V4L2 sub-devices as well as video nodes. + +.. kernel-figure:: ipu6_isys_graph.svg + :alt: ipu6 isys media graph with multiple streams support + + IPU6 ISYS media graph with multiple streams support + +The graph has been produced using the following command: + +.. code-block:: none + + fdp -Gsplines=true -Tsvg < dot > dot.svg + +Capturing frames with IPU6 ISYS +------------------------------- + +IPU6 ISYS is used to capture frames from the camera sensors connected to the +CSI2 ports. The supported input formats of ISYS are listed in table below: + +.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| + +.. flat-table:: + :header-rows: 1 + + * - IPU6 ISYS supported input formats + + * - RGB565, RGB888 + + * - UYVY8, YUYV8 + + * - RAW8, RAW10, RAW12 + +.. _ipu6_isys_capture_examples: + +Examples +~~~~~~~~ + +Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this +machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can +generate images at sBGGR10 with resolution 1280x800. + +Using the media controller APIs, we can configure ov01a10 sensor by +media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. + +.. code-block:: none + + # Example 1 capture frame from ov01a10 camera sensor + # This example assumes /dev/media0 as the IPU ISYS media device + export MDEV=/dev/media0 + + # Establish the link for the media devices using media-ctl + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" + + # Set the format for the media devices + media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" + +Once the media pipeline is configured, desired sensor specific settings +(such as exposure and gain settings) can be set, using the yavta tool. + +e.g + +.. code-block:: none + + # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 + export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") + + yavta -w 0x009e0903 400 $SDEV + yavta -w 0x009e0913 1000 $SDEV + yavta -w 0x009e0911 2000 $SDEV + +Once the desired sensor settings are set, frame captures can be done as below. + +e.g + +.. code-block:: none + + yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ + -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") + +With the above command, 10 frames are captured at 1280x800 resolution with +sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. + +Here is another example of IPU6 ISYS RAW and metadata capture from camera +sensor ov2740 on Lenovo X1 Yoga laptop. + +.. code-block:: none + + media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]" + media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[5]" + media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[5]" + + # set routing + media-ctl -v -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]" + + media-ctl -v "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]" + media-ctl -v "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]" + media-ctl -v "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]" + media-ctl -v "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]" + + CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0") + ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \ + -f SGRBG10 ${CAPTURE_DEV} + + CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1") + ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \ + --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META} + +References +========== + +.. [#f1] https://git.ideasonboard.org/media-ctl.git +.. [#f2] https://git.ideasonboard.org/yavta.git diff --git a/Documentation/admin-guide/media/ipu6_isys_graph.svg b/Documentation/admin-guide/media/ipu6_isys_graph.svg new file mode 100644 index 000000000000..c8539ef320d2 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6_isys_graph.svg @@ -0,0 +1,548 @@ + + + + + + +board + + + +n00000001 + +Intel IPU6 ISYS Capture 0 +/dev/video0 + + + +n00000005 + +Intel IPU6 ISYS Capture 1 +/dev/video1 + + + +n00000009 + +Intel IPU6 ISYS Capture 2 +/dev/video2 + + + +n0000000d + +Intel IPU6 ISYS Capture 3 +/dev/video3 + + + +n00000011 + +Intel IPU6 ISYS Capture 4 +/dev/video4 + + + +n00000015 + +Intel IPU6 ISYS Capture 5 +/dev/video5 + + + +n00000019 + +Intel IPU6 ISYS Capture 6 +/dev/video6 + + + +n0000001d + +Intel IPU6 ISYS Capture 7 +/dev/video7 + + + +n00000021 + +Intel IPU6 ISYS Capture 8 +/dev/video8 + + + +n00000025 + +Intel IPU6 ISYS Capture 9 +/dev/video9 + + + +n00000029 + +Intel IPU6 ISYS Capture 10 +/dev/video10 + + + +n0000002d + +Intel IPU6 ISYS Capture 11 +/dev/video11 + + + +n00000031 + +Intel IPU6 ISYS Capture 12 +/dev/video12 + + + +n00000035 + +Intel IPU6 ISYS Capture 13 +/dev/video13 + + + +n00000039 + +Intel IPU6 ISYS Capture 14 +/dev/video14 + + + +n0000003d + +Intel IPU6 ISYS Capture 15 +/dev/video15 + + + +n00000041 + +Intel IPU6 ISYS Capture 16 +/dev/video16 + + + +n00000045 + +Intel IPU6 ISYS Capture 17 +/dev/video17 + + + +n00000049 + +Intel IPU6 ISYS Capture 18 +/dev/video18 + + + +n0000004d + +Intel IPU6 ISYS Capture 19 +/dev/video19 + + + +n00000051 + +Intel IPU6 ISYS Capture 20 +/dev/video20 + + + +n00000055 + +Intel IPU6 ISYS Capture 21 +/dev/video21 + + + +n00000059 + +Intel IPU6 ISYS Capture 22 +/dev/video22 + + + +n0000005d + +Intel IPU6 ISYS Capture 23 +/dev/video23 + + + +n00000061 + +Intel IPU6 ISYS Capture 24 +/dev/video24 + + + +n00000065 + +Intel IPU6 ISYS Capture 25 +/dev/video25 + + + +n00000069 + +Intel IPU6 ISYS Capture 26 +/dev/video26 + + + +n0000006d + +Intel IPU6 ISYS Capture 27 +/dev/video27 + + + +n00000071 + +Intel IPU6 ISYS Capture 28 +/dev/video28 + + + +n00000075 + +Intel IPU6 ISYS Capture 29 +/dev/video29 + + + +n00000079 + +Intel IPU6 ISYS Capture 30 +/dev/video30 + + + +n0000007d + +Intel IPU6 ISYS Capture 31 +/dev/video31 + + + +n00000081 + +0 + +Intel IPU6 CSI2 0 +/dev/v4l-subdev0 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n00000081:port1->n00000001 + + + + + +n00000081:port2->n00000005 + + + + + +n00000081:port3->n00000009 + + + + + +n00000081:port4->n0000000d + + + + + +n00000081:port5->n00000011 + + + + + +n00000081:port6->n00000015 + + + + + +n00000081:port7->n00000019 + + + + + +n00000081:port8->n0000001d + + + + + +n0000008b + +0 + +Intel IPU6 CSI2 1 +/dev/v4l-subdev1 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n0000008b:port1->n00000021 + + + + + +n0000008b:port2->n00000025 + + + + + +n0000008b:port3->n00000029 + + + + + +n0000008b:port4->n0000002d + + + + + +n0000008b:port5->n00000031 + + + + + +n0000008b:port6->n00000035 + + + + + +n0000008b:port7->n00000039 + + + + + +n0000008b:port8->n0000003d + + + + + +n00000095 + +0 + +Intel IPU6 CSI2 2 +/dev/v4l-subdev2 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n00000095:port1->n00000041 + + + + + +n00000095:port2->n00000045 + + + + + +n00000095:port3->n00000049 + + + + + +n00000095:port4->n0000004d + + + + + +n00000095:port5->n00000051 + + + + + +n00000095:port6->n00000055 + + + + + +n00000095:port7->n00000059 + + + + + +n00000095:port8->n0000005d + + + + + +n0000009f + +0 + +Intel IPU6 CSI2 3 +/dev/v4l-subdev3 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n0000009f:port1->n00000061 + + + + + +n0000009f:port2->n00000065 + + + + + +n0000009f:port3->n00000069 + + + + + +n0000009f:port4->n0000006d + + + + + +n0000009f:port5->n00000071 + + + + + +n0000009f:port6->n00000075 + + + + + +n0000009f:port7->n00000079 + + + + + +n0000009f:port8->n0000007d + + + + + +n000000e9 + +1 + +2 + +ov2740 4-0036 +/dev/v4l-subdev4 + +0 + + + +n000000e9:port0->n0000008b:port0 + + + + + diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index f4bb2605f07e..4120eded9a13 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -16,6 +16,7 @@ Video4Linux (V4L) driver-specific documentation imx imx7 ipu3 + ipu6-isys ivtv mgb4 omap3isp