From patchwork Wed May 29 15:28:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679110 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8C449C27C43 for ; Wed, 29 May 2024 15:30:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hwAcBQYIHHgQlumNU3Y3Zja2VUzxGxWQQlSCLqPppP0=; b=uhHVu2JJt/DYzi cQlTNJwpFtC9fHdNJMpKnietW8SoDyciLWPUKU4o7Oeg8CoVJ8jczIv9i+iD5JgSO2t1nm8IVVRAE RwP9B7j0jOt8PeB4k+IXTQWJblhoMecJ4mrNrTdYV6RJZvUJMoRO11JLVusc+BDPoptvH86VC35da dCgODomkNc7byJ07+3GFEqJQQnJ/grWvg0vuEPNrSZT7/Iu9/v/Z3H4LtrwriRd6qSdc40IEyLlcy ut1kPaBLNL19miWBIWluNB7FSLOjrNdHj9ecCNflDAg+wB7pFvmf7KFIXlS8AnyB/IUhGj8NBUBwq 7WFvqtSOwZozDsUeb+Fw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFQ-00000004hPy-49GM; Wed, 29 May 2024 15:29:53 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLEx-00000004h2c-3Edi for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:25 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DEAAD2B5A; Wed, 29 May 2024 17:29:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996556; bh=AUeHCqKJUsyxUH9rTn0sI/xVufnVxR6ce7+vDVqRXqo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RQCJaWh2kB4/0gygzqXnhd/2zjD005vpRv4CTTOkSyigh+f2wXqe9UnrbJ7nNML59 iEUS1prE0Ss+D6zg010CM7NfszgLZbTUReTzv6xea6TbPrzGqU9+gqRno0E7b/SSxB JGbfr2BSv9lTLI4BH4SjnBBDw7HfttdXAWObGgis= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 01/16] media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code Date: Wed, 29 May 2024 16:28:43 +0100 Message-Id: <20240529152858.183799-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082924_009517_BC394491 X-CRM114-Status: GOOD ( 10.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Mali-C55 ISP by ARM requires 20-bits per colour channel input on the bus. Add a new media bus format code to represent it. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - none Changes in v4: - None Changes in v3: - None Changes in v2: - none .../media/v4l/subdev-formats.rst | 168 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 3 +- 2 files changed, 170 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index d2a6cd2e1eb2..8d164a9a9e15 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -2224,6 +2224,174 @@ The following table list existing packed 48bit wide RGB formats. \endgroup +The following table list existing packed 60bit wide RGB formats. + +.. tabularcolumns:: |p{4.0cm}|p{0.7cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}| + +.. _v4l2-mbus-pixelcode-rgb-60: + +.. raw:: latex + + \begingroup + \tiny + \setlength{\tabcolsep}{2pt} + +.. flat-table:: 60bit RGB formats + :header-rows: 3 + :stub-columns: 0 + :widths: 36 7 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 + + * - Identifier + - Code + - + - :cspan:`31` Data organization + * - + - + - Bit + - + - + - + - + - 59 + - 58 + - 57 + - 56 + - 55 + - 54 + - 53 + - 52 + - 51 + - 50 + - 49 + - 48 + - 47 + - 46 + - 45 + - 44 + - 43 + - 42 + - 41 + - 40 + - 39 + - 38 + - 37 + - 36 + - 35 + - 34 + - 33 + - 32 + * - + - + - + - 31 + - 30 + - 29 + - 28 + - 27 + - 26 + - 25 + - 24 + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-RGB202020-1X60: + + - MEDIA_BUS_FMT_RGB202020_1X60 + - 0x1026 + - + - + - + - + - + - r\ :sub:`19` + - r\ :sub:`18` + - r\ :sub:`17` + - r\ :sub:`16` + - r\ :sub:`15` + - r\ :sub:`14` + - r\ :sub:`13` + - r\ :sub:`12` + - r\ :sub:`11` + - r\ :sub:`10` + - r\ :sub:`8` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` + - g\ :sub:`19` + - g\ :sub:`18` + - g\ :sub:`17` + - g\ :sub:`16` + - g\ :sub:`15` + - g\ :sub:`14` + - g\ :sub:`13` + - g\ :sub:`12` + * - + - + - + - g\ :sub:`11` + - g\ :sub:`10` + - g\ :sub:`9` + - g\ :sub:`8` + - g\ :sub:`7` + - g\ :sub:`6` + - g\ :sub:`5` + - g\ :sub:`4` + - g\ :sub:`3` + - g\ :sub:`2` + - g\ :sub:`1` + - g\ :sub:`0` + - b\ :sub:`19` + - b\ :sub:`18` + - b\ :sub:`17` + - b\ :sub:`16` + - b\ :sub:`15` + - b\ :sub:`14` + - b\ :sub:`13` + - b\ :sub:`12` + - b\ :sub:`11` + - b\ :sub:`10` + - b\ :sub:`9` + - b\ :sub:`8` + - b\ :sub:`7` + - b\ :sub:`6` + - b\ :sub:`5` + - b\ :sub:`4` + - b\ :sub:`3` + - b\ :sub:`2` + - b\ :sub:`1` + - b\ :sub:`0` + +.. raw:: latex + + \endgroup + On LVDS buses, usually each sample is transferred serialized in seven time slots per pixel clock, on three (18-bit) or four (24-bit) differential data pairs at the same time. The remaining bits are used diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index d4c1d991014b..49be328d9a3b 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -34,7 +34,7 @@ #define MEDIA_BUS_FMT_FIXED 0x0001 -/* RGB - next is 0x1026 */ +/* RGB - next is 0x1027 */ #define MEDIA_BUS_FMT_RGB444_1X12 0x1016 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE 0x1001 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE 0x1002 @@ -72,6 +72,7 @@ #define MEDIA_BUS_FMT_RGB888_1X36_CPADLO 0x1021 #define MEDIA_BUS_FMT_RGB121212_1X36 0x1019 #define MEDIA_BUS_FMT_RGB161616_1X48 0x101a +#define MEDIA_BUS_FMT_RGB202020_1X60 0x1026 /* YUV (including grey) - next is 0x202f */ #define MEDIA_BUS_FMT_Y8_1X8 0x2001 From patchwork Wed May 29 15:28:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679112 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 16D32C27C50 for ; Wed, 29 May 2024 15:30:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QihNpwlxQcGGlD7fdWhY57rkmgwZzZDQ4QdfnDHnh5g=; b=tn2XVxXjhHucLB twDQE+6pJsJwoXH8NhEY5rPcTH4EbQsfZZHA6hkWrQjiGL639D8fDJsHOsWcEBrCyF3IGoXn7a0lR vV5arbJeKQFhlGiOpHne6N01EqjzLOGt+0a8sGgXToxY+UJglYl8xKZtb9QntGiWdfs/r6LS6U+On e6DFcCi9Lc0ULfm+7Xojb9e14GS6ZyxYZ/lMSRKkanhOU/VC+ElLmaMrQVnwbDGfGgankQgnWGCWq scFk7fDfSy21qASlU6dNPKK2QOnQ+pZmKLGsnK2bX8qYq1UfaxLzqszQKJLQIobWTF+sHFgEs+09E 5ai2bz8VuJgRwJiThYjw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFU-00000004hSR-3Dcr; Wed, 29 May 2024 15:29:56 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLEx-00000004h2e-443s for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:25 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 81E3A2D54; Wed, 29 May 2024 17:29:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996557; bh=Hu4aMLJwuBDXxhTlXJSWkF/966g2zaAJ82YLQ6LeSCo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MDcpA+GxrBxEcR9Ygi4jH63JQYbtO4E9gZTZxgRNxljp4209yycPKkLx+MLmN1m0o D+OvrFJLZtmo6s0Kjla71NGkBVwkwVUfgEiIgPzO5zKFCxdSxyRxOU+C7NnruUlhG8 AhIeN0l1Q+GKokXlVb0IG84Ud7dTdTyB/ZnvqoSE= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 02/16] media: uapi: Add 20-bit bayer formats Date: Wed, 29 May 2024 16:28:44 +0100 Message-Id: <20240529152858.183799-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082924_177299_9D811383 X-CRM114-Status: UNSURE ( 9.27 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Mali-C55 requires input data be in 20-bit format, MSB aligned. Add some new media bus format macros to represent that input format. Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - New patch .../media/v4l/subdev-formats.rst | 100 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 6 +- 2 files changed, 105 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index 8d164a9a9e15..f986dfc52879 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -3445,6 +3445,106 @@ organization is given as an example for the first pixel only. - r\ :sub:`2` - r\ :sub:`1` - r\ :sub:`0` + * .. _MEDIA-BUS-FMT-SBGGR20-1X20: + + - MEDIA_BUS_FMT_SBGGR20_1X20 + - 0x3021 + - + - r\ :sub:`19` + - r\ :sub:`18` + - r\ :sub:`17` + - r\ :sub:`16` + - r\ :sub:`15` + - r\ :sub:`14` + - r\ :sub:`13` + - r\ :sub:`12` + - r\ :sub:`11` + - r\ :sub:`10` + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` + * .. _MEDIA-BUS-FMT-SGBRG20-1X20: + + - MEDIA_BUS_FMT_SGBRG20_1X20 + - 0x3022 + - + - r\ :sub:`19` + - r\ :sub:`18` + - r\ :sub:`17` + - r\ :sub:`16` + - r\ :sub:`15` + - r\ :sub:`14` + - r\ :sub:`13` + - r\ :sub:`12` + - r\ :sub:`11` + - r\ :sub:`10` + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` + * .. _MEDIA-BUS-FMT-SGRBG20-1X20: + + - MEDIA_BUS_FMT_SGRBG20_1X20 + - 0x3023 + - + - r\ :sub:`19` + - r\ :sub:`18` + - r\ :sub:`17` + - r\ :sub:`16` + - r\ :sub:`15` + - r\ :sub:`14` + - r\ :sub:`13` + - r\ :sub:`12` + - r\ :sub:`11` + - r\ :sub:`10` + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` + * .. _MEDIA-BUS-FMT-SRGGB20-1X20: + + - MEDIA_BUS_FMT_SRGGB20_1X20 + - 0x3024 + - + - r\ :sub:`19` + - r\ :sub:`18` + - r\ :sub:`17` + - r\ :sub:`16` + - r\ :sub:`15` + - r\ :sub:`14` + - r\ :sub:`13` + - r\ :sub:`12` + - r\ :sub:`11` + - r\ :sub:`10` + - r\ :sub:`9` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` .. raw:: latex diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index 49be328d9a3b..b6acf8c8e383 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -122,7 +122,7 @@ #define MEDIA_BUS_FMT_YUV16_1X48 0x202a #define MEDIA_BUS_FMT_UYYVYY16_0_5X48 0x202b -/* Bayer - next is 0x3021 */ +/* Bayer - next is 0x3025 */ #define MEDIA_BUS_FMT_SBGGR8_1X8 0x3001 #define MEDIA_BUS_FMT_SGBRG8_1X8 0x3013 #define MEDIA_BUS_FMT_SGRBG8_1X8 0x3002 @@ -155,6 +155,10 @@ #define MEDIA_BUS_FMT_SGBRG16_1X16 0x301e #define MEDIA_BUS_FMT_SGRBG16_1X16 0x301f #define MEDIA_BUS_FMT_SRGGB16_1X16 0x3020 +#define MEDIA_BUS_FMT_SBGGR20_1X20 0x3021 +#define MEDIA_BUS_FMT_SGBRG20_1X20 0x3022 +#define MEDIA_BUS_FMT_SGRBG20_1X20 0x3023 +#define MEDIA_BUS_FMT_SRGGB20_1X20 0x3024 /* JPEG compressed formats - next is 0x4002 */ #define MEDIA_BUS_FMT_JPEG_1X8 0x4001 From patchwork Wed May 29 15:28:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 639B3C25B75 for ; Wed, 29 May 2024 15:30:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pMqxevzkGcUo01E6rHilmQ5edvIDeQxtlpgP2PXHr1M=; b=eWo/zpFDf/i1d+ 9CdkbtX56cr/P8CVqffjkZDjm8xJ7s8WwqHJ/CfNjyItSvzqVeEaaFbB/gb8YpF2TfoDm4deFp04N /8PYJtuIae0R3E5YKkIX/XI51sKFfVIntaqHQGVQCyntX/UvVdSOMnp4FLEkd9LpZOhkzF8fyEGL7 MgZ9CG9WoFKQCfeDgh9o3aY+uLbt4iemwQKhoRQ1KQFUjVeIAGMdj0OGnrtsnsfT9VmPIdqg3vSEQ KnQjORvmD7Q2euF0t8HawbCh1V1a3IAQeFGR9ft2Ne31tbdgeJXdBvh4HU65cD5N+7ioW6Dg7OAyg ORCMOfwadHO5ZvrvEyVg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFT-00000004hRt-3D35; Wed, 29 May 2024 15:29:55 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLEy-00000004h2n-10Oi for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:25 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2CE2E2D5F; Wed, 29 May 2024 17:29:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996557; bh=4Zk7tRiwTOL78Byp+CPsuM4Rdg2v2J5MGqG9hkA39w8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VF7GQzVaaltwEkTJA/N3hr95DHYNRicmt4hoDxVJrJmVE8lYVwBNCWwWaWfBTJodz AkpdYNLncSDWNnBskcev/K3q7+zDcCBREMFEyIbOO7xbbNWTqVsBdRPEuDxa8ymR5c hXsqaRkoKHTQEhnzU3KsVzbLN8TFFIOf2pUay0jg= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 03/16] media: v4l2-common: Add RAW16 format info Date: Wed, 29 May 2024 16:28:45 +0100 Message-Id: <20240529152858.183799-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082924_477856_BACA3F34 X-CRM114-Status: UNSURE ( 8.39 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add entries to v4l2_format_info describing the 16-bit bayer formats so that they can be used in drivers. Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - New patch drivers/media/v4l2-core/v4l2-common.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c index 4165c815faef..c5d5704af5ee 100644 --- a/drivers/media/v4l2-core/v4l2-common.c +++ b/drivers/media/v4l2-core/v4l2-common.c @@ -331,6 +331,10 @@ const struct v4l2_format_info *v4l2_format_info(u32 format) { .format = V4L2_PIX_FMT_SGBRG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_SGRBG12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_SRGGB12, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_SBGGR16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_SGBRG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_SGRBG16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_SRGGB16, .pixel_enc = V4L2_PIXEL_ENC_BAYER, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, }; unsigned int i; From patchwork Wed May 29 15:28:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 36ECEC25B75 for ; Wed, 29 May 2024 15:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WMM16Ay9yQj1QuAHIi9wwgaLfkpCeO2WADW8eqfmMRc=; b=rNqLvwlcq/B5cm 1B86vc57jKpd+cBXELrVg556oHlKrI2kv7d+71T1RUcrHiG5StJgwtMeGtXJFQJg8sP6QX4EvkUuv lWLXCkuvON7Tm95l8snGGoGZCfjpLL+nRtzza8PI+Z/xTy482pQwoyhqQUA5IYk3RrOdBskxjeq2z cErjsMVxxAIQ8P1DN2iiy3ilK+lS3gGEUwfSYuJaocWB/7iVe9KJGV5Q7gp2Q6Jj6TmDPuNE2z1by 8Fx84BhT3lkglboNZcvlpJTwPPpM9iSbWjJgTrCtjqUdO5s3hToIGZ/zGH+0L5Dapl2G33KrnJXg6 BiXn74W1eqvd8Ju13ymA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF3-00000004h9p-1sb7; Wed, 29 May 2024 15:29:29 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLEx-00000004h3A-1FTi for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:24 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C88ED2D60; Wed, 29 May 2024 17:29:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996558; bh=k3Tloi/UosPGf8DVTncqKsVn+k3gnL7IL/lSFQm1p8w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i3y9iaIVUuaMZdRtaJocZthydhkHKsQnnMwLhthe781qpEcaIFOqaQ3ljFhx0HbYa w63xuib992ZUvjo61Ntxsj7cR4cU1NJAfdQl3isT+C+rjv7NPRr0Ubg6aImlJOF3VT ktZ1qAelMa98+t4v+hSz39cflCR0PLvYO+1bjD4I= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 04/16] dt-bindings: media: Add bindings for ARM mali-c55 Date: Wed, 29 May 2024 16:28:46 +0100 Message-Id: <20240529152858.183799-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082923_538172_31E4A12B X-CRM114-Status: GOOD ( 15.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add the yaml binding for ARM's Mali-C55 Image Signal Processor. Reviewed-by: Krzysztof Kozlowski Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - None Changes in v4: - Switched to port instead of ports Changes in v3: - Dropped the video clock as suggested by Laurent. I didn't retain it for the purposes of the refcount since this driver will call .s_stream() for the sensor driver which will refcount the clock anyway. - Clarified that the port is a parallel input port rather (Sakari) Changes in v2: - Added clocks information - Fixed the warnings raised by Rob .../bindings/media/arm,mali-c55.yaml | 66 +++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml diff --git a/Documentation/devicetree/bindings/media/arm,mali-c55.yaml b/Documentation/devicetree/bindings/media/arm,mali-c55.yaml new file mode 100644 index 000000000000..9cc2481f2da3 --- /dev/null +++ b/Documentation/devicetree/bindings/media/arm,mali-c55.yaml @@ -0,0 +1,66 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/arm,mali-c55.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: ARM Mali-C55 Image Signal Processor + +maintainers: + - Daniel Scally + - Jacopo Mondi + +properties: + compatible: + const: arm,mali-c55 + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: ISP AXI clock + - description: ISP AHB-lite clock + + clock-names: + items: + - const: aclk + - const: hclk + + port: + $ref: /schemas/graph.yaml#/properties/port + description: Input parallel video bus + + properties: + endpoint: + $ref: /schemas/graph.yaml#/properties/endpoint + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - port + +additionalProperties: false + +examples: + - | + mali_c55: isp@400000 { + compatible = "arm,mali-c55"; + reg = <0x400000 0x200000>; + clocks = <&clk 0>, <&clk 1>; + clock-names = "aclk", "hclk"; + interrupts = <0>; + + port { + isp_in: endpoint { + remote-endpoint = <&csi2_rx_out>; + }; + }; + }; +... From patchwork Wed May 29 15:28:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0A0AC27C44 for ; Wed, 29 May 2024 15:30:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5S1bnuN1Gqk5vaIN924C0gtRbJ2Bjiv9Zb38nX/3Pj0=; b=WEeZiPR1MG0E6I 8Lo87T4QULucuN1G82AEbatypy4ZG+PnskYMI9ga/92oDj4u5gXpme2qn7pCrwfxyzxcz6b4ICefN IfGXcIUwMv4cO7kOtQzxU0S+9MgBlO/4jiWVsuJtFjeSaILOjD8VyAP2/0O4X2Ed3WRsPZgOewP3N 0n0JgLTmYuZUojzH6v3+9RhTBGEaSl6XNK85kap2WGkeo8pTbzhhdi9GtQp5goazbTf48CcUj57Dg HFFubcDWxqeEUeceZWzDOLgi4oAYC/MEYBtCU8TypJ/FsDk/iZJp1dmfuuuOprRj1StMHnLy+9LRj YjqOXDD6ymaWErM+Widg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFY-00000004hV2-3uOb; Wed, 29 May 2024 15:30:00 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLEz-00000004h5T-3nuN for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:29 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6A80B2F85; Wed, 29 May 2024 17:29:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996559; bh=Yv7QXnVagU+JLcv3JRD/F/UNXc3K3jdQ3sKQTmLwfZ0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SlYmpdmJBV2xwg1p1E0b471RROE4MsrUc1UWvXniDxFOXsr5h8QKpXtxUfzEPNKEy mY8PI/z8YrdRlPerNkoaxCehz7EbfePRUyKl/2EMtAxl3GrQvy2A8/vtjInu6dVOU6 sNrWsVAYqUwo1eBrSXrbHzsepPaO8anWV+qHOLMo= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 06/16] media: Documentation: Add Mali-C55 ISP Documentation Date: Wed, 29 May 2024 16:28:48 +0100 Message-Id: <20240529152858.183799-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082926_281306_497ED7EA X-CRM114-Status: GOOD ( 31.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a documentation page for the mali-c55 driver, which gives a brief overview of the hardware and explains how to use the driver's capture devices and the crop/scaler functions. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None Changes in v4: - None Changes in v3: - Documented the synchronised buffer sequence numbers (Sakari) - Clarified that the downscale pipe cannot output raw data, the ISP'S resolution limits and choice of media bus format code (Kieran) Changes in v2: - none .../admin-guide/media/mali-c55-graph.dot | 19 + Documentation/admin-guide/media/mali-c55.rst | 333 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 353 insertions(+) create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot create mode 100644 Documentation/admin-guide/media/mali-c55.rst diff --git a/Documentation/admin-guide/media/mali-c55-graph.dot b/Documentation/admin-guide/media/mali-c55-graph.dot new file mode 100644 index 000000000000..0775ba42bf4c --- /dev/null +++ b/Documentation/admin-guide/media/mali-c55-graph.dot @@ -0,0 +1,19 @@ +digraph board { + rankdir=TB + n00000001 [label="{{} | mali-c55 tpg\n/dev/v4l-subdev0 | { 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port0 -> n00000003:port0 [style=dashed] + n00000003 [label="{{ 0} | mali-c55 isp\n/dev/v4l-subdev1 | { 1 | 2}}", shape=Mrecord, style=filled, fillcolor=green] + n00000003:port1 -> n00000007:port0 [style=bold] + n00000003:port2 -> n00000007:port2 [style=bold] + n00000003:port1 -> n0000000b:port0 [style=bold] + n00000007 [label="{{ 0 | 2} | mali-c55 resizer fr\n/dev/v4l-subdev2 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000007:port1 -> n0000000e [style=bold] + n0000000b [label="{{ 0} | mali-c55 resizer ds\n/dev/v4l-subdev3 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n0000000b:port1 -> n00000012 [style=bold] + n0000000e [label="mali-c55 fr\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n00000012 [label="mali-c55 ds\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000022 [label="{{ 0} | csi2-rx\n/dev/v4l-subdev4 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000022:port1 -> n00000003:port0 + n00000027 [label="{{} | imx415 1-001a\n/dev/v4l-subdev5 | { 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000027:port0 -> n00000022:port0 [style=bold] +} \ No newline at end of file diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst new file mode 100644 index 000000000000..cf4176cb1e44 --- /dev/null +++ b/Documentation/admin-guide/media/mali-c55.rst @@ -0,0 +1,333 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================== +ARM Mali-C55 Image Signal Processor driver +========================================== + +Introduction +============ + +This file documents the driver for ARM's Mali-C55 Image Signal Processor. The +driver is located under drivers/media/platform/arm/mali-c55. + +The Mali-C55 ISP receives data in either raw Bayer format or RGB/YUV format from +sensors through either a parallel interface or a memory bus before processing it +and outputting it through an internal DMA engine. Two output pipelines are +possible (though one may not be fitted, depending on the implementation). These +are referred to as "Full resolution" and "Downscale", but the naming is historic +and both pipes are capable of cropping/scaling operations. The full resolution +pipe is also capable of outputting RAW data, bypassing much of the ISP's +processing. The downscale pipe cannot output RAW data. An integrated test +pattern generator can be used to drive the ISP and produce image data in the +absence of a connected camera sensor. The driver module is named mali_c55, and +is enabled through the CONFIG_VIDEO_MALI_C55 config option. + +The driver implements V4L2, Media Controller and V4L2 Subdevice interfaces and +expects camera sensors connected to the ISP to have V4L2 subdevice interfaces. + +Mali-C55 ISP hardware +===================== + +A high level functional view of the Mali-C55 ISP is presented below. The ISP +takes input from either a live source or through a DMA engine for memory input, +depending on the SoC integration.:: + + +---------+ +----------+ +--------+ + | Sensor |--->| CSI-2 Rx | "Full Resolution" | DMA | + +---------+ +----------+ |\ Output +--->| Writer | + | | \ | +--------+ + | | \ +----------+ +------+---> Streaming I/O + +------------+ +------->| | | | | + | | | |-->| Mali-C55 |--+ + | DMA Reader |--------------->| | | ISP | | + | | | / | | | +---> Streaming I/O + +------------+ | / +----------+ | | + |/ +------+ + | +--------+ + +--->| DMA | + "Downscaled" | Writer | + Output +--------+ + +Media Controller Topology +========================= + +An example of the ISP's topology (as implemented in a system with an IMX415 +camera sensor and generic CSI-2 receiver) is below: + + +.. kernel-figure:: mali-c55-graph.dot + :alt: mali-c55-graph.dot + :align: center + +The driver has 4 V4L2 subdevices: + +- `mali_c55 isp`: Responsible for configuring input crop and color space + conversion +- `mali_c55 tpg`: The test pattern generator, emulating a camera sensor. +- `mali_c55 resizer fr`: The Full-Resolution pipe resizer +- `mali_c55 resizer ds`: The Downscale pipe resizer + +The driver has 2 V4L2 video devices: + +- `mali-c55 fr`: The full-resolution pipe's capture device +- `mali-c55 ds`: The downscale pipe's capture device + +Frame sequences are synchronised across to two capture devices, meaning if one +pipe is started later than the other the sequence numbers returned in its +buffers will match those of the other pipe rather than starting from zero. + +Frame sequences are synchronised across to two capture devices, meaning if one +pipe is started later than the other the sequence numbers returned in its +buffers will match those of the other pipe rather than starting from zero. + +Idiosyncrasies +-------------- + +**mali-c55 isp** +The `mali-c55 isp` subdevice has a single sink pad to which all sources of data +should be connected. The active source is selected by enabling the appropriate +media link and disabling all others. The ISP has two source pads, reflecting the +different paths through which it can internally route data. Tap points within +the ISP allow users to divert data to avoid processing by some or all of the +hardware's processing steps. The diagram below is intended only to highlight how +the bypassing works and is not a true reflection of those processing steps; for +a high-level functional block diagram see ARM's developer page for the +ISP [3]_:: + + +--------------------------------------------------------------+ + | Possible Internal ISP Data Routes | + | +------------+ +----------+ +------------+ | + +---+ | | | | | Colour | +---+ + | 0 |--+-->| Processing |->| Demosaic |->| Space |--->| 1 | + +---+ | | | | | | Conversion | +---+ + | | +------------+ +----------+ +------------+ | + | | +---+ + | +---------------------------------------------------| 2 | + | +---+ + | | + +--------------------------------------------------------------+ + + +.. flat-table:: + :header-rows: 1 + + * - Pad + - Direction + - Purpose + + * - 0 + - sink + - Data input, connected to the TPG and camera sensors + + * - 1 + - source + - RGB/YUV data, connected to the FR and DS V4L2 subdevices + + * - 2 + - source + - RAW bayer data, connected to the FR V4L2 subdevices + +The ISP is limited to both input and output resolutions between 640x480 and +8192x8192, and this is reflected in the ISP and resizer subdevice's .set_fmt() +operations. + +**mali-c55 resizer fr** +The `mali-c55 resizer fr` subdevice has two _sink_ pads to reflect the different +insertion points in the hardware (either RAW or demosaiced data): + +.. flat-table:: + :header-rows: 1 + + * - Pad + - Direction + - Purpose + + * - 0 + - sink + - Data input connected to the ISP's demosaiced stream. + + * - 1 + - source + - Data output connected to the capture video device + + * - 2 + - sink + - Data input connected to the ISP's raw data stream + +The data source in use is selected through the routing API; two routes each of a +single stream are available: + +.. flat-table:: + :header-rows: 1 + + * - Sink Pad + - Source Pad + - Purpose + + * - 0 + - 1 + - Demosaiced data route + + * - 2 + - 1 + - Raw data route + + +If the demosaiced route is active then the FR pipe is only capable of output +in RGB/YUV formats. If the raw route is active then the output reflects the +input (which may be either Bayer or RGB/YUV data). + +Using the driver to capture video +================================= + +Using the media controller APIs we can configure the input source and ISP to +capture images in a variety of formats. In the examples below, configuring the +media graph is done with the v4l-utils [1]_ package's media-ctl utility. +Capturing the images is done with yavta [2]_. + +Configuring the input source +---------------------------- + +The first step is to set the input source that we wish by enabling the correct +media link. Using the example topology above, we can select the TPG as follows: + +.. code-block:: none + + media-ctl -l "'lte-csi2-rx':1->'mali-c55 isp':0[0]" + media-ctl -l "'mali-c55 tpg':0->'mali-c55 isp':0[1]" + +Capturing bayer data from the source and processing to RGB/YUV +-------------------------------------------------------------- +To capture 1920x1080 bayer data from the source and push it through the ISP's +full processing pipeline, we configure the data formats appropriately on the +source, ISP and resizer subdevices and set the FR resizer's routing to select +processed data. The media bus format on the resizer's source pad will be either +RGB121212_1X36 or YUV10_1X30, depending on whether you want to capture RGB or +YUV. The ISP's debayering block outputs RGB data natively, setting the source +pad format to YUV10_1X30 enables the colour space conversion block. + +In this example we target RGB565 output, so select RGB121212_1X36 as the resizer +source pad's format: + +.. code-block:: none + + # Set formats on the TPG and ISP + media-ctl -V "'mali-c55 tpg':0[fmt:SRGGB16_1X16/1920x1080]" + media-ctl -V "'mali-c55 isp':0[fmt:SRGGB16_1X16/1920x1080]" + media-ctl -V "'mali-c55 isp':1[fmt:SRGGB16_1X16/1920x1080]" + + # Set routing on the FR resizer + media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[1],2/0->1/0[0]]" + + # Set format on the resizer, must be done AFTER the routing. + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/1920x1080]" + +The downscale output can also be used to stream data at the same time. In this +case since only processed data can be captured through the downscale output no +routing need be set: + +.. code-block:: none + + # Set format on the resizer + media-ctl -V "'mali-c55 resizer ds':1[fmt:RGB121212_1X36/1920x1080]" + +Following which images can be captured from both the FR and DS output's video +devices (simultaneously, if desired): + +.. code-block:: none + + yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 + yavta -f RGB565 -s 1920x1080 -c10 /dev/video1 + +Cropping the image +~~~~~~~~~~~~~~~~~~ + +Both the full resolution and downscale pipes can crop to a minimum resolution of +640x480. To crop the image simply configure the resizer's sink pad's crop and +compose rectangles and set the format on the video device: + +.. code-block:: none + + media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(480,270)/640x480 compose:(0,0)/640x480]" + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" + yavta -f RGB565 -s 640x480 -c10 /dev/video0 + +Downscaling the image +~~~~~~~~~~~~~~~~~~~~~ + +Both the full resolution and downscale pipes can downscale the image by up to 8x +provided the minimum 640x480 resolution is adhered to. For the best image result +the scaling ratio for each dimension should be the same. To configure scaling we +use the compose rectangle on the resizer's sink pad: + +.. code-block:: none + + media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(0,0)/1920x1080 compose:(0,0)/640x480]" + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" + yavta -f RGB565 -s 640x480 -c10 /dev/video0 + +Capturing images in YUV formats +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If we need to output YUV data rather than RGB the color space conversion block +needs to be active, which is achieved by setting MEDIA_BUS_FMT_YUV10_1X30 on the +resizer's source pad. We can then configure a capture format like NV12 (here in +its multi-planar variant) + +.. code-block:: none + + media-ctl -V "'mali-c55 resizer fr':1[fmt:YUV10_1X30/1920x1080]" + yavta -f NV12M -s 1920x1080 -c10 /dev/video0 + +Capturing RGB data from the source and processing it with the resizers +---------------------------------------------------------------------- + +The Mali-C55 ISP can work with sensors capable of outputting RGB data. In this +case although none of the image quality blocks would be used it can still +crop/scale the data in the usual way. + +To achieve this, the ISP's sink pad's format is set to +MEDIA_BUS_FMT_RGB202020_1X60 - this reflects the format that data must be in to +work with the ISP. Converting the camera sensor's output to that format is the +responsibility of external hardware. + +In this example we ask the test pattern generator to give us RGB data instead of +bayer. + +.. code-block:: none + + media-ctl -V "'mali-c55 tpg':0[fmt:RGB202020_1X60/1920x1080]" + media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]" + +Cropping or scaling the data can be done in exactly the same way as outlined +earlier. + +Capturing raw data from the source and outputting it unmodified +----------------------------------------------------------------- + +The ISP can additionally capture raw data from the source and output it on the +full resolution pipe only, completely unmodified. In this case the downscale +pipe can still process the data normally and be used at the same time. + +To configure raw bypass the FR resizer's subdevice's routing table needs to be +configured, followed by formats in the appropriate places: + +.. code-block:: none + + # We need to configure the routing table for the resizer to use the bypass + # path along with set formats on the resizer's bypass sink pad. Doing this + # necessitates a single media-ctl command, as multiple calls to the program + # reset the routing table. + media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[0],2/0->1/0[1]]"\ + -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080],"\ + "'mali-c55 resizer fr':2[fmt:RGB202020_1X60/1920x1080],"\ + "'mali-c55 resizer fr':1[fmt:RGB202020_1X60/1920x1080]" + + # Set format on the video device and stream + yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 + +References +========== +.. [1] https://git.linuxtv.org/v4l-utils.git/ +.. [2] https://git.ideasonboard.org/yavta.git +.. [3] https://developer.arm.com/Processors/Mali-C55 diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 4120eded9a13..1d9485860d93 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -18,6 +18,7 @@ Video4Linux (V4L) driver-specific documentation ipu3 ipu6-isys ivtv + mali-c55 mgb4 omap3isp omap4_camera From patchwork Wed May 29 15:28:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2C00C41513 for ; Wed, 29 May 2024 15:30:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DzfNQM218eY6tgjWkP97C/EiJPxcATmQ6oCHhBmqM54=; b=yKlVta8ufmVaQP UzDUdXbMRK3sEHp5JIsE1Vf9JHIVWOp0XxPOwlJAx5yX0/sofrFpLWskDLGXU4xG42OYSyZsyg1AP F+hQNtqWWdmkEal9aDBgcKNIh/1lVbZmAdu/It1BhTHCLv2hsl8fQ4MZ4jlV/zpgS2kX1Eqdt7HvR bnXJG6uG2SkbbNgndWKbM8F9ErlhnrLCNNHS5xuRqnI2G0aMVEEV9InKMAkWkVfGsr0JtE3nht97x ksIfa5kRFTZ4EJrxpPVSaqXPOWbpOaJWfWm/KARBH6PvaHiwdpHr7GONYY/UmLreU70fDidmt9qNm ucOZoNTHOWBMFJFJxguw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFV-00000004hSv-2ymo; Wed, 29 May 2024 15:29:57 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF0-00000004h5d-0uX3 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:29 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1203B2F5E; Wed, 29 May 2024 17:29:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996560; bh=JH2skSJCn7g7zFUA49cvwzmxePyqLTzxCN/8oa1/aD8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RMzzxoIuFuDrvlpm/XiuQHAi3T35609N7t/jHqw2UZ9QUk2o7IWza86Z6r+JX3GvO 4pPZtA8XnHCDj9xMa5DI8GwDeynmBdeG4agh6LD79aG5X75IH45Fn5rw+kIfpyTr7m zEivsiODnH4F5EqmgTQ4+Pn/kq4FGuzqzdrEI1Dw= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 07/16] MAINTAINERS: Add entry for mali-c55 driver Date: Wed, 29 May 2024 16:28:49 +0100 Message-Id: <20240529152858.183799-8-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082926_522337_96B5924A X-CRM114-Status: UNSURE ( 9.15 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a MAINTAINERS entry for the mali-c55 driver and its associated documentation. Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - None Changes in v4: - None Changes in v3: - none Changes in v2: - none MAINTAINERS | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index ef6be9d95143..c2883b30e0ca 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1688,6 +1688,16 @@ F: Documentation/gpu/panfrost.rst F: drivers/gpu/drm/panfrost/ F: include/uapi/drm/panfrost_drm.h +ARM MALI-C55 ISP DRIVER +M: Daniel Scally +M: Jacopo Mondi +L: linux-media@vger.kernel.org +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/admin-guide/media/mali-c55.rst +F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml +F: drivers/media/platform/arm/mali-c55/ + ARM MALI-DP DRM DRIVER M: Liviu Dudau S: Supported From patchwork Wed May 29 15:28:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7D97C27C44 for ; Wed, 29 May 2024 15:30:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DKfJa3N8wFFJCx45HLXKm9uszpNsOzO8q4qUPtO8sx0=; b=ckm9LVNzhnY6P5 m+3rJErPrH/peV78SWGAXLNQx6c0ol4lstvbRi3Y2wk+TxLSe0vob5F+DHBR5bODh1nENYJThWCjp R7AB0TGSS+oPiLS8CwIYZXaZyiQLbzKeOipB82fAX16yPt5ZO7zvrwPgNCSHQBY9y0bySVJovGPFn QR8qjGKSlZrH433ATmSYl7+yG0gpl829f5OTI/xpgNa9DLFpYMxO2AOC87hsim0LkRXy8lrakjdK8 aMuQ1WiGVIEC2NebiB/UIWcbRjCOLKXjHGB6ZJ/IJN8qIAUgAv67CCxQp6qMuDwGxCtaSB4iEv9VU SUDkfJvJUT9i9USB5rPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFX-00000004hTX-09iF; Wed, 29 May 2024 15:29:59 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF0-00000004h5f-165S for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:29 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B089B2F02; Wed, 29 May 2024 17:29:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996561; bh=KwJPgkAHbNdTZbB7Q4OzS2Oh35PIaDmwAfmk17g7+J0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WR5WvRscN2cukN1BAhe7rJexrYaaqSizn3DUgk1tJdNnRbgqUvYpfwE7CSdWKKF/H vtCwyCpoaC/iItCcJt4Agpuj+QcTMjGlCbPbBHMY3cbAxJtUuk+KAU3kMgk/SK4dcw wO1A+wk1mIaseifIYDdGNpcKQttnP7sL5ICiQ1S4= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 08/16] media: Add MALI_C55_3A_STATS meta format Date: Wed, 29 May 2024 16:28:50 +0100 Message-Id: <20240529152858.183799-9-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082926_564399_071DA54D X-CRM114-Status: UNSURE ( 9.79 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a new meta format for the Mali-C55 ISP's 3A Statistics along with a new descriptor entry. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - New patch drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 2 ++ 2 files changed, 3 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 4c76d17b4629..0d00b0476762 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break; case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A Parameters"; break; case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A Statistics"; break; + case V4L2_META_FMT_MALI_C55_3A_STATS: descr = "ARM Mali-C55 ISP 3A Statistics"; break; case V4L2_PIX_FMT_NV12_8L128: descr = "NV12 (8x128 Linear)"; break; case V4L2_PIX_FMT_NV12M_8L128: descr = "NV12M (8x128 Linear)"; break; case V4L2_PIX_FMT_NV12_10BE_8L128: descr = "10-bit NV12 (8x128 Linear, BE)"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index fe6b67e83751..021424fdaf68 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -841,6 +841,8 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_MALI_C55_3A_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */ + #ifdef __KERNEL__ /* * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when From patchwork Wed May 29 15:28:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679116 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C10FDC27C43 for ; Wed, 29 May 2024 15:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nHd4n9duqEOUsCEKdikFSI3EcDpE7Gts5ttWJ8Tb4jA=; b=M3V1TNE99MPO1k sXfnNakuhn8wrMoBDsOgT8WEaVQN9WRuPqSa9bFe83ekkjbDjW48nrCur0XIewvuRNTUJOKbh9hLP PDxDRtH8ZdPPTz5/BThCnocHxAc9Z/Qku6EfFgbRpAIAoIjEyyOEGEP0c43sJbB3949sKvyb9d9Vw 8jNF0o8RI5WyJJZFQ2aIMfMA0ODeRJdjg/KuYIp2QzE7fRQkGmqqk2dXzAgTsTx19arDBE/+UGiyG uvaOO13pGyPeYQuzQuaYjW7/NM8MRGzzCLidQbMmYMVhCaWju7pVwf3afe+Vi4x2oUgQPrBvB91V2 gx44rbzQVzsiqyn2mHgA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFc-00000004hYL-42pF; Wed, 29 May 2024 15:30:04 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF0-00000004h5u-48ov for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:31 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 555C0149B; Wed, 29 May 2024 17:29:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996561; bh=Y7gSxGZFvqzOyZhtrRGLbuiPPT+Zuok4YJ5gnG2tU8k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eECbv65MfhaWKPR9CvXUGG0t292iI863/4QRGfZuBSKEgcyNRlkVganpojf7N6PrP d45+A95nBxNmg2yYNXh7iAV49SYeOKWrhkgXb3SqDwiUV9KTCwosIE06jwr2iJvD4c IzVbIqcrNI5BSKydVpcSPwag7vwqd0j9j9fm0mq0= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 09/16] media: uapi: Add 3a stats buffer for mali-c55 Date: Wed, 29 May 2024 16:28:51 +0100 Message-Id: <20240529152858.183799-10-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082927_661922_346A4707 X-CRM114-Status: GOOD ( 25.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a header that describes the format of the 3A statistics buffers for the mali-c55 ISP. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - New patch .../uapi/linux/media/arm/mali-c55-config.h | 182 ++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 include/uapi/linux/media/arm/mali-c55-config.h diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h new file mode 100644 index 000000000000..8fb89af6c874 --- /dev/null +++ b/include/uapi/linux/media/arm/mali-c55-config.h @@ -0,0 +1,182 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * ARM Mali-C55 ISP Driver - Userspace API + * + * Copyright (C) 2023 Ideas on Board Oy + */ + +#ifndef __UAPI_MALI_C55_CONFIG_H +#define __UAPI_MALI_C55_CONFIG_H + +#include + +/* + * Frames are split into zones of almost equal width and height - a zone is a + * rectangular tile of a frame. The metering blocks within the ISP collect + * aggregated statistics per zone. A maximum of 15x15 zones can be configured, + * and so the statistics buffer within the hardware is sized to accommodate + * that. + * + * The utilised number of zones is runtime configurable. + */ +#define MALI_C55_MAX_ZONES 225 + +/** + * struct mali_c55_ae_1024bin_hist - Auto Exposure 1024-bin histogram statistics + * + * @bins: 1024 element array of 16-bit pixel counts. + * + * The 1024-bin histogram module collects image-global but zone-weighted + * intensity distributions of pixels in fixed-width bins. The modules can be + * configured into different "plane modes" which affect the contents of the + * collected statistics. In plane mode 0, pixel intensities are taken regardless + * of colour plane into a single 1024-bin histogram with a bin width of 4. In + * plane mode 1, four 256-bin histograms with a bin width of 16 are collected - + * one for each CFA colour plane. In plane modes 4, 5, 6 and 7 two 512-bin + * histograms with a bin width of 8 are collected - in each mode one of the + * colour planes is collected into the first histogram and all the others are + * combined into the second. The histograms are stored consecutively in the bins + * array. + * + * The 16-bit pixel counts are stored as a 4-bit exponent in the most + * significant bits followed by a 12-bit mantissa. Conversion to a usable + * format can be done according to the following pseudo-code:: + * + * if (e == 0) { + * bin = m * 2; + * } else { + * bin = (m + 4096) x 2^e + * } + * + * where + * e is the exponent value in range 0..15 + * m is the mantissa value in range 0..4095 + * + * The pixels used in calculating the statistics can be masked using three + * methods: + * + * 1. Pixels can be skipped in X and Y directions independently. + * 2. Minimum/Maximum intensities can be configured + * 3. Zones can be differentially weighted, including 0 weighted to mask them + * + * The data for this histogram can be collected from different tap points in the + * ISP depending on configuration - after the white balance or digital gain + * blocks, or immediately after the input crossbar. + */ +struct mali_c55_ae_1024bin_hist { + __u16 bins[1024]; +} __attribute__((packed)); + +/** + * struct mali_c55_ae_5bin_hist - Auto Exposure 5-bin histogram statistics + * + * @hist0: 16-bit normalised pixel count for the 0th intensity bin + * @hist1: 16-bit normalised pixel count for the 1st intensity bin + * @hist3: 16-bit normalised pixel count for the 3rd intensity bin + * @hist4: 16-bit normalised pixel count for the 4th intensity bin + * + * The ISP generates a 5-bin histogram of normalised pixel counts within bins of + * pixel intensity for each of 225 possible zones within a frame. The centre bin + * of the histogram for each zone is not available from the hardware and must be + * calculated by subtracting the values of hist0, hist1, hist3 and hist4 from + * 0xffff as in the following equation: + * + * hist2 = 0xffff - (hist0 + hist1 + hist3 + hist4) + */ +struct mali_c55_ae_5bin_hist { + __u16 hist0; + __u16 hist1; + __u16 hist3; + __u16 hist4; +} __attribute__((packed)); + +/** + * struct mali_c55_awb_average_ratios - Auto White Balance colour ratios + * + * @avg_rg_gr: Average R/G or G/R ratio in Q4.8 format. + * @avg_bg_br: Average B/G or B/R ratio in Q4.8 format. + * @num_pixels: The number of pixels used in the AWB calculation + * + * The ISP calculates and collects average colour ratios for each zone in an + * image and stores them in Q4.8 format (the lowest 8 bits are fractional, with + * bits [11:8] representing the integer). The exact ratios collected (either + * R/G, B/G or G/R, B/R) are configurable through the parameters buffer. The + * value of the 4 high bits is undefined. + */ +struct mali_c55_awb_average_ratios { + __u16 avg_rg_gr; + __u16 avg_bg_br; + __u32 num_pixels; +} __attribute__((packed)); + +/** + * struct mali_c55_af_statistics - Auto Focus edge and intensity statistics + * + * @intensity_stats: Packed mantissa and exponent value for pixel intensity + * @edge_stats: Packed mantissa and exponent values for edge intensity + * + * The ISP collects the squared sum of pixel intensities for each zone within a + * configurable Region of Interest on the frame. Additionally, the same data are + * collected after being passed through a bandpass filter which removes high and + * low frequency components - these are referred to as the edge statistics. + * + * The intensity and edge statistics for a zone can be used to calculate the + * contrast information for a zone + * + * C = E2 / I2 + * + * Where I2 is the intensity statistic for a zone and E2 is the edge statistic + * for that zone. Optimum focus is reached when C is at its maximum. + * + * The intensity and edge statistics are stored packed into a non-standard 16 + * bit floating point format, where the 7 most significant bits represent the + * exponent and the 9 least significant bits the mantissa. This format can be + * unpacked with the following pseudocode:: + * + * if (e == 0) { + * x = m; + * } else { + * x = 2^e-1 x (m + 2^9) + * } + * + * where + * e is the exponent value in range 0..128 + * m is the mantissa value in range 0..511 + */ +struct mali_c55_af_statistics { + __u16 intensity_stats; + __u16 edge_stats; +} __attribute__((packed)); + +/** + * struct mali_c55_stats_buffer - 3A statistics for the mali-c55 ISP + * + * @ae_1024bin_hist: 1024-bin frame-global pixel intensity histogram + * @iridix_1024bin_hist: Post-Iridix block 1024-bin histogram + * @ae_5bin_hists: 5-bin pixel intensity histograms for AEC + * @reserved1: Undefined buffer space + * @awb_ratios: Color balance ratios for Auto White Balance + * @reserved2: Undefined buffer space + * @af_statistics: Pixel intensity statistics for Auto Focus + * @reserved3: Undefined buffer space + * + * This struct describes the metering statistics space in the Mali-C55 ISP's + * hardware in its entirety. The space between each defined area is marked as + * "unknown" and may not be 0, but should not be used. The @ae_5bin_hists, + * @awb_ratios and @af_statistics members are arrays of statistics per-zone. + * The zones are arranged in the array in raster order starting from the top + * left corner of the image. + */ + +struct mali_c55_stats_buffer { + struct mali_c55_ae_1024bin_hist ae_1024bin_hist; + struct mali_c55_ae_1024bin_hist iridix_1024bin_hist; + struct mali_c55_ae_5bin_hist ae_5bin_hists[MALI_C55_MAX_ZONES]; + __u32 reserved1[14]; + struct mali_c55_awb_average_ratios awb_ratios[MALI_C55_MAX_ZONES]; + __u32 reserved2[14]; + struct mali_c55_af_statistics af_statistics[MALI_C55_MAX_ZONES]; + __u32 reserved3[15]; +} __attribute__((packed)); + +#endif /* __UAPI_MALI_C55_CONFIG_H */ From patchwork Wed May 29 15:28:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A9D80C25B75 for ; Wed, 29 May 2024 15:30:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ErjoUqzZNfbbpigh/pvKDFkjVZrtVAsUkxA2ZumvmMs=; b=4eOXCc83GIeiMU k5+2/AFTTOdPDVWjMqaNpZOWnpBwUq0eKrYuDrvNqcaHMMHTuZkV28IhC4KyvOABglyc9eEXepep2 L5MIxmihMhMkwmRWrYXrGiNccx4Ki+kyeDxCsaV0uu4SVQIvje322r8MeScvWQfgjx0Rg/LvDPzIs K4lKYTxcZm+utMdlLLlrJo5IzJnBNV1BqTf95gG52N2xyWcHYJDRZaL1ElwSJDtucxzxjFzGVbuwv s/hi59RMJE+VO/yZRkeUgrQk2j8y93+9tdReou0JHKuVjVVhEPh0c7rxb9SYdhvKK4b09ZEK2AsXY m/5YGWgr92yfwQpRIbEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFv-00000004hnm-2tN3; Wed, 29 May 2024 15:30:23 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF1-00000004h6a-2T4p for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:34 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ED6692F62; Wed, 29 May 2024 17:29:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996562; bh=XxcGSyPAiUZ/hh5eIE+k+9lAAocmer1k9qTXVmiX/bw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uFxecbGh6SzBSyIAXuF66EmBBRlZJeU6JfZJ8AgMuGkl/n/vyLmIO/qteYxwWg7up f6rKNQhxDxXRUwrOO6JRDhJkQRj+xOZqNA5u9UnA+3AOc5o6Lz/Rh7VEVqRUp+yGO3 e9aC1V2cSLQglEmJ4KwCVJzooQubiProqYaAfI7k= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 10/16] media: platform: Add mali-c55 3a stats devnode Date: Wed, 29 May 2024 16:28:52 +0100 Message-Id: <20240529152858.183799-11-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082928_643206_71861008 X-CRM114-Status: GOOD ( 24.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a new code file to govern the 3a statistics capture node. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - New patch drivers/media/platform/arm/mali-c55/Makefile | 3 +- .../platform/arm/mali-c55/mali-c55-common.h | 28 ++ .../platform/arm/mali-c55/mali-c55-core.c | 15 + .../platform/arm/mali-c55/mali-c55-isp.c | 1 + .../arm/mali-c55/mali-c55-registers.h | 3 + .../platform/arm/mali-c55/mali-c55-stats.c | 350 ++++++++++++++++++ 6 files changed, 399 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile index 77dcb2fbf0f4..cd5a64bf0c62 100644 --- a/drivers/media/platform/arm/mali-c55/Makefile +++ b/drivers/media/platform/arm/mali-c55/Makefile @@ -4,6 +4,7 @@ mali-c55-y := mali-c55-capture.o \ mali-c55-core.o \ mali-c55-isp.o \ mali-c55-tpg.o \ - mali-c55-resizer.o + mali-c55-resizer.o \ + mali-c55-stats.o obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h index 2d0c4d152beb..44119e04009b 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h @@ -79,6 +79,7 @@ enum mali_c55_isp_pads { MALI_C55_ISP_PAD_SINK_VIDEO, MALI_C55_ISP_PAD_SOURCE, MALI_C55_ISP_PAD_SOURCE_BYPASS, + MALI_C55_ISP_PAD_SOURCE_3A, MALI_C55_ISP_NUM_PADS, }; @@ -194,6 +195,28 @@ struct mali_c55_cap_dev { bool streaming; }; +struct mali_c55_stats_buf { + struct vb2_v4l2_buffer vb; + spinlock_t lock; + unsigned int segments_remaining; + struct list_head queue; + bool failed; +}; + +struct mali_c55_stats { + struct mali_c55 *mali_c55; + struct video_device vdev; + struct dma_chan *channel; + struct vb2_queue queue; + struct media_pad pad; + struct mutex lock; + + struct { + spinlock_t lock; + struct list_head queue; + } buffers; +}; + enum mali_c55_config_spaces { MALI_C55_CONFIG_PING, MALI_C55_CONFIG_PONG, @@ -224,6 +247,7 @@ struct mali_c55 { struct mali_c55_isp isp; struct mali_c55_resizer resizers[MALI_C55_NUM_RZRS]; struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS]; + struct mali_c55_stats stats; struct list_head contexts; enum mali_c55_config_spaces next_config; @@ -245,6 +269,8 @@ int mali_c55_register_resizers(struct mali_c55 *mali_c55); void mali_c55_unregister_resizers(struct mali_c55 *mali_c55); int mali_c55_register_capture_devs(struct mali_c55 *mali_c55); void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55); +int mali_c55_register_stats(struct mali_c55 *mali_c55); +void mali_c55_unregister_stats(struct mali_c55 *mali_c55); struct mali_c55_ctx *mali_c55_get_active_context(struct mali_c55 *mali_c55); void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev, enum mali_c55_planes plane); @@ -262,5 +288,7 @@ mali_c55_isp_fmt_next(const struct mali_c55_isp_fmt *fmt); bool mali_c55_isp_is_format_supported(unsigned int mbus_code); #define for_each_mali_isp_fmt(fmt)\ for ((fmt) = NULL; ((fmt) = mali_c55_isp_fmt_next((fmt)));) +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55, + enum mali_c55_config_spaces cfg_space); #endif /* _MALI_C55_COMMON_H */ diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c index 50caf5ee7474..9ea70010876c 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c @@ -337,6 +337,16 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55) } } + ret = media_create_pad_link(&mali_c55->isp.sd.entity, + MALI_C55_ISP_PAD_SOURCE_3A, + &mali_c55->stats.vdev.entity, 0, + MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE); + if (ret) { + dev_err(mali_c55->dev, + "failed to link ISP and 3a stats node\n"); + goto err_remove_links; + } + return 0; err_remove_links: @@ -350,6 +360,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55) mali_c55_unregister_isp(mali_c55); mali_c55_unregister_resizers(mali_c55); mali_c55_unregister_capture_devs(mali_c55); + mali_c55_unregister_stats(mali_c55); } static int mali_c55_register_entities(struct mali_c55 *mali_c55) @@ -372,6 +383,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55) if (ret) goto err_unregister_entities; + ret = mali_c55_register_stats(mali_c55); + if (ret) + goto err_unregister_entities; + ret = mali_c55_create_links(mali_c55); if (ret) goto err_unregister_entities; diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c index ea8b7b866e7a..94876fba3353 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c @@ -564,6 +564,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55) isp->pads[MALI_C55_ISP_PAD_SINK_VIDEO].flags = MEDIA_PAD_FL_SINK; isp->pads[MALI_C55_ISP_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE; isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE; + isp->pads[MALI_C55_ISP_PAD_SOURCE_3A].flags = MEDIA_PAD_FL_SOURCE; ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS, isp->pads); diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h index cb27abde2aa5..eb3719245ec3 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h @@ -68,6 +68,9 @@ #define MALI_C55_VC_START(v) ((v) & 0xffff) #define MALI_C55_VC_SIZE(v) (((v) & 0xffff) << 16) +#define MALI_C55_REG_1024BIN_HIST 0x054a8 +#define MALI_C55_1024BIN_HIST_SIZE 4096 + /* Ping/Pong Configuration Space */ #define MALI_C55_REG_BASE_ADDR 0x18e88 #define MALI_C55_REG_BYPASS_0 0x18eac diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c new file mode 100644 index 000000000000..aa40480ed814 --- /dev/null +++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c @@ -0,0 +1,350 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * ARM Mali-C55 ISP Driver - 3A Statistics capture device + * + * Copyright (C) 2023 Ideas on Board Oy + */ + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "mali-c55-common.h" +#include "mali-c55-registers.h" + +static unsigned int metering_space_addrs[] = { + [MALI_C55_CONFIG_PING] = 0x095AC, + [MALI_C55_CONFIG_PONG] = 0x2156C, +}; + +static int mali_c55_stats_enum_fmt_meta_cap(struct file *file, void *fh, + struct v4l2_fmtdesc *f) +{ + if (f->index || f->type != V4L2_BUF_TYPE_META_CAPTURE) + return -EINVAL; + + f->pixelformat = V4L2_META_FMT_MALI_C55_3A_STATS; + + return 0; +} + +static int mali_c55_stats_g_fmt_meta_cap(struct file *file, void *fh, + struct v4l2_format *f) +{ + static const struct v4l2_meta_format mfmt = { + .dataformat = V4L2_META_FMT_MALI_C55_3A_STATS, + .buffersize = sizeof(struct mali_c55_stats_buffer) + }; + + if (f->type != V4L2_BUF_TYPE_META_CAPTURE) + return -EINVAL; + + f->fmt.meta = mfmt; + + return 0; +} + +static int mali_c55_stats_querycap(struct file *file, + void *priv, struct v4l2_capability *cap) +{ + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver)); + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card)); + + return 0; +} + +static const struct v4l2_ioctl_ops mali_c55_stats_v4l2_ioctl_ops = { + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, + .vidioc_enum_fmt_meta_cap = mali_c55_stats_enum_fmt_meta_cap, + .vidioc_g_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap, + .vidioc_s_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap, + .vidioc_try_fmt_meta_cap = mali_c55_stats_g_fmt_meta_cap, + .vidioc_querycap = mali_c55_stats_querycap, + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +static const struct v4l2_file_operations mali_c55_stats_v4l2_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = video_ioctl2, + .open = v4l2_fh_open, + .release = vb2_fop_release, + .poll = vb2_fop_poll, + .mmap = vb2_fop_mmap, +}; + +static int +mali_c55_stats_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, + unsigned int *num_planes, unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct mali_c55_stats *stats = vb2_get_drv_priv(q); + + if (*num_planes && *num_planes > 1) + return -EINVAL; + + if (sizes[0] && sizes[0] != sizeof(struct mali_c55_stats_buffer)) + return -EINVAL; + + *num_planes = 1; + sizes[0] = sizeof(struct mali_c55_stats_buffer); + + if (stats->channel) + alloc_devs[0] = stats->channel->device->dev; + + return 0; +} + +static void mali_c55_stats_buf_queue(struct vb2_buffer *vb) +{ + struct mali_c55_stats *stats = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct mali_c55_stats_buf *buf = container_of(vbuf, + struct mali_c55_stats_buf, vb); + + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_stats_buffer)); + buf->segments_remaining = 2; + buf->failed = false; + + spin_lock(&stats->buffers.lock); + list_add_tail(&buf->queue, &stats->buffers.queue); + spin_unlock(&stats->buffers.lock); +} + +static void mali_c55_stats_stop_streaming(struct vb2_queue *q) +{ + struct mali_c55_stats *stats = vb2_get_drv_priv(q); + struct mali_c55_stats_buf *buf, *tmp; + + spin_lock(&stats->buffers.lock); + + list_for_each_entry_safe(buf, tmp, &stats->buffers.queue, queue) { + list_del(&buf->queue); + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + } + + spin_unlock(&stats->buffers.lock); +} + +static const struct vb2_ops mali_c55_stats_vb2_ops = { + .queue_setup = mali_c55_stats_queue_setup, + .buf_queue = mali_c55_stats_buf_queue, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, + .stop_streaming = mali_c55_stats_stop_streaming, +}; + +static void +mali_c55_stats_metering_complete(void *param, + const struct dmaengine_result *result) +{ + struct mali_c55_stats_buf *buf = param; + + spin_lock(&buf->lock); + + if (buf->failed) + goto out_unlock; + + buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns(); + + if (result->result != DMA_TRANS_NOERROR) { + buf->failed = true; + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + goto out_unlock; + } + + if (!--buf->segments_remaining) + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE); + +out_unlock: + spin_unlock(&buf->lock); +} + +static int mali_c55_stats_dma_xfer(struct mali_c55_stats *stats, dma_addr_t src, + dma_addr_t dst, + struct mali_c55_stats_buf *buf, + size_t length, + void (*callback)(void *, const struct dmaengine_result *result)) +{ + struct dma_async_tx_descriptor *tx; + dma_cookie_t cookie; + + tx = dmaengine_prep_dma_memcpy(stats->channel, dst, src, length, 0); + if (!tx) { + dev_err(stats->mali_c55->dev, "failed to prep stats DMA\n"); + return -EIO; + } + + tx->callback_result = callback; + tx->callback_param = buf; + + cookie = dmaengine_submit(tx); + if (dma_submit_error(cookie)) { + dev_err(stats->mali_c55->dev, "failed to submit stats DMA\n"); + return -EIO; + } + + dma_async_issue_pending(stats->channel); + return 0; +} + +void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55, + enum mali_c55_config_spaces cfg_space) +{ + struct mali_c55_ctx *ctx = mali_c55_get_active_context(mali_c55); + struct mali_c55_stats *stats = &mali_c55->stats; + struct mali_c55_stats_buf *buf = NULL; + dma_addr_t src, dst; + int ret; + + spin_lock(&stats->buffers.lock); + if (!list_empty(&stats->buffers.queue)) { + buf = list_first_entry(&stats->buffers.queue, + struct mali_c55_stats_buf, queue); + list_del(&buf->queue); + } + spin_unlock(&stats->buffers.lock); + + if (!buf) + return; + + buf->vb.sequence = mali_c55->isp.frame_sequence; + + /* + * There are infact two noncontiguous sections of the ISP's + * memory space that hold statistics for 3a algorithms to use. A + * section in each config space and a global section holding + * histograms which is double buffered and so holds data for the + * last frame. We need to read both. + */ + src = ctx->base + MALI_C55_REG_1024BIN_HIST; + dst = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0); + + ret = mali_c55_stats_dma_xfer(stats, src, dst, buf, + MALI_C55_1024BIN_HIST_SIZE, + mali_c55_stats_metering_complete); + if (ret) + goto err_fail_buffer; + + src = ctx->base + metering_space_addrs[cfg_space]; + dst += MALI_C55_1024BIN_HIST_SIZE; + + ret = mali_c55_stats_dma_xfer( + stats, src, dst, buf, + sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE, + mali_c55_stats_metering_complete); + if (ret) { + dmaengine_terminate_sync(stats->channel); + goto err_fail_buffer; + } + + return; + +err_fail_buffer: + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); +} + +void mali_c55_unregister_stats(struct mali_c55 *mali_c55) +{ + struct mali_c55_stats *stats = &mali_c55->stats; + + if (!video_is_registered(&stats->vdev)) + return; + + vb2_video_unregister_device(&stats->vdev); + media_entity_cleanup(&stats->vdev.entity); + dma_release_channel(stats->channel); + mutex_destroy(&stats->lock); +} + +int mali_c55_register_stats(struct mali_c55 *mali_c55) +{ + struct mali_c55_stats *stats = &mali_c55->stats; + struct video_device *vdev = &stats->vdev; + struct vb2_queue *vb2q = &stats->queue; + dma_cap_mask_t mask; + int ret; + + mutex_init(&stats->lock); + INIT_LIST_HEAD(&stats->buffers.queue); + + dma_cap_zero(mask); + dma_cap_set(DMA_MEMCPY, mask); + + stats->channel = dma_request_channel(mask, 0, NULL); + if (!stats->channel) { + ret = -ENODEV; + goto err_destroy_mutex; + } + + stats->pad.flags = MEDIA_PAD_FL_SINK; + ret = media_entity_pads_init(&stats->vdev.entity, 1, &stats->pad); + if (ret) + goto err_release_dma_channel; + + vb2q->type = V4L2_BUF_TYPE_META_CAPTURE; + vb2q->io_modes = VB2_MMAP | VB2_DMABUF; + vb2q->drv_priv = stats; + vb2q->mem_ops = &vb2_dma_contig_memops; + vb2q->ops = &mali_c55_stats_vb2_ops; + vb2q->buf_struct_size = sizeof(struct mali_c55_stats_buf); + vb2q->min_queued_buffers = 1; + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + vb2q->lock = &stats->lock; + vb2q->dev = mali_c55->dev; + + ret = vb2_queue_init(vb2q); + if (ret) { + dev_err(mali_c55->dev, "stats vb2 queue init failed\n"); + goto err_cleanup_entity; + } + + strscpy(stats->vdev.name, "mali-c55 3a stats", sizeof(stats->vdev.name)); + vdev->release = video_device_release_empty; + vdev->fops = &mali_c55_stats_v4l2_fops; + vdev->ioctl_ops = &mali_c55_stats_v4l2_ioctl_ops; + vdev->lock = &stats->lock; + vdev->v4l2_dev = &mali_c55->v4l2_dev; + vdev->queue = &stats->queue; + vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING; + vdev->vfl_dir = VFL_DIR_RX; + video_set_drvdata(vdev, stats); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) { + dev_err(mali_c55->dev, + "failed to register stats video device\n"); + goto err_release_vb2q; + } + + stats->mali_c55 = mali_c55; + + return 0; + +err_release_vb2q: + vb2_queue_release(vb2q); +err_cleanup_entity: + media_entity_cleanup(&stats->vdev.entity); +err_release_dma_channel: + dma_release_channel(stats->channel); +err_destroy_mutex: + mutex_destroy(&stats->lock); + + return ret; +} From patchwork Wed May 29 15:28:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679114 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9DD60C25B75 for ; Wed, 29 May 2024 15:30:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VN1+I59atpKlIZ1mqak9VGgWDdjkPAae6vnPiqhmKsI=; b=LeZaX29UnJ7GyJ vGVQh5zHInCZmX4wowcndHoGbTeRshyO0+ddi2Lsh4QwXvQkoBvksxa/wauDc7Rw/UYjOxrzkVHUr 0aG/q5UDzeZFp6CxVV8dT+k5zNPjfF28aMQE33wnj1WRD41Z3sGw6kXDK7kE5+xXjVEe1qA/axvCV ctzMq30ONHdvEdzXUfQERlvs4OBusPdYd2HouHhLYmNgwqj9mRPMXW0kTABdHIABKLlC9o2Ei84da yARiReqLdBleFEpOYykRBoWKsujVLRj+YiwW3TFS49mIhHYebT1Ydj34KRy36B1bLZfAZiaWjeJ9T CmcCrAgWlrM/OYPn5Xfg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFa-00000004hWO-2cUH; Wed, 29 May 2024 15:30:02 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF1-00000004h72-2Zva for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:30 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9573F2F79; Wed, 29 May 2024 17:29:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996563; bh=1VZKD/7iXkmlYhIgC1bcHRQA7bBFUpGOqE0+BrQtj9I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vElfIQmjIYkFvXgK2ynoHG95MW/051It+iKrA7QK6wtQtq+8GAcKljmKHLvEFRvBq 0Ntx2QRKNe9MWPsO58ULGqUKe1HWgFSDd3EVInPFEuogPbIyp1JBrBay0p7J0+DuqH 1Q5tdWCRmb3nuS199C/6CkazSTsx/8LylaXni5aA= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 11/16] media: platform: Fill stats buffer on ISP_START Date: Wed, 29 May 2024 16:28:53 +0100 Message-Id: <20240529152858.183799-12-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082928_569688_0E276B72 X-CRM114-Status: GOOD ( 14.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On ISP_START, fill the stats buffer by reading out the metering space in the ISP's memory. This is done for the non-active config just as the dma transfer of the registers is. To acheive that, move the checking of the current config outside of mali_c55_swap_next_config() so we can use it for both functions. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - New patch .../platform/arm/mali-c55/mali-c55-core.c | 35 ++++++++++++++----- 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c index 9ea70010876c..2cf8b1169604 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c @@ -420,15 +420,9 @@ static u32 mali_c55_check_hwcfg(struct mali_c55 *mali_c55) return version; } -static void mali_c55_swap_next_config(struct mali_c55 *mali_c55) +static void mali_c55_swap_next_config(struct mali_c55 *mali_c55, u32 next_config) { struct mali_c55_ctx *ctx = mali_c55_get_active_context(mali_c55); - u32 curr_config, next_config; - - curr_config = mali_c55_read(mali_c55, MALI_C55_REG_PING_PONG_READ, false); - curr_config = (curr_config & MALI_C55_REG_PING_PONG_READ_MASK) - >> (ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1); - next_config = curr_config ^ 1; mali_c55_update_bits(mali_c55, MALI_C55_REG_MCU_CONFIG, MALI_C55_REG_MCU_CONFIG_WRITE_MASK, next_config); @@ -440,6 +434,7 @@ static irqreturn_t mali_c55_isr(int irq, void *context) { struct device *dev = context; struct mali_c55 *mali_c55 = dev_get_drvdata(dev); + u32 curr_config, next_config; u32 interrupt_status; unsigned int i, j; @@ -465,7 +460,31 @@ static irqreturn_t mali_c55_isr(int irq, void *context) for (j = i; j < MALI_C55_NUM_CAP_DEVS; j++) mali_c55_set_next_buffer(&mali_c55->cap_devs[j]); - mali_c55_swap_next_config(mali_c55); + /* + * When the ISP starts a frame we have some work to do: + * + * 1. Copy over the config for the **next** frame + * 2. Read out the metering stats for the **last** frame + */ + + curr_config = mali_c55_read(mali_c55, + MALI_C55_REG_PING_PONG_READ, + false); + curr_config &= MALI_C55_REG_PING_PONG_READ_MASK; + curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1; + next_config = curr_config ^ 1; + + /* + * The ordering of these two is currently important as + * mali_c55_stats_fill_buffer() is asynchronous whereas + * mali_c55_swap_next_config() is not. + * + * TODO: Should mali_c55_swap_next_config() be async? + */ + mali_c55_stats_fill_buffer(mali_c55, + next_config ? MALI_C55_CONFIG_PING : + MALI_C55_CONFIG_PONG); + mali_c55_swap_next_config(mali_c55, next_config); break; case MALI_C55_IRQ_ISP_DONE: From patchwork Wed May 29 15:28:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA05DC27C43 for ; Wed, 29 May 2024 15:30:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=I4sZqtTB6Q2OOFHD6NecU3UHg7sqh36DGDBbd2kGu1s=; b=oq461j6HnjAduB laf/zFeklqGPpY0JARHTyVrU3ogOkD4kc+8KKK/3vYzFx3jsRg32M5Du0qu33hmjFGZkSaHmSVyZ8 I2aq5XIdRL5MoKW4lVucMGYFtJ93PBfVkNgdEAfeewKDEy2fDtEONHJVHQ3p+3YMXe+Dwrgm8eClM 9z7zImALv53aIW5fAc52wxoB2zTibxQHOLkHRbfcRPZrZSYdsxULP0W9htnh9Q2SNrxqTwLHRxE4N kOjjWBBSw0AxAB0IocrtcdzsyHfKl9umHTVJBPFq6TJt0tNhx6m8DHzV4YI9ZOnsgbIf4PCehaANx bxA6JndSNn98Ka0g/15w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFm-00000004hh9-47Go; Wed, 29 May 2024 15:30:15 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF2-00000004h7S-1igg for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:34 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3926B2F86; Wed, 29 May 2024 17:29:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996563; bh=ZiO//0gHAM83B+PZxsN+u0NS0mRuPH0tT19a/LMI4rM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EyhYkb+yzYewzndGbS6PY6qFvwM5zWNqIbYLwoMO9Wz99db1OyRnFovA93zccXlPE rtODjoR4i4UMR00Hn83f/JPAJMdbjtMAPuY5e2+aBH5A4unziMBhSHRAYHu1bZS5rn vu6MQSnG/Ssq+U3fMpU77OPRJvv3r8ClWp601j/I= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 12/16] Documentation: mali-c55: Add Statistics documentation Date: Wed, 29 May 2024 16:28:54 +0100 Message-Id: <20240529152858.183799-13-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082928_746387_C9E6E784 X-CRM114-Status: GOOD ( 17.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add documentation explaining the ability to capture statistics from the mali-c55 driver's new V4L2 device, as well as the various tap points from which those statistics can be drawn in the ISP's processing flow. Additionally add a page detailing the new V4L2 meta format for the mali-c55 statistics. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - New patch Documentation/admin-guide/media/mali-c55.rst | 60 ++++++++++++++++++- .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-arm-mali-c55.rst | 28 +++++++++ 3 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst index cf4176cb1e44..b75437f6e96a 100644 --- a/Documentation/admin-guide/media/mali-c55.rst +++ b/Documentation/admin-guide/media/mali-c55.rst @@ -67,10 +67,11 @@ The driver has 4 V4L2 subdevices: - `mali_c55 resizer fr`: The Full-Resolution pipe resizer - `mali_c55 resizer ds`: The Downscale pipe resizer -The driver has 2 V4L2 video devices: +The driver has 3 V4L2 video devices: - `mali-c55 fr`: The full-resolution pipe's capture device - `mali-c55 ds`: The downscale pipe's capture device +- `mali-c55 3a stats`: The 3A statistics capture device Frame sequences are synchronised across to two capture devices, meaning if one pipe is started later than the other the sequence numbers returned in its @@ -326,6 +327,63 @@ configured, followed by formats in the appropriate places: # Set format on the video device and stream yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 +.. _mali-c55-3a-stats: + +Capturing ISP Statistics +======================== + +The ISP is capable of producing statistics for consumption by image processing +algorithms running in userspace. These statistics can be captured by queueing +buffers to the `mali-c55 3a stats` V4L2 Device whilst the ISP is streaming. Only +the :ref:`V4L2_META_FMT_MALI_C55_3A_STATS ` +format is supported, so no format-setting need be done: + +.. code-block:: none + + # We assume the media graph has been configured to support RGB565 capture + # from the mali-c55 fr V4L2 Device, which is at /dev/video0. The statistics + # V4L2 device is at /dev/video3 + + yavta -f RGB565 -s 1920x1080 -c32 /dev/video0 && \ + yavta -c10 -F /dev/video3 + +The layout of the buffer is described by :c:type:`mali_c55_stats_buffer`, +but broadly statistics are generated to support three image processing +algorithms; AEXP (Auto-Exposure), AWB (Auto-White Balance) and AF (Auto-Focus). +These stats can be drawn from various places in the Mali C55 ISP pipeline, known +as "tap points". This high-level block diagram is intended to explain where in +the processing flow the statistics can be drawn from:: + + +--> AEXP-2 +----> AEXP-1 +--> AF-0 + | +----> AF-1 | + | | | + +---------+ | +--------------+ | +--------------+ | + | Input +-+-->+ Digital Gain +---+-->+ Black Level +---+---+ + +---------+ +--------------+ +--------------+ | + +-----------------------------------------------------------------+ + | + | +--------------+ +---------+ +----------------+ + +-->| Sinter Noise +-+ White +--+--->| Lens Shading +--+---------------+ + | Reduction | | Balance | | | | | | + +--------------+ +---------+ | +----------------+ | | + +---> AEXP-0 (A) +--> AEXP-0 (B) | + +--------------------------------------------------------------------------+ + | + | +----------------+ +--------------+ +----------------+ + +-->| Tone mapping +-+--->| Demosaicing +->+ Purple Fringe +-+-----------+ + | | | +--------------+ | Correction | | | + +----------------+ +-> AEXP-IRIDIX +----------------+ +---> AWB-0 | + +----------------------------------------------------------------------------+ + | +-------------+ +-------------+ + +------------------->| Colour +---+--->| Output | + | Correction | | | Pipelines | + +-------------+ | +-------------+ + +--> AWB-1 + +At present all statistics are drawn from the 0th tap point for each algorithm; +I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF +statistics from AF-0. In the future this will be configurable. + References ========== .. [1] https://git.linuxtv.org/v4l-utils.git/ diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst index c23aac823d2c..73c05cc35321 100644 --- a/Documentation/userspace-api/media/v4l/meta-formats.rst +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst @@ -15,6 +15,7 @@ These formats are used for the :ref:`metadata` interface only. metafmt-d4xx metafmt-generic metafmt-intel-ipu3 + metafmt-arm-mali-c55 metafmt-rkisp1 metafmt-uvc metafmt-vivid diff --git a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst new file mode 100644 index 000000000000..219a5dd42d70 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst @@ -0,0 +1,28 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _v4l2-meta-fmt-mali-c55-3a-stats: + +**************************************** +V4L2_META_FMT_MALI_C55_3A_STATS ('C55S') +**************************************** + +3A Statistics +============= +The ISP device collects different statistics over an input bayer frame. Those +statistics can be obtained by userspace from the +:ref:`mali-c55 3a stats ` metadata capture video node, using +the :c:type:`v4l2_meta_format` interface. The buffer contains a single instance +of the C structure :c:type:`mali_c55_stats_buffer` defined in +``mali-c55-config.h``, so the structure can be obtained from the buffer by: + +.. code-block:: C + + struct mali_c55_stats_buffer *stats = + (struct mali_c55_stats_buffer *)buf; + +For details of the statistics see :c:type:`mali_c55_stats_buffer`. + +Arm Mali-C55 uAPI data types +============================ + +.. kernel-doc:: include/uapi/linux/media/arm/mali-c55-config.h From patchwork Wed May 29 15:28:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31979C25B75 for ; Wed, 29 May 2024 15:30:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SZuolUd/H8/o5uvqtZKNPP2/Ei0ab+EXanQ1l9EzRoc=; b=mJnEZX8JSLwtsR +HV6Xytp8u8otXqlQanv7TIFdBxNzdwTnTh8aVpyBnnqbGM72JOE2fJgyyyP8BytqSqHpC6DnbGWW /P0CbP3/Io9ZnGQS7ngYqzWWfbILuJwYxkEtwpNjwb7M6O24Ff+4V4e0cwGDGb8eYjUS6eMHHs5Ii WsIo/ieoAR+BWjIAy5KiPKuni1C7zQLNdr96UcZUwPfHHq+zCBs/EIVu44Sj5na7SzSI7Cbak+v9i WOqxXJfR/0zRLLzW2QdTAJENjkA6Phz1Hs4nGigw9WYgfMEFX/+P1Asf8QCPiE6eapkSAPcbE+yqv FKyfTQpEThpb3Ra6Iokw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLFg-00000004hbI-2PAc; Wed, 29 May 2024 15:30:08 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF3-00000004h7m-0ROV for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:34 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D36942F89; Wed, 29 May 2024 17:29:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996564; bh=QqaKyYHvtgjn6nCjsW06V2tgVeolZTDMF2g07H8vRfo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pWXJUDUSCxnd1iROwRRmFRvCBGl6zPOe/nUnJD6vpKn8nu93xrqQBNlLAADoKTWCL IO5z/bMqs9q7qu0H4N+4V+wCAUZnWFf4i6yi0/HIOZuO0DOZXA3yYfcQTSOUrR4qy3 PvOrAbg5yG2R8VvfLN9TNbLN4JxDjBDYiut623s8= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 13/16] media: mali-c55: Add image formats for Mali-C55 parameters buffer Date: Wed, 29 May 2024 16:28:55 +0100 Message-Id: <20240529152858.183799-14-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082929_361400_08506391 X-CRM114-Status: UNSURE ( 9.81 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Jacopo Mondi Add a new V4L2 meta format code for the Mali-C55 parameters Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - New patch drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 2 files changed, 2 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 0d00b0476762..4e3b5e16571c 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break; case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A Parameters"; break; case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A Statistics"; break; + case V4L2_META_FMT_MALI_C55_PARAMS: descr = "ARM Mali-C55 ISP Parameters"; break; case V4L2_META_FMT_MALI_C55_3A_STATS: descr = "ARM Mali-C55 ISP 3A Statistics"; break; case V4L2_PIX_FMT_NV12_8L128: descr = "NV12 (8x128 Linear)"; break; case V4L2_PIX_FMT_NV12M_8L128: descr = "NV12M (8x128 Linear)"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 021424fdaf68..e63d5e76d21e 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -841,6 +841,7 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_MALI_C55_PARAMS v4l2_fourcc('C', '5', '5', 'P') /* ARM Mali-C55 Parameters */ #define V4L2_META_FMT_MALI_C55_3A_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */ #ifdef __KERNEL__ From patchwork Wed May 29 15:28:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 946B4C25B7E for ; Wed, 29 May 2024 15:39:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cC7bt0VRYZ8hXIlE9bt66/rTbeCLgR3GrDq9AcNs6Dg=; b=XG5b3yYTYJ7HQ9 QfoSSQLeQ43yQIG9d0I3o3uNH71tuJxSofVi3H0d9EAuEIQBZGufoMrJW+j0zv2FLzTsZuHZWue1T IYmLlvAuj81gG6BnPGS6MaLqAL0jXiYLs36dQL2SGeV3lpU8XyMs0oZ2NrJHAqii121Lf/Hx3/Pvt +r9e3/HmJ4SRHEgd31vITm0sDHL/X0aNwujV9KivsAEpUbH9gy1j7fd/8fS+Te75KKPrBCaYqOC8u 19tuIyE7rfU2zaPU+QSY7ijhELgPU50X78RnF8kexo6oPtWTYiZuiEjrX7V1sUSPA+X1/m9ySYIdb hAoXFAeH+yEOHJv4BJCg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLOI-00000004jaR-02rU; Wed, 29 May 2024 15:39:02 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLOC-00000004jXo-43Di for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:38:59 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 856BB2F8E; Wed, 29 May 2024 17:29:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996565; bh=WovLB8lOhufb1//Lf4DA4asm2rqwLu8GYezDV4//2Sc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YPU8AA2THUBSXSR5v0zLAMspGntMWI3kBMVTrsNb9n7dz4Ng4imDQmPbDEra7d+lV 1cWGEAdmsA941aD+PKtycWwREoXJzqJjrWoDNc4tHduvy3On+QIpQpCyGtomWPsOC9 CBvqvO3JF2PnaevAPM48NbIBll9NaoF22/fn1V8Q= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 14/16] media: uapi: Add parameters structs to mali-c55-config.h Date: Wed, 29 May 2024 16:28:56 +0100 Message-Id: <20240529152858.183799-15-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_083857_313609_604B18D3 X-CRM114-Status: GOOD ( 24.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add structures describing the ISP parameters to mali-c55-config.h Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - New patch .../uapi/linux/media/arm/mali-c55-config.h | 669 ++++++++++++++++++ 1 file changed, 669 insertions(+) diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h index 8fb89af6c874..fce14bc74f4a 100644 --- a/include/uapi/linux/media/arm/mali-c55-config.h +++ b/include/uapi/linux/media/arm/mali-c55-config.h @@ -179,4 +179,673 @@ struct mali_c55_stats_buffer { __u32 reserved3[15]; } __attribute__((packed)); +/** + * enum mali_c55_param_buffer_version - Mali-C55 parameters block versioning + * + * @MALI_C55_PARAM_BUFFER_V0: First version of Mali-C55 parameters block + */ +enum mali_c55_param_buffer_version { + MALI_C55_PARAM_BUFFER_V0, +}; + +/** + * enum mali_c55_param_block_type - Enumeration of Mali-C55 parameter blocks + * + * This enumeration defines the types of Mali-C55 parameters block. Each block + * configures a specific processing block of the Mali-C55 ISP. The block + * type allows the driver to correctly interpret the parameters block data. + * + * It is the responsibility of userspace to correctly set the type of each + * parameters block. + * + * @MALI_C55_PARAM_BLOCK_SENSOR_OFFS: Sensor pre-shading black level offset + * @MALI_C55_PARAM_BLOCK_AEXP_HIST: Auto-exposure 1024-bin histogram + * configuration + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST: Post-Iridix auto-exposure 1024-bin + * histogram configuration + * @MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS: Auto-exposure 1024-bin histogram + * weighting + * @MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS: Post-Iridix auto-exposure 1024-bin + * histogram weighting + * @MALI_C55_PARAM_BLOCK_DIGITAL_GAIN: Digital gain + * @MALI_C55_PARAM_BLOCK_AWB_GAINS: Auto-white balance gains + * @MALI_C55_PARAM_BLOCK_AWB_CONFIG: Auto-white balance statistics config + * @MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP: Auto-white balance gains for AEXP-0 tap + * @MALI_C55_PARAM_MESH_SHADING_CONFIG : Mesh shading tables configuration + * @MALI_C55_PARAM_MESH_SHADING_SELECTION: Mesh shading table selection + * @MALI_C55_PARAM_BLOCK_SENTINEL: First non-valid block index + */ +enum mali_c55_param_block_type { + MALI_C55_PARAM_BLOCK_SENSOR_OFFS, + MALI_C55_PARAM_BLOCK_AEXP_HIST, + MALI_C55_PARAM_BLOCK_AEXP_IHIST, + MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS, + MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS, + MALI_C55_PARAM_BLOCK_DIGITAL_GAIN, + MALI_C55_PARAM_BLOCK_AWB_GAINS, + MALI_C55_PARAM_BLOCK_AWB_CONFIG, + MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP, + MALI_C55_PARAM_MESH_SHADING_CONFIG, + MALI_C55_PARAM_MESH_SHADING_SELECTION, + MALI_C55_PARAM_BLOCK_SENTINEL, +}; + +/** + * struct mali_c55_params_block_header - Mali-C55 parameter block header + * + * This structure represents the common part of all the ISP configuration + * blocks. Each parameters block shall embed an instance of this structure type + * as its first member, followed by the block-specific configuration data. The + * driver inspects this common header to discern the block type and its size and + * properly handle the block content by casting it to the correct block-specific + * type. + * + * The @type field is one of the values enumerated by + * :c:type:`mali_c55_param_block_type` and specifies how the data should be + * interpreted by the driver. The @size field specifies the size of the + * parameters block and is used by the driver for validation purposes. The + * @enabled field specifies if the ISP block should be enabled (and configured + * according to the provided parameters) or disabled. + * + * .. code-block:: c + * + * struct mali_c55_params_block_header *block = ...; + * + * switch (block->type) { + * case MALI_C55_PARAM_BLOCK_SENSOR_OFFS: + * struct mali_c55_params_sensor_off_preshading *sensor_offs = + * (struct mali_c55_params_sensor_off_preshading *)block; + * + * if (block->size != + * sizeof(struct mali_c55_params_sensor_off_preshading)) + * return -EINVAL; + * + * handle_sensor_offs(sensor_offs); + * break; + * + * case MALI_C55_PARAM_BLOCK_AEXP_HIST: + * struct mali_c55_params_aexp_hist *aexp_hist = + * (struct mali_c55_params_aexp_hist)block; + * + * if (block->size != sizeof(mali_c55_params_aexp_hist)) + * return -EINVAL; + * + * handle_aexp_hist(aesp_hist); + * break; + * + * ... + * + * } + * + * Userspace is responsible for correctly populating the parameters block header + * fields (@type, @enabled and @size) and correctly populate the block-specific + * parameters. + * + * For example: + * + * .. code-block:: c + * + * void populate_sensor_offs(struct mali_c55_params_block_header *block) { + * block->type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; + * block->enabled = true; + * block->size = sizeof(struct mali_c55_params_sensor_off_preshading); + * + * struct mali_c55_params_sensor_off_preshading *sensor_offs = + * (struct mali_c55_params_sensor_off_preshading *)block; + * + * sensor_offs->chan00 = offset00; + * sensor_offs->chan01 = offset01; + * sensor_offs->chan10 = offset10; + * sensor_offs->chan11 = offset11; + * } + * + * @type: The parameters block type (enum mali_c55_param_block_type) + * @enabled: Block enabled/disabled flag + * @size: Size (in bytes) of the parameters block + */ +struct mali_c55_params_block_header { + enum mali_c55_param_block_type type; + bool enabled; + size_t size; +}; + +/** + * struct mali_c55_params_sensor_off_preshading - offset subtraction for each + * color channel + * + * Provides removal of the sensor black level from the sensor data. Separate + * offsets are provided for each of the four Bayer component color channels + * which are defaulted to R, Gr, Gb, B. + * + * @header: The Mali-C55 parameters block header + * @chan00: Offset for color channel 00 (default: R) + * @chan01: Offset for color channel 01 (default: Gr) + * @chan10: Offset for color channel 10 (default: Gb) + * @chan11: Offset for color channel 11 (default: B) + */ +struct mali_c55_params_sensor_off_preshading { + struct mali_c55_params_block_header header; + __u32 chan00; + __u32 chan01; + __u32 chan10; + __u32 chan11; +}; + +/** + * enum mali_c55_aexp_hist_tap_points - Tap points for the AEXP histogram + * @MALI_C55_AEXP_HIST_TAP_WB: After static white balance + * @MALI_C55_AEXP_HIST_TAP_FS: After WDR Frame Stitch + * @MALI_C55_AEXP_HIST_TAP_TPG: After the test pattern generator + */ +enum mali_c55_aexp_hist_tap_points { + MALI_C55_AEXP_HIST_TAP_WB = 0, + MALI_C55_AEXP_HIST_TAP_FS, + MALI_C55_AEXP_HIST_TAP_TPG, +}; + +/** + * enum mali_c55_aexp_skip_x - Horizontal pixel skipping + * @MALI_C55_AEXP_SKIP_X_EVERY_2ND: Collect every 2nd pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_3RD: Collect every 3rd pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_4TH: Collect every 4th pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_5TH: Collect every 5th pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_8TH: Collect every 8th pixel horizontally + * @MALI_C55_AEXP_SKIP_X_EVERY_9TH: Collect every 9th pixel horizontally + */ +enum mali_c55_aexp_skip_x { + MALI_C55_AEXP_SKIP_X_EVERY_2ND, + MALI_C55_AEXP_SKIP_X_EVERY_3RD, + MALI_C55_AEXP_SKIP_X_EVERY_4TH, + MALI_C55_AEXP_SKIP_X_EVERY_5TH, + MALI_C55_AEXP_SKIP_X_EVERY_8TH, + MALI_C55_AEXP_SKIP_X_EVERY_9TH +}; + +/** + * enum mali_c55_aexp_skip_y - Vertical pixel skipping + * @MALI_C55_AEXP_SKIP_Y_ALL: Collect every single pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_2ND: Collect every 2nd pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_3RD: Collect every 3rd pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_4TH: Collect every 4th pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_5TH: Collect every 5th pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_8TH: Collect every 8th pixel vertically + * @MALI_C55_AEXP_SKIP_Y_EVERY_9TH: Collect every 9th pixel vertically + */ +enum mali_c55_aexp_skip_y { + MALI_C55_AEXP_SKIP_Y_ALL, + MALI_C55_AEXP_SKIP_Y_EVERY_2ND, + MALI_C55_AEXP_SKIP_Y_EVERY_3RD, + MALI_C55_AEXP_SKIP_Y_EVERY_4TH, + MALI_C55_AEXP_SKIP_Y_EVERY_5TH, + MALI_C55_AEXP_SKIP_Y_EVERY_8TH, + MALI_C55_AEXP_SKIP_Y_EVERY_9TH +}; + +/** + * enum mali_c55_aexp_row_column_offset - Start from the first or second row or + * column + * @MALI_C55_AEXP_FIRST_ROW_OR_COL: Start from the first row / column + * @MALI_C55_AEXP_SECOND_ROW_OR_COL: Start from the second row / column + */ +enum mali_c55_aexp_row_column_offset { + MALI_C55_AEXP_FIRST_ROW_OR_COL = 1, + MALI_C55_AEXP_SECOND_ROW_OR_COL = 2, +}; + +/** + * enum mali_c55_aexp_hist_plane_mode - Mode for the AEXP Histograms + * @MALI_C55_AEXP_HIST_COMBINED: All color planes in one 1024-bin histogram + * @MALI_C55_AEXP_HIST_SEPARATE: Each color plane in one 256-bin histogram with a bin width of 16 + * @MALI_C55_AEXP_HIST_FOCUS_00: Top left plane in the first bank, rest in second bank + * @MALI_C55_AEXP_HIST_FOCUS_01: Top right plane in the first bank, rest in second bank + * @MALI_C55_AEXP_HIST_FOCUS_10: Bottom left plane in the first bank, rest in second bank + * @MALI_C55_AEXP_HIST_FOCUS_11: Bottom right plane in the first bank, rest in second bank + * + * In the "focus" modes statistics are collected into two 512-bin histograms + * with a bin width of 8. One colour plane is in the first histogram with the + * remainder combined into the second. The four options represent which of the + * four positions in a bayer pattern are the focused plane. + */ +enum mali_c55_aexp_hist_plane_mode { + MALI_C55_AEXP_HIST_COMBINED = 0, + MALI_C55_AEXP_HIST_SEPARATE = 1, + MALI_C55_AEXP_HIST_FOCUS_00 = 4, + MALI_C55_AEXP_HIST_FOCUS_01 = 5, + MALI_C55_AEXP_HIST_FOCUS_10 = 6, + MALI_C55_AEXP_HIST_FOCUS_11 = 7, +}; + +/** + * struct mali_c55_params_aexp_hist - configuration for AEXP metering hists + * + * This struct allows users to configure the 1024-bin AEXP histograms. Broadly + * speaking the parameters allow you to mask particular regions of the image and + * to select different kinds of histogram. + * + * @header: The Mali-C55 parameters block header + * @skip_x: Horizontal decimation. See enum mali_c55_aexp_skip_x + * @offset_x: Column to start from. See enum mali_c55_aexp_row_column_offset + * @skip_y: Vertical decimation. See enum mali_c55_aexp_skip_y + * @offset_y: Row to start from. See enum mali_c55_aexp_row_column_offset + * @scale_bottom: scale of bottom half of range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x + * @scale_top: scale of top half of range: 0=1x ,1=2x, 2=4x, 4=8x, 4=16x + * @plane_mode: Plane separation mode. See enum mali_c55_aexp_hist_plane_mode + * @tap_point: Tap point for histogram from enum mali_c55_aexp_hist_tap_points. + * This parameter is unused for the post-Iridix Histogram + */ +struct mali_c55_params_aexp_hist { + struct mali_c55_params_block_header header; + __u8 skip_x; + __u8 offset_x; + __u8 skip_y; + __u8 offset_y; + __u8 scale_bottom; + __u8 scale_top; + __u8 plane_mode; + __u8 tap_point; +}; + +/** + * struct mali_c55_params_aexp_weights - Array of weights for AEXP metering + * + * This struct allows users to configure the weighting for both of the 1024-bin + * AEXP histograms. The pixel data collected for each zone is multiplied by the + * corresponding weight from this array, which may be zero if the intention is + * to mask off the zone entirely. + * + * @header: The Mali-C55 parameters block header + * @nodes_used_horiz: Number of active zones horizontally [0..15] + * @nodes_used_vert: Number of active zones vertically [0..15] + * @zone_weights: Zone weighting. Index is row*col where 0,0 is the top + * left zone continuing in raster order. Each zone can be + * weighted in the range [0..15]. The number of rows and + * columns is defined by @nodes_used_vert and + * @nodes_used_horiz + */ +struct mali_c55_params_aexp_weights { + struct mali_c55_params_block_header header; + __u8 nodes_used_horiz; + __u8 nodes_used_vert; + __u8 zone_weights[MALI_C55_MAX_ZONES]; +}; + +/** + * struct mali_c55_params_digital_gain - Digital gain value + * + * This struct carries a digital gain value to set in the ISP + * + * @header: The Mali-C55 parameters block header + * @gain: The digital gain value to apply, in Q5.8 format. + */ +struct mali_c55_params_digital_gain { + struct mali_c55_params_block_header header; + __u16 gain; +}; + +/** + * enum mali_c55_awb_stats_mode - Statistics mode for AWB + * @MALI_C55_AWB_MODE_GRBR: Statistics collected as Green/Red and Blue/Red ratios + * @MALI_C55_AWB_MODE_RGBG: Statistics collected as Red/Green and Blue/Green ratios + */ +enum mali_c55_awb_stats_mode { + MALI_C55_AWB_MODE_GRBR = 0, + MALI_C55_AWB_MODE_RGBG, +}; + +/** + * struct mali_c55_params_awb_gains - Gain settings for auto white balance + * + * This struct allows users to configure the gains for auto-white balance. There + * are four gain settings corresponding to each colour channel in the bayer + * domain. Although named generically, the association between the gain applied + * and the colour channel is done automatically within the ISP depending on the + * input format, and so the following mapping always holds true:: + * + * gain00 = R + * gain01 = Gr + * gain10 = Gb + * gain11 = B + * + * All of the gains are stored in Q4.8 format. + * + * @header: The Mali-C55 parameters block header + * @gain00: Multiplier for colour channel 00 + * @gain01: Multiplier for colour channel 01 + * @gain10: Multiplier for colour channel 10 + * @gain11: Multiplier for colour channel 11 + */ +struct mali_c55_params_awb_gains { + struct mali_c55_params_block_header header; + __u16 gain00; + __u16 gain01; + __u16 gain10; + __u16 gain11; +}; + +/** + * enum mali_c55_params_awb_tap_points - Tap points for the AWB statistics + * @MALI_C55_AWB_STATS_TAP_PF: Immediately after the Purple Fringe block + * @MALI_C55_AWB_STATS_TAP_CNR: Immediately after the CNR block + */ +enum mali_c55_params_awb_tap_points { + MALI_C55_AWB_STATS_TAP_PF = 0, + MALI_C55_AWB_STATS_TAP_CNR, +}; + +/** + * struct mali_c55_params_awb_config - Stats settings for auto-white balance + * + * This struct allows the configuration of the statistics generated for auto + * white balance. Pixel intensity limits can be set to exclude overly bright or + * dark regions of an image from the statistics entirely. Colour ratio minima + * and maxima can be set to discount pixels who's ratios fall outside the + * defined boundaries; there are two sets of registers to do this - the + * "min/max" ratios which bound a region and the "high/low" ratios which further + * trim the upper and lower ratios. For example with the boundaries configured + * as follows, only pixels whos colour ratios falls into the region marked "A" + * would be counted:: + * + * cr_high + * 2.0 | | + * | cb_max --> _________________________v_____ + * 1.8 | | \ | + * | | \ | + * 1.6 | | \ | + * | | \ | + * c 1.4 | cb_low -->|\ A \|<-- cb_high + * b | | \ | + * 1.2 | | \ | + * r | | \ | + * a 1.0 | cb_min --> |____\_________________________| + * t | ^ ^ ^ + * i 0.8 | | | | + * o | cr_min | cr_max + * s 0.6 | | + * | cr_low + * 0.4 | + * | + * 0.2 | + * | + * 0.0 |_______________________________________________________________ + * 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 + * cr ratios + * + * @header: The Mali-C55 parameters block header + * @tap_point: The tap point from enum mali_c55_params_awb_tap_points + * @stats_mode: AWB statistics collection mode, see :c:type:`mali_c55_awb_stats_mode` + * @white_level: Upper pixel intensity (I.E. raw pixel values) limit + * @black_level: Lower pixel intensity (I.E. raw pixel values) limit + * @cr_max: Maximum R/G ratio (Q4.8 format) + * @cr_min: Minimum R/G ratio (Q4.8 format) + * @cb_max: Maximum B/G ratio (Q4.8 format) + * @cb_min: Minimum B/G ratio (Q4.8 format) + * @nodes_used_horiz: Number of active zones horizontally [0..15] + * @nodes_used_vert: Number of active zones vertically [0..15] + * @cr_high: R/G ratio trim high (Q4.8 format) + * @cr_low: R/G ratio trim low (Q4.8 format) + * @cb_high: B/G ratio trim high (Q4.8 format) + * @cb_low: B/G ratio trim low (Q4.8 format) + */ +struct mali_c55_params_awb_config { + struct mali_c55_params_block_header header; + __u8 tap_point; + __u8 stats_mode; + __u16 white_level; + __u16 black_level; + __u16 cr_max; + __u16 cr_min; + __u16 cb_max; + __u16 cb_min; + __u8 nodes_used_horiz; + __u8 nodes_used_vert; + __u16 cr_high; + __u16 cr_low; + __u16 cb_high; + __u16 cb_low; +}; + +#define MALI_C55_NUM_MESH_SHADING_ELEMENTS 3072 + +/** + * struct mali_c55_params_mesh_shading_config - Mesh shading configuration + * + * The mesh shading correction module allows programming a separate table of + * either 16x16 or 32x32 node coefficients for 3 different light sources. The + * final correction coefficients applied are computed by blending the + * coefficients from two tables together. + * + * A page of 1024 32-bit integers is associated to each colour channel, with + * pages stored consecutively in memory. Each 32-bit integer packs 3 8-bit + * correction coefficients for a single node, one for each of the three light + * sources. The 8 most significant bits are unused. The following table + * describes the layout:: + * + * +----------- Page (Colour Plane) 0 -------------+ + * | @mesh[i] | Mesh Point | Bits | Light Source | + * +-----------+------------+-------+--------------+ + * | 0 | 0,0 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | 1 | 0,1 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | ... | ... | ... | ... | + * +-----------+------------+-------+--------------+ + * | 1023 | 31,31 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +----------- Page (Colour Plane) 1 -------------+ + * | @mesh[i] | Mesh Point | Bits | Light Source | + * +-----------+------------+-------+--------------+ + * | 1024 | 0,0 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | 1025 | 0,1 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | ... | ... | ... | ... | + * +-----------+------------+-------+--------------+ + * | 2047 | 31,31 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +----------- Page (Colour Plane) 2 -------------+ + * | @mesh[i] | Mesh Point | Bits | Light Source | + * +-----------+------------+-------+--------------+ + * | 2048 | 0,0 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | 2049 | 0,1 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * | ... | ... | ... | ... | + * +-----------+------------+-------+--------------+ + * | 3071 | 31,31 | 16,23 | LS2 | + * | | | 08-15 | LS1 | + * | | | 00-07 | LS0 | + * +-----------+------------+-------+--------------+ + * + * The @mesh_scale member determines the precision and minimum and maximum gain. + * For example if @mesh_scale is 0 and therefore selects 0 - 2x gain, a value of + * 0 in a coefficient means 0.0 gain, a value of 128 means 1.0 gain and 255 + * means 2.0 gain. + * + * @header: The Mali-C55 parameters block header + * @mesh_show: Output the mesh data rather than image data + * @mesh_scale: Set the precision and maximum gain range of mesh shading + * - 0 = 0-2x gain + * - 1 = 0-4x gain + * - 2 = 0-8x gain + * - 3 = 0-16x gain + * - 4 = 1-2x gain + * - 5 = 1-3x gain + * - 6 = 1-5x gain + * - 7 = 1-9x gain + * @mesh_page_r: Mesh page select for red colour plane [0..2] + * @mesh_page_g: Mesh page select for green colour plane [0..2] + * @mesh_page_b: Mesh page select for blue colour plane [0..2] + * @mesh_width: Number of horizontal nodes minus 1 [15,31] + * @mesh_height: Number of vertical nodes minus 1 [15,31] + * @mesh: Mesh shading correction tables + */ +struct mali_c55_params_mesh_shading_config { + struct mali_c55_params_block_header header; + bool mesh_show; + __u8 mesh_scale; + __u8 mesh_page_r; + __u8 mesh_page_g; + __u8 mesh_page_b; + __u8 mesh_width; + __u8 mesh_height; + __u32 mesh[MALI_C55_NUM_MESH_SHADING_ELEMENTS]; +}; + +/** enum mali_c55_params_mesh_alpha_bank - Mesh shading table bank selection + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 - Select Light Sources 0 and 1 + * @MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 - Select Light Sources 1 and 2 + * @MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 - Select Light Sources 0 and 2 + */ +enum mali_c55_params_mesh_alpha_bank { + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS1 = 0, + MALI_C55_MESH_ALPHA_BANK_LS1_AND_LS2 = 1, + MALI_C55_MESH_ALPHA_BANK_LS0_AND_LS2 = 4 +}; + +/** + * struct mali_c55_params_mesh_shading_selection - Mesh table selection + * + * The module computes the final correction coefficients by blending the ones + * from two light source tables, which are selected (independently for each + * colour channel) by the @mesh_alpha_bank_r/g/b fields. + * + * The final blended coefficients for each node are calculated using the + * following equation: + * + * Final coefficient = (a x LS\ :sub:`b`\ + (256 - a) x LS\ :sub:`a`\) / 256 + * + * Where a is the @mesh_alpha_r/g/b value, and LS\ :sub:`a`\ and LS\ :sub:`b`\ + * are the node cofficients for the two tables selected by the + * @mesh_alpha_bank_r/g/b value. + * + * The scale of the applied correction may also be controlled by tuning the + * @mesh_strength member. This is a modifier to the final coefficients which can + * be used to globally reduce the gains applied. + * + * @header: The Mali-C55 parameters block header + * @mesh_alpha_bank_r: Red mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`) + * @mesh_alpha_bank_g: Green mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`) + * @mesh_alpha_bank_b: Blue mesh table select (c:type:`enum mali_c55_params_mesh_alpha_bank`) + * @mesh_alpha_r: Blend coefficient for R [0..255] + * @mesh_alpha_g: Blend coefficient for G [0..255] + * @mesh_alpha_b: Blend coefficient for B [0..255] + * @mesh_strength: Mesh strength in Q4.12 format [0..4096] + */ +struct mali_c55_params_mesh_shading_selection { + struct mali_c55_params_block_header header; + __u8 mesh_alpha_bank_r; + __u8 mesh_alpha_bank_g; + __u8 mesh_alpha_bank_b; + __u8 mesh_alpha_r; + __u8 mesh_alpha_g; + __u8 mesh_alpha_b; + __u16 mesh_strength; +}; + +/** + * define MALI_C55_PARAMS_MAX_SIZE - Maximum size of all Mali C55 Parameters + * + * Though the parameters for the Mali-C55 are passed as optional blocks, the + * driver still needs to know the absolute maximum size so that it can allocate + * a buffer sized appropriately to accomodate userspace attempting to set all + * possible parameters in a single frame. + */ +#define MALI_C55_PARAMS_MAX_SIZE \ + sizeof(struct mali_c55_params_sensor_off_preshading) + \ + sizeof(struct mali_c55_params_aexp_hist) + \ + sizeof(struct mali_c55_params_aexp_weights) + \ + sizeof(struct mali_c55_params_aexp_hist) + \ + sizeof(struct mali_c55_params_aexp_weights) + \ + sizeof(struct mali_c55_params_digital_gain) + \ + sizeof(struct mali_c55_params_awb_gains) + \ + sizeof(struct mali_c55_params_awb_config) + \ + sizeof(struct mali_c55_params_awb_gains) + \ + sizeof(struct mali_c55_params_mesh_shading_config) + \ + sizeof(struct mali_c55_params_mesh_shading_selection) + +/** + * struct mali_c55_params_buffer - 3A configuration parameters + * + * This struct contains the configuration parameters of the Mali-C55 ISP + * algorithms, serialized by userspace into an opaque data buffer. Each + * configuration parameter block is represented by a block-specific structure + * which contains a :c:type:`mali_c55_params_block_header` entry as first + * member. Userspace populates the @data buffer with configuration parameters + * for the blocks that it intends to configure. As a consequence, the data + * buffer effective size changes according to the number of ISP blocks that + * userspace intends to configure. + * + * The parameters buffer is versioned by the @version field to allow modifying + * and extending its definition. Userspace should populate the @version field to + * inform the driver about the version it intends to use. The driver will parse + * and handle the @data buffer according to the data layout specific to the + * indicated revision and return an error if the desired revision is not + * supported. + * + * For each ISP block that userspace wants to configure, a block-specific + * structure is appended to the @data buffer, one after the other without gaps + * in between nor overlaps. Userspace shall populate the @total_size field with + * the effective size, in bytes, of the @data buffer. + * + * The expected memory layout of the parameters buffer is:: + * + * +-------------------- struct mali_c55_params_buffer ------------------+ + * | version = MALI_C55_PARAM_BUFFER_V0; | + * | total_size = sizeof(struct mali_c55_params_sensor_off_preshading) | + * | sizeof(struct mali_c55_params_aexp_hist); | + * | +------------------------- data ---------------------------------+ | + * | | +--------- struct mali_c55_params_sensor_off_preshading ------+ | | + * | | | +-------- struct mali_c55_params_block_header header -----+ | | | + * | | | | type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; | | | | + * | | | | enabled = 1; | | | | + * | | | | size = | | | | + * | | | | sizeof(struct mali_c55_params_sensor_off_preshading);| | | | + * | | | +---------------------------------------------------------+ | | | + * | | | chan00 = ...; | | | + * | | | chan01 = ...; | | | + * | | | chan10 = ...; | | | + * | | | chan11 = ...; | | | + * | | +------------ struct mali_c55_params_aexp_hist ---------------+ | | + * | | | +-------- struct mali_c55_params_block_header header -----+ | | | + * | | | | type = MALI_C55_PARAM_BLOCK_AEXP_HIST; | | | | + * | | | | enabled = 1; | | | | + * | | | | size = sizeof(struct mali_c55_params_aexp_hist); | | | | + * | | | +---------------------------------------------------------+ | | | + * | | | skip_x = ...; | | | + * | | | offset_x = ...; | | | + * | | | skip_y = ...; | | | + * | | | offset_y = ...; | | | + * | | | scale_bottom = ...; | | | + * | | | scale_top = ...; | | | + * | | | plane_mode = ...; | | | + * | | | tap_point = ...; | | | + * | | +-------------------------------------------------------------+ | | + * | +-----------------------------------------------------------------+ | + * +---------------------------------------------------------------------+ + * + * @version: The Mali-C55 parameters buffer version + * @total_size: The Mali-C55 configuration data effective size, excluding this + * header + * @data: The Mali-C55 configuration blocks data + */ +struct mali_c55_params_buffer { + enum mali_c55_param_buffer_version version; + size_t total_size; + __u8 data[MALI_C55_PARAMS_MAX_SIZE]; +}; + #endif /* __UAPI_MALI_C55_CONFIG_H */ From patchwork Wed May 29 15:28:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8657BC25B75 for ; Wed, 29 May 2024 15:30:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xd1dwo1d8Ha/OUKk7956FlcLXWhvTnKAoF5A3c3ijus=; b=Eol+g+7OL1q6Os BoRV/GXMJmDMjVM4zlaB4ZZD2FR9b7MzblffFzBjva6YNV2woU/RQbvohiLtLgCVwecCiE5wQTTvt qxWfAWvrE4qQv+Dq6/8e1/eanYrjYHwkHWM5jSJeXg8heouZQSEPRVc7weSlySHVoLhjOYaM+iiGD b1csEh49ksAjzf3Rgf1lugL+tozpAEXOcKv5LgDvXwv/ETFhVjZuRh3xDWUaiu+Y0p25e5vwUpssS Wfyb6p+QRsY42VXoYMRNJ0SB+YxaOUf7yzc+uwCDa3eJwI9WkKEyA52YKoR7ZJQkPsUt3CYWzdEB3 sEFDDvdyA78jKWZ8lWrA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLG8-00000004hxU-1DN6; Wed, 29 May 2024 15:30:36 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF4-00000004hAO-3Sd4 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:40 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2F6F72F8F; Wed, 29 May 2024 17:29:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996565; bh=QSWCetl8QuRYivGAmsu6PYODKxPaUG0eotqt5EB1m/c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mtpnjI7I7jHapzKe90YKgKpbUP3LCdhuLx0k4l/ptqyHy2I1dIdhWooZnS04/FlzB RuAUExH4RkFJmrcFcqnseUp9VkJs4LoStl3Vj+P3YuEqd1bwZjYYRThfU+8onn0gi8 nBPI9tFTgJu9M3iPba24DsewENxYjfEVjPY1M3is= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 15/16] media: platform: Add mali-c55 parameters video node Date: Wed, 29 May 2024 16:28:57 +0100 Message-Id: <20240529152858.183799-16-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082931_422906_27E01230 X-CRM114-Status: GOOD ( 30.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a new code file to the mali-c55 driver that registers an output video node for userspace to queue buffers of parameters to. Handlers are included to program the statistics generation plus the white balance, black level correction and mesh shading correction blocks. Update the rest of the driver to register and link the new video node Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - New patch drivers/media/platform/arm/mali-c55/Makefile | 1 + .../platform/arm/mali-c55/mali-c55-common.h | 18 + .../platform/arm/mali-c55/mali-c55-core.c | 24 + .../platform/arm/mali-c55/mali-c55-isp.c | 16 +- .../platform/arm/mali-c55/mali-c55-params.c | 615 ++++++++++++++++++ .../arm/mali-c55/mali-c55-registers.h | 104 +++ 6 files changed, 777 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c diff --git a/drivers/media/platform/arm/mali-c55/Makefile b/drivers/media/platform/arm/mali-c55/Makefile index cd5a64bf0c62..b2443f2d416a 100644 --- a/drivers/media/platform/arm/mali-c55/Makefile +++ b/drivers/media/platform/arm/mali-c55/Makefile @@ -5,6 +5,7 @@ mali-c55-y := mali-c55-capture.o \ mali-c55-isp.o \ mali-c55-tpg.o \ mali-c55-resizer.o \ + mali-c55-params.o \ mali-c55-stats.o obj-$(CONFIG_VIDEO_MALI_C55) += mali-c55.o diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-common.h b/drivers/media/platform/arm/mali-c55/mali-c55-common.h index 44119e04009b..565d98acfcdd 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-common.h +++ b/drivers/media/platform/arm/mali-c55/mali-c55-common.h @@ -80,6 +80,7 @@ enum mali_c55_isp_pads { MALI_C55_ISP_PAD_SOURCE, MALI_C55_ISP_PAD_SOURCE_BYPASS, MALI_C55_ISP_PAD_SOURCE_3A, + MALI_C55_ISP_PAD_SINK_PARAMS, MALI_C55_ISP_NUM_PADS, }; @@ -217,6 +218,19 @@ struct mali_c55_stats { } buffers; }; +struct mali_c55_params { + struct mali_c55 *mali_c55; + struct video_device vdev; + struct vb2_queue queue; + struct media_pad pad; + struct mutex lock; + + struct { + spinlock_t lock; + struct list_head queue; + } buffers; +}; + enum mali_c55_config_spaces { MALI_C55_CONFIG_PING, MALI_C55_CONFIG_PONG, @@ -247,6 +261,7 @@ struct mali_c55 { struct mali_c55_isp isp; struct mali_c55_resizer resizers[MALI_C55_NUM_RZRS]; struct mali_c55_cap_dev cap_devs[MALI_C55_NUM_CAP_DEVS]; + struct mali_c55_params params; struct mali_c55_stats stats; struct list_head contexts; @@ -271,6 +286,8 @@ int mali_c55_register_capture_devs(struct mali_c55 *mali_c55); void mali_c55_unregister_capture_devs(struct mali_c55 *mali_c55); int mali_c55_register_stats(struct mali_c55 *mali_c55); void mali_c55_unregister_stats(struct mali_c55 *mali_c55); +int mali_c55_register_params(struct mali_c55 *mali_c55); +void mali_c55_unregister_params(struct mali_c55 *mali_c55); struct mali_c55_ctx *mali_c55_get_active_context(struct mali_c55 *mali_c55); void mali_c55_set_plane_done(struct mali_c55_cap_dev *cap_dev, enum mali_c55_planes plane); @@ -290,5 +307,6 @@ bool mali_c55_isp_is_format_supported(unsigned int mbus_code); for ((fmt) = NULL; ((fmt) = mali_c55_isp_fmt_next((fmt)));) void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55, enum mali_c55_config_spaces cfg_space); +void mali_c55_params_write_config(struct mali_c55 *mali_c55); #endif /* _MALI_C55_COMMON_H */ diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c index 2cf8b1169604..6acee3edd03f 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c +++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c @@ -347,6 +347,17 @@ static int mali_c55_create_links(struct mali_c55 *mali_c55) goto err_remove_links; } + ret = media_create_pad_link(&mali_c55->params.vdev.entity, 0, + &mali_c55->isp.sd.entity, + MALI_C55_ISP_PAD_SINK_PARAMS, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + if (ret) { + dev_err(mali_c55->dev, + "failed to link ISP and parameters video node\n"); + goto err_remove_links; + } + return 0; err_remove_links: @@ -360,6 +371,7 @@ static void mali_c55_unregister_entities(struct mali_c55 *mali_c55) mali_c55_unregister_isp(mali_c55); mali_c55_unregister_resizers(mali_c55); mali_c55_unregister_capture_devs(mali_c55); + mali_c55_unregister_params(mali_c55); mali_c55_unregister_stats(mali_c55); } @@ -383,6 +395,10 @@ static int mali_c55_register_entities(struct mali_c55 *mali_c55) if (ret) goto err_unregister_entities; + ret = mali_c55_register_params(mali_c55); + if (ret) + goto err_unregister_entities; + ret = mali_c55_register_stats(mali_c55); if (ret) goto err_unregister_entities; @@ -474,6 +490,14 @@ static irqreturn_t mali_c55_isr(int irq, void *context) curr_config >>= ffs(MALI_C55_REG_PING_PONG_READ_MASK) - 1; next_config = curr_config ^ 1; + /* + * Write the configuration parameters received from + * userspace into the configuration buffer, which will + * be transferred to the 'next' active config space at + * by mali_c55_swap_next_config(). + */ + mali_c55_params_write_config(mali_c55); + /* * The ordering of these two is currently important as * mali_c55_stats_fill_buffer() is asynchronous whereas diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c index 94876fba3353..8c2b45bfd82d 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c @@ -146,6 +146,7 @@ static int mali_c55_isp_start(struct mali_c55 *mali_c55) cfg->encoding == V4L2_PIXEL_ENC_RGB ? MALI_C55_ISP_RAW_BYPASS_BYPASS_MASK : 0x00); + mali_c55_params_write_config(mali_c55); ret = mali_c55_config_write(ctx, MALI_C55_CONFIG_PING); if (ret) { dev_err(mali_c55->dev, "failed to DMA config\n"); @@ -455,8 +456,20 @@ static const struct v4l2_subdev_internal_ops mali_c55_isp_internal_ops = { .init_state = mali_c55_isp_init_state, }; +static int mali_c55_subdev_link_validate(struct media_link *link) +{ + /* + * Skip validation for the parameters sink pad, as the source is not + * a subdevice. + */ + if (link->sink->index == MALI_C55_ISP_PAD_SINK_PARAMS) + return 0; + + return v4l2_subdev_link_validate(link); +} + static const struct media_entity_operations mali_c55_isp_media_ops = { - .link_validate = v4l2_subdev_link_validate, + .link_validate = mali_c55_subdev_link_validate, }; static int mali_c55_isp_notifier_bound(struct v4l2_async_notifier *notifier, @@ -565,6 +578,7 @@ int mali_c55_register_isp(struct mali_c55 *mali_c55) isp->pads[MALI_C55_ISP_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE; isp->pads[MALI_C55_ISP_PAD_SOURCE_BYPASS].flags = MEDIA_PAD_FL_SOURCE; isp->pads[MALI_C55_ISP_PAD_SOURCE_3A].flags = MEDIA_PAD_FL_SOURCE; + isp->pads[MALI_C55_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK; ret = media_entity_pads_init(&sd->entity, MALI_C55_ISP_NUM_PADS, isp->pads); diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c new file mode 100644 index 000000000000..049a7b8e4861 --- /dev/null +++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c @@ -0,0 +1,615 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * ARM Mali-C55 ISP Driver - Configuration parameters output device + * + * Copyright (C) 2024 Ideas on Board Oy + */ +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "mali-c55-common.h" +#include "mali-c55-registers.h" + +typedef void (*mali_c55_block_handler)(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block); + +struct mali_c55_block_handler { + size_t size; + mali_c55_block_handler handler; +}; + +static void mali_c55_params_sensor_offs(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_sensor_off_preshading *p = + (struct mali_c55_params_sensor_off_preshading *)block; + __u32 global_offset; + + if (!block->enabled) + return; + + if (!(p->chan00 || p->chan01 || p->chan10 || p->chan11)) + return; + + mali_c55_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_00, + p->chan00 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK); + mali_c55_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_01, + p->chan01 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK); + mali_c55_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_10, + p->chan10 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK); + mali_c55_write(mali_c55, MALI_C55_REG_SENSOR_OFF_PRE_SHA_11, + p->chan11 & MALI_C55_SENSOR_OFF_PRE_SHA_MASK); + + /* + * The average offset is applied as a global offset for the digital + * gain block + */ + global_offset = (p->chan00 + p->chan01 + p->chan10 + p->chan11) >> 2; + mali_c55_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN_OFFSET, + MALI_C55_DIGITAL_GAIN_OFFSET_MASK, global_offset); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_BYPASS_3, + MALI_C55_REG_BYPASS_3_SENSOR_OFFSET_PRE_SH, 0x00); +} + +static void mali_c55_params_aexp_hist(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + u32 disable_mask = block->type == MALI_C55_PARAM_BLOCK_AEXP_HIST ? + MALI_C55_AEXP_HIST_DISABLE_MASK : + MALI_C55_AEXP_IHIST_DISABLE_MASK; + u32 base = block->type == MALI_C55_PARAM_BLOCK_AEXP_HIST ? + MALI_C55_REG_AEXP_HIST_BASE : + MALI_C55_REG_AEXP_IHIST_BASE; + struct mali_c55_params_aexp_hist *params = + (struct mali_c55_params_aexp_hist *)block; + + if (!block->enabled) { + mali_c55_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG, + disable_mask, true); + return; + } + + mali_c55_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG, + disable_mask, false); + + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET, + MALI_C55_AEXP_HIST_SKIP_X_MASK, params->skip_x); + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET, + MALI_C55_AEXP_HIST_OFFSET_X_MASK, params->offset_x); + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET, + MALI_C55_AEXP_HIST_SKIP_Y_MASK, params->skip_y); + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SKIP_OFFSET, + MALI_C55_AEXP_HIST_OFFSET_Y_MASK, params->offset_y); + + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET, + MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK, params->scale_bottom); + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_SCALE_OFFSET, + MALI_C55_AEXP_HIST_SCALE_TOP_MASK, params->scale_top); + + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET, + MALI_C55_AEXP_HIST_PLANE_MODE_MASK, params->plane_mode); + + if (block->type == MALI_C55_PARAM_BLOCK_AEXP_HIST) + mali_c55_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG, + MALI_C55_AEXP_HIST_SWITCH_MASK, + params->tap_point); +} + +static void +mali_c55_params_aexp_hist_weights(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_aexp_weights *params = + (struct mali_c55_params_aexp_weights *)block; + u32 base; + + if (!block->enabled) + return; + + base = block->type == MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS ? + MALI_C55_REG_AEXP_HIST_BASE : + MALI_C55_REG_AEXP_IHIST_BASE; + + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET, + MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK, params->nodes_used_horiz); + mali_c55_update_bits(mali_c55, base + MALI_C55_AEXP_HIST_NODES_USED_OFFSET, + MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK, params->nodes_used_vert); + + /* + * The zone weights array is a 225-element array of u8 values, but that + * is a bit annoying to handle given the ISP expects 32-bit writes. We + * just reinterpret it as a 57-element array of 32-bit values for the + * purposes of this transaction (the 3 bytes of additional space at the + * end of the write is just padding for the array of weights in the ISP + * memory space anyway, so there's no risk of overwriting other + * registers). + */ + for (unsigned int i = 0; i < 57; i++) { + u32 val = ((u32 *)params->zone_weights)[i] + & MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK; + u32 addr = base + MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET + (4 * i); + + mali_c55_write(mali_c55, addr, val); + } +} + +static void mali_c55_params_digital_gain(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_digital_gain *dgain = + (struct mali_c55_params_digital_gain *)block; + + /* + * If the block is flagged as disabled we write a gain of 1.0, which in + * Q5.8 format is 256. + */ + mali_c55_update_bits(mali_c55, MALI_C55_REG_DIGITAL_GAIN, + MALI_C55_DIGITAL_GAIN_MASK, + block->enabled ? dgain->gain : 256); +} + +static void mali_c55_params_awb_gains(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_awb_gains *gains = + (struct mali_c55_params_awb_gains *)block; + + /* + * There are two places AWB gains can be set in the ISP; one affects the + * image output data and the other affects the statistics for the + * AEXP-0 tap point. + */ + u32 addr1 = block->type = MALI_C55_PARAM_BLOCK_AWB_GAINS ? + MALI_C55_REG_AWB_GAINS1 : + MALI_C55_REG_AWB_GAINS1_AEXP; + u32 addr2 = block->type = MALI_C55_PARAM_BLOCK_AWB_GAINS ? + MALI_C55_REG_AWB_GAINS2 : + MALI_C55_REG_AWB_GAINS2_AEXP; + + mali_c55_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN00_MASK, + gains->gain00); + mali_c55_update_bits(mali_c55, addr1, MALI_C55_AWB_GAIN01_MASK, + gains->gain01); + mali_c55_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN10_MASK, + gains->gain10); + mali_c55_update_bits(mali_c55, addr2, MALI_C55_AWB_GAIN11_MASK, + gains->gain11); +} + +static void mali_c55_params_awb_config(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_awb_config *params = + (struct mali_c55_params_awb_config *)block; + + if (!block->enabled) { + mali_c55_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG, + MALI_C55_AWB_DISABLE_MASK, true); + return; + } + + mali_c55_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG, + MALI_C55_AWB_DISABLE_MASK, false); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_STATS_MODE, + MALI_C55_AWB_STATS_MODE_MASK, params->stats_mode); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_WHITE_LEVEL, + MALI_C55_AWB_WHITE_LEVEL_MASK, params->white_level); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_BLACK_LEVEL, + MALI_C55_AWB_BLACK_LEVEL_MASK, params->black_level); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MAX, + MALI_C55_AWB_CR_MAX_MASK, params->cr_max); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CR_MIN, + MALI_C55_AWB_CR_MIN_MASK, params->cr_min); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MAX, + MALI_C55_AWB_CB_MAX_MASK, params->cb_max); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CB_MIN, + MALI_C55_AWB_CB_MIN_MASK, params->cb_min); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED, + MALI_C55_AWB_NODES_USED_HORIZ_MASK, + params->nodes_used_horiz); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_NODES_USED, + MALI_C55_AWB_NODES_USED_VERT_MASK, + params->nodes_used_vert); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CR_HIGH, + MALI_C55_AWB_CR_HIGH_MASK, params->cr_high); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CR_LOW, + MALI_C55_AWB_CR_LOW_MASK, params->cr_low); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CB_HIGH, + MALI_C55_AWB_CB_HIGH_MASK, params->cb_high); + mali_c55_update_bits(mali_c55, MALI_C55_REG_AWB_CB_LOW, + MALI_C55_AWB_CB_LOW_MASK, params->cb_low); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_METERING_CONFIG, + MALI_C55_AWB_SWITCH_MASK, params->tap_point); +} + +static void mali_c55_params_lsc_config(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_mesh_shading_config *params = + (struct mali_c55_params_mesh_shading_config *)block; + unsigned int i; + u32 addr; + + if (!block->enabled) { + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_ENABLE_MASK, false); + return; + } + + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_ENABLE_MASK, true); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_MESH_SHOW, params->mesh_show); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_SCALE_MASK, + params->mesh_scale); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_PAGE_R_MASK, + params->mesh_page_r); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_PAGE_G_MASK, + params->mesh_page_g); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_PAGE_B_MASK, + params->mesh_page_b); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_MESH_WIDTH_MASK, + params->mesh_width); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_CONFIG, + MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK, + params->mesh_height); + + for (i = 0; i < MALI_C55_NUM_MESH_SHADING_ELEMENTS; i++) { + addr = MALI_C55_REG_MESH_SHADING_TABLES + (i * 4); + mali_c55_write(mali_c55, addr, params->mesh[i]); + } +} + +static void mali_c55_params_lsc_selection(struct mali_c55 *mali_c55, + struct mali_c55_params_block_header *block) +{ + struct mali_c55_params_mesh_shading_selection *params = + (struct mali_c55_params_mesh_shading_selection *)block; + + if (!block->enabled) + return; + + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK, + MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK, + params->mesh_alpha_bank_r); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK, + MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK, + params->mesh_alpha_bank_g); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA_BANK, + MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK, + params->mesh_alpha_bank_b); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA, + MALI_C55_MESH_SHADING_ALPHA_R_MASK, + params->mesh_alpha_r); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA, + MALI_C55_MESH_SHADING_ALPHA_G_MASK, + params->mesh_alpha_g); + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_ALPHA, + MALI_C55_MESH_SHADING_ALPHA_B_MASK, + params->mesh_alpha_b); + + mali_c55_update_bits(mali_c55, MALI_C55_REG_MESH_SHADING_MESH_STRENGTH, + MALI_c55_MESH_STRENGTH_MASK, + params->mesh_strength); +} + +static const struct mali_c55_block_handler mali_c55_block_handlers[] = { + [MALI_C55_PARAM_BLOCK_SENSOR_OFFS] = { + .size = sizeof(struct mali_c55_params_sensor_off_preshading), + .handler = &mali_c55_params_sensor_offs, + }, + [MALI_C55_PARAM_BLOCK_AEXP_HIST] = { + .size = sizeof(struct mali_c55_params_aexp_hist), + .handler = &mali_c55_params_aexp_hist, + }, + [MALI_C55_PARAM_BLOCK_AEXP_IHIST] = { + .size = sizeof(struct mali_c55_params_aexp_hist), + .handler = &mali_c55_params_aexp_hist, + }, + [MALI_C55_PARAM_BLOCK_AEXP_HIST_WEIGHTS] = { + .size = sizeof(struct mali_c55_params_aexp_weights), + .handler = &mali_c55_params_aexp_hist_weights, + }, + [MALI_C55_PARAM_BLOCK_AEXP_IHIST_WEIGHTS] = { + .size = sizeof(struct mali_c55_params_aexp_weights), + .handler = &mali_c55_params_aexp_hist_weights, + }, + [MALI_C55_PARAM_BLOCK_DIGITAL_GAIN] = { + .size = sizeof(struct mali_c55_params_digital_gain), + .handler = &mali_c55_params_digital_gain, + }, + [MALI_C55_PARAM_BLOCK_AWB_GAINS] = { + .size = sizeof(struct mali_c55_params_awb_gains), + .handler = &mali_c55_params_awb_gains, + }, + [MALI_C55_PARAM_BLOCK_AWB_CONFIG] = { + .size = sizeof(struct mali_c55_params_awb_config), + .handler = &mali_c55_params_awb_config, + }, + [MALI_C55_PARAM_BLOCK_AWB_GAINS_AEXP] = { + .size = sizeof(struct mali_c55_params_awb_gains), + .handler = &mali_c55_params_awb_gains, + }, + [MALI_C55_PARAM_MESH_SHADING_CONFIG] = { + .size = sizeof(struct mali_c55_params_mesh_shading_config), + .handler = &mali_c55_params_lsc_config, + }, + [MALI_C55_PARAM_MESH_SHADING_SELECTION] = { + .size = sizeof(struct mali_c55_params_mesh_shading_selection), + .handler = &mali_c55_params_lsc_selection, + }, +}; + +static int mali_c55_params_enum_fmt_meta_out(struct file *file, void *fh, + struct v4l2_fmtdesc *f) +{ + if (f->index || f->type != V4L2_BUF_TYPE_META_OUTPUT) + return -EINVAL; + + f->pixelformat = V4L2_META_FMT_MALI_C55_PARAMS; + + return 0; +} + +static int mali_c55_params_g_fmt_meta_out(struct file *file, void *fh, + struct v4l2_format *f) +{ + static const struct v4l2_meta_format mfmt = { + .dataformat = V4L2_META_FMT_MALI_C55_PARAMS, + .buffersize = sizeof(struct mali_c55_params_buffer), + }; + + if (f->type != V4L2_BUF_TYPE_META_OUTPUT) + return -EINVAL; + + f->fmt.meta = mfmt; + + return 0; +} + +static int mali_c55_params_querycap(struct file *file, + void *priv, struct v4l2_capability *cap) +{ + strscpy(cap->driver, MALI_C55_DRIVER_NAME, sizeof(cap->driver)); + strscpy(cap->card, "ARM Mali-C55 ISP", sizeof(cap->card)); + + return 0; +} + +static const struct v4l2_ioctl_ops mali_c55_params_v4l2_ioctl_ops = { + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, + .vidioc_enum_fmt_meta_out = mali_c55_params_enum_fmt_meta_out, + .vidioc_g_fmt_meta_out = mali_c55_params_g_fmt_meta_out, + .vidioc_s_fmt_meta_out = mali_c55_params_g_fmt_meta_out, + .vidioc_try_fmt_meta_out = mali_c55_params_g_fmt_meta_out, + .vidioc_querycap = mali_c55_params_querycap, + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +static const struct v4l2_file_operations mali_c55_params_v4l2_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = video_ioctl2, + .open = v4l2_fh_open, + .release = vb2_fop_release, + .poll = vb2_fop_poll, + .mmap = vb2_fop_mmap, +}; + +static int +mali_c55_params_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, + unsigned int *num_planes, unsigned int sizes[], + struct device *alloc_devs[]) +{ + if (*num_planes && *num_planes > 1) + return -EINVAL; + + if (sizes[0] && sizes[0] != sizeof(struct mali_c55_params_buffer)) + return -EINVAL; + + *num_planes = 1; + sizes[0] = sizeof(struct mali_c55_params_buffer); + + return 0; +} + +static void mali_c55_params_buf_queue(struct vb2_buffer *vb) +{ + struct mali_c55_params *params = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct mali_c55_buffer *buf = container_of(vbuf, + struct mali_c55_buffer, vb); + + vb2_set_plane_payload(vb, 0, sizeof(struct mali_c55_params_buffer)); + + spin_lock(¶ms->buffers.lock); + list_add_tail(&buf->queue, ¶ms->buffers.queue); + spin_unlock(¶ms->buffers.lock); +} + +static void mali_c55_params_stop_streaming(struct vb2_queue *q) +{ + struct mali_c55_params *params = vb2_get_drv_priv(q); + struct mali_c55_buffer *buf, *tmp; + + spin_lock(¶ms->buffers.lock); + + list_for_each_entry_safe(buf, tmp, ¶ms->buffers.queue, queue) { + list_del(&buf->queue); + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + } + + spin_unlock(¶ms->buffers.lock); +} + +static const struct vb2_ops mali_c55_params_vb2_ops = { + .queue_setup = mali_c55_params_queue_setup, + .buf_queue = mali_c55_params_buf_queue, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, + .stop_streaming = mali_c55_params_stop_streaming, +}; + +void mali_c55_params_write_config(struct mali_c55 *mali_c55) +{ + struct mali_c55_params *params = &mali_c55->params; + enum vb2_buffer_state state = VB2_BUF_STATE_DONE; + struct mali_c55_params_buffer *config; + struct mali_c55_buffer *buf; + size_t block_offset = 0; + + spin_lock(¶ms->buffers.lock); + + buf = list_first_entry_or_null(¶ms->buffers.queue, + struct mali_c55_buffer, queue); + if (buf) + list_del(&buf->queue); + spin_unlock(¶ms->buffers.lock); + + if (!buf) + return; + + buf->vb.sequence = mali_c55->isp.frame_sequence; + config = vb2_plane_vaddr(&buf->vb.vb2_buf, 0); + + if (config->total_size > MALI_C55_PARAMS_MAX_SIZE) { + dev_dbg(mali_c55->dev, "Invalid parameters buffer size %lu\n", + config->total_size); + state = VB2_BUF_STATE_ERROR; + goto err_buffer_done; + } + + /* Walk the list of parameter blocks and process them. */ + while (block_offset < config->total_size) { + const struct mali_c55_block_handler *block_handler; + struct mali_c55_params_block_header *block; + + block = (struct mali_c55_params_block_header *) + &config->data[block_offset]; + + if (block->type >= MALI_C55_PARAM_BLOCK_SENTINEL) { + dev_dbg(mali_c55->dev, "Invalid parameters block type\n"); + state = VB2_BUF_STATE_ERROR; + goto err_buffer_done; + } + + block_handler = &mali_c55_block_handlers[block->type]; + if (block->size != block_handler->size) { + dev_dbg(mali_c55->dev, "Invalid parameters block size\n"); + state = VB2_BUF_STATE_ERROR; + goto err_buffer_done; + } + + block_handler->handler(mali_c55, block); + + block_offset += block->size; + } + +err_buffer_done: + vb2_buffer_done(&buf->vb.vb2_buf, state); +} + +void mali_c55_unregister_params(struct mali_c55 *mali_c55) +{ + struct mali_c55_params *params = &mali_c55->params; + + if (!video_is_registered(¶ms->vdev)) + return; + + vb2_video_unregister_device(¶ms->vdev); + media_entity_cleanup(¶ms->vdev.entity); + mutex_destroy(¶ms->lock); +} + +int mali_c55_register_params(struct mali_c55 *mali_c55) +{ + struct mali_c55_params *params = &mali_c55->params; + struct video_device *vdev = ¶ms->vdev; + struct vb2_queue *vb2q = ¶ms->queue; + int ret; + + mutex_init(¶ms->lock); + INIT_LIST_HEAD(¶ms->buffers.queue); + + params->pad.flags = MEDIA_PAD_FL_SOURCE; + ret = media_entity_pads_init(¶ms->vdev.entity, 1, ¶ms->pad); + if (ret) + goto err_destroy_mutex; + + vb2q->type = V4L2_BUF_TYPE_META_OUTPUT; + vb2q->io_modes = VB2_MMAP | VB2_DMABUF; + vb2q->drv_priv = params; + vb2q->mem_ops = &vb2_dma_contig_memops; + vb2q->ops = &mali_c55_params_vb2_ops; + vb2q->buf_struct_size = sizeof(struct mali_c55_buffer); + vb2q->min_queued_buffers = 1; + vb2q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + vb2q->lock = ¶ms->lock; + vb2q->dev = mali_c55->dev; + + ret = vb2_queue_init(vb2q); + if (ret) { + dev_err(mali_c55->dev, "params vb2 queue init failed\n"); + goto err_cleanup_entity; + } + + strscpy(params->vdev.name, "mali-c55 3a params", + sizeof(params->vdev.name)); + vdev->release = video_device_release_empty; + vdev->fops = &mali_c55_params_v4l2_fops; + vdev->ioctl_ops = &mali_c55_params_v4l2_ioctl_ops; + vdev->lock = ¶ms->lock; + vdev->v4l2_dev = &mali_c55->v4l2_dev; + vdev->queue = ¶ms->queue; + vdev->device_caps = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING; + vdev->vfl_dir = VFL_DIR_TX; + video_set_drvdata(vdev, params); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) { + dev_err(mali_c55->dev, + "failed to register params video device\n"); + goto err_release_vb2q; + } + + params->mali_c55 = mali_c55; + + return 0; + +err_release_vb2q: + vb2_queue_release(vb2q); +err_cleanup_entity: + media_entity_cleanup(¶ms->vdev.entity); +err_destroy_mutex: + mutex_destroy(¶ms->lock); + + return ret; +} diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h index eb3719245ec3..8e6a801077ed 100644 --- a/drivers/media/platform/arm/mali-c55/mali-c55-registers.h +++ b/drivers/media/platform/arm/mali-c55/mali-c55-registers.h @@ -119,6 +119,19 @@ #define MALI_C55_REG_ACTIVE_HEIGHT_MASK 0xffff0000 #define MALI_C55_REG_BAYER_ORDER 0x18e8c #define MALI_C55_BAYER_ORDER_MASK GENMASK(1, 0) + +#define MALI_C55_REG_METERING_CONFIG 0x18ed0 +#define MALI_C55_5BIN_HIST_DISABLE_MASK BIT(0) +#define MALI_C55_5BIN_HIST_SWITCH_MASK GENMASK(2, 1) +#define MALI_C55_AF_DISABLE_MASK BIT(4) +#define MALI_C55_AF_SWITCH_MASK BIT(5) +#define MALI_C55_AWB_DISABLE_MASK BIT(8) +#define MALI_C55_AWB_SWITCH_MASK BIT(9) +#define MALI_C55_AEXP_HIST_DISABLE_MASK BIT(12) +#define MALI_C55_AEXP_HIST_SWITCH_MASK GENMASK(14, 13) +#define MALI_C55_AEXP_IHIST_DISABLE_MASK BIT(16) +#define MALI_C55_AEXP_SRC_MASK BIT(24) + #define MALI_C55_REG_TPG_CH0 0x18ed8 #define MALI_C55_TEST_PATTERN_ON_OFF BIT(0) #define MALI_C55_TEST_PATTERN_RGB_MASK BIT(1) @@ -138,6 +151,11 @@ #define MALI_C55_REG_CONFIG_SPACES_OFFSET 0x0ab6c #define MALI_C55_CONFIG_SPACE_SIZE 0x1231c +#define MALI_C55_REG_DIGITAL_GAIN 0x1926c +#define MALI_C55_DIGITAL_GAIN_MASK GENMASK(12, 0) +#define MALI_C55_REG_DIGITAL_GAIN_OFFSET 0x19270 +#define MALI_C55_DIGITAL_GAIN_OFFSET_MASK GENMASK(19, 0) + #define MALI_C55_REG_SINTER_CONFIG 0x19348 #define MALI_C55_SINTER_VIEW_FILTER_MASK GENMASK(1, 0) #define MALI_C55_SINTER_SCALE_MODE_MASK GENMASK(3, 2) @@ -146,6 +164,46 @@ #define MALI_C55_SINTER_INT_SELECT_MASK BIT(6) #define MALI_C55_SINTER_RM_ENABLE_MASK BIT(7) +/* Black Level Correction Configuration */ +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_00 0x1abcc +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_01 0x1abd0 +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_10 0x1abd4 +#define MALI_C55_REG_SENSOR_OFF_PRE_SHA_11 0x1abd8 +#define MALI_C55_SENSOR_OFF_PRE_SHA_MASK 0xfffff + +/* Lens Mesh Shading Configuration */ +#define MALI_C55_REG_MESH_SHADING_TABLES 0x13074 +#define MALI_C55_REG_MESH_SHADING_CONFIG 0x1abfc +#define MALI_C55_MESH_SHADING_ENABLE_MASK BIT(0) +#define MALI_C55_MESH_SHADING_MESH_SHOW BIT(1) +#define MALI_C55_MESH_SHADING_SCALE_MASK GENMASK(4, 2) +#define MALI_C55_MESH_SHADING_PAGE_R_MASK GENMASK(9, 8) +#define MALI_C55_MESH_SHADING_PAGE_G_MASK GENMASK(11, 10) +#define MALI_C55_MESH_SHADING_PAGE_B_MASK GENMASK(13, 12) +#define MALI_C55_MESH_SHADING_MESH_WIDTH_MASK GENMASK(21, 16) +#define MALI_C55_MESH_SHADING_MESH_HEIGHT_MASK GENMASK(29, 24) + +#define MALI_C55_REG_MESH_SHADING_ALPHA_BANK 0x1ac04 +#define MALI_C55_MESH_SHADING_ALPHA_BANK_R_MASK GENMASK(2, 0) +#define MALI_C55_MESH_SHADING_ALPHA_BANK_G_MASK GENMASK(5, 3) +#define MALI_C55_MESH_SHADING_ALPHA_BANK_B_MASK GENMASK(8, 6) +#define MALI_C55_REG_MESH_SHADING_ALPHA 0x1ac08 +#define MALI_C55_MESH_SHADING_ALPHA_R_MASK GENMASK(7, 0) +#define MALI_C55_MESH_SHADING_ALPHA_G_MASK GENMASK(15, 8) +#define MALI_C55_MESH_SHADING_ALPHA_B_MASK GENMASK(23, 16) +#define MALI_C55_REG_MESH_SHADING_MESH_STRENGTH 0x1ac0c +#define MALI_c55_MESH_STRENGTH_MASK GENMASK(15, 0) + +/* AWB Gains Configuration */ +#define MALI_C55_REG_AWB_GAINS1 0x1ac10 +#define MALI_C55_AWB_GAIN00_MASK GENMASK(11, 0) +#define MALI_C55_AWB_GAIN01_MASK GENMASK(27, 16) +#define MALI_C55_REG_AWB_GAINS2 0x1ac14 +#define MALI_C55_AWB_GAIN10_MASK GENMASK(11, 0) +#define MALI_C55_AWB_GAIN11_MASK GENMASK(27, 16) +#define MALI_C55_REG_AWB_GAINS1_AEXP 0x1ac18 +#define MALI_C55_REG_AWB_GAINS2_AEXP 0x1ac1c + /* Colour Correction Matrix Configuration */ #define MALI_C55_REG_CCM_ENABLE 0x1b07c #define MALI_C55_CCM_ENABLE_MASK BIT(0) @@ -168,6 +226,52 @@ #define MALI_C55_REG_CCM_ANTIFOG_OFFSET_B 0x1b0c8 #define MALI_C55_CCM_ANTIFOG_OFFSET_MASK GENMASK(11, 0) +/* AWB Statistics Configuration */ +#define MALI_C55_REG_AWB_STATS_MODE 0x1b29c +#define MALI_C55_AWB_STATS_MODE_MASK BIT(0) +#define MALI_C55_REG_AWB_WHITE_LEVEL 0x1b2a0 +#define MALI_C55_AWB_WHITE_LEVEL_MASK GENMASK(9, 0) +#define MALI_C55_REG_AWB_BLACK_LEVEL 0x1b2a4 +#define MALI_C55_AWB_BLACK_LEVEL_MASK GENMASK(9, 0) +#define MALI_C55_REG_AWB_CR_MAX 0x1b2a8 +#define MALI_C55_AWB_CR_MAX_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_CR_MIN 0x1b2ac +#define MALI_C55_AWB_CR_MIN_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_CB_MAX 0x1b2b0 +#define MALI_C55_AWB_CB_MAX_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_CB_MIN 0x1b2b4 +#define MALI_C55_AWB_CB_MIN_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_NODES_USED 0x1b2c4 +#define MALI_C55_AWB_NODES_USED_HORIZ_MASK GENMASK(7, 0) +#define MALI_C55_AWB_NODES_USED_VERT_MASK GENMASK(15, 8) +#define MALI_C55_REG_AWB_CR_HIGH 0x1b2c8 +#define MALI_C55_AWB_CR_HIGH_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_CR_LOW 0x1b2cc +#define MALI_C55_AWB_CR_LOW_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_CB_HIGH 0x1b2d0 +#define MALI_C55_AWB_CB_HIGH_MASK GENMASK(11, 0) +#define MALI_C55_REG_AWB_CB_LOW 0x1b2d4 +#define MALI_C55_AWB_CB_LOW_MASK GENMASK(11, 0) + +/* AEXP Metering Histogram Configuration */ +#define MALI_C55_REG_AEXP_HIST_BASE 0x1b730 +#define MALI_C55_REG_AEXP_IHIST_BASE 0x1bbac +#define MALI_C55_AEXP_HIST_SKIP_OFFSET 0 +#define MALI_C55_AEXP_HIST_SKIP_X_MASK GENMASK(2, 0) +#define MALI_C55_AEXP_HIST_OFFSET_X_MASK BIT(3) +#define MALI_C55_AEXP_HIST_SKIP_Y_MASK GENMASK(6, 4) +#define MALI_C55_AEXP_HIST_OFFSET_Y_MASK BIT(7) +#define MALI_C55_AEXP_HIST_SCALE_OFFSET 4 +#define MALI_C55_AEXP_HIST_SCALE_BOTTOM_MASK GENMASK(3, 0) +#define MALI_C55_AEXP_HIST_SCALE_TOP_MASK GENMASK(7, 4) +#define MALI_C55_AEXP_HIST_PLANE_MODE_OFFSET 16 +#define MALI_C55_AEXP_HIST_PLANE_MODE_MASK GENMASK(2, 0) +#define MALI_C55_AEXP_HIST_NODES_USED_OFFSET 52 +#define MALI_C55_AEXP_HIST_NODES_USED_HORIZ_MASK GENMASK(7, 0) +#define MALI_C55_AEXP_HIST_NODES_USED_VERT_MASK GENMASK(15, 8) +#define MALI_C55_AEXP_HIST_ZONE_WEIGHTS_OFFSET 56 +#define MALI_C55_AEXP_HIST_ZONE_WEIGHT_MASK 0x0f0f0f0f + /* * The Mali-C55 ISP has up to two output pipes; known as full resolution and * down scaled. The register space for these is laid out identically, but offset From patchwork Wed May 29 15:28:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 13679120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 73F30C25B75 for ; Wed, 29 May 2024 15:30:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LSdNd/2vi+oO31tvG9+h5TivUU0VM9jPySGDsiUNHy8=; b=InAR6tptiUlHo6 Lyc1+j29yrqf+6ke6qvqMs7sx7cmANUuRmE1jyh6bfpv6FbfHK0RwwyaUMQnyKtyIiOfsgcp8lezu YQ4Ur7a7SPg2+Lz8vAcyGg1pN+HdoCTedbTpkGVPbu3nTTfuDqH5pX/1PJAzkvG1x/1cqpJ0AUThL 338aImVYdlLrsdMyV7b0OxnMx1iUh93DFzmIfkmSdJxQcuvFk7cHvXUBqk2SJRJ5Keg/QYE1U18Q7 +m7LSIe90eGrrdQlTpYMvx6EBoXj1cNPILGN6aMJCQUplK9596k3ezn3EqT5886xMfFkFs5532VGx xe32isMbKP1OiejKcL1w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLG3-00000004huC-2yaz; Wed, 29 May 2024 15:30:31 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF5-00000004hAv-1zb9 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:37 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CFC872F93; Wed, 29 May 2024 17:29:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996566; bh=IkfvO15HaTn0N2u4HLoTFNIjbEmbpUy6xdmqh81I+l0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TuA2HcvCF309lnNCv6/df0lKuwIO47MD3BMEoOPooKhpH5gSN3XFszFrZSdA7s+OS TVLU4HB7/9QQPJPRqio2QloD5gg7MQXcFcPqGkthX9ugOJy33Kgc0PSgnzM+iok7FR AMUvHl3Pgln42c7k0QJOY4Pi5U310/ZbWZ+PL0Us= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 16/16] Documentation: mali-c55: Document the mali-c55 parameter setting Date: Wed, 29 May 2024 16:28:58 +0100 Message-Id: <20240529152858.183799-17-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529152858.183799-1-dan.scally@ideasonboard.com> References: <20240529152858.183799-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082931_795599_FE632AE0 X-CRM114-Status: GOOD ( 15.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Document the mali-c55 parameter setting by expanding the relevant pages in both admin-guide/ and userspace-api/. Acked-by: Nayden Kanchev Co-developed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - New patch Documentation/admin-guide/media/mali-c55.rst | 19 +++++- .../media/v4l/metafmt-arm-mali-c55.rst | 65 ++++++++++++++++++- 2 files changed, 79 insertions(+), 5 deletions(-) diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst index b75437f6e96a..c5e6ac826c99 100644 --- a/Documentation/admin-guide/media/mali-c55.rst +++ b/Documentation/admin-guide/media/mali-c55.rst @@ -380,9 +380,24 @@ the processing flow the statistics can be drawn from:: +-------------+ | +-------------+ +--> AWB-1 -At present all statistics are drawn from the 0th tap point for each algorithm; +By default all statistics are drawn from the 0th tap point for each algorithm; I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF -statistics from AF-0. In the future this will be configurable. +statistics from AF-0. This is configurable for AEXP and AWB statsistics through +programming the ISP's parameters. + +.. _mali-c55-3a-params: + +Programming ISP Parameters +========================== + +The ISP can be programmed with various parameters from userspace to apply to the +hardware before and during video stream. This allows userspace to dynamically +change values such as black level, white balance and lens shading gains and so +on. + +The buffer format and how to populate it are described by the +:ref:`V4L2_META_FMT_MALI_C55_PARAMS ` format, +which should be set as the data format for the `mali-c55 3a params` video node. References ========== diff --git a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst index 219a5dd42d70..c359d2c83858 100644 --- a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst +++ b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst @@ -1,10 +1,11 @@ .. SPDX-License-Identifier: GPL-2.0 +.. _v4l2-meta-fmt-mali-c55-params: .. _v4l2-meta-fmt-mali-c55-3a-stats: -**************************************** -V4L2_META_FMT_MALI_C55_3A_STATS ('C55S') -**************************************** +******************************************************************************** +V4L2_META_FMT_MALI_C55_3A_STATS ('C55S'), V4L2_META_FMT_MALI_C55_PARAMS ('C55P') +******************************************************************************** 3A Statistics ============= @@ -22,6 +23,64 @@ of the C structure :c:type:`mali_c55_stats_buffer` defined in For details of the statistics see :c:type:`mali_c55_stats_buffer`. +Configuration Parameters +======================== + +The configuration parameters are passed to the +:ref:`mali-c55 3a params ` metadata output video node, using +the :c:type:`v4l2_meta_format` interface. Rather than a single struct containing +sub-structs for each configurable area of the ISP, parameters for the Mali-C55 +are defined as distinct structs or "blocks" which may be added to the data +member of struct mali_c55_params_buffer. Userspace is responsible for populating +the data member with the blocks that need to be configured by the driver, but +need not populate it with **all** the blocks, or indeed with any at all if there +are no configuration changes to make. Populated blocks **must** be consecutive +in the buffer. To assist both userspace and the driver in identifying the +blocks each block-specific struct should embed +struct mali_c55_params_block_header as its first member and userspace must +populate the type member with a value from enum mali_c55_param_block_type. Once +the blocks have been populated into the data buffer, the combined size of all +populated blocks should be set in the total_size member of +struct mali_c55_params_buffer. For example: + +.. code-block:: c + + struct mali_c55_params_buffer *params = + (struct mali_c55_params_buffer *)buffer; + + params->version = MALI_C55_PARAM_BUFFER_V0; + + void *data = (void *)params->data; + + struct mali_c55_params_awb_gains *gains = + (struct mali_c55_params_awb_gains *)data; + + gains->header.type = MALI_C55_PARAM_BLOCK_AWB_GAINS; + gains->header.enabled = true; + gains->header.size = sizeof(struct mali_c55_params_awb_gains); + + gains->gain00 = 256; + gains->gain00 = 256; + gains->gain00 = 256; + gains->gain00 = 256; + + data += sizeof(struct mali_c55_params_awb_gains) + + struct mali_c55_params_sensor_off_preshading *blc = + (struct mali_c55_params_sensor_off_preshading *)data; + + blc->header.type = MALI_C55_PARAM_BLOCK_SENSOR_OFFS; + blc->header.enabled = true; + blc->header.size = sizeof(struct mali_c55_params_sensor_off_preshading); + + blc->chan00 = 51200; + blc->chan01 = 51200; + blc->chan10 = 51200; + blc->chan11 = 51200; + + params->total_size = sizeof(struct mali_c55_params_awb_gains) + + sizeof(struct mali_c55_params_sensor_off_preshading); + Arm Mali-C55 uAPI data types ============================