From patchwork Wed May 29 15:28:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 13679107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08B71C27C43 for ; Wed, 29 May 2024 15:29:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=2Swqo57rjlmVXoe7LerodAG7xkf2teWsonFWGRokGoE=; b=Yb3KL6xMsUR5fh enV/B0XFgWkBgB9E+W1saVVxT60fktrXYsQJvhordjaVayHWZpBterKKYrrmVSYu0uXUz0xi8bSO3 nQu9wfy2ip2N6mpEZVd/9Ok3EW+Gsf9fSS06lQCCSO4weUcQVsaidxLAE2uaQvQ8ttEV1mnwX3H7l Mq6LNuR2o3MCxHPJOomFkmxp7yI2Hl2q3Dw5THA6/+kYcHBDjF+6Eze7aF3Gwmofbob7aFqY00eEj 0WNnEEGLCZs7YuocErBrj8UDPS5x6Mp9XFubSttT+AWSEuZIeZNJrT0NzJjbIni83/1SmEldKotYp Tw5FiXpvERjVpYcWkruQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLF0-00000004h5z-2Arf; Wed, 29 May 2024 15:29:26 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCLEw-00000004h1o-34OH for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 15:29:24 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 40A57149B; Wed, 29 May 2024 17:29:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1716996555; bh=MR+691o6QI2VzlxrZXSGEXZ0LSPiIz9Rv0+QjuaM6Fw=; h=From:To:Cc:Subject:Date:From; b=rhq6ZqFbqxkDrz1yBsliaYmL5/5jeII5lp0vlwGowdOizeL3FZGrg/zi2bfTzgUrX +9xiHMlnbQNgYIEdQ7M6nLPdwyghWvK+J4ItJ1kmWGhBHd+3gG1cS4sHrFHu0UL5D2 kblQpU53/anikhWSeRwrsfit3F2RwaA/UUNjSmZk= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi, dan.scally@ideasonboard.com Subject: [PATCH v5 00/16] Add Arm Mali-C55 Image Signal Processor Driver Date: Wed, 29 May 2024 16:28:42 +0100 Message-Id: <20240529152858.183799-1-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_082922_985370_671B05BD X-CRM114-Status: GOOD ( 17.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hello all This patchset introduces a driver for Arm's Mali-C55 Image Signal Processor. The driver uses the V4L2 / media controller API and implements both of the ISP's capture pipelines allowing a range of output formats plus downscaling and cropping. The capture pipelines are named "Full resolution" and "Downscale" and so abbreviated FR and DS throughout the driver. The driver exposes 4 V4L2 subdevices: - mali-c55 isp: input data formatting - mali-c55 tpg: test pattern generator (modeled as a camera sensor entity) - mali-c55 resizer fr: downscale / crop and format setting for the FR pipe - mali-c55 resizer ds: downscale / crop and format setting for the DS pipe Along with 4 V4L2 Video devices: - mali-c55 fr: Capture device for the full resolution pipe - mali-c55 ds: Capture device for the downscale pipe - mali-c55 3a stats: Capture device for statistics to support 3A algorithms - mali-c55 3a params: Output device for parameter buffers to configure the ISP Support is implemented in the parameters video device code for many of the ISP'S hardware blocks, but not yet all of them. The buffer format is (as far as I am aware anyway) a novel design that we intend to be extensible so that support for the C55's remaining hardware blocks can be added later. Patches 1, 4, 5, 6 and 7 have already had 4 versions on the mailing list...I decided to post the additional work on the driver as extra patches rather than merge them all into the existing series as it's already a lot of code to review and I hoped that that might make it a little easier...if I'm wrong and that's not liked I can just squash them into a much smaller series. Thanks Dan Daniel Scally (15): media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code media: uapi: Add 20-bit bayer formats media: v4l2-common: Add RAW16 format info dt-bindings: media: Add bindings for ARM mali-c55 media: mali-c55: Add Mali-C55 ISP driver media: Documentation: Add Mali-C55 ISP Documentation MAINTAINERS: Add entry for mali-c55 driver media: Add MALI_C55_3A_STATS meta format media: uapi: Add 3a stats buffer for mali-c55 media: platform: Add mali-c55 3a stats devnode media: platform: Fill stats buffer on ISP_START Documentation: mali-c55: Add Statistics documentation media: uapi: Add parameters structs to mali-c55-config.h media: platform: Add mali-c55 parameters video node Documentation: mali-c55: Document the mali-c55 parameter setting Jacopo Mondi (1): media: mali-c55: Add image formats for Mali-C55 parameters buffer .../admin-guide/media/mali-c55-graph.dot | 19 + Documentation/admin-guide/media/mali-c55.rst | 406 ++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + .../bindings/media/arm,mali-c55.yaml | 66 ++ .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-arm-mali-c55.rst | 87 ++ .../media/v4l/subdev-formats.rst | 268 +++++ MAINTAINERS | 10 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile | 1 + drivers/media/platform/arm/Kconfig | 5 + drivers/media/platform/arm/Makefile | 2 + drivers/media/platform/arm/mali-c55/Kconfig | 18 + drivers/media/platform/arm/mali-c55/Makefile | 11 + .../platform/arm/mali-c55/mali-c55-capture.c | 951 ++++++++++++++++++ .../platform/arm/mali-c55/mali-c55-common.h | 312 ++++++ .../platform/arm/mali-c55/mali-c55-core.c | 825 +++++++++++++++ .../platform/arm/mali-c55/mali-c55-isp.c | 626 ++++++++++++ .../platform/arm/mali-c55/mali-c55-params.c | 615 +++++++++++ .../arm/mali-c55/mali-c55-registers.h | 365 +++++++ .../arm/mali-c55/mali-c55-resizer-coefs.h | 382 +++++++ .../platform/arm/mali-c55/mali-c55-resizer.c | 779 ++++++++++++++ .../platform/arm/mali-c55/mali-c55-stats.c | 350 +++++++ .../platform/arm/mali-c55/mali-c55-tpg.c | 402 ++++++++ drivers/media/v4l2-core/v4l2-common.c | 4 + drivers/media/v4l2-core/v4l2-ioctl.c | 2 + include/uapi/linux/media-bus-format.h | 9 +- .../uapi/linux/media/arm/mali-c55-config.h | 851 ++++++++++++++++ include/uapi/linux/videodev2.h | 3 + 29 files changed, 7370 insertions(+), 2 deletions(-) create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot create mode 100644 Documentation/admin-guide/media/mali-c55.rst create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst create mode 100644 drivers/media/platform/arm/Kconfig create mode 100644 drivers/media/platform/arm/Makefile create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig create mode 100644 drivers/media/platform/arm/mali-c55/Makefile create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer-coefs.h create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c create mode 100644 include/uapi/linux/media/arm/mali-c55-config.h