From patchwork Tue Aug 23 17:18:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 915F0C32789 for ; Tue, 23 Aug 2022 18:54:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233001AbiHWSyD (ORCPT ); Tue, 23 Aug 2022 14:54:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233288AbiHWSxp (ORCPT ); Tue, 23 Aug 2022 14:53:45 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D377205D8 for ; Tue, 23 Aug 2022 10:18:49 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E430B484; Tue, 23 Aug 2022 19:18:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275127; bh=qc4dX5X6pcswRfuYM0u0Kk5nQTzPLRvQQ3wiK+TRz2g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jm/670bQrEC9gfWreLhXIviKU0Q3dz7ikUl+rxizGDmJIE6C4W7wVvlD53NaXThxB BSCeafu7i0BGJFJYSKIv9nh5D3IMA6gRBxFXf5waTwYxlhwY0ztYRKUvVoiImsjyVF 1uxyXj1zDkdXLQyLpnyEBih3mINTT4ULdRR48jbM= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 1/9] media: rkisp1: Initialize color space on ISP sink and source pads Date: Tue, 23 Aug 2022 20:18:32 +0300 Message-Id: <20220823171840.8958-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Initialize the four color space fields on the sink and source video pads of the ISP in the .init_cfg() operation. As the main use case for the ISP is to convert Bayer data to YUV, select a raw color space on the sink pad and a limited range quantization of SYCC on the source pad by default. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- Changes since v1: - Mention ISP in the subject line --- drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index a3c7d4d88387..a52b22824739 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -553,12 +553,17 @@ static int rkisp1_isp_init_config(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *sink_fmt, *src_fmt; struct v4l2_rect *sink_crop, *src_crop; + /* Video. */ sink_fmt = v4l2_subdev_get_try_format(sd, sd_state, RKISP1_ISP_PAD_SINK_VIDEO); sink_fmt->width = RKISP1_DEFAULT_WIDTH; sink_fmt->height = RKISP1_DEFAULT_HEIGHT; sink_fmt->field = V4L2_FIELD_NONE; sink_fmt->code = RKISP1_DEF_SINK_PAD_FMT; + sink_fmt->colorspace = V4L2_COLORSPACE_RAW; + sink_fmt->xfer_func = V4L2_XFER_FUNC_NONE; + sink_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + sink_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; sink_crop = v4l2_subdev_get_try_crop(sd, sd_state, RKISP1_ISP_PAD_SINK_VIDEO); @@ -571,11 +576,16 @@ static int rkisp1_isp_init_config(struct v4l2_subdev *sd, RKISP1_ISP_PAD_SOURCE_VIDEO); *src_fmt = *sink_fmt; src_fmt->code = RKISP1_DEF_SRC_PAD_FMT; + src_fmt->colorspace = V4L2_COLORSPACE_SRGB; + src_fmt->xfer_func = V4L2_XFER_FUNC_SRGB; + src_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + src_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; src_crop = v4l2_subdev_get_try_crop(sd, sd_state, RKISP1_ISP_PAD_SOURCE_VIDEO); *src_crop = *sink_crop; + /* Parameters and statistics. */ sink_fmt = v4l2_subdev_get_try_format(sd, sd_state, RKISP1_ISP_PAD_SINK_PARAMS); src_fmt = v4l2_subdev_get_try_format(sd, sd_state, From patchwork Tue Aug 23 17:18:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8D1CC32772 for ; Tue, 23 Aug 2022 18:54:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233656AbiHWSyN (ORCPT ); Tue, 23 Aug 2022 14:54:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230426AbiHWSxz (ORCPT ); Tue, 23 Aug 2022 14:53:55 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EC53E1142 for ; Tue, 23 Aug 2022 10:19:07 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 455636BB; Tue, 23 Aug 2022 19:18:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275128; bh=dpRQoQdidwsPDR2uIIecPAZtZqaBPiziKux6rvilmys=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gdu7QqWoyzEVt/xGlp0het0DkeCBsTQXiT5wVn+VKoxNwlxxyGcyDXVoFZ7BE+itM Ujg3npeLKScHmatkuY8+2fAZoGq43HbTbuHkN/tsKk9rBnHFT+CvHVGDDmBKZ+yExC 4oc1Z6sH+yA1WRc8DTKZ60c0VjREPETBuJtu1+L0= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 2/9] media: rkisp1: Allow setting color space on ISP sink pad Date: Tue, 23 Aug 2022 20:18:33 +0300 Message-Id: <20220823171840.8958-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ISP accepts different color spaces on its input: for YUV input, it doesn't set any restrictions, and for Bayer inputs, any color primaries or transfer function can be accepted (YCbCr encoding isn't applicable there, and quantization range can only be full). Allow setting a color space on the ISP sink pad, with the aforementioned restrictions. The settings don't influence hardware yet (only the YUV quantization range will, anything else has no direct effect on the ISP configuration), but can already be set to allow color space information to be coherent across the ISP sink link. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Dafna Hirschfeld --- Changes since v1: - Fix swapped default color spaces for YUV and Bayer - Improve coherency in usage of ternary operator ? : --- .../platform/rockchip/rkisp1/rkisp1-isp.c | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index a52b22824739..b5bdf427c7e1 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -705,6 +705,7 @@ static void rkisp1_isp_set_sink_fmt(struct rkisp1_isp *isp, const struct rkisp1_mbus_info *mbus_info; struct v4l2_mbus_framefmt *sink_fmt; struct v4l2_rect *sink_crop; + bool is_yuv; sink_fmt = rkisp1_isp_get_pad_fmt(isp, sd_state, RKISP1_ISP_PAD_SINK_VIDEO, @@ -725,6 +726,36 @@ static void rkisp1_isp_set_sink_fmt(struct rkisp1_isp *isp, RKISP1_ISP_MIN_HEIGHT, RKISP1_ISP_MAX_HEIGHT); + /* + * Adjust the color space fields. Accept any color primaries and + * transfer function for both YUV and Bayer. For YUV any YCbCr encoding + * and quantization range is also accepted. For Bayer formats, the YCbCr + * encoding isn't applicable, and the quantization range can only be + * full. + */ + is_yuv = mbus_info->pixel_enc == V4L2_PIXEL_ENC_YUV; + + sink_fmt->colorspace = format->colorspace ? : + (is_yuv ? V4L2_COLORSPACE_SRGB : + V4L2_COLORSPACE_RAW); + sink_fmt->xfer_func = format->xfer_func ? : + V4L2_MAP_XFER_FUNC_DEFAULT(sink_fmt->colorspace); + if (is_yuv) { + sink_fmt->ycbcr_enc = format->ycbcr_enc ? : + V4L2_MAP_YCBCR_ENC_DEFAULT(sink_fmt->colorspace); + sink_fmt->quantization = format->quantization ? : + V4L2_MAP_QUANTIZATION_DEFAULT(false, sink_fmt->colorspace, + sink_fmt->ycbcr_enc); + } else { + /* + * The YCbCr encoding isn't applicable for non-YUV formats, but + * V4L2 has no "no encoding" value. Hardcode it to Rec. 601, it + * should be ignored by userspace. + */ + sink_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + sink_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; + } + *format = *sink_fmt; /* Propagate to in crop */ From patchwork Tue Aug 23 17:18:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF4F1C32772 for ; Tue, 23 Aug 2022 18:54:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232674AbiHWSyY (ORCPT ); Tue, 23 Aug 2022 14:54:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231685AbiHWSyF (ORCPT ); Tue, 23 Aug 2022 14:54:05 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B481E9924 for ; Tue, 23 Aug 2022 10:19:35 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 92B816DB; Tue, 23 Aug 2022 19:18:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275129; bh=RIFI48othOgoauVe4eRMKYWJE+4l2r3GttCN6Go7NqI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AKV2i3MRUu7HaiAIc70QYrlM/KIHmFBGr93ZxkNsEEw0BJ5Do2Ae1ilP3pXaTlqmw Ma3E5zdFBK9l9NeClajgAZCD2wOlq08OFrsukZFYxMVRCzSi7LNnT6BNhBJYGx0GGF lRDzjVjsl3XJhCq0wWgNksn6iDurkX6OMZbuO40c= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 3/9] media: rkisp1: Fix source pad format configuration Date: Tue, 23 Aug 2022 20:18:34 +0300 Message-Id: <20220823171840.8958-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ISP converts Bayer data to YUV when operating normally, and can also operate in pass-through mode where the input and output formats must match. Converting from YUV to Bayer isn't possible. If such an invalid configuration is attempted, adjust it by copying the sink pad media bus code to the source pad. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- .../platform/rockchip/rkisp1/rkisp1-isp.c | 40 +++++++++++++++---- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index b5bdf427c7e1..d34f32271d74 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -604,23 +604,43 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, struct v4l2_mbus_framefmt *format, unsigned int which) { - const struct rkisp1_mbus_info *mbus_info; + const struct rkisp1_mbus_info *sink_info; + const struct rkisp1_mbus_info *src_info; + struct v4l2_mbus_framefmt *sink_fmt; struct v4l2_mbus_framefmt *src_fmt; const struct v4l2_rect *src_crop; + sink_fmt = rkisp1_isp_get_pad_fmt(isp, sd_state, + RKISP1_ISP_PAD_SINK_VIDEO, which); src_fmt = rkisp1_isp_get_pad_fmt(isp, sd_state, RKISP1_ISP_PAD_SOURCE_VIDEO, which); src_crop = rkisp1_isp_get_pad_crop(isp, sd_state, RKISP1_ISP_PAD_SOURCE_VIDEO, which); + /* + * Media bus code. The ISP can operate in pass-through mode (Bayer in, + * Bayer out or YUV in, YUV out) or process Bayer data to YUV, but + * can't convert from YUV to Bayer. + */ + sink_info = rkisp1_mbus_info_get_by_code(sink_fmt->code); + src_fmt->code = format->code; - mbus_info = rkisp1_mbus_info_get_by_code(src_fmt->code); - if (!mbus_info || !(mbus_info->direction & RKISP1_ISP_SD_SRC)) { + src_info = rkisp1_mbus_info_get_by_code(src_fmt->code); + if (!src_info || !(src_info->direction & RKISP1_ISP_SD_SRC)) { src_fmt->code = RKISP1_DEF_SRC_PAD_FMT; - mbus_info = rkisp1_mbus_info_get_by_code(src_fmt->code); + src_info = rkisp1_mbus_info_get_by_code(src_fmt->code); } - if (which == V4L2_SUBDEV_FORMAT_ACTIVE) - isp->src_fmt = mbus_info; + + if (sink_info->pixel_enc == V4L2_PIXEL_ENC_YUV && + src_info->pixel_enc == V4L2_PIXEL_ENC_BAYER) { + src_fmt->code = sink_fmt->code; + src_info = sink_info; + } + + /* + * The source width and height must be identical to the source crop + * size. + */ src_fmt->width = src_crop->width; src_fmt->height = src_crop->height; @@ -630,14 +650,18 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, */ if (format->flags & V4L2_MBUS_FRAMEFMT_SET_CSC && format->quantization == V4L2_QUANTIZATION_FULL_RANGE && - mbus_info->pixel_enc == V4L2_PIXEL_ENC_YUV) + src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) src_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; - else if (mbus_info->pixel_enc == V4L2_PIXEL_ENC_YUV) + else if (src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) src_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; else src_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; *format = *src_fmt; + + /* Store the source format info when setting the active format. */ + if (which == V4L2_SUBDEV_FORMAT_ACTIVE) + isp->src_fmt = src_info; } static void rkisp1_isp_set_src_crop(struct rkisp1_isp *isp, From patchwork Tue Aug 23 17:18:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E50CDC32772 for ; Tue, 23 Aug 2022 18:54:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233604AbiHWSyc (ORCPT ); Tue, 23 Aug 2022 14:54:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230472AbiHWSyJ (ORCPT ); Tue, 23 Aug 2022 14:54:09 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14B3AEA8B8 for ; Tue, 23 Aug 2022 10:19:46 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DDC849C4; Tue, 23 Aug 2022 19:18:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275131; bh=WeVhIIrOGSxD0QDS3twAok28MZFQgk9/L8LNeg/uP5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mnZgWudn2X4/jhB3tTQP6nmjtN4RFvG5ZawL822nzIhxNhpQHTv5fZvH6mp7HpQvO v+tpePb29+2rUm4Zg01xtd2n0gfSlAh8t9iSWlkO5BNiEbrPO02b1yoO13FoHpWBzP F+68JAO0w/JnwHhZmVGCSEoO59zcu9E1dpKH6Y0M= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 4/9] media: rkisp1: Allow setting all color space fields on ISP source pad Date: Tue, 23 Aug 2022 20:18:35 +0300 Message-Id: <20220823171840.8958-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ISP output color space is configured through the ISP source pad. At the moment, only the quantization can be set. Extend it to the three other color space fields: - The ycbcr_enc field will be used to configure the RGB to YUV matrix (currently hardcoded to Rec. 601). - The colorspace (which controls the color primaries) and xfer_func fields will not be used to configure the ISP, as the corresponding hardware blocks (the cross-talk RGB to RGB matrix and the tone mapping curve) are programmed directly by applications through ISP parameters. Nonetheless, those two fields should be set by applications to match the ISP configuration, in order to propagate the correct color space down the pipeline up to the capture video nodes. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Dafna Hirschfeld --- Changes since v1: - Fix quantization setting that was set on sink pad by mistake --- .../platform/rockchip/rkisp1/rkisp1-isp.c | 55 ++++++++++++++++--- 1 file changed, 48 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index d34f32271d74..7869f0cced62 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -609,6 +609,7 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, struct v4l2_mbus_framefmt *sink_fmt; struct v4l2_mbus_framefmt *src_fmt; const struct v4l2_rect *src_crop; + bool set_csc; sink_fmt = rkisp1_isp_get_pad_fmt(isp, sd_state, RKISP1_ISP_PAD_SINK_VIDEO, which); @@ -645,20 +646,60 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, src_fmt->height = src_crop->height; /* - * The CSC API is used to allow userspace to force full - * quantization on YUV formats. + * Copy the color space for the sink pad. When converting from Bayer to + * YUV, default to a limited quantization range. */ - if (format->flags & V4L2_MBUS_FRAMEFMT_SET_CSC && - format->quantization == V4L2_QUANTIZATION_FULL_RANGE && + src_fmt->colorspace = sink_fmt->colorspace; + src_fmt->xfer_func = sink_fmt->xfer_func; + src_fmt->ycbcr_enc = sink_fmt->ycbcr_enc; + + if (sink_info->pixel_enc == V4L2_PIXEL_ENC_BAYER && src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) - src_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; - else if (src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) src_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; else - src_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; + src_fmt->quantization = sink_fmt->quantization; + + /* + * Allow setting the source color space fields when the SET_CSC flag is + * set and the source format is YUV. If the sink format is YUV, don't + * set the color primaries, transfer function or YCbCr encoding as the + * ISP is bypassed in that case and passes YUV data through without + * modifications. + * + * The color primaries and transfer function are configured through the + * cross-talk matrix and tone curve respectively. Settings for those + * hardware blocks are conveyed through the ISP parameters buffer, as + * they need to combine color space information with other image tuning + * characteristics and can't thus be computed by the kernel based on the + * color space. The source pad colorspace and xfer_func fields are thus + * ignored by the driver, but can be set by userspace to propagate + * accurate color space information down the pipeline. + */ + set_csc = !!(format->flags & V4L2_MBUS_FRAMEFMT_SET_CSC); + + if (set_csc && src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) { + if (sink_info->pixel_enc == V4L2_PIXEL_ENC_BAYER) { + if (format->colorspace != V4L2_COLORSPACE_DEFAULT) + src_fmt->colorspace = format->colorspace; + if (format->xfer_func != V4L2_XFER_FUNC_DEFAULT) + src_fmt->xfer_func = format->xfer_func; + if (format->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) + src_fmt->ycbcr_enc = format->ycbcr_enc; + } + + if (format->quantization != V4L2_QUANTIZATION_DEFAULT) + src_fmt->quantization = format->quantization; + } *format = *src_fmt; + /* + * Restore the SET_CSC flag if it was set to indicate support for the + * CSC setting API. + */ + if (set_csc) + format->flags |= V4L2_MBUS_FRAMEFMT_SET_CSC; + /* Store the source format info when setting the active format. */ if (which == V4L2_SUBDEV_FORMAT_ACTIVE) isp->src_fmt = src_info; From patchwork Tue Aug 23 17:18:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FE85C32772 for ; Tue, 23 Aug 2022 18:54:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232149AbiHWSyl (ORCPT ); Tue, 23 Aug 2022 14:54:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232401AbiHWSyT (ORCPT ); Tue, 23 Aug 2022 14:54:19 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC816816B6 for ; Tue, 23 Aug 2022 10:20:07 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3BC0DB2A; Tue, 23 Aug 2022 19:18:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275132; bh=6kNtGNzhNvTD/yxpiyeUnP/C7xOxnLCzh+3Gy0PiM8A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=v+UM2xtl7xUXP3bT0A3hV1vtGtJBzulxe6VKOGIJeySzC+2lvvSmaGigsI8GL8S7X frI8i2t7EpZtbuPzUAXy+HXb69GVbIaKtnVrLNkpAURCG9Ik05qMS4jrrGjfN07QZC UOzj4Klhnp6v+G7JpRMlKFLAo9/FEH6KhGnWzoTU= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 5/9] media: rkisp1: Configure quantization using ISP source pad Date: Tue, 23 Aug 2022 20:18:36 +0300 Message-Id: <20220823171840.8958-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The rkisp1_config_isp() function uses the format on the sink pad of the ISP to configure quantization at the output of the ISP. This is incorrect, as hinted by the src_frm variable name that stores the format. Fix it by using the source pad. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index 7869f0cced62..babf88066c2e 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -341,7 +341,7 @@ static int rkisp1_config_isp(struct rkisp1_isp *isp, struct v4l2_mbus_framefmt *src_frm; src_frm = rkisp1_isp_get_pad_fmt(isp, NULL, - RKISP1_ISP_PAD_SINK_VIDEO, + RKISP1_ISP_PAD_SOURCE_VIDEO, V4L2_SUBDEV_FORMAT_ACTIVE); rkisp1_params_configure(&rkisp1->params, sink_fmt->bayer_pat, src_frm->quantization); From patchwork Tue Aug 23 17:18:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D799C32774 for ; Tue, 23 Aug 2022 18:54:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232587AbiHWSym (ORCPT ); Tue, 23 Aug 2022 14:54:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232447AbiHWSyU (ORCPT ); Tue, 23 Aug 2022 14:54:20 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01F7211D535 for ; Tue, 23 Aug 2022 10:20:01 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B226FE1E; Tue, 23 Aug 2022 19:18:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275134; bh=ei00uxRrPfVtT4n0RYTb5ZUOk9kkdzGSUKVGJZnvSoM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OdD9eqVLle7qyTj+HNqmKTt53/iGk9JFY0/VJk/sg2HMvTe9M5H/V1FvHxbo7axBl k/XaXG92Bl5yL28R1CItiGUDAIm8mLmr84UfwQezseSGpR0TYwn1MmUqpKKkmmV6OP Pk6pqn4bu25nvrMZm9jTBMXpz3Ei1SlJT4vUYoyw= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 6/9] media: rkisp1: Don't pass the quantization to rkisp1_csm_config() Date: Tue, 23 Aug 2022 20:18:37 +0300 Message-Id: <20220823171840.8958-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The rkisp1_csm_config() function takes a pointer to the rkisp1_params structure which contains the quantization value. There's no need to pass it separately to the function. Drop it from the function parameters. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- drivers/media/platform/rockchip/rkisp1/rkisp1-params.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c index 8b4eea77af0d..163419624370 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c @@ -1076,7 +1076,7 @@ static void rkisp1_ie_enable(struct rkisp1_params *params, bool en) } } -static void rkisp1_csm_config(struct rkisp1_params *params, bool full_range) +static void rkisp1_csm_config(struct rkisp1_params *params) { static const u16 full_range_coeff[] = { 0x0026, 0x004b, 0x000f, @@ -1090,7 +1090,7 @@ static void rkisp1_csm_config(struct rkisp1_params *params, bool full_range) }; unsigned int i; - if (full_range) { + if (params->quantization == V4L2_QUANTIZATION_FULL_RANGE) { for (i = 0; i < ARRAY_SIZE(full_range_coeff); i++) rkisp1_write(params->rkisp1, RKISP1_CIF_ISP_CC_COEFF_0 + i * 4, @@ -1562,11 +1562,7 @@ static void rkisp1_params_config_parameter(struct rkisp1_params *params) rkisp1_param_set_bits(params, RKISP1_CIF_ISP_HIST_PROP_V10, rkisp1_hst_params_default_config.mode); - /* set the range */ - if (params->quantization == V4L2_QUANTIZATION_FULL_RANGE) - rkisp1_csm_config(params, true); - else - rkisp1_csm_config(params, false); + rkisp1_csm_config(params); spin_lock_irq(¶ms->config_lock); From patchwork Tue Aug 23 17:18:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 684D7C32772 for ; Tue, 23 Aug 2022 18:54:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233074AbiHWSyu (ORCPT ); Tue, 23 Aug 2022 14:54:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230466AbiHWSy2 (ORCPT ); Tue, 23 Aug 2022 14:54:28 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 825C6816B9 for ; Tue, 23 Aug 2022 10:20:18 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 256C0EF2; Tue, 23 Aug 2022 19:18:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275135; bh=PEo66cr/iALjZ+PwUPkXm9qqrciKduhUGDqzHmg7kuM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=loY8MvXHngDVOLL9FS5d3IuGTGTCtCxQb+L89n0ydoLH8ezrSdRSAO74Y8B9IT7gm 7K6JB/8f2on8jAOIwtCfGsoO7xVU451FQVxqbwwmlMudrcl3i+4QTeuQVH4kzawe+n +dcUX8HvqSbq7kCjuVJNL2MjEgFGm5jw69FS5nVA= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 7/9] media: rkisp1: Configure CSM based on YCbCr encoding Date: Tue, 23 Aug 2022 20:18:38 +0300 Message-Id: <20220823171840.8958-8-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The driver currently only implements the Rec. 601 YCbCr encoding, extend it with support for the other encodings defined by V4L2 (Rec. 709, Rec. 2020 and SMPTE240m). The coefficients have been calculated by rounding the floating point values to the nearest Q1.7 fixed-point value, adjusting the rounding to ensure that the sum of each line in the matrix is preserved to avoid overflows. At the hardware level, the RGB to YUV conversion matrix is fully configurable, custom encoding could be supported by extending the ISP parameters if desired. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- .../platform/rockchip/rkisp1/rkisp1-common.h | 5 +- .../platform/rockchip/rkisp1/rkisp1-isp.c | 3 +- .../platform/rockchip/rkisp1/rkisp1-params.c | 97 +++++++++++++++---- 3 files changed, 84 insertions(+), 21 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h index 589999020a16..1383c13e22b8 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h @@ -393,6 +393,7 @@ struct rkisp1_params { struct v4l2_format vdev_fmt; enum v4l2_quantization quantization; + enum v4l2_ycbcr_encoding ycbcr_encoding; enum rkisp1_fmt_raw_pat_type raw_type; }; @@ -595,10 +596,12 @@ const struct rkisp1_mbus_info *rkisp1_mbus_info_get_by_code(u32 mbus_code); * @params: pointer to rkisp1_params. * @bayer_pat: the bayer pattern on the isp video sink pad * @quantization: the quantization configured on the isp's src pad + * @ycbcr_encoding: the ycbcr_encoding configured on the isp's src pad */ void rkisp1_params_configure(struct rkisp1_params *params, enum rkisp1_fmt_raw_pat_type bayer_pat, - enum v4l2_quantization quantization); + enum v4l2_quantization quantization, + enum v4l2_ycbcr_encoding ycbcr_encoding); /* rkisp1_params_disable - disable all parameters. * This function is called by the isp entity upon stream start diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index babf88066c2e..20c01e0e2e17 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -344,7 +344,8 @@ static int rkisp1_config_isp(struct rkisp1_isp *isp, RKISP1_ISP_PAD_SOURCE_VIDEO, V4L2_SUBDEV_FORMAT_ACTIVE); rkisp1_params_configure(&rkisp1->params, sink_fmt->bayer_pat, - src_frm->quantization); + src_frm->quantization, + src_frm->ycbcr_enc); } return 0; diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c index 163419624370..246a6faa1fc1 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c @@ -1078,37 +1078,94 @@ static void rkisp1_ie_enable(struct rkisp1_params *params, bool en) static void rkisp1_csm_config(struct rkisp1_params *params) { - static const u16 full_range_coeff[] = { - 0x0026, 0x004b, 0x000f, - 0x01ea, 0x01d6, 0x0040, - 0x0040, 0x01ca, 0x01f6 + struct csm_coeffs { + u16 limited[9]; + u16 full[9]; }; - static const u16 limited_range_coeff[] = { - 0x0021, 0x0040, 0x000d, - 0x01ed, 0x01db, 0x0038, - 0x0038, 0x01d1, 0x01f7, + static const struct csm_coeffs rec601_coeffs = { + .limited = { + 0x0021, 0x0042, 0x000d, + 0x01ed, 0x01db, 0x0038, + 0x0038, 0x01d1, 0x01f7, + }, + .full = { + 0x0026, 0x004b, 0x000f, + 0x01ea, 0x01d6, 0x0040, + 0x0040, 0x01ca, 0x01f6, + }, }; + static const struct csm_coeffs rec709_coeffs = { + .limited = { + 0x0018, 0x0050, 0x0008, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + .full = { + 0x001b, 0x005c, 0x0009, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c6, 0x01fa, + }, + }; + static const struct csm_coeffs rec2020_coeffs = { + .limited = { + 0x001d, 0x004c, 0x0007, + 0x01f0, 0x01d8, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + .full = { + 0x0022, 0x0057, 0x0008, + 0x01ee, 0x01d2, 0x0040, + 0x0040, 0x01c5, 0x01fb, + }, + }; + static const struct csm_coeffs smpte240m_coeffs = { + .limited = { + 0x0018, 0x004f, 0x000a, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01ce, 0x01fa, + }, + .full = { + 0x001b, 0x005a, 0x000b, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c7, 0x01f9, + }, + }; + + const struct csm_coeffs *coeffs; + const u16 *csm; unsigned int i; + switch (params->ycbcr_encoding) { + case V4L2_YCBCR_ENC_601: + default: + coeffs = &rec601_coeffs; + break; + case V4L2_YCBCR_ENC_709: + coeffs = &rec709_coeffs; + break; + case V4L2_YCBCR_ENC_BT2020: + coeffs = &rec2020_coeffs; + break; + case V4L2_YCBCR_ENC_SMPTE240M: + coeffs = &smpte240m_coeffs; + break; + } + if (params->quantization == V4L2_QUANTIZATION_FULL_RANGE) { - for (i = 0; i < ARRAY_SIZE(full_range_coeff); i++) - rkisp1_write(params->rkisp1, - RKISP1_CIF_ISP_CC_COEFF_0 + i * 4, - full_range_coeff[i]); - + csm = coeffs->full; rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL, RKISP1_CIF_ISP_CTRL_ISP_CSM_Y_FULL_ENA | RKISP1_CIF_ISP_CTRL_ISP_CSM_C_FULL_ENA); } else { - for (i = 0; i < ARRAY_SIZE(limited_range_coeff); i++) - rkisp1_write(params->rkisp1, - RKISP1_CIF_ISP_CC_COEFF_0 + i * 4, - limited_range_coeff[i]); - + csm = coeffs->limited; rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL, RKISP1_CIF_ISP_CTRL_ISP_CSM_Y_FULL_ENA | RKISP1_CIF_ISP_CTRL_ISP_CSM_C_FULL_ENA); } + + for (i = 0; i < 9; i++) + rkisp1_write(params->rkisp1, RKISP1_CIF_ISP_CC_COEFF_0 + i * 4, + csm[i]); } /* ISP De-noise Pre-Filter(DPF) function */ @@ -1574,9 +1631,11 @@ static void rkisp1_params_config_parameter(struct rkisp1_params *params) void rkisp1_params_configure(struct rkisp1_params *params, enum rkisp1_fmt_raw_pat_type bayer_pat, - enum v4l2_quantization quantization) + enum v4l2_quantization quantization, + enum v4l2_ycbcr_encoding ycbcr_encoding) { params->quantization = quantization; + params->ycbcr_encoding = ycbcr_encoding; params->raw_type = bayer_pat; rkisp1_params_config_parameter(params); } From patchwork Tue Aug 23 17:18:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BC90C32772 for ; Tue, 23 Aug 2022 18:54:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233595AbiHWSyz (ORCPT ); Tue, 23 Aug 2022 14:54:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231603AbiHWSyb (ORCPT ); Tue, 23 Aug 2022 14:54:31 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C0E565540 for ; Tue, 23 Aug 2022 10:20:18 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8387B10CD; Tue, 23 Aug 2022 19:18:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275136; bh=e9dDoI7js6GAclo5QIceH/HdFmi9cGjnukCULiCyneE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Slke7OUD7FbrSdZFj81M/mBKdABPf/5k+HpkHY9ASG1pWVacc8K+e9YIMJGX+3ZvQ vR7DTrXG/uISbi1PPykZJpOoAZITKxonwaKDsGIdBaVOsHMl/LEBKmEXOADmpam/uv 2LQgR74uSFQGir8B5gWc3/eGOZbdkgyq2LQprGX0= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 8/9] media: rkisp1: Initialize color space on resizer sink and source pads Date: Tue, 23 Aug 2022 20:18:39 +0300 Message-Id: <20220823171840.8958-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Initialize the four color space fields on the sink and source video pads of the resizer in the .init_cfg() operation. The resizer can't perform any color space conversion, so set the sink and source color spaces to the same defaults, which match the ISP source video pad default. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Dafna Hirschfeld --- drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c index becef04fdf2d..6f6ec00b63b8 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c @@ -430,6 +430,10 @@ static int rkisp1_rsz_init_config(struct v4l2_subdev *sd, sink_fmt->height = RKISP1_DEFAULT_HEIGHT; sink_fmt->field = V4L2_FIELD_NONE; sink_fmt->code = RKISP1_DEF_FMT; + sink_fmt->colorspace = V4L2_COLORSPACE_SRGB; + sink_fmt->xfer_func = V4L2_XFER_FUNC_SRGB; + sink_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + sink_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; sink_crop = v4l2_subdev_get_try_crop(sd, sd_state, RKISP1_RSZ_PAD_SINK); From patchwork Tue Aug 23 17:18:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 12952488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1D28C32772 for ; Tue, 23 Aug 2022 18:54:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233971AbiHWSy6 (ORCPT ); Tue, 23 Aug 2022 14:54:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231523AbiHWSyf (ORCPT ); Tue, 23 Aug 2022 14:54:35 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA88E10BE30 for ; Tue, 23 Aug 2022 10:20:27 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DBB6B143C; Tue, 23 Aug 2022 19:18:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275138; bh=NF7VI93ZIiaX4q4lsScRC/VEsYG6gzShH0NgacxKOsc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sbRuxN5ySKO3Z43ZFVdBMowctqdLAspqCITSfhVtMsfHq6/UIH5cI5BwSVo/6Ap/0 LUtFX8RXp87zL2CKhs/HmZXgv0S6mVX1aDgWcPjYL+I630dlGGwUyJg1NiiB3Dg3tj g70P7CFLViAExmWCDF7siPmB7likOdyr55ZuPihc= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 9/9] media: rkisp1: Allow setting color space on resizer sink pad Date: Tue, 23 Aug 2022 20:18:40 +0300 Message-Id: <20220823171840.8958-10-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The resizer doesn't deal with color spaces, so it can accept any color space on its input, and propagates it unchanged to its output. When operating with a Bayer input format (in pass-through mode) further restrict the YCbCr encoding and quantization to Rec 601 and full range respectively, as for raw data the former ought to be ignored and the latter is always full range. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Dafna Hirschfeld --- .../platform/rockchip/rkisp1/rkisp1-resizer.c | 41 +++++++++++++++++-- 1 file changed, 38 insertions(+), 3 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c index 6f6ec00b63b8..891a622124e2 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c @@ -526,6 +526,7 @@ static void rkisp1_rsz_set_sink_fmt(struct rkisp1_resizer *rsz, const struct rkisp1_mbus_info *mbus_info; struct v4l2_mbus_framefmt *sink_fmt, *src_fmt; struct v4l2_rect *sink_crop; + bool is_yuv; sink_fmt = rkisp1_rsz_get_pad_fmt(rsz, sd_state, RKISP1_RSZ_PAD_SINK, which); @@ -547,9 +548,6 @@ static void rkisp1_rsz_set_sink_fmt(struct rkisp1_resizer *rsz, if (which == V4L2_SUBDEV_FORMAT_ACTIVE) rsz->pixel_enc = mbus_info->pixel_enc; - /* Propagete to source pad */ - src_fmt->code = sink_fmt->code; - sink_fmt->width = clamp_t(u32, format->width, RKISP1_ISP_MIN_WIDTH, RKISP1_ISP_MAX_WIDTH); @@ -557,8 +555,45 @@ static void rkisp1_rsz_set_sink_fmt(struct rkisp1_resizer *rsz, RKISP1_ISP_MIN_HEIGHT, RKISP1_ISP_MAX_HEIGHT); + /* + * Adjust the color space fields. Accept any color primaries and + * transfer function for both YUV and Bayer. For YUV any YCbCr encoding + * and quantization range is also accepted. For Bayer formats, the YCbCr + * encoding isn't applicable, and the quantization range can only be + * full. + */ + is_yuv = mbus_info->pixel_enc == V4L2_PIXEL_ENC_YUV; + + sink_fmt->colorspace = format->colorspace ? : + (is_yuv ? V4L2_COLORSPACE_SRGB : + V4L2_COLORSPACE_RAW); + sink_fmt->xfer_func = format->xfer_func ? : + V4L2_MAP_XFER_FUNC_DEFAULT(sink_fmt->colorspace); + if (is_yuv) { + sink_fmt->ycbcr_enc = format->ycbcr_enc ? : + V4L2_MAP_YCBCR_ENC_DEFAULT(sink_fmt->colorspace); + sink_fmt->quantization = format->quantization ? : + V4L2_MAP_QUANTIZATION_DEFAULT(false, sink_fmt->colorspace, + sink_fmt->ycbcr_enc); + } else { + /* + * The YCbCr encoding isn't applicable for non-YUV formats, but + * V4L2 has no "no encoding" value. Hardcode it to Rec. 601, it + * should be ignored by userspace. + */ + sink_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + sink_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; + } + *format = *sink_fmt; + /* Propagate the media bus code and color space to the source pad. */ + src_fmt->code = sink_fmt->code; + src_fmt->colorspace = sink_fmt->colorspace; + src_fmt->xfer_func = sink_fmt->xfer_func; + src_fmt->ycbcr_enc = sink_fmt->ycbcr_enc; + src_fmt->quantization = sink_fmt->quantization; + /* Update sink crop */ rkisp1_rsz_set_sink_crop(rsz, sd_state, sink_crop, which); }