From patchwork Tue Jun 16 20:14:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11608581 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6D6590 for ; Tue, 16 Jun 2020 20:15:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A77C208D5 for ; Tue, 16 Jun 2020 20:15:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="n2WHA01s" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731631AbgFPUPl (ORCPT ); Tue, 16 Jun 2020 16:15:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731620AbgFPUPh (ORCPT ); Tue, 16 Jun 2020 16:15:37 -0400 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8EBAC061573 for ; Tue, 16 Jun 2020 13:15:36 -0700 (PDT) Received: by mail-ed1-x542.google.com with SMTP id c35so142502edf.5 for ; Tue, 16 Jun 2020 13:15:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UhE+zbVBx6mzDVILaZnHei28SizFjJx0SU7vvL+dRhE=; b=n2WHA01s9EZkeKKyqUap0wZLNfqCP7Rl1UOcp9o8SfsO8C8DO9Qu0GWnbIyUDh4kRe 8Cn6uwIh0W2kzAOPd2AjYqzYNsBJ/ToYFVs1CQl/Fc53WSPAgf4NT7hcT/0Q2qSea7AG mwSJ8BfyRgVLqjsgFIk/BtIkoacdZinAe2hy1qjQiGfUtJPF93N63BUBYkLNEU1qlYvt 3mtjqBQA+yyK3HzSbCfJfj9LRaje4tQxjbEZ99tSpyqb11dvyP4L7WOxGkNfVHtoXW58 qLMaC8Je3VE7+eVC3xn7Dzd4SoPl3gFmL9TKAR9esxd/HIgx9OU+lNFWt6xeUa1t9/Wl R04w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UhE+zbVBx6mzDVILaZnHei28SizFjJx0SU7vvL+dRhE=; b=P6L3WSJ666+15xC2UFl1QPcVvYgA42TIKwKq22EtqUbavxP6SJ/zNDSZ+mqdJ+bTZy jiqrLXejwP2INyAqOcI0+gA1zf72ZPRNt1jPS+MfVI+vIOzHJZT2vxsqSvBceru6uf2H jbqUQoRmYjTPfvE6AxxR8H9TSSUpD9eFLlfNt0JbLqUx9A3Z9qwHPnt/zoxDxuYjzXRx 8lYqzcAnhb71jo3UmjK6qn2kZUKkof9FG8Dxy62AePnav9hn4RsCyihNo2uFycXZKFiC Wsnm+KKRalqeL5s8qurN77nff37bfEmmwLWrQ9ejRxTVuP6OvAECet2FuQ2nhgGhGTbc BcLQ== X-Gm-Message-State: AOAM5326ehAb/WYttPFalw8MS5lIK8R5IN1jjrq0fxrcG0yDti62tk/A 237KDA8OQrW0hPu0jq4iCQpILg== X-Google-Smtp-Source: ABdhPJwZhGuBTTi2k/dcfpp0D9MSWNcg/wp+Nt6SmME+tlAslpJVGt7nDZbYn6k9lKEKWfk9nUcqoA== X-Received: by 2002:a05:6402:cbc:: with SMTP id cn28mr4273902edb.220.1592338535462; Tue, 16 Jun 2020 13:15:35 -0700 (PDT) Received: from localhost.localdomain (212-5-158-38.ip.btc-net.bg. [212.5.158.38]) by smtp.gmail.com with ESMTPSA id b14sm11602658ejq.105.2020.06.16.13.15.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2020 13:15:34 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org Cc: Hans Verkuil , Ezequiel Garcia , Maheshwar Ajja , Mauro Carvalho Chehab , Stanimir Varbanov Subject: [PATCH v2 1/4] media: v4l2-ctrls: Add encoder constant quality control Date: Tue, 16 Jun 2020 23:14:43 +0300 Message-Id: <20200616201446.15996-2-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200616201446.15996-1-stanimir.varbanov@linaro.org> References: <20200616201446.15996-1-stanimir.varbanov@linaro.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Maheshwar Ajja When V4L2_CID_MPEG_VIDEO_BITRATE_MODE value is V4L2_MPEG_VIDEO_BITRATE_MODE_CQ, encoder will produce constant quality output indicated by V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY control value. Encoder will choose appropriate quantization parameter and bitrate to produce requested frame quality level. Signed-off-by: Maheshwar Ajja Signed-off-by: Stanimir Varbanov Reviewed-by: Hans Verkuil --- .../userspace-api/media/v4l/ext-ctrls-codec.rst | 10 ++++++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++ include/uapi/linux/v4l2-controls.h | 2 ++ 3 files changed, 14 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index d0d506a444b1..b9d3f7ae6486 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -581,6 +581,8 @@ enum v4l2_mpeg_video_bitrate_mode - - Variable bitrate * - ``V4L2_MPEG_VIDEO_BITRATE_MODE_CBR`` - Constant bitrate + * - ``V4L2_MPEG_VIDEO_BITRATE_MODE_CQ`` + - Constant quality @@ -592,6 +594,14 @@ enum v4l2_mpeg_video_bitrate_mode - the average video bitrate. It is ignored if the video bitrate mode is set to constant bitrate. +``V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY (integer)`` + Constant quality level control. This control is applicable when + ``V4L2_CID_MPEG_VIDEO_BITRATE_MODE`` value is + ``V4L2_MPEG_VIDEO_BITRATE_MODE_CQ``. Valid range is 1 to 100 + where 1 indicates lowest quality and 100 indicates highest quality. + Encoder will decide the appropriate quantization parameter and + bitrate to produce requested frame quality. + ``V4L2_CID_MPEG_VIDEO_TEMPORAL_DECIMATION (integer)`` For every captured frame, skip this many subsequent frames (default 0). diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index 3f3fbcd60cc6..bc00d02e411f 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -200,6 +200,7 @@ const char * const *v4l2_ctrl_get_menu(u32 id) static const char * const mpeg_video_bitrate_mode[] = { "Variable Bitrate", "Constant Bitrate", + "Constant Quality", NULL }; static const char * const mpeg_stream_type[] = { @@ -832,6 +833,7 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_GOP_CLOSURE: return "Video GOP Closure"; case V4L2_CID_MPEG_VIDEO_PULLDOWN: return "Video Pulldown"; case V4L2_CID_MPEG_VIDEO_BITRATE_MODE: return "Video Bitrate Mode"; + case V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY: return "Constant Quality"; case V4L2_CID_MPEG_VIDEO_BITRATE: return "Video Bitrate"; case V4L2_CID_MPEG_VIDEO_BITRATE_PEAK: return "Video Peak Bitrate"; case V4L2_CID_MPEG_VIDEO_TEMPORAL_DECIMATION: return "Video Temporal Decimation"; diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h index 62271418c1be..0f7e4388dcce 100644 --- a/include/uapi/linux/v4l2-controls.h +++ b/include/uapi/linux/v4l2-controls.h @@ -375,6 +375,7 @@ enum v4l2_mpeg_video_aspect { enum v4l2_mpeg_video_bitrate_mode { V4L2_MPEG_VIDEO_BITRATE_MODE_VBR = 0, V4L2_MPEG_VIDEO_BITRATE_MODE_CBR = 1, + V4L2_MPEG_VIDEO_BITRATE_MODE_CQ = 2, }; #define V4L2_CID_MPEG_VIDEO_BITRATE (V4L2_CID_MPEG_BASE+207) #define V4L2_CID_MPEG_VIDEO_BITRATE_PEAK (V4L2_CID_MPEG_BASE+208) @@ -742,6 +743,7 @@ enum v4l2_cid_mpeg_video_hevc_size_of_length_field { #define V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L6_BR (V4L2_CID_MPEG_BASE + 642) #define V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES (V4L2_CID_MPEG_BASE + 643) #define V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR (V4L2_CID_MPEG_BASE + 644) +#define V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY (V4L2_CID_MPEG_BASE + 645) /* MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */ #define V4L2_CID_MPEG_CX2341X_BASE (V4L2_CTRL_CLASS_MPEG | 0x1000) From patchwork Tue Jun 16 20:14:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11608593 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 355856A2 for ; Tue, 16 Jun 2020 20:16:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 17309208D5 for ; Tue, 16 Jun 2020 20:16:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="rAxw4zi0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731643AbgFPUP6 (ORCPT ); Tue, 16 Jun 2020 16:15:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731603AbgFPUPj (ORCPT ); Tue, 16 Jun 2020 16:15:39 -0400 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F30DAC061573 for ; Tue, 16 Jun 2020 13:15:38 -0700 (PDT) Received: by mail-ej1-x644.google.com with SMTP id dr13so23034076ejc.3 for ; Tue, 16 Jun 2020 13:15:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=l53FcUBXaXLsNwgF2kGygWkBGpRAyHzlKoeXvC/PqGA=; b=rAxw4zi0hLcxHYTInF92hJRCjCBbw+BklYODRAJ21O8G7VkpRU/GTSeDLJJEqLcnbs ipQtZ4RT2rgM2QszCTKAwS+Q/IBUOouTa68FQ4/WjUVH7C/+7+R9fvMqfisdfakolmrd 9+pVRXLDmdjzZ0oKHpqbpMK4M3a6ZoJlXQqaxImAfJyn+UNqe0ZOgjiL0zWfC9BLiwcS ZD3uzVwpSl2zjW1vE8jAGTQKAL6IEFbFqIZgzFIGNv8WHxJ0eTpijAY1niSYvawDJX8F JUSjUZ6IVbm38w2CDj8RE6nsXhB5xEANK1T3WnDgZ1kQGjfyMTHao32ZlnQK1S/mqtin 0MFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=l53FcUBXaXLsNwgF2kGygWkBGpRAyHzlKoeXvC/PqGA=; b=JPZ3yW7gx5ZbzZuR4unkVAES5YLx8x9f/50qv9O1BFdB8jEb6BBShDp/c8fU+MSS7M a3S0qvDkaou+kqb75Ogi/n7gKoBlHwH9q7Ji2keXstPz6CvYsaRejp93P1keqyrZwi76 1TKsKaeqSsZUpvoSfTAXJrJkEzdEUiZj8sbGOTGxOmqO134c5lvdHPObbiddNSINLDSL b2eS7j/cN2NhWiSIQTohhwwbWoi69PT2+z4Hw4r7Yq+PP5Ko1dZvFYVgzkMxp4sy3c0u jS3AEyl0NuxpSFjqmpfL7/WnCQh+/HMlNOb4PuUyxGeU4oPro1nQ21UJARQ4g2/n6YOP 0j3g== X-Gm-Message-State: AOAM532GeLVyfApucQgTj28Gs1FoCuxi9RvOtkTqaiV3qJ2Onz9gCgIk f1t42IJXhwngv0cGQNf+7qopdA== X-Google-Smtp-Source: ABdhPJyC48lSDgSusZjdt7wuW3QqHCOrlYimc3jhj7e/M/SEHRc420ooTefdfmy/P4U8y1k3DVVwmA== X-Received: by 2002:a17:906:f2d9:: with SMTP id gz25mr4644585ejb.467.1592338537663; Tue, 16 Jun 2020 13:15:37 -0700 (PDT) Received: from localhost.localdomain (212-5-158-38.ip.btc-net.bg. [212.5.158.38]) by smtp.gmail.com with ESMTPSA id b14sm11602658ejq.105.2020.06.16.13.15.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2020 13:15:37 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org Cc: Hans Verkuil , Ezequiel Garcia , Maheshwar Ajja , Mauro Carvalho Chehab , Stanimir Varbanov Subject: [PATCH v2 2/4] venus: venc: Add support for constant quality control Date: Tue, 16 Jun 2020 23:14:44 +0300 Message-Id: <20200616201446.15996-3-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200616201446.15996-1-stanimir.varbanov@linaro.org> References: <20200616201446.15996-1-stanimir.varbanov@linaro.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Adds implementation of V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY v4l control when the bitrate mode is CQ. Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/core.h | 1 + drivers/media/platform/qcom/venus/hfi_cmds.c | 37 ++++++++++++++++++- .../media/platform/qcom/venus/hfi_helper.h | 10 ++++- drivers/media/platform/qcom/venus/venc.c | 14 ++++++- .../media/platform/qcom/venus/venc_ctrls.c | 8 +++- 5 files changed, 66 insertions(+), 4 deletions(-) diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h index 7118612673c9..1bac30d4cf50 100644 --- a/drivers/media/platform/qcom/venus/core.h +++ b/drivers/media/platform/qcom/venus/core.h @@ -201,6 +201,7 @@ struct venc_controls { u32 bitrate; u32 bitrate_peak; u32 rc_enable; + u32 const_quality; u32 h264_i_period; u32 h264_entropy_mode; diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/venus/hfi_cmds.c index c67e412f8201..7022368c1e63 100644 --- a/drivers/media/platform/qcom/venus/hfi_cmds.c +++ b/drivers/media/platform/qcom/venus/hfi_cmds.c @@ -640,6 +640,7 @@ static int pkt_session_set_property_1x(struct hfi_session_set_property_pkt *pkt, case HFI_RATE_CONTROL_CBR_VFR: case HFI_RATE_CONTROL_VBR_CFR: case HFI_RATE_CONTROL_VBR_VFR: + case HFI_RATE_CONTROL_CQ: break; default: ret = -EINVAL; @@ -1218,6 +1219,37 @@ pkt_session_set_property_4xx(struct hfi_session_set_property_pkt *pkt, return 0; } +static int +pkt_session_set_property_6xx(struct hfi_session_set_property_pkt *pkt, + void *cookie, u32 ptype, void *pdata) +{ + void *prop_data; + + if (!pkt || !cookie || !pdata) + return -EINVAL; + + prop_data = &pkt->data[1]; + + pkt->shdr.hdr.size = sizeof(*pkt); + pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_PROPERTY; + pkt->shdr.session_id = hash32_ptr(cookie); + pkt->num_properties = 1; + pkt->data[0] = ptype; + + switch (ptype) { + case HFI_PROPERTY_CONFIG_HEIC_FRAME_QUALITY: { + struct hfi_heic_frame_quality *in = pdata, *cq = prop_data; + + cq->frame_quality = in->frame_quality; + pkt->shdr.hdr.size += sizeof(u32) + sizeof(*cq); + break; + } default: + return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata); + } + + return 0; +} + int pkt_session_get_property(struct hfi_session_get_property_pkt *pkt, void *cookie, u32 ptype) { @@ -1236,7 +1268,10 @@ int pkt_session_set_property(struct hfi_session_set_property_pkt *pkt, if (hfi_ver == HFI_VERSION_3XX) return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata); - return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata); + if (hfi_ver == HFI_VERSION_4XX) + return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata); + + return pkt_session_set_property_6xx(pkt, cookie, ptype, pdata); } void pkt_set_version(enum hfi_version version) diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h index f6613df1d16b..758c70ac26fd 100644 --- a/drivers/media/platform/qcom/venus/hfi_helper.h +++ b/drivers/media/platform/qcom/venus/hfi_helper.h @@ -231,6 +231,7 @@ #define HFI_RATE_CONTROL_VBR_CFR 0x1000003 #define HFI_RATE_CONTROL_CBR_VFR 0x1000004 #define HFI_RATE_CONTROL_CBR_CFR 0x1000005 +#define HFI_RATE_CONTROL_CQ 0x1000008 #define HFI_VIDEO_CODEC_H264 0x00000002 #define HFI_VIDEO_CODEC_H263 0x00000004 @@ -504,6 +505,7 @@ #define HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER 0x200600b #define HFI_PROPERTY_CONFIG_VENC_LTRPERIOD 0x200600c #define HFI_PROPERTY_CONFIG_VENC_PERF_MODE 0x200600e +#define HFI_PROPERTY_CONFIG_HEIC_FRAME_QUALITY 0x2006014 /* * HFI_PROPERTY_PARAM_VPE_COMMON_START @@ -520,7 +522,8 @@ enum hfi_version { HFI_VERSION_1XX, HFI_VERSION_3XX, - HFI_VERSION_4XX + HFI_VERSION_4XX, + HFI_VERSION_6XX, }; struct hfi_buffer_info { @@ -725,6 +728,11 @@ struct hfi_quality_vs_speed { u32 quality_vs_speed; }; +struct hfi_heic_frame_quality { + u32 frame_quality; + u32 reserved[3]; +}; + struct hfi_quantization { u32 qp_i; u32 qp_p; diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c index feed648550d1..802d608a00f9 100644 --- a/drivers/media/platform/qcom/venus/venc.c +++ b/drivers/media/platform/qcom/venus/venc.c @@ -740,14 +740,26 @@ static int venc_set_properties(struct venus_inst *inst) rate_control = HFI_RATE_CONTROL_OFF; else if (ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) rate_control = HFI_RATE_CONTROL_VBR_CFR; - else + else if (ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) rate_control = HFI_RATE_CONTROL_CBR_CFR; + else if (ctr->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) + rate_control = HFI_RATE_CONTROL_CQ; ptype = HFI_PROPERTY_PARAM_VENC_RATE_CONTROL; ret = hfi_session_set_property(inst, ptype, &rate_control); if (ret) return ret; + if (rate_control == HFI_RATE_CONTROL_CQ && ctr->const_quality) { + struct hfi_heic_frame_quality quality = {}; + + ptype = HFI_PROPERTY_CONFIG_HEIC_FRAME_QUALITY; + quality.frame_quality = ctr->const_quality; + ret = hfi_session_set_property(inst, ptype, &quality); + if (ret) + return ret; + } + if (!ctr->bitrate) bitrate = 64000; else diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/venus/venc_ctrls.c index 8362dde7949e..97a1e821c07e 100644 --- a/drivers/media/platform/qcom/venus/venc_ctrls.c +++ b/drivers/media/platform/qcom/venus/venc_ctrls.c @@ -202,6 +202,9 @@ static int venc_op_s_ctrl(struct v4l2_ctrl *ctrl) case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE: ctr->rc_enable = ctrl->val; break; + case V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY: + ctr->const_quality = ctrl->val; + break; default: return -EINVAL; } @@ -217,7 +220,7 @@ int venc_ctrl_init(struct venus_inst *inst) { int ret; - ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 31); + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 32); if (ret) return ret; @@ -357,6 +360,9 @@ int venc_ctrl_init(struct venus_inst *inst) v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE, 0, 1, 1, 1); + v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY, 0, 100, 1, 0); + ret = inst->ctrl_handler.error; if (ret) goto err; From patchwork Tue Jun 16 20:14:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11608583 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADB5A6A2 for ; Tue, 16 Jun 2020 20:15:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9072D20B1F for ; Tue, 16 Jun 2020 20:15:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="TvS+lpRe" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731680AbgFPUPp (ORCPT ); Tue, 16 Jun 2020 16:15:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731669AbgFPUPl (ORCPT ); Tue, 16 Jun 2020 16:15:41 -0400 Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com [IPv6:2a00:1450:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47469C0613EE for ; Tue, 16 Jun 2020 13:15:41 -0700 (PDT) Received: by mail-ej1-x643.google.com with SMTP id o15so23050226ejm.12 for ; Tue, 16 Jun 2020 13:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gW9cbjjRPD6R6Z7AF3iL+tqbW8lQcn4mBsQDieJZv9Y=; b=TvS+lpRerMK7Vh9j8U/PXM8SvfVymFr6pX9tI7Shj29sprrHvMvzGEFsqWa8uXWTkR fOVEMyyl0EL25CtoL9PLNJS1BNdetkq7c5E9CpjAA8Ih08ptnMxWahoK/yYrRPsxaf7o vBtX/zJSXErob5isSOg8B063lsyxaN10ZtQ8oRHs25gZjXwJ8ISzStb9G2xesomdiiE/ qfpNMoTc28Z7YRuPZC3AwKvAfV69ApepJGlGKe2hvVdJkGJlQeqpdZNYrkpbfFG74gxm hvQFytLZ7F2UwRii9qRLXt/bFCBm8/g695mr1gVqnG03s8/Fb42CuQL+VlLhNmbecWtl Ga5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gW9cbjjRPD6R6Z7AF3iL+tqbW8lQcn4mBsQDieJZv9Y=; b=jmyz8k8Ho3lriVcIK9eQ8ACUUwGVxs2z/dILRZbs3sfRWUvRLrtVxSoRGJlg3y9M8Q 2TueesOapwPAmxWnbeMlc1fBhdRV9IxmgMZq9GEZsecQJdnZoSmcDb4NWF0Eeb+kvxxn qe8OzrGAyy0C5eObNq6eiR7VA4PoE5w1plm9m37R9pBy/3VM6I1591MTeZ4w9q3QsdEy FosIw1Pq5v7t4DUiaCiSNjUHVzkHWwZMt2RPxo6IVHPa5Tubz/doaMq/qElJeRr0DdtD l/+RyjFLlLqFXnnZxcofIZ1bLMz/+0aLlVnEWASyjSue9qO1NXfEDfmNN7yiK8YAAWxm /LUg== X-Gm-Message-State: AOAM5329vspurw60uU10ULEyajMCCSmx3iQvdaMTK7Z2cwzYz1l3VPof 1dCvjEF5cB4LVtpH1IBM+Il+gA== X-Google-Smtp-Source: ABdhPJzgXkbMUwl8DMfZ408p8n1EGyMsTX1YMSgHzEF6jce2krk5jx3jQ+aRMOjy2tB0tYG6LH3ywA== X-Received: by 2002:a17:906:7253:: with SMTP id n19mr4557425ejk.31.1592338539978; Tue, 16 Jun 2020 13:15:39 -0700 (PDT) Received: from localhost.localdomain (212-5-158-38.ip.btc-net.bg. [212.5.158.38]) by smtp.gmail.com with ESMTPSA id b14sm11602658ejq.105.2020.06.16.13.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2020 13:15:39 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org Cc: Hans Verkuil , Ezequiel Garcia , Maheshwar Ajja , Mauro Carvalho Chehab , Stanimir Varbanov Subject: [PATCH v2 3/4] v4l2-ctrl: Add control for intra only decode Date: Tue, 16 Jun 2020 23:14:45 +0300 Message-Id: <20200616201446.15996-4-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200616201446.15996-1-stanimir.varbanov@linaro.org> References: <20200616201446.15996-1-stanimir.varbanov@linaro.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org This adds a new decoder control to instruct the decoders to produce on its output intra frames only. Usually in this mode decoders might lower the count of output decoder buffers and hence reduce memory usage. Signed-off-by: Stanimir Varbanov --- .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++ drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++ include/uapi/linux/v4l2-controls.h | 1 + 3 files changed, 12 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index b9d3f7ae6486..d7f34596f95b 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -652,6 +652,15 @@ enum v4l2_mpeg_video_bitrate_mode - otherwise the decoder expects a single frame in per buffer. Applicable to the decoder, all codecs. +``V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY (boolean)`` + If enabled the decoder should start decoding only intra frames. The + decoder consume first input buffer for progressive stream (or first + two buffers for interlace). Decoder might not allocate more output + buffers than it is required to consume one input frame. Usually the + decoder input buffers will contain only intra frames but it is not + mandatory. This control could be used for thumbnails generation. + Applicable to the decoder, all codecs. + ``V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE (boolean)`` Enable writing sample aspect ratio in the Video Usability Information. Applicable to the H264 encoder. diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index bc00d02e411f..2b1fb8dcd360 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -846,6 +846,7 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE: return "H264 MB Level Rate Control"; case V4L2_CID_MPEG_VIDEO_HEADER_MODE: return "Sequence Header Mode"; case V4L2_CID_MPEG_VIDEO_MAX_REF_PIC: return "Max Number of Reference Pics"; + case V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY: return "Decode intra frames only"; case V4L2_CID_MPEG_VIDEO_H263_I_FRAME_QP: return "H263 I-Frame QP Value"; case V4L2_CID_MPEG_VIDEO_H263_P_FRAME_QP: return "H263 P-Frame QP Value"; case V4L2_CID_MPEG_VIDEO_H263_B_FRAME_QP: return "H263 B-Frame QP Value"; @@ -1197,6 +1198,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE: case V4L2_CID_MPEG_VIDEO_MPEG4_QPEL: case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: + case V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY: case V4L2_CID_WIDE_DYNAMIC_RANGE: case V4L2_CID_IMAGE_STABILIZATION: case V4L2_CID_RDS_RECEPTION: diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h index 0f7e4388dcce..c64471e64aa7 100644 --- a/include/uapi/linux/v4l2-controls.h +++ b/include/uapi/linux/v4l2-controls.h @@ -744,6 +744,7 @@ enum v4l2_cid_mpeg_video_hevc_size_of_length_field { #define V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES (V4L2_CID_MPEG_BASE + 643) #define V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR (V4L2_CID_MPEG_BASE + 644) #define V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY (V4L2_CID_MPEG_BASE + 645) +#define V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY (V4L2_CID_MPEG_BASE + 646) /* MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */ #define V4L2_CID_MPEG_CX2341X_BASE (V4L2_CTRL_CLASS_MPEG | 0x1000) From patchwork Tue Jun 16 20:14:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 11608589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9C6C214DD for ; Tue, 16 Jun 2020 20:15:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 82C88208B3 for ; Tue, 16 Jun 2020 20:15:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="gy3NOO0T" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731722AbgFPUP4 (ORCPT ); Tue, 16 Jun 2020 16:15:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731642AbgFPUPn (ORCPT ); Tue, 16 Jun 2020 16:15:43 -0400 Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E754C0613F0 for ; Tue, 16 Jun 2020 13:15:43 -0700 (PDT) Received: by mail-ed1-x541.google.com with SMTP id g1so137497edv.6 for ; Tue, 16 Jun 2020 13:15:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AZqdoAEGz2z2dhDjwXWaMZ24wdl6oc5L+Rca1tEh8dc=; b=gy3NOO0TVFbQO8JeVz50/yi9A6AS7UoNAvAWS0yUYYqBasL4KYmzd6LX0YSCcoYzih WZAXIMLWxXdlpY/bvSoQGSycZqpuRHWIneQo9OcdzFjNUomCLSSqIOPp+mszgZqoPk58 11vlXMHdE8MlJ4HgS2M268UOIQg/vg94l2gEPrHYgzwAlfNPcT+aLo+uzZqgkLwSOejQ 7pTvUWtbeRhK8owv8gttck7ySAvgy2te3vvK1RRACY0AitSSke1l+HqJN1XmObrNj9pk 9Gt8EA3JTwxuKVwPRdbirVeQDhKiVXmFX2yjEqB5roJ5YX8jD5XeOYouqsyg4Mbq/taa zXVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AZqdoAEGz2z2dhDjwXWaMZ24wdl6oc5L+Rca1tEh8dc=; b=Xyp0KLZ4E+SRPqv4O/B5UsZXSgZs/4bKJzM1oqHaZ0XYN2qK0CabD9bCqXDe+dVidK 8ljXE6PVrAf2yEVPQXJfdZZkq9FUEFN8FNcZ/CX+YuCoxWWhAzUfKK3A5bO+DEzWvFqa Xoay48seu8IItFzPAo19jT3QLN86VamUNK0rT731Xo6X14CSZgiu8WOAMV7h3b6dk/rZ kUm7WBsDMWlxjwUr+hLbzbkh70JmAS9Di6ud874OCtkjFXKUW2IzriPWlF58FQMHeUNV HjVViB558Nda0H2aXymJsyUnSQTQM+ffYb4WKwj8nQRN3Oav1mZTR1h6wNQ1zXZttoZg 6u7w== X-Gm-Message-State: AOAM533OhKX3WtuzK17EA8RCpHC0172bnCl8yE0hDH6dABj5AXtHCwty kHa/PQW3P8Cy7rovblytB/DzqA== X-Google-Smtp-Source: ABdhPJxIsyuMwyfFK+YWY/8UN0ykI79Dmd+IfHF88uSKNdHy2vmbGNi43R1JVapD/sonJonUkmDEIg== X-Received: by 2002:aa7:da8c:: with SMTP id q12mr4234503eds.385.1592338542217; Tue, 16 Jun 2020 13:15:42 -0700 (PDT) Received: from localhost.localdomain (212-5-158-38.ip.btc-net.bg. [212.5.158.38]) by smtp.gmail.com with ESMTPSA id b14sm11602658ejq.105.2020.06.16.13.15.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2020 13:15:41 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org Cc: Hans Verkuil , Ezequiel Garcia , Maheshwar Ajja , Mauro Carvalho Chehab , Stanimir Varbanov Subject: [PATCH v2 4/4] venus: vdec: Add support for decode intra frames only Date: Tue, 16 Jun 2020 23:14:46 +0300 Message-Id: <20200616201446.15996-5-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200616201446.15996-1-stanimir.varbanov@linaro.org> References: <20200616201446.15996-1-stanimir.varbanov@linaro.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Adds support in the decoder for intra frames only decode. The implementation in the Venus use HFI property for thumbnail generation to lower memory usage and when the control is enabled the number of decoder output buffers for progressive stream will be one (for interlace two). We assume that the client will queue on the decoder input intra frames only but this is not mandatory. If the client queue non-intra frames on decoder input they will be returned on decoder output with an error. Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/core.h | 1 + drivers/media/platform/qcom/venus/vdec.c | 7 +++++++ drivers/media/platform/qcom/venus/vdec_ctrls.c | 9 ++++++++- 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h index 1bac30d4cf50..b9a3b9ca6ae1 100644 --- a/drivers/media/platform/qcom/venus/core.h +++ b/drivers/media/platform/qcom/venus/core.h @@ -191,6 +191,7 @@ struct vdec_controls { u32 post_loop_deb_mode; u32 profile; u32 level; + bool intra_only; }; struct venc_controls { diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c index 7c4c483d5438..aa68cefcae96 100644 --- a/drivers/media/platform/qcom/venus/vdec.c +++ b/drivers/media/platform/qcom/venus/vdec.c @@ -625,6 +625,13 @@ static int vdec_set_properties(struct venus_inst *inst) return ret; } + if (ctr->intra_only) { + ptype = HFI_PROPERTY_PARAM_VDEC_THUMBNAIL_MODE; + ret = hfi_session_set_property(inst, ptype, &en); + if (ret) + return ret; + } + return 0; } diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/venus/vdec_ctrls.c index 3a963cbd342a..96ca8d9dd22e 100644 --- a/drivers/media/platform/qcom/venus/vdec_ctrls.c +++ b/drivers/media/platform/qcom/venus/vdec_ctrls.c @@ -28,6 +28,9 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL: ctr->level = ctrl->val; break; + case V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY: + ctr->intra_only = ctrl->val; + break; default: return -EINVAL; } @@ -86,7 +89,7 @@ int vdec_ctrl_init(struct venus_inst *inst) struct v4l2_ctrl *ctrl; int ret; - ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 7); + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 8); if (ret) return ret; @@ -141,6 +144,10 @@ int vdec_ctrl_init(struct venus_inst *inst) if (ctrl) ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE; + v4l2_ctrl_new_std(&inst->ctrl_handler, &vdec_ctrl_ops, + V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY, + 0, 1, 1, 0); + ret = inst->ctrl_handler.error; if (ret) { v4l2_ctrl_handler_free(&inst->ctrl_handler);