From patchwork Mon Jul 6 21:54:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646981 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EF1114E3 for ; Mon, 6 Jul 2020 21:55:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D79520715 for ; Mon, 6 Jul 2020 21:55:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="CFjJyI82" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727908AbgGFVzX (ORCPT ); Mon, 6 Jul 2020 17:55:23 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:64851 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726835AbgGFVye (ORCPT ); Mon, 6 Jul 2020 17:54:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=TGA8bCtzG3W48MXoVBxFSHeewOIBjbl37ylvpTpVch8=; b=CFjJyI82kqHAAybk35Iw5Oxm6ciN3WXKnqdGBPs6ot03enrioSCkp+e7N7T6q7aas32L Hr264zsRRdggLlNKE+Xlb8Ji2bhg87huQr7K/Y3Fgy+H61iSc9DLbgPOZCt5uN+JkZ1/2F yiAEFNJ4JNO25EH2SUKRFh6Nit9l2iHHA= Received: by filterdrecv-p3iad2-5b55dcd864-d2lx7 with SMTP id filterdrecv-p3iad2-5b55dcd864-d2lx7-18-5F039D9A-4 2020-07-06 21:54:34.170135634 +0000 UTC m=+881111.822229131 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id lAitVEpTTyqzql48WShlmQ Mon, 06 Jul 2020 21:54:33.919 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 02/12] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Mon, 06 Jul 2020 21:54:34 +0000 (UTC) Message-Id: <20200706215430.22859-3-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0hxOVj4gd+wIhsMeaZ4/uq2uZ67RzqPtTDrFqEKz5LOLXtj/Ff4sWlE2++AwGx5hzQFsz0qLPWfM26W5k7Zgkfyh+NQuvGT7gW2DAY1rpO85eI33a6lEOTqA9+YLNLQDIn/SyuQAwxxdfwUEHZjGtPF1aeFRJW398HZXeXBbhXhwXqVn3GtEYT6z2RN1oiunIJg== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OPTUPT buffer resolution. Signed-off-by: Jonas Karlman --- Changes in v2: - New patch --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 55dc27171ce4..4ab8f7e0566b 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -235,6 +235,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Mon Jul 6 21:54:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646975 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 834B0739 for ; Mon, 6 Jul 2020 21:55:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 688E120723 for ; Mon, 6 Jul 2020 21:55:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="ZnsptEwt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727110AbgGFVyh (ORCPT ); Mon, 6 Jul 2020 17:54:37 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:19208 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726805AbgGFVyg (ORCPT ); Mon, 6 Jul 2020 17:54:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=cwF57Yd3nZtjPAkGbWz1sj11wL8vtzFB8jLVo6QFcIM=; b=ZnsptEwt5pk9pQUUQHZnrPCxrlJvV+eFjTyh1ZD2Km2EfnA6etRAM9ci095i2iX274kw JqmldnNPnoJ7rpPoqky+64n47oh5tPjVWsaAyp9ldu7XMImg0Nslxaz/zTHbmGZQ+OrOel Yiwo0ePnEgtTrKU3udyk36z/3+Y5yRq6M= Received: by filterdrecv-p3iad2-5b55dcd864-n86tl with SMTP id filterdrecv-p3iad2-5b55dcd864-n86tl-21-5F039D9A-2C 2020-07-06 21:54:34.700927518 +0000 UTC m=+881118.466369360 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id b1_Boy5iS3SQY-lAi3uGrA Mon, 06 Jul 2020 21:54:34.331 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 03/12] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Mon, 06 Jul 2020 21:54:34 +0000 (UTC) Message-Id: <20200706215430.22859-4-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h9qXHT5HF1Lnoqt7is33+HITW1dRwIOmFbYeLljfuRRo1rVg5ZzV/t652lKedAkyvqU9lOpy+i46k2Wq6oCefRoj/mUHcLgrZqUsRKRbfIcDaQvUHsBWlegHjsEeSw8PHRs1CBoV7qZSG4fu/qnZ1I+d4Aeu5VPCbrmj6WWwfHt+hBX5bwgftZe+PU4JTr2VBw== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The width and height in mbs is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. Signed-off-by: Jonas Karlman --- Changes in v2: - Validate against coded_fmt instead of decoded_fmt - Validation moved to rkvdec_try_ctrl --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++++++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index f0cfed84d60d..3498e9eec3d8 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 4ab8f7e0566b..7a9f78bc0a55 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,8 +29,11 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + if (ctrl->id == V4L2_CID_MPEG_VIDEO_H264_SPS) { const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p; + unsigned int width, height; /* * TODO: The hardware supports 10-bit and 4:2:2 profiles, * but it's currently broken in the driver. @@ -45,6 +48,13 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) if (sps->bit_depth_luma_minus8 != 0) /* Only 8-bit is supported */ return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; } return 0; } From patchwork Mon Jul 6 21:54:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646977 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 373E2739 for ; Mon, 6 Jul 2020 21:55:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E00320773 for ; Mon, 6 Jul 2020 21:55:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="hGRFbE36" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727102AbgGFVyh (ORCPT ); Mon, 6 Jul 2020 17:54:37 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:3927 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727038AbgGFVyf (ORCPT ); Mon, 6 Jul 2020 17:54:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=oQDyTyk1JzCRldIwq9SBmCjNp7iNMJTbJziH/eYuor0=; b=hGRFbE36/plQ2L4O1mrPAGqBLc6ZqlUA/94+IA9YgLQrhlkO9XI/DfxPby1W2liHgk/q eaL1wqNghDqNJOB7EUz0WdAzeqLgjfmSrdnplHYYX/C8Aqzb3xGF2llFZs8ZZJe+TzuGx5 /5fEK6nUoKiTSqeZNB8NMsTaiqeBhlllE= Received: by filterdrecv-p3iad2-5b55dcd864-pbk2w with SMTP id filterdrecv-p3iad2-5b55dcd864-pbk2w-17-5F039D9A-68 2020-07-06 21:54:34.983629525 +0000 UTC m=+881115.107033651 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id J3M56gfURVypOe2JLmhI8A Mon, 06 Jul 2020 21:54:34.738 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 04/12] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Mon, 06 Jul 2020 21:54:35 +0000 (UTC) Message-Id: <20200706215430.22859-5-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h1efvHYa9/ptEut3+r5e7y4PIlttPYGcFfLug8A7ah0XXMgBi9nG7DhjCn8wOxRy91+3nOQ5fIrAIRprtQBVJN6mWi2zr7qEmqiwnLV2V0zQQW3nNnNOWLF8JkYd0e+2OC2K2avq0ZyDmwfImmZ01ijzDXjj7nSoy+XHGYhkPDZnhyyumkpEzwRY036mHLd0wQ== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The luma and chroma bit depth fields in the pps packet is 3 bits wide. 8 is wrongly added to the bit depth value written to these 3-bit fields. Because only the 3 LSB is written the hardware is configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Signed-off-by: Jonas Karlman Reviewed-by: Ezequiel Garcia --- Changes in v2: - Collect r-b tag --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3498e9eec3d8..6576b4a101ae 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Mon Jul 6 21:54:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646979 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A306D739 for ; Mon, 6 Jul 2020 21:55:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7FC8320674 for ; Mon, 6 Jul 2020 21:55:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="eHn0cmBP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726915AbgGFVzI (ORCPT ); Mon, 6 Jul 2020 17:55:08 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:30076 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727076AbgGFVyg (ORCPT ); Mon, 6 Jul 2020 17:54:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=f4gyh5BbpSNrHqLWYRO43egmkaJ0ArYD/IydQmU1/fw=; b=eHn0cmBPH0r8LYqSgN26UoBdh2iFUGj66BfDvhV62GrcFwE782/WYQD6g4VtA3H79JkA wtH3tuRapEcsLXxz4l6unM/ufOSH/cKb66sqyYhMTWPmkbFYkQBYDR0flAKG9hquSiGeqT 3rpePa3ymbqPD8WcmMmkxK6Rhr7bQGloo= Received: by filterdrecv-p3iad2-5b55dcd864-l2x6n with SMTP id filterdrecv-p3iad2-5b55dcd864-l2x6n-19-5F039D9B-18 2020-07-06 21:54:35.393097943 +0000 UTC m=+881114.024848071 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id 3AFt_t2JSWu7UKeWJ0wpVA Mon, 06 Jul 2020 21:54:35.147 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 05/12] media: rkvdec: h264: Do not override output buffer sizeimage Date: Mon, 06 Jul 2020 21:54:35 +0000 (UTC) Message-Id: <20200706215430.22859-6-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h/ZQMlPTDcLuCQ0tVzeXPkGBV4qNq3S0YEgC/N9LaI6/aUWSS1BRUTm0RQgkvcSeULS8rxgiQ9lbE/drjo+gB1I5RdKoM8lHIY1UeTIVvFDiIy5N7cVfWF1tfyRNacrz1VJSopsR5dH0yoVC6sDpvzGSS859Q3uw9B0r2lQxWdZ9bSeTiWKnmomdu3AIAmDMLw== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The output buffer sizeimage is currently forced to 2 bytes per pixel, this can lead to high memory usage for 4K content when multiple output buffers is created by userspace. Do not override output buffer sizeimage and let userspace have control of output buffer sizeimage by only setting sizeimage if none is provided. Signed-off-by: Jonas Karlman --- Changes in v2: - New patch --- drivers/staging/media/rkvdec/rkvdec-h264.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 6576b4a101ae..3a85545bcb38 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -1012,8 +1012,9 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx, struct v4l2_pix_format_mplane *fmt = &f->fmt.pix_mp; fmt->num_planes = 1; - fmt->plane_fmt[0].sizeimage = fmt->width * fmt->height * - RKVDEC_H264_MAX_DEPTH_IN_BYTES; + if (!fmt->plane_fmt[0].sizeimage) + fmt->plane_fmt[0].sizeimage = fmt->width * fmt->height * + RKVDEC_H264_MAX_DEPTH_IN_BYTES; return 0; } From patchwork Mon Jul 6 21:54:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646935 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B0DF1398 for ; Mon, 6 Jul 2020 21:54:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 583CD20723 for ; Mon, 6 Jul 2020 21:54:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="Jcfg7XAT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727826AbgGFVyj (ORCPT ); Mon, 6 Jul 2020 17:54:39 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:5084 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727082AbgGFVyi (ORCPT ); Mon, 6 Jul 2020 17:54:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=7H3DbOIR6DWUdA/y3YLDXdf//aCB9yI7dsoADDEwozU=; b=Jcfg7XATDP7G5Kwyih6QhhwruQJRtV9idRe038JFEN351FtgQQXLiJfGxT/ho3rhlynJ FIUsYcz7F6k1Jw1dOSuPBBs06rF0gCKlKxEflkOZtNItAs+ula4k++tDOefCY2ct9qnO0+ bLBVpEthk8ObvphiWk2PL7N5nGCLN3ACI= Received: by filterdrecv-p3iad2-5b55dcd864-v6r54 with SMTP id filterdrecv-p3iad2-5b55dcd864-v6r54-19-5F039D9B-54 2020-07-06 21:54:35.80985375 +0000 UTC m=+881109.656863597 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id gfC6wkyfTOGWv03XGF9v8A Mon, 06 Jul 2020 21:54:35.558 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 06/12] media: v4l2-common: Add helpers to calculate bytesperline and sizeimage Date: Mon, 06 Jul 2020 21:54:35 +0000 (UTC) Message-Id: <20200706215430.22859-7-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h3y1x8235EXAV4YClet+SGrNaA0R6qJhUAIyAfAHdHztlPiXwwTHlNjo7TXKaduOpyw+hl91q4Pr44PzU91Fvzm53nHJXSVWXyXN4JbMVDsJpbnbQN0fIsnoIumkt/cAy6oxxgBAJkqsFjcI2WP/AuVLf/fXNNV25ZYfJ2zJIF7Ph9lHWf1VaMZrI0Te665U/A== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add helper functions to calculate plane bytesperline and sizeimage, these new helpers consider block width and height when calculating plane bytesperline and sizeimage. This prepare support for new pixel formats added in next patch that make use of block width and height. Signed-off-by: Jonas Karlman --- drivers/media/v4l2-core/v4l2-common.c | 77 +++++++++++++-------------- 1 file changed, 38 insertions(+), 39 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c index 3dc17ebe14fa..4102c373b48a 100644 --- a/drivers/media/v4l2-core/v4l2-common.c +++ b/drivers/media/v4l2-core/v4l2-common.c @@ -333,6 +333,33 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf return info->block_h[plane]; } +static inline unsigned int v4l2_format_plane_width(const struct v4l2_format_info *info, int plane, + unsigned int width) +{ + unsigned int hdiv = plane ? info->hdiv : 1; + unsigned int bytes = DIV_ROUND_UP(width * info->bpp[plane], + v4l2_format_block_width(info, plane) * + v4l2_format_block_height(info, plane)); + + return DIV_ROUND_UP(bytes, hdiv); +} + +static inline unsigned int v4l2_format_plane_height(const struct v4l2_format_info *info, int plane, + unsigned int height) +{ + unsigned int vdiv = plane ? info->vdiv : 1; + unsigned int lines = ALIGN(height, v4l2_format_block_height(info, plane)); + + return DIV_ROUND_UP(lines, vdiv); +} + +static inline unsigned int v4l2_format_plane_size(const struct v4l2_format_info *info, int plane, + unsigned int width, unsigned int height) +{ + return v4l2_format_plane_width(info, plane, width) * + v4l2_format_plane_height(info, plane, height); +} + void v4l2_apply_frmsize_constraints(u32 *width, u32 *height, const struct v4l2_frmsize_stepwise *frmsize) { @@ -368,37 +395,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt, if (info->mem_planes == 1) { plane = &pixfmt->plane_fmt[0]; - plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0]; + plane->bytesperline = v4l2_format_plane_width(info, 0, width); plane->sizeimage = 0; - for (i = 0; i < info->comp_planes; i++) { - unsigned int hdiv = (i == 0) ? 1 : info->hdiv; - unsigned int vdiv = (i == 0) ? 1 : info->vdiv; - unsigned int aligned_width; - unsigned int aligned_height; - - aligned_width = ALIGN(width, v4l2_format_block_width(info, i)); - aligned_height = ALIGN(height, v4l2_format_block_height(info, i)); - - plane->sizeimage += info->bpp[i] * - DIV_ROUND_UP(aligned_width, hdiv) * - DIV_ROUND_UP(aligned_height, vdiv); - } + for (i = 0; i < info->comp_planes; i++) + plane->sizeimage += + v4l2_format_plane_size(info, i, width, height); } else { for (i = 0; i < info->comp_planes; i++) { - unsigned int hdiv = (i == 0) ? 1 : info->hdiv; - unsigned int vdiv = (i == 0) ? 1 : info->vdiv; - unsigned int aligned_width; - unsigned int aligned_height; - - aligned_width = ALIGN(width, v4l2_format_block_width(info, i)); - aligned_height = ALIGN(height, v4l2_format_block_height(info, i)); - plane = &pixfmt->plane_fmt[i]; plane->bytesperline = - info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv); - plane->sizeimage = - plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv); + v4l2_format_plane_width(info, i, width); + plane->sizeimage = plane->bytesperline * + v4l2_format_plane_height(info, i, height); } } return 0; @@ -422,22 +431,12 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat, pixfmt->width = width; pixfmt->height = height; pixfmt->pixelformat = pixelformat; - pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0]; + pixfmt->bytesperline = v4l2_format_plane_width(info, 0, width); pixfmt->sizeimage = 0; - for (i = 0; i < info->comp_planes; i++) { - unsigned int hdiv = (i == 0) ? 1 : info->hdiv; - unsigned int vdiv = (i == 0) ? 1 : info->vdiv; - unsigned int aligned_width; - unsigned int aligned_height; - - aligned_width = ALIGN(width, v4l2_format_block_width(info, i)); - aligned_height = ALIGN(height, v4l2_format_block_height(info, i)); - - pixfmt->sizeimage += info->bpp[i] * - DIV_ROUND_UP(aligned_width, hdiv) * - DIV_ROUND_UP(aligned_height, vdiv); - } + for (i = 0; i < info->comp_planes; i++) + pixfmt->sizeimage += + v4l2_format_plane_size(info, i, width, height); return 0; } EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt); From patchwork Mon Jul 6 21:54:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646945 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C9C814E3 for ; Mon, 6 Jul 2020 21:54:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5764B2075B for ; Mon, 6 Jul 2020 21:54:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="dv2HPvdD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727850AbgGFVyo (ORCPT ); Mon, 6 Jul 2020 17:54:44 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:9245 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727090AbgGFVyj (ORCPT ); Mon, 6 Jul 2020 17:54:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:mime-version:to:cc:content-type: content-transfer-encoding; s=001; bh=YoXiEJGtZe0e7X5GwX7ycoiC6AsWzUaJUSnHJI7q4k8=; b=dv2HPvdDZVTAlgJZMaQHJulOWjoTNOr+HwhYhrMsYt15lNSk3Es+IUgUFOT+bjplpNbJ 9a4kcAgjJaXxXnpzA8gcYsTq3RfaScoxecrWS/hBoxIVRB0LhTG4N4dxBT9EN9DaAv+wTL WCb9pBCC+OCG+dpNyvUNxF5jUiiCxNxDk= Received: by filterdrecv-p3iad2-5b55dcd864-m99xc with SMTP id filterdrecv-p3iad2-5b55dcd864-m99xc-18-5F039D9C-C 2020-07-06 21:54:36.217052106 +0000 UTC m=+881110.822000320 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id 3XDDoPchSpu615aeBNfrVQ Mon, 06 Jul 2020 21:54:35.967 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 07/12] media: v4l2: Add NV15 and NV20 pixel formats Date: Mon, 06 Jul 2020 21:54:36 +0000 (UTC) Message-Id: <20200706215430.22859-8-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> MIME-Version: 1.0 X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0hxCdMn1E/szv+Qbpqh40sX18D23R7F+1AriqcTnttXyDVUnNOtMcFGomDQge966iQXmD1rtNAfmZU8StJOiHx7WHkw/bQ7Bq+QcbNUpoqoM3JQgmmWsSHy0aN9bJQdVmeU59vrNA+9hMdDXk/FoPdD+S+/Mh58zqQTyPYx6IN3pVJ9eX1RX2PkAwjE7sPov2eQ== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add NV15 and NV20 pixel formats used by the Rockchip Video Decoder for 10-bit buffers. NV15 and NV20 is a packed 10-bit 4:2:0/4:2:2 semi-planar Y/CbCr format similar to P010 and P210 but has no padding between components. Instead, luminance and chrominance samples are grouped into 4s so that each group is packed into an integer number of bytes: YYYY = UVUV = 4 * 10 bits = 40 bits = 5 bytes The '15' and '20' suffix refers to the optimum effective bits per pixel which is achieved when the total number of luminance samples is a multiple of 8 for NV15 and 4 for NV20. Signed-off-by: Jonas Karlman --- .../userspace-api/media/v4l/pixfmt-nv15.rst | 101 ++++++++++++++++++ .../userspace-api/media/v4l/pixfmt-nv20.rst | 99 +++++++++++++++++ .../userspace-api/media/v4l/yuv-formats.rst | 2 + drivers/media/v4l2-core/v4l2-common.c | 3 + drivers/media/v4l2-core/v4l2-ioctl.c | 2 + include/uapi/linux/videodev2.h | 3 + 6 files changed, 210 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-nv15.rst create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-nv20.rst diff --git a/Documentation/userspace-api/media/v4l/pixfmt-nv15.rst b/Documentation/userspace-api/media/v4l/pixfmt-nv15.rst new file mode 100644 index 000000000000..d059db58c6e0 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/pixfmt-nv15.rst @@ -0,0 +1,101 @@ +.. Permission is granted to copy, distribute and/or modify this +.. document under the terms of the GNU Free Documentation License, +.. Version 1.1 or any later version published by the Free Software +.. Foundation, with no Invariant Sections, no Front-Cover Texts +.. and no Back-Cover Texts. A copy of the license is included at +.. Documentation/userspace-api/media/fdl-appendix.rst. +.. +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections + +.. _V4L2-PIX-FMT-NV15: + +************************** +V4L2_PIX_FMT_NV15 ('NV15') +************************** + +Format with ½ horizontal and vertical chroma resolution, also known as +YUV 4:2:0. One luminance and one chrominance plane with alternating +chroma samples similar to ``V4L2_PIX_FMT_NV12`` but with 10-bit samples +that are grouped into four and packed into five bytes. + +The '15' suffix refers to the optimum effective bits per pixel which is +achieved when the total number of luminance samples is a multiple of 8. + + +Description +=========== + +This is a packed 10-bit two-plane version of the YUV 4:2:0 format. The +three components are separated into two sub-images or planes. The Y plane +is first. The Y plane has five bytes per each group of four pixels. A +combined CbCr plane immediately follows the Y plane in memory. The CbCr +plane is the same width, in bytes, as the Y plane (and of the image), but +is half as tall in pixels. Each CbCr pair belongs to four pixels. For +example, Cb\ :sub:`00`/Cr\ :sub:`00` belongs to Y'\ :sub:`00`, +Y'\ :sub:`01`, Y'\ :sub:`10`, Y'\ :sub:`11`. + +If the Y plane has pad bytes after each row, then the CbCr plane has as +many pad bytes after its rows. + +**Byte Order.** +Little endian. Each cell is one byte. Pixels cross the byte boundary. + + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + + * - start + 0: + - Y'\ :sub:`00[7:0]` + - Y'\ :sub:`01[5:0]`\ Y'\ :sub:`00[9:8]` + - Y'\ :sub:`02[3:0]`\ Y'\ :sub:`01[9:6]` + - Y'\ :sub:`03[1:0]`\ Y'\ :sub:`02[9:4]` + - Y'\ :sub:`03[9:2]` + * - start + 5: + - Y'\ :sub:`10[7:0]` + - Y'\ :sub:`11[5:0]`\ Y'\ :sub:`10[9:8]` + - Y'\ :sub:`12[3:0]`\ Y'\ :sub:`11[9:6]` + - Y'\ :sub:`13[1:0]`\ Y'\ :sub:`12[9:4]` + - Y'\ :sub:`13[9:2]` + * - start + 10: + - Cb'\ :sub:`00[7:0]` + - Cr'\ :sub:`00[5:0]`\ Cb'\ :sub:`00[9:8]` + - Cb'\ :sub:`01[3:0]`\ Cr'\ :sub:`00[9:6]` + - Cr'\ :sub:`01[1:0]`\ Cb'\ :sub:`01[9:4]` + - Cr'\ :sub:`01[9:2]` + + +**Color Sample Location:** + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + + * - + - 0 + - + - 1 + - 2 + - + - 3 + * - 0 + - Y + - + - Y + - Y + - + - Y + * - + - + - C + - + - + - C + - + * - 1 + - Y + - + - Y + - Y + - + - Y diff --git a/Documentation/userspace-api/media/v4l/pixfmt-nv20.rst b/Documentation/userspace-api/media/v4l/pixfmt-nv20.rst new file mode 100644 index 000000000000..a8123be0baa3 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/pixfmt-nv20.rst @@ -0,0 +1,99 @@ +.. Permission is granted to copy, distribute and/or modify this +.. document under the terms of the GNU Free Documentation License, +.. Version 1.1 or any later version published by the Free Software +.. Foundation, with no Invariant Sections, no Front-Cover Texts +.. and no Back-Cover Texts. A copy of the license is included at +.. Documentation/userspace-api/media/fdl-appendix.rst. +.. +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections + +.. _V4L2-PIX-FMT-NV20: + +************************** +V4L2_PIX_FMT_NV20 ('NV20') +************************** + +Format with ½ horizontal chroma resolution, also known as YUV 4:2:2. +One luminance and one chrominance plane with alternating chroma samples +similar to ``V4L2_PIX_FMT_NV16`` but with 10-bit samples +that are grouped into four and packed into five bytes. + +The '20' suffix refers to the optimum effective bits per pixel which is +achieved when the total number of luminance samples is a multiple of 4. + + +Description +=========== + +This is a packed 10-bit two-plane version of the YUV 4:2:2 format. The +three components are separated into two sub-images or planes. The Y plane +is first. The Y plane has five bytes per each group of four pixels. A +combined CbCr plane immediately follows the Y plane in memory. The CbCr +plane is the same width and height, in bytes, as the Y plane (and of the +image). Each CbCr pair belongs to two pixels. For example, +Cb\ :sub:`00`/Cr\ :sub:`00` belongs to Y'\ :sub:`00`, Y'\ :sub:`01`. + +If the Y plane has pad bytes after each row, then the CbCr plane has as +many pad bytes after its rows. + +**Byte Order.** +Little endian. Each cell is one byte. Pixels cross the byte boundary. + + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + + * - start + 0: + - Y'\ :sub:`00[7:0]` + - Y'\ :sub:`01[5:0]`\ Y'\ :sub:`00[9:8]` + - Y'\ :sub:`02[3:0]`\ Y'\ :sub:`01[9:6]` + - Y'\ :sub:`03[1:0]`\ Y'\ :sub:`02[9:4]` + - Y'\ :sub:`03[9:2]` + * - start + 5: + - Y'\ :sub:`10[7:0]` + - Y'\ :sub:`11[5:0]`\ Y'\ :sub:`10[9:8]` + - Y'\ :sub:`12[3:0]`\ Y'\ :sub:`11[9:6]` + - Y'\ :sub:`13[1:0]`\ Y'\ :sub:`12[9:4]` + - Y'\ :sub:`13[9:2]` + * - start + 10: + - Cb'\ :sub:`00[7:0]` + - Cr'\ :sub:`00[5:0]`\ Cb'\ :sub:`00[9:8]` + - Cb'\ :sub:`01[3:0]`\ Cr'\ :sub:`00[9:6]` + - Cr'\ :sub:`01[1:0]`\ Cb'\ :sub:`01[9:4]` + - Cr'\ :sub:`01[9:2]` + * - start + 15: + - Cb'\ :sub:`10[7:0]` + - Cr'\ :sub:`10[5:0]`\ Cb'\ :sub:`10[9:8]` + - Cb'\ :sub:`11[3:0]`\ Cr'\ :sub:`10[9:6]` + - Cr'\ :sub:`11[1:0]`\ Cb'\ :sub:`11[9:4]` + - Cr'\ :sub:`11[9:2]` + + +**Color Sample Location:** + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + + * - + - 0 + - + - 1 + - 2 + - + - 3 + * - 0 + - Y + - C + - Y + - Y + - C + - Y + * - 1 + - Y + - C + - Y + - Y + - C + - Y diff --git a/Documentation/userspace-api/media/v4l/yuv-formats.rst b/Documentation/userspace-api/media/v4l/yuv-formats.rst index 8ee92d0cd769..7cca883f178a 100644 --- a/Documentation/userspace-api/media/v4l/yuv-formats.rst +++ b/Documentation/userspace-api/media/v4l/yuv-formats.rst @@ -61,4 +61,6 @@ to brightness information. pixfmt-nv16 pixfmt-nv16m pixfmt-nv24 + pixfmt-nv15 + pixfmt-nv20 pixfmt-m420 diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c index 4102c373b48a..0caac755d303 100644 --- a/drivers/media/v4l2-core/v4l2-common.c +++ b/drivers/media/v4l2-core/v4l2-common.c @@ -267,6 +267,9 @@ const struct v4l2_format_info *v4l2_format_info(u32 format) { .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_NV15, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 5, 0, 0 }, .hdiv = 2, .vdiv = 2, .block_w = { 4, 2, 0, 0 }, .block_h = { 1, 1, 0, 0 } }, + { .format = V4L2_PIX_FMT_NV20, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 5, 0, 0 }, .hdiv = 2, .vdiv = 1, .block_w = { 4, 2, 0, 0 }, .block_h = { 1, 1, 0, 0 } }, + { .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 }, { .format = V4L2_PIX_FMT_YVU410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 }, { .format = V4L2_PIX_FMT_YUV411P, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 1 }, diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 02bfef0da76d..4657274bb37a 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1315,6 +1315,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break; case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break; case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break; + case V4L2_PIX_FMT_NV15: descr = "10-bit Y/CbCr 4:2:0 (Packed)"; break; + case V4L2_PIX_FMT_NV20: descr = "10-bit Y/CbCr 4:2:2 (Packed)"; break; case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break; case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break; case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 303805438814..bd23aeaf0706 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -610,6 +610,9 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */ #define V4L2_PIX_FMT_NV42 v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb 4:4:4 */ +#define V4L2_PIX_FMT_NV15 v4l2_fourcc('N', 'V', '1', '5') /* 15 Y/CbCr 4:2:0 10-bit packed */ +#define V4L2_PIX_FMT_NV20 v4l2_fourcc('N', 'V', '2', '0') /* 20 Y/CbCr 4:2:2 10-bit packed */ + /* two non contiguous planes - one Y, one Cr + Cb interleaved */ #define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */ #define V4L2_PIX_FMT_NV21M v4l2_fourcc('N', 'M', '2', '1') /* 21 Y/CrCb 4:2:0 */ From patchwork Mon Jul 6 21:54:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646967 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2892739 for ; Mon, 6 Jul 2020 21:54:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CA2D920723 for ; Mon, 6 Jul 2020 21:54:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="i61qMz44" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727817AbgGFVyi (ORCPT ); Mon, 6 Jul 2020 17:54:38 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:38039 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727098AbgGFVyh (ORCPT ); Mon, 6 Jul 2020 17:54:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=3CQUn2PVnW9X4yx0d9/3EzKXGgYmdWlnwnt1CztSmW0=; b=i61qMz44kVFyABbirdM2EF8Jyzb44X8XPtXarmhQL879GU1X0fXluEIb//DemiFh5ROO ZW+V/BOsglT7Mdzutu+Ylp7JsuCWqoXQaKUHsQ/FfsZWH3Gb93Mq1JfJKt6q2r2sfeUlQn C7PjOdWu7UsZZ40yqdPoEOoqZJt48eyJ4= Received: by filterdrecv-p3iad2-5b55dcd864-n86tl with SMTP id filterdrecv-p3iad2-5b55dcd864-n86tl-21-5F039D9C-3C 2020-07-06 21:54:36.626597881 +0000 UTC m=+881120.392039716 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id F70ZXhhbSZ6A7Rdnhu03sw Mon, 06 Jul 2020 21:54:36.378 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 08/12] media: rkvdec: h264: Use bytesperline and buffer height to calculate stride Date: Mon, 06 Jul 2020 21:54:36 +0000 (UTC) Message-Id: <20200706215430.22859-9-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h9ohZspGJZfMni/fzfWWRa+I4LBx+40cubwsiXB2hHCNd1DojwFpL+Iri5OLgLH8uykEtrqZ2jmg9+1iGqlJnYZ+Myirhs03kOYhGIbk0UpihSYPRDqSQ+gpOtL/Q8YZaKhU4lT5Z5ChyOkDGOmFjPt3RDYqLoA6e4I+q7CTaVMrC/CGyCo2Jk6KbFzzVrRsnw== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Use bytesperline and buffer height to calculate the strides configured. This does not really change anything other than ensuring the bytesperline that is signaled to userspace matches what is configured in HW. Signed-off-by: Jonas Karlman --- Changes in v2: - Drop code refactoring --- drivers/staging/media/rkvdec/rkvdec-h264.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3a85545bcb38..10756b9d6118 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -891,9 +891,9 @@ static void config_registers(struct rkvdec_ctx *ctx, dma_addr_t rlc_addr; dma_addr_t refer_addr; u32 rlc_len; - u32 hor_virstride = 0; - u32 ver_virstride = 0; - u32 y_virstride = 0; + u32 hor_virstride; + u32 ver_virstride; + u32 y_virstride; u32 yuv_virstride = 0; u32 offset; dma_addr_t dst_addr; @@ -904,8 +904,8 @@ static void config_registers(struct rkvdec_ctx *ctx, f = &ctx->decoded_fmt; dst_fmt = &f->fmt.pix_mp; - hor_virstride = (sps->bit_depth_luma_minus8 + 8) * dst_fmt->width / 8; - ver_virstride = round_up(dst_fmt->height, 16); + hor_virstride = dst_fmt->plane_fmt[0].bytesperline; + ver_virstride = dst_fmt->height; y_virstride = hor_virstride * ver_virstride; if (sps->chroma_format_idc == 0) From patchwork Mon Jul 6 21:54:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646937 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 398BB739 for ; Mon, 6 Jul 2020 21:54:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 204C82075B for ; Mon, 6 Jul 2020 21:54:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="gegNJQ/C" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727099AbgGFVyk (ORCPT ); Mon, 6 Jul 2020 17:54:40 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:46354 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727116AbgGFVyi (ORCPT ); Mon, 6 Jul 2020 17:54:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=Xti8MeJ8Buq5Qftfoor5F4zHqB4uXzWh3iX3dz9i1n4=; b=gegNJQ/Cza48PHNmiYol3bMb+VwjtoSy8ATO5peZAUPptJePkso14bAVCxgL0ato3Lkx pSXIu3UiEpUw5Kejcn/EsclTy+rKbN/VF6Z0t6fnQg7Td+f4gzleRFwmWbw7tR8018x8lp k7/SIprTDr9p0n6hdCq4ItuThlP7SeJyE= Received: by filterdrecv-p3iad2-5b55dcd864-hp5f5 with SMTP id filterdrecv-p3iad2-5b55dcd864-hp5f5-19-5F039D9D-4 2020-07-06 21:54:37.156872496 +0000 UTC m=+881115.200300484 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id juJseSgoSt6z8Gm-IMNBng Mon, 06 Jul 2020 21:54:36.786 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 09/12] media: rkvdec: Extract rkvdec_fill_decoded_pixfmt helper method Date: Mon, 06 Jul 2020 21:54:37 +0000 (UTC) Message-Id: <20200706215430.22859-10-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0hzopxZlLT9tDvbnpZje3k9acDUv2WkNylHW0j6kACYxx49WD22f7CEROr37QhI9pBQcxUTEwTpXNASxS4ZrEhaaWeLaGUuDAXWbudvhVNNORh4QR5v5u/9ynSn1LXVzQDtLTCYBZaFukhMLLiGvbm7H6Aqe4d+Sknnz94wqL0c2a9/HPnoT2QPWgd1ZFE460vQ== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This extract setting decoded pixfmt into a helper method, current code is replaced with a call to the new helper method. The helper method is also called from a new function in next patch. Signed-off-by: Jonas Karlman --- Changes in v2: - New patch --- drivers/staging/media/rkvdec/rkvdec.c | 29 ++++++++++++++------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 7a9f78bc0a55..911132be6d50 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -27,6 +27,17 @@ #include "rkvdec.h" #include "rkvdec-regs.h" +static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx, + struct v4l2_pix_format_mplane *pix_mp) +{ + v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, + pix_mp->width, pix_mp->height); + pix_mp->plane_fmt[0].sizeimage += 128 * + DIV_ROUND_UP(pix_mp->width, 16) * + DIV_ROUND_UP(pix_mp->height, 16); + pix_mp->field = V4L2_FIELD_NONE; +} + static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); @@ -179,13 +190,9 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx) rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]); f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; - v4l2_fill_pixfmt_mp(&f->fmt.pix_mp, - ctx->coded_fmt_desc->decoded_fmts[0], - ctx->coded_fmt.fmt.pix_mp.width, - ctx->coded_fmt.fmt.pix_mp.height); - f->fmt.pix_mp.plane_fmt[0].sizeimage += 128 * - DIV_ROUND_UP(f->fmt.pix_mp.width, 16) * - DIV_ROUND_UP(f->fmt.pix_mp.height, 16); + f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width; + f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height; + rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp); } static int rkvdec_enum_framesizes(struct file *file, void *priv, @@ -251,13 +258,7 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, &pix_mp->height, &coded_desc->frmsize); - v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, - pix_mp->width, pix_mp->height); - pix_mp->plane_fmt[0].sizeimage += - 128 * - DIV_ROUND_UP(pix_mp->width, 16) * - DIV_ROUND_UP(pix_mp->height, 16); - pix_mp->field = V4L2_FIELD_NONE; + rkvdec_fill_decoded_pixfmt(ctx, pix_mp); return 0; } From patchwork Mon Jul 6 21:54:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646949 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D5F514E3 for ; Mon, 6 Jul 2020 21:54:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6C4FA2073E for ; Mon, 6 Jul 2020 21:54:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="YhYy5chJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727858AbgGFVyp (ORCPT ); Mon, 6 Jul 2020 17:54:45 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:19624 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727124AbgGFVyj (ORCPT ); Mon, 6 Jul 2020 17:54:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=3J91EmTF5OcN7m3vDKl6Rjiu026EUSnJD5AzCjIGOG8=; b=YhYy5chJSocG6eniupnaylgEcjEdI83n6ieim4g31wWdZYhp0g+7BoESXr5eYcYxMDG4 IRSbLeFN32P97k/+Q0FpnS2e2YVRMjxnY7SZFK+6zFi24sjpJCtrt1NDE2v7/BParEVZ4I eamt6gmNiFKkl3Sm+ZsXeeOGvM6KAkfVg= Received: by filterdrecv-p3iad2-5b55dcd864-v6r54 with SMTP id filterdrecv-p3iad2-5b55dcd864-v6r54-19-5F039D9D-2F 2020-07-06 21:54:37.447138052 +0000 UTC m=+881111.294147889 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id Dz87MUcKQ8qbnGWsGSIFHg Mon, 06 Jul 2020 21:54:37.197 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 10/12] media: rkvdec: Lock capture pixel format in s_ctrl and s_fmt Date: Mon, 06 Jul 2020 21:54:37 +0000 (UTC) Message-Id: <20200706215430.22859-11-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h0i3HAJ5P855+2MmHPYyDRJwSFB674zB7UomfNLPdwR8Qrr2woWvwvrSny51hdKo9SN0lZ7AGR1S425aXw+20Se4Hu8mX157gdpbgopgsEbC5hJT/qPSCjBGsGQZmKsB8tp111LkbMNC3SxPoTjXfIjtorkOGHjrK/4kwwyhQp4RQb5iUpzS5hjJUckph7I6Gw== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add an optional valid_fmt operation that should return the valid pixelformat of CAPTURE buffers. This is used in next patch to ensure correct pixelformat is used for 10-bit and 4:2:2 content. Signed-off-by: Jonas Karlman --- Changes in v2: - Reworked validation of pixelformat Limitations: - Only a single valid format is supported, technically both CbCr and CrCb format variants could be supported by HW. Example call order using ffmpeg for a 10-bit H.264 608x480 video: 0) Open /dev/video node rkvdec_reset_decoded_fmt: current valid_fmt= new valid_fmt= rkvdec_fill_decoded_pixfmt: pixelformat=NV12 width=64 height=48 rkvdec_s_ctrl: current valid_fmt= new valid_fmt=NV12 rkvdec_fill_decoded_pixfmt: pixelformat=NV12 width=64 height=48 1) Set the coded format on the OUTPUT queue via VIDIOC_S_FMT() rkvdec_enum_output_fmt: index=0 pixelformat=S264 rkvdec_s_output_fmt: pixelformat=S264 width=608 height=480 valid_fmt=NV12 rkvdec_reset_decoded_fmt: current valid_fmt=NV12 new valid_fmt= rkvdec_fill_decoded_pixfmt: pixelformat=NV12 width=640 height=480 2) Call VIDIOC_S_EXT_CTRLS() to set all the controls rkvdec_try_ctrl: current valid_fmt= rkvdec_s_ctrl: current valid_fmt= new valid_fmt=NV15 rkvdec_fill_decoded_pixfmt: pixelformat=NV15 width=640 height=480 3) Call VIDIOC_G_FMT() for CAPTURE queue rkvdec_g_capture_fmt: pixelformat=NV15 width=640 height=480 4) Enumerate CAPTURE formats via VIDIOC_ENUM_FMT() on the CAPTURE queue rkvdec_enum_capture_fmt: index=0 pixelformat=NV15 valid_fmt=NV15 5) Choose a different CAPTURE format than suggested via VIDIOC_G_FMT() rkvdec_s_capture_fmt: pixelformat=NV15 width=608 height=480 valid_fmt=NV15 rkvdec_fill_decoded_pixfmt: pixelformat=NV15 width=640 height=480 rkvdec_g_capture_fmt: pixelformat=NV15 width=640 height=480 --- drivers/staging/media/rkvdec/rkvdec.c | 59 ++++++++++++++++++++++++--- drivers/staging/media/rkvdec/rkvdec.h | 2 + 2 files changed, 55 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 911132be6d50..11a88cb6407d 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -38,6 +38,16 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx, pix_mp->field = V4L2_FIELD_NONE; } +static u32 rkvdec_valid_fmt(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + const struct rkvdec_coded_fmt_desc *coded_desc = ctx->coded_fmt_desc; + + if (coded_desc->ops->valid_fmt) + return coded_desc->ops->valid_fmt(ctx, ctrl); + + return ctx->valid_fmt; +} + static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); @@ -60,6 +70,10 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) /* Only 8-bit is supported */ return -EINVAL; + if (ctx->valid_fmt && ctx->valid_fmt != rkvdec_valid_fmt(ctx, ctrl)) + /* Only current valid format */ + return -EINVAL; + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; height = (sps->pic_height_in_map_units_minus1 + 1) * 16; @@ -70,8 +84,27 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) return 0; } +static int rkvdec_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + + if (ctrl->id == V4L2_CID_MPEG_VIDEO_H264_SPS && !ctx->valid_fmt) { + ctx->valid_fmt = rkvdec_valid_fmt(ctx, ctrl); + if (ctx->valid_fmt) { + struct v4l2_pix_format_mplane *pix_mp; + + pix_mp = &ctx->decoded_fmt.fmt.pix_mp; + pix_mp->pixelformat = ctx->valid_fmt; + rkvdec_fill_decoded_pixfmt(ctx, pix_mp); + } + } + + return 0; +} + static const struct v4l2_ctrl_ops rkvdec_ctrl_ops = { .try_ctrl = rkvdec_try_ctrl, + .s_ctrl = rkvdec_s_ctrl, }; static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = { @@ -188,6 +221,7 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx) { struct v4l2_format *f = &ctx->decoded_fmt; + ctx->valid_fmt = 0; rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]); f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width; @@ -243,13 +277,17 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, if (WARN_ON(!coded_desc)) return -EINVAL; - for (i = 0; i < coded_desc->num_decoded_fmts; i++) { - if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat) - break; - } + if (ctx->valid_fmt) { + pix_mp->pixelformat = ctx->valid_fmt; + } else { + for (i = 0; i < coded_desc->num_decoded_fmts; i++) { + if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat) + break; + } - if (i == coded_desc->num_decoded_fmts) - pix_mp->pixelformat = coded_desc->decoded_fmts[0]; + if (i == coded_desc->num_decoded_fmts) + pix_mp->pixelformat = coded_desc->decoded_fmts[0]; + } /* Always apply the frmsize constraint of the coded end. */ pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); @@ -324,6 +362,7 @@ static int rkvdec_s_capture_fmt(struct file *file, void *priv, return ret; ctx->decoded_fmt = *f; + ctx->valid_fmt = f->fmt.pix_mp.pixelformat; return 0; } @@ -413,6 +452,14 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv, if (WARN_ON(!ctx->coded_fmt_desc)) return -EINVAL; + if (ctx->valid_fmt) { + if (f->index) + return -EINVAL; + + f->pixelformat = ctx->valid_fmt; + return 0; + } + if (f->index >= ctx->coded_fmt_desc->num_decoded_fmts) return -EINVAL; diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2fc9f46b6910..50e67401fdbe 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -64,6 +64,7 @@ vb2_to_rkvdec_decoded_buf(struct vb2_buffer *buf) struct rkvdec_coded_fmt_ops { int (*adjust_fmt)(struct rkvdec_ctx *ctx, struct v4l2_format *f); + u32 (*valid_fmt)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl); int (*start)(struct rkvdec_ctx *ctx); void (*stop)(struct rkvdec_ctx *ctx); int (*run)(struct rkvdec_ctx *ctx); @@ -97,6 +98,7 @@ struct rkvdec_ctx { struct v4l2_fh fh; struct v4l2_format coded_fmt; struct v4l2_format decoded_fmt; + u32 valid_fmt; const struct rkvdec_coded_fmt_desc *coded_fmt_desc; struct v4l2_ctrl_handler ctrl_hdl; struct rkvdec_dev *dev; From patchwork Mon Jul 6 21:54:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646955 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF3F0739 for ; Mon, 6 Jul 2020 21:54:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6D3220715 for ; Mon, 6 Jul 2020 21:54:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="HhG+fJE6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727864AbgGFVyr (ORCPT ); Mon, 6 Jul 2020 17:54:47 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:6357 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726918AbgGFVyj (ORCPT ); Mon, 6 Jul 2020 17:54:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=MLSvldzejjVa6mA9VLILpE7cVCqoIBR9HapVYyZOR0A=; b=HhG+fJE6+169yi9zVjcph7xe0etGhuWflJ+VOPLB8FnwVd8+BPow5Rjb07Y0m2p64WQh wi36rKVnhhaloYOKpMWy/02kOIbxpcwd6v+sXuq+T7a6QO6pRJOCPGGeA28ByK73wirGmU euyTQ7c9Ac+mVw36Lcdl3Nwop6dX8/s0A= Received: by filterdrecv-p3iad2-5b55dcd864-hp5f5 with SMTP id filterdrecv-p3iad2-5b55dcd864-hp5f5-19-5F039D9D-44 2020-07-06 21:54:37.854603941 +0000 UTC m=+881115.898031932 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id Rz21i4k0TPihwDSj8S233A Mon, 06 Jul 2020 21:54:37.607 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 11/12] media: rkvdec: h264: Support High 10 and 4:2:2 profiles Date: Mon, 06 Jul 2020 21:54:37 +0000 (UTC) Message-Id: <20200706215430.22859-12-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h+R/N4SrW2dJiDYTGDah4WIX6jGHzzO2ZlGh8MC6B6jPMIEM1idY6ip1c4tSpok6dKXGLm4Xw3VNEcUURkA+L3RUifV0mgenX8GJ4namVxJwbi7SwFSl3xnMbi+claxe5gGGTi3d+V5XsAhoDpwKd76fJjNisKKR9Lo1BcXiyVea8De36ktqLa5bUSr3J8bnBg== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add support and enable decoding of H264 High 10 and 4:2:2 profiles. Decoded CAPTURE buffer width is aligned to 64 pixels to accommodate HW requirement on 10-bit format buffers. The new valid_fmt operation is implemented and return a valid pixelformat for the provided SPS control. Signed-off-by: Jonas Karlman --- Changes in v2: - Only align decoded buffer instead of using frmsize step_width --- drivers/staging/media/rkvdec/rkvdec-h264.c | 20 ++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 19 +++++++++---------- 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 10756b9d6118..0757fc97d1ff 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -1018,6 +1018,25 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx, return 0; } +static u32 rkvdec_h264_valid_fmt(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p; + + if (sps->bit_depth_luma_minus8 == 0) { + if (sps->chroma_format_idc == 2) + return V4L2_PIX_FMT_NV16; + else + return V4L2_PIX_FMT_NV12; + } else if (sps->bit_depth_luma_minus8 == 2) { + if (sps->chroma_format_idc == 2) + return V4L2_PIX_FMT_NV20; + else + return V4L2_PIX_FMT_NV15; + } + + return 0; +} + static int rkvdec_h264_start(struct rkvdec_ctx *ctx) { struct rkvdec_dev *rkvdec = ctx->dev; @@ -1125,6 +1144,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { .adjust_fmt = rkvdec_h264_adjust_fmt, + .valid_fmt = rkvdec_h264_valid_fmt, .start = rkvdec_h264_start, .stop = rkvdec_h264_stop, .run = rkvdec_h264_run, diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 11a88cb6407d..4faee9262392 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -31,7 +31,7 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, - pix_mp->width, pix_mp->height); + ALIGN(pix_mp->width, 64), pix_mp->height); pix_mp->plane_fmt[0].sizeimage += 128 * DIV_ROUND_UP(pix_mp->width, 16) * DIV_ROUND_UP(pix_mp->height, 16); @@ -55,19 +55,15 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) if (ctrl->id == V4L2_CID_MPEG_VIDEO_H264_SPS) { const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p; unsigned int width, height; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ + + if (sps->chroma_format_idc > 2) + /* Only 4:0:0, 4:2:0 and 4:2:2 are supported */ return -EINVAL; if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) /* Luma and chroma bit depth mismatch */ return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ + if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2) + /* Only 8-bit and 10-bit is supported */ return -EINVAL; if (ctx->valid_fmt && ctx->valid_fmt != rkvdec_valid_fmt(ctx, ctrl)) @@ -157,6 +153,9 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = { static const u32 rkvdec_h264_decoded_fmts[] = { V4L2_PIX_FMT_NV12, + V4L2_PIX_FMT_NV15, + V4L2_PIX_FMT_NV16, + V4L2_PIX_FMT_NV20, }; static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { From patchwork Mon Jul 6 21:54:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonas Karlman X-Patchwork-Id: 11646947 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CAA6D739 for ; Mon, 6 Jul 2020 21:54:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8E022073E for ; Mon, 6 Jul 2020 21:54:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kwiboo.se header.i=@kwiboo.se header.b="S1nVBMGm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727853AbgGFVyp (ORCPT ); Mon, 6 Jul 2020 17:54:45 -0400 Received: from o1.b.az.sendgrid.net ([208.117.55.133]:38039 "EHLO o1.b.az.sendgrid.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727789AbgGFVyj (ORCPT ); Mon, 6 Jul 2020 17:54:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kwiboo.se; h=from:subject:in-reply-to:references:to:cc:content-type: content-transfer-encoding; s=001; bh=xRvXleIsFrOUoVEFb2+EqS1sO8vNP09eztv2NUch3P4=; b=S1nVBMGmkIVtJMfuJDtwUDlmFM36++kjO/Sbbw3SBF8V8MLUQ0Wa2NB0F8t1ZP48hTSJ JisXget+1VHsk17FOWWgMnzDtNWN/XDp5tbZ99jItemPZevfJGfHzlI2LtJbd5FtUnKbPi Gy8qguUJHcFQ6DKTvydNbrTVCU1K60Pos= Received: by filterdrecv-p3iad2-5b55dcd864-5dqzw with SMTP id filterdrecv-p3iad2-5b55dcd864-5dqzw-19-5F039D9E-D 2020-07-06 21:54:38.266225202 +0000 UTC m=+881111.140990039 Received: from bionic.localdomain (unknown) by ismtpd0008p1lon1.sendgrid.net (SG) with ESMTP id RkkRZ2UwQbil5L0N__xREg Mon, 06 Jul 2020 21:54:38.020 +0000 (UTC) From: Jonas Karlman Subject: [PATCH v2 12/12] media: rkvdec: h264: Support profile and level controls Date: Mon, 06 Jul 2020 21:54:38 +0000 (UTC) Message-Id: <20200706215430.22859-13-jonas@kwiboo.se> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706215430.22859-1-jonas@kwiboo.se> References: <20200701215616.30874-1-jonas@kwiboo.se> <20200706215430.22859-1-jonas@kwiboo.se> X-SG-EID: TdbjyGynYnRZWhH+7lKUQJL+ZxmxpowvO2O9SQF5CwCVrYgcwUXgU5DKUU3QxAfZekEeQsTe+RrMu3cja6a0h1vz2//hk378RB7OvuU+Unwr2NFFYBDDmKvAj6aOu7u4GSrColBrbVrhTX5qJmcTnPslZyASlbiJvcXj2p9JfMzqQjp2c5Dwej9bazdn0FDJ3De97GWZ5P2JxHPOyDbQEAv+dtSFg9upo7V2PjhaujJhfQ+LfJPCspTtgQ+wyWhRQGbSEJS+zCDI3TbCnYaHwg== To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jonas Karlman , Ezequiel Garcia , Hans Verkuil , Nicolas Dufresne , Tomasz Figa , Alexandre Courbot Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The Rockchip Video Decoder used in RK3399 supports H.264 profiles from Baseline to High 4:2:2 up to Level 5.1, except for the Extended profile. Expose the V4L2_CID_MPEG_VIDEO_H264_PROFILE and the V4L2_CID_MPEG_VIDEO_H264_LEVEL control, so that userspace can query the driver for the list of supported profiles and level. Signed-off-by: Jonas Karlman Reviewed-by: Ezequiel Garcia --- Changes in v2: - Moved to end - Collect r-b tag --- drivers/staging/media/rkvdec/rkvdec.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 4faee9262392..b21031535330 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -144,6 +144,19 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = { .cfg.def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .cfg.max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + { + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422, + .cfg.menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .cfg.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + }, + { + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_LEVEL, + .cfg.min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0, + .cfg.max = V4L2_MPEG_VIDEO_H264_LEVEL_5_1, + }, }; static const struct rkvdec_ctrls rkvdec_h264_ctrls = {