From patchwork Mon Mar 28 19:59:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F189CC433EF for ; Mon, 28 Mar 2022 19:59:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343985AbiC1UBe (ORCPT ); Mon, 28 Mar 2022 16:01:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343970AbiC1UBe (ORCPT ); Mon, 28 Mar 2022 16:01:34 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54CE439166; Mon, 28 Mar 2022 12:59:52 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id D9B6F1F4384C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497591; bh=juEkTDE+SY0qwuOHnUrLzTo/uQ/rPpurP5PmYNSVfck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iDK0M7FyPd5O9ZPH6cann5jLnLLjQxkwxZS6CxveFOrYR2TgBiSEl3kWREbifBLE5 HZRuUeEO9kEeU/TrdUJ9PrBWxVgDCny/0Y81gl86ehzEsOcPhiqoSaXJripdv6E71x 0/25h8I8CAdgcpPLWjeE19aES5+LvahKaZ5cD1qVXJg+Zx8TeFAdwyMEPjV/k3svr+ lcehnmWXJt8Ck9Bmp0H1zf/LUbVDh/KbR2MIF9h2fOQBKaxcPWGqsLYOMxy1jFN4mH 7U+Vs2qTQY+N/5Stg8DeyjEX6RuMi0RR/lqfzNFIq5yT+qFvI0KHGVelygcwbOopyu SSNsN4gBJVs5Q== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v1 01/24] media: h264: Increase reference lists size to 32 Date: Mon, 28 Mar 2022 15:59:13 -0400 Message-Id: <20220328195936.82552-2-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is to accommodate support for field decoding, which splits the top and the bottom reference into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 0618c6f52214..8d750ee69e74 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -210,7 +210,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -233,9 +233,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2bc6b8f088f5..292aaaabaf24 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Mon Mar 28 19:59:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7001DC4332F for ; Mon, 28 Mar 2022 19:59:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344001AbiC1UBf (ORCPT ); Mon, 28 Mar 2022 16:01:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343979AbiC1UBe (ORCPT ); Mon, 28 Mar 2022 16:01:34 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79427393D4; Mon, 28 Mar 2022 12:59:53 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id AA34B1F43850 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497592; bh=Wt3ayzGsMmxeOdSu0lSUaLnN2BhIuPB/Wbnfn/jKVUM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DcQ+A85Uofv5sKjmZcd8r3GNXf8kJWW9PmI2jPYDVediCtxMop1xKNQx3h6tko46+ oRoeTTQnq9vKhLlyxErUsuhz4Cou+Ze4fV9pcfU+yTzFsHI4rcZ8UgxT4s1YDn4lA4 jAbO+TeggaVC02YTFXR3D8hRANdosjKEd91v86cmya+ss3yxYeEkeJftPpZc5g2JXK ZxVLdUJaq78GdJQc+/BGFUx1ErE1FO5htmvgQAOJoqjZGogkRZFx3h05+ZyvQoBjzN qeaxVwDifn5SufOPKSwiIO3Vx+K392YfnvE93Rz8QU3WyhNQX/xbNhkUinRLL+85c1 jer60pS+HQhgQ== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 02/24] media: doc: Document dual use of H.264 pic_num/frame_num Date: Mon, 28 Mar 2022 15:59:14 -0400 Message-Id: <20220328195936.82552-3-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org These two fields needs documentation as they have dual meaning. It is also confusing since pic_num is a derived value from frame_num, so this should help application developpers. If we ever need to make a V2 of this API, I would suggest to remove pic_num entirely. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- .../media/v4l/ext-ctrls-codec-stateless.rst | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst index 6541e4c32b26..f634f20bcfbe 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst @@ -649,10 +649,16 @@ Stateless Codec Control ID :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64. * - __u32 - ``pic_num`` - - + - For short term reference, this should match the derived value PicNum + (8-28) and for long term references it should match the derived value + LongTermPicNum (8-29). Note that pic_num is the same as FrameNumWrap + for frame decoding. * - __u16 - ``frame_num`` - - + - For short term references, this should match the frame_num value from + the slice header syntax (the driver will wrap the value if neeeded). For + long term references, this should be set to the value of + long_term_frame_idx describes in the dec_ref_pic_marking() syntax. * - __u8 - ``fields`` - Specifies how the DPB entry is referenced. See :ref:`Reference Fields ` From patchwork Mon Mar 28 19:59:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEABEC433EF for ; Mon, 28 Mar 2022 19:59:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235927AbiC1UBh (ORCPT ); Mon, 28 Mar 2022 16:01:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344003AbiC1UBf (ORCPT ); Mon, 28 Mar 2022 16:01:35 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F24163969A; Mon, 28 Mar 2022 12:59:54 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id D7E931F43853 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497593; bh=koeYZVYa1e9V9VAWFVOsniYBHyi6nWtDcIkC0przTEE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n4LqLgpBupA7Vw6AD5z/wVRO3g7+he2wEA288iD6Ho4mfTQVWnM+FTZpAmFZPZDaH oj4IlQXm7SvqKU/H10sqMz9poEwuzZUeiG8gEE5WzvHIvwCs7WZRCcgSfye+qzqCfR EoHcip48VZ/+lhfc0OdQ+tsFwhTfX/RxMezGwkbxBdD6s2MPplK0ASSQ54/PjzgfCw avIr1tSLDsp4s9vUcNp4o92zyvncm2ExFJAxNiqgzqwdLtEvnEJO9t1et5oL61egqu r9mDlWWDSqwT3dXjH0VK0+X7DkXmvrAh6x4UzBbdnW5aC27R57F6deeFO515CWc3xf Kh5Hc7D/2cwPQ== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 03/24] media: h264: Avoid wrapping long_term_frame_idx Date: Mon, 28 Mar 2022 15:59:15 -0400 Message-Id: <20220328195936.82552-4-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org For long term reference, frame_num is set to long_term_frame_idx which does not require wrapping. This if fixed by observation, no directly related issue have been found yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 8d750ee69e74..aebed1cbe05a 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -57,8 +57,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * '8.2.4.1 Decoding process for picture numbers' of the spec. * TODO: This logic will have to be adjusted when we start * supporting interlaced content. + * For long term reference, frame_num is set to + * long_term_frame_idx which requires no wrapping. */ - if (dpb[i].frame_num > cur_frame_num) + if (!b->refs[i].longterm && dpb[i].frame_num > cur_frame_num) b->refs[i].frame_num = (int)dpb[i].frame_num - max_frame_num; else From patchwork Mon Mar 28 19:59:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D88DC433F5 for ; Mon, 28 Mar 2022 20:00:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344158AbiC1UBr (ORCPT ); Mon, 28 Mar 2022 16:01:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344057AbiC1UBh (ORCPT ); Mon, 28 Mar 2022 16:01:37 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F31543969A; Mon, 28 Mar 2022 12:59:55 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 3554D1F43859 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497594; bh=WOqr7O+kEBWDQ82IgnDtk4f1Hxn2t1rG144dWJ0+7Co=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cz72vvSxzdruPPJ8pEtYfNdPVPTUte/ufE9GzUc+AXcriDMH0TI+o/84Ii9w3hPOl gt5u+JSyJsd5Z6aG/Eg0JdSsgz8FdcmFD2WaRi8S8omaLlUeSOMddEbng9F8UMZO5P kvgM4lkscZatvCDNoFSCPe51n0uAYafRMkkmdLjhhjcXD109gBa3DfnjCo+TYmW1/D lqPeoP70MbWqAkwNBLA5819DUrfC+rNWuF0K0rMkmArcnc/Syt+XsZ+m0Ek/Fd4DSg utYttlwNVvg+CK7t4PS2goLlFmz/+980G61TyAO9xPQZi/Ovsdrq/NyQotDj9zbxpT xkAHFQdYQokOg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 04/24] media: h264: Store current picture fields Date: Mon, 28 Mar 2022 15:59:16 -0400 Message-Id: <20220328195936.82552-5-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This information, also called picture structure will be needed to construct reference list when decoding field. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 10 +++++++--- include/media/v4l2-h264.h | 4 ++++ 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index aebed1cbe05a..4c6bfb057bda 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -34,13 +34,17 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, cur_frame_num = dec_params->frame_num; memset(b, 0, sizeof(*b)); - if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) + if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) { b->cur_pic_order_count = min(dec_params->bottom_field_order_cnt, dec_params->top_field_order_cnt); - else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) + b->cur_pic_fields = V4L2_H264_FRAME_REF; + } else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) { b->cur_pic_order_count = dec_params->bottom_field_order_cnt; - else + b->cur_pic_fields = V4L2_H264_BOTTOM_FIELD_REF; + } else { b->cur_pic_order_count = dec_params->top_field_order_cnt; + b->cur_pic_fields = V4L2_H264_TOP_FIELD_REF; + } for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { u32 pic_order_count; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index e282fb16ac58..e165a54c68fa 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -21,6 +21,7 @@ * @refs.longterm: set to true for a long term reference * @refs: array of references * @cur_pic_order_count: picture order count of the frame being decoded + * @cur_pic_fields: fields present in the frame being decoded * @unordered_reflist: unordered list of references. Will be used to generate * ordered P/B0/B1 lists * @num_valid: number of valid references in the refs array @@ -36,7 +37,10 @@ struct v4l2_h264_reflist_builder { u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; + s32 cur_pic_order_count; + u8 cur_pic_fields; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; From patchwork Mon Mar 28 19:59:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A9E3C433EF for ; Mon, 28 Mar 2022 20:00:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344191AbiC1UBo (ORCPT ); Mon, 28 Mar 2022 16:01:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344093AbiC1UBi (ORCPT ); Mon, 28 Mar 2022 16:01:38 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4711939BA0; Mon, 28 Mar 2022 12:59:57 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 837671F43850 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497596; bh=d2iOirXh/xNpYRw3u+9Ch8gJt9EKy5UNHmbX/3uP0rM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GiCybdep6uR+UxOBoW/R7m3rKtcVLxeWXr53R8PMdDrGyaSGT2JsVNw5vzzbTqmbx cjxT3gwE5ErWxroUdy8C3ActoeLGDS/UVG27g3kXgnnNmOc6bIkLDXc1S8g6AvNWCk nNX2PorvE/1TAd6QmqSNUsROMIH8L0CUKRUdoxc4uURp7CBJwEMGSOhZeBKVdnVVWC EbUsaVgBH9Ow9wKWLdYt4ihHPkfh+n962FapAEVxaHnxaRS/LFXLYPbcFSYW6ijOxZ SE13vVQtsu02WyQRqMm28kKOdwR6FqqI+BFeGRr0FLDmml/jhQ15Xx5heuZY0+MrM1 lHkhdobb72xYA== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 05/24] media: h264: Store all fields into the unordered list Date: Mon, 28 Mar 2022 15:59:17 -0400 Message-Id: <20220328195936.82552-6-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When the current picture is a field, store each field into the unordered_list. Doing this required preserving the top and bottom field poc. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 65 +++++++++++++++++++++-------- include/media/v4l2-h264.h | 6 ++- 2 files changed, 51 insertions(+), 20 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 4c6bfb057bda..d5698c981973 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -47,8 +47,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, } for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { - u32 pic_order_count; - if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; @@ -59,8 +57,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, /* * Handle frame_num wraparound as described in section * '8.2.4.1 Decoding process for picture numbers' of the spec. - * TODO: This logic will have to be adjusted when we start - * supporting interlaced content. * For long term reference, frame_num is set to * long_term_frame_idx which requires no wrapping. */ @@ -70,17 +66,33 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, else b->refs[i].frame_num = dpb[i].frame_num; - if (dpb[i].fields == V4L2_H264_FRAME_REF) - pic_order_count = min(dpb[i].top_field_order_cnt, - dpb[i].bottom_field_order_cnt); - else if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) - pic_order_count = dpb[i].bottom_field_order_cnt; - else - pic_order_count = dpb[i].top_field_order_cnt; + b->refs[i].top_field_order_cnt = dpb[i].top_field_order_cnt; + b->refs[i].bottom_field_order_cnt = dpb[i].bottom_field_order_cnt; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + u8 fields = V4L2_H264_FRAME_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + continue; + } + + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) { + u8 fields = V4L2_H264_TOP_FIELD_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + } - b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid].index = i; - b->num_valid++; + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) { + u8 fields = V4L2_H264_BOTTOM_FIELD_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + } } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) @@ -88,6 +100,23 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); +static s32 v4l2_h264_get_poc(const struct v4l2_h264_reflist_builder *b, + const struct v4l2_h264_reference *ref) +{ + switch (ref->fields) { + case V4L2_H264_FRAME_REF: + return min(b->refs[ref->index].top_field_order_cnt, + b->refs[ref->index].bottom_field_order_cnt); + case V4L2_H264_TOP_FIELD_REF: + return b->refs[ref->index].top_field_order_cnt; + case V4L2_H264_BOTTOM_FIELD_REF: + return b->refs[ref->index].bottom_field_order_cnt; + } + + /* not reached */ + return 0; +} + static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const void *data) { @@ -150,8 +179,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, builder->refs[idxb].pic_num ? -1 : 1; - poca = builder->refs[idxa].pic_order_count; - pocb = builder->refs[idxb].pic_order_count; + poca = v4l2_h264_get_poc(builder, ptra); + pocb = v4l2_h264_get_poc(builder, ptrb); /* * Short term pics with POC < cur POC first in POC descending order @@ -195,8 +224,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, builder->refs[idxb].pic_num ? -1 : 1; - poca = builder->refs[idxa].pic_order_count; - pocb = builder->refs[idxb].pic_order_count; + poca = v4l2_h264_get_poc(builder, ptra); + pocb = v4l2_h264_get_poc(builder, ptrb); /* * Short term pics with POC > cur POC first in POC ascending order diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index e165a54c68fa..4cef717b3f18 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -15,7 +15,8 @@ /** * struct v4l2_h264_reflist_builder - Reference list builder object * - * @refs.pic_order_count: reference picture order count + * @refs.top_field_order_cnt: top field order count + * @refs.bottom_field_order_cnt: bottom field order count * @refs.frame_num: reference frame number * @refs.pic_num: reference picture number * @refs.longterm: set to true for a long term reference @@ -32,7 +33,8 @@ */ struct v4l2_h264_reflist_builder { struct { - s32 pic_order_count; + s32 top_field_order_cnt; + s32 bottom_field_order_cnt; int frame_num; u32 pic_num; u16 longterm : 1; From patchwork Mon Mar 28 19:59:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794178 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53881C433EF for ; Mon, 28 Mar 2022 20:00:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344194AbiC1UBt (ORCPT ); Mon, 28 Mar 2022 16:01:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344121AbiC1UBj (ORCPT ); Mon, 28 Mar 2022 16:01:39 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F2913A182; Mon, 28 Mar 2022 12:59:58 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C0B8A1F4385C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497597; bh=ApaE5zfnpwhITRSMiCOdS9C9/LuBBqBopAyX2XsBfWg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PSXKoDQJZDTTaRsrGv649xU/szo8tanzvCPuSi+xps6sTEKXhd4yKDZvwuAUl2kQA oHRGgk71YG7EjQRVeIOxmINFWWIxUMt0bkw45uFawLVYMFbrDG8yOXDyW3sKVv5r38 7ZIXBC6HPYUxhNwyjU9zDN8A7ApQAntOyiRpW+TbEES5cjzvMszTBpJPYC92/idt8A IpD/qKn9BCU4huQy0pu6ipxHD136ERDnG4f49XlevIat5voijIb9mMz2DjBYDmcU94 Km/6fNpV2LZAdbbgcCXOwcFkNbLGkrjsC1Pfhloq13CeiuuJSXUz8IjJCjslqbQ/7U sTp/x5ippYLgA== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 06/24] media: v4l2: Trace calculated p/b0/b1 initial reflist Date: Mon, 28 Mar 2022 15:59:18 -0400 Message-Id: <20220328195936.82552-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is crucial in verifying that the ordering is right, specially with more complex sorting needed for field decoding. Signed-off-by: Nicolas Dufresne Tested-by: Sebastian Fricke Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index d5698c981973..c9e18fd51d78 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +static char ref_type_to_char (u8 ref_type) +{ + switch (ref_type) { + case V4L2_H264_FRAME_REF: + return 'f'; + case V4L2_H264_TOP_FIELD_REF: + return 't'; + case V4L2_H264_BOTTOM_FIELD_REF: + return 'b'; + } + + return '?'; +} + +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char *out_str, const int len) +{ + int n = 0, i; + + n += snprintf(out_str + n, len - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + /* this is pic_num for frame and frame_num (wrapped) for field, + * but for frame pic_num is equal to frame_num (wrapped). + */ + int frame_num = builder->refs[reflist[i].index].frame_num; + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(out_str + n, len - n, "%i%c%c|", + frame_num, longterm ? 'l' : 's', + ref_type_to_char (reflist[i].fields)); + } + + return out_str; +} + +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist) +{ + char buf[1024]; + + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n", + builder->cur_pic_order_count, + ref_type_to_char(builder->cur_pic_fields), + format_ref_list_p(builder, reflist, buf, sizeof(buf))); +} + +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char *out_str, const int len) +{ + int n = 0, i; + + n += snprintf(out_str + n, len - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + int frame_num = builder->refs[reflist[i].index].frame_num; + u32 poc = v4l2_h264_get_poc(builder, reflist + i); + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(out_str + n, len - n, "%i%c%c|", + longterm ? frame_num : poc, + longterm ? 'l' : 's', + ref_type_to_char(reflist[i].fields)); + } + + return out_str; +} + +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, u8 list_num) +{ + char buf[1024]; + + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s", + list_num, builder->cur_pic_order_count, + ref_type_to_char (builder->cur_pic_fields), + format_ref_list_b(builder, reflist, buf, sizeof(buf))); +} + /** * v4l2_h264_build_p_ref_list() - Build the P reference list * @@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sizeof(builder->unordered_reflist[0]) * builder->num_valid); sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); + + print_ref_list_b(builder, b0_reflist, 0); + print_ref_list_b(builder, b1_reflist, 1); } EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists); From patchwork Mon Mar 28 19:59:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F82AC433FE for ; Mon, 28 Mar 2022 20:00:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344202AbiC1UBu (ORCPT ); Mon, 28 Mar 2022 16:01:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344160AbiC1UBn (ORCPT ); Mon, 28 Mar 2022 16:01:43 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D57DE3A5CF; Mon, 28 Mar 2022 12:59:59 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1D4421F4385A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497598; bh=cei8NQmJ3TQDsmsIXeEFyhyT52qbmE8s5HKgYlTiliQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KYfn2tqpyXuUTYu+iBNMPH3apa1ZTg6CQYlSGTN5gC840rvAg6XvtPcNTypmcsTxE NQeW4/eTrrvoKkBGI/hg/Wbe41Ucv7xsss7v0chUyvj8DMC77XSY+Lb/AKE9sfklXG 4BfeR+e7YA0l6059BP8Agpps6MHiqlhReR5HMPYbsE9kfpLSGAU/GBX3WetSdMTqhL zcrnXghE9nYXwW1OYorY2HUTBSy8SGDdinNmqzBj6vcrToIMehSmiJMHJVlx9HZnOd L4zDm7TGImyyNmI6zZ6cFp3hZNRMsQJjPB2EpFuPzDNe3StMTyXRFRbYTjjPbOlRmx kUxtCxRlCj+9g== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 07/24] media: h264: Sort p/b reflist using frame_num Date: Mon, 28 Mar 2022 15:59:19 -0400 Message-Id: <20220328195936.82552-8-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In the reference list builder, frame_num referers to FrameNumWrap in the spec, which is the same as the pic_num for frame decoding. The same applied for long_tern_pic_num and long_ter_frame_idx. Sort all type of references by frame_num so the sort can be reused for field reflist were the sorting is done using frame_num instead. In short, pic_num is never actually used for building reference lists. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 22 ++++++++++++---------- include/media/v4l2-h264.h | 2 -- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index c9e18fd51d78..c3fad382882d 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -50,7 +50,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - b->refs[i].pic_num = dpb[i].pic_num; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) b->refs[i].longterm = true; @@ -139,15 +138,18 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, } /* - * Short term pics in descending pic num order, long term ones in - * ascending order. + * For frames, short term pics are in descending pic num order and long + * term ones in ascending order. For field this is the same direction + * but with frame_num (wrapped). As for frame the pic_num and frame_num + * are the same (see formula (8-28) and (8-29)) use the frame_num and share + * this funciton between frame and field reflist. */ if (!builder->refs[idxa].longterm) return builder->refs[idxb].frame_num < builder->refs[idxa].frame_num ? -1 : 1; - return builder->refs[idxa].pic_num < builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < builder->refs[idxb].frame_num ? -1 : 1; } @@ -173,10 +175,10 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, return 1; } - /* Long term pics in ascending pic num order. */ + /* Long term pics in ascending frame num order. */ if (builder->refs[idxa].longterm) - return builder->refs[idxa].pic_num < - builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < + builder->refs[idxb].frame_num ? -1 : 1; poca = v4l2_h264_get_poc(builder, ptra); @@ -218,10 +220,10 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return 1; } - /* Long term pics in ascending pic num order. */ + /* Long term pics in ascending frame num order. */ if (builder->refs[idxa].longterm) - return builder->refs[idxa].pic_num < - builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < + builder->refs[idxb].frame_num ? -1 : 1; poca = v4l2_h264_get_poc(builder, ptra); diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4cef717b3f18..0d9eaa956123 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -18,7 +18,6 @@ * @refs.top_field_order_cnt: top field order count * @refs.bottom_field_order_cnt: bottom field order count * @refs.frame_num: reference frame number - * @refs.pic_num: reference picture number * @refs.longterm: set to true for a long term reference * @refs: array of references * @cur_pic_order_count: picture order count of the frame being decoded @@ -36,7 +35,6 @@ struct v4l2_h264_reflist_builder { s32 top_field_order_cnt; s32 bottom_field_order_cnt; int frame_num; - u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; From patchwork Mon Mar 28 19:59:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F219C433FE for ; Mon, 28 Mar 2022 20:00:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344183AbiC1UB4 (ORCPT ); Mon, 28 Mar 2022 16:01:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344131AbiC1UBn (ORCPT ); Mon, 28 Mar 2022 16:01:43 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A036D3A730; Mon, 28 Mar 2022 13:00:01 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 4EFD31F4386D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497600; bh=vByQV7rCpRfvNGtGYzI2vxESufjfaYFSOaqSafHc0Zo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AD3UX9iFdiyFU1ERpNORrTzFj7a34p+IuYkWRoz3FspWwoYs2DJFs5h8TYtLeCWrA 156jZRkmHbeiPv4tTDwgOSlpaNM9OYSQnahvWiidfWyncGtPrapG/uBhO/lr6/JofG wp99iuELgfA+prTZQsQ5aJtzIe7MoMkC8XuUkc+ig0v+UY4YSp7H09ZRvA+DqdvHG4 VuNJuLL25t2S2wSHYEmSazuRRyz0NYGS5T4fkm1V7Sz4S1jn23mSg9Yt7l/GEecuzL jN2yoKyFbE12FNITrbV5A9Zvl30RVGInPXI5ECGqG3EavwYtiYh2kHZua7lo8FCQTT Aw3UXC/4mhaOg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 08/24] media: v4l2: Reorder field reflist Date: Mon, 28 Mar 2022 15:59:20 -0400 Message-Id: <20220328195936.82552-9-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As per spec, the field refslist requires interleaving top and bottom field in a specific way that does not fit inside the sort operation. Reorder in-place the references so that their parity sart with the same parity as the current picture and do that for both short and longterm references separately. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 45 +++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index c3fad382882d..2f7ee8d5479f 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -243,6 +243,43 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +/* + * The references need to be reorder so that reference are alternating between + * top and bottom ref starting with the current picture parity. This have to be + * done for short term and long term references separately. + */ +static void reorder_field_reflist(const struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + struct v4l2_h264_reference tmplist[V4L2_H264_REF_LIST_LEN]; + u8 lt, i = 0, j = 0, k = 0; + + memcpy(tmplist, reflist, sizeof(tmplist[0]) * b->num_valid); + + for (lt = 0; lt <= 1; lt++) { + do { + for (; i < b->num_valid && b->refs[tmplist[i].index].longterm == lt; i++) { + if (tmplist[i].fields == b->cur_pic_fields) { + reflist[k] = tmplist[i]; + k++; + i++; + break; + } + } + + for (; j < b->num_valid && b->refs[tmplist[j].index].longterm == lt; j++) { + if (tmplist[j].fields != b->cur_pic_fields) { + reflist[k] = tmplist[j]; + k++; + j++; + break; + } + } + } while ((i < b->num_valid && b->refs[tmplist[i].index].longterm == lt) || + (j < b->num_valid && b->refs[tmplist[j].index].longterm == lt)); + } +} + static char ref_type_to_char (u8 ref_type) { switch (ref_type) { @@ -345,6 +382,9 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) + reorder_field_reflist(builder, reflist); + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -378,6 +418,11 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, sort_r(b1_reflist, builder->num_valid, sizeof(*b1_reflist), v4l2_h264_b1_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) { + reorder_field_reflist(builder, b0_reflist); + reorder_field_reflist(builder, b1_reflist); + } + if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); From patchwork Mon Mar 28 19:59:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8C5CC433F5 for ; Mon, 28 Mar 2022 20:00:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344229AbiC1UB5 (ORCPT ); Mon, 28 Mar 2022 16:01:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344187AbiC1UBn (ORCPT ); Mon, 28 Mar 2022 16:01:43 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 639003AA61; Mon, 28 Mar 2022 13:00:02 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 806C31F4385C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497601; bh=+jEmz225xAmiHcP4mdC1OAJ4YlOO5CterzYv8eeAA5E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cyMq7dIKVPjnU9VEcGNat7wGGQwWR70hwnpinaQEVDKQ7OchlkgRT/CGTxaCw4T22 cgBTQ0KZNS+Xni2qoiQV9aVafUgx/tPn9sM/X955kAokyAbl9NAYYq5GkBDWhSvm3p Efl+35d4w2rypramDPJlxs4fdmFqNRDCqEBsX5siFN7p210uMleMBw6swP46jS5qRw YSLqKmS4JtkWf5/9HVQfU3/16TUFgY9sevpMuZVBexeUva7PcARnb3RXzBo7kJs1CB GfO66jRmf3HnW1Fna7VnUxlUvFcBNMprMrbspBWJ3Q5q87wOsNnkF+3huQcKwFc00f TNCAilpx27h2g== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 09/24] media: v4l2-mem2mem: Fix typo in trace message Date: Mon, 28 Mar 2022 15:59:21 -0400 Message-Id: <20220328195936.82552-10-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org On -> One Signed-off-by: Nicolas Dufresne --- drivers/media/v4l2-core/v4l2-mem2mem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 675e22895ebe..53c2332d5cbd 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -316,7 +316,7 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, } if (m2m_ctx->job_flags & TRANS_QUEUED) { - dprintk("On job queue already\n"); + dprintk("One job queue already\n"); goto job_unlock; } From patchwork Mon Mar 28 19:59:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 604C7C433EF for ; Mon, 28 Mar 2022 20:00:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344250AbiC1UCB (ORCPT ); Mon, 28 Mar 2022 16:02:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344213AbiC1UBz (ORCPT ); Mon, 28 Mar 2022 16:01:55 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AACA33A1BE; Mon, 28 Mar 2022 13:00:03 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id BF52A1F43867 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497602; bh=HlQF5QYc2YwGYm21E6HNcjkx9wicYLtIvyiKegTR5UM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X2V9/AoW0YdfJtrSr0L83RDfv81sqKRWMuoCG6uFzKhVuId0g7jU0+lCWSjHFtvDt 3wzVTlRR05iMdPpkcowX+LYBbNam7WzQ+LjM+8QGAw3nbqf7wKLL0e07ceJvgMJf8Q eUc16uV2Y1opzL3MKU8z58oj3mCfhCEim9J5lGrNfg/RMhRCqtGj/VNrk/HoAaSjd4 ubhJ7Em0sfWafUoDLriWEgwNCCu8FDfDe8huEPeMlqjkMZoJQDHtxdXJICTgNGcHx+ LsUrjfRvKShqcJhGpE4QJwQg2uUzK/0wKDS0MOB5vJuXDR82STUSF0egw3UDQfcdo0 u5NxEVnZZY4DQ== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 10/24] media: v4l2-mem2mem: Trace on implicit un-hold Date: Mon, 28 Mar 2022 15:59:22 -0400 Message-Id: <20220328195936.82552-11-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org If the timestamp of the src buffer differs from the timestamp of a held dst buffer, the held buffer is implicitly removed and marked as done. Add a trace to help debugging if someone hits that case. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-mem2mem.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 53c2332d5cbd..d3e739ef16ef 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -336,6 +336,7 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, if (src && dst && dst->is_held && dst->vb2_buf.copied_timestamp && dst->vb2_buf.timestamp != src->vb2_buf.timestamp) { + dprintk("src and dst timestamp mismatch, removing held capture buffer.\n"); dst->is_held = false; v4l2_m2m_dst_buf_remove(m2m_ctx); v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE); From patchwork Mon Mar 28 19:59:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA614C433FE for ; Mon, 28 Mar 2022 20:00:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344280AbiC1UCD (ORCPT ); Mon, 28 Mar 2022 16:02:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344222AbiC1UBz (ORCPT ); Mon, 28 Mar 2022 16:01:55 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2815C3B029; Mon, 28 Mar 2022 13:00:05 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 0B5E21F43877 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497604; bh=RrVHdlTOtkGYblUxkaOs8WkHp5+WDfAbFJ5ZboXk3tI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mCPzYBRYAl61tvf8GTMcmBxGdN6e6GKkQ7iLBhSYfMXSI5wCKOV47aKWEpmb2TCa7 iTwLoVmFRuhRbO20PyOhJeRXqBoOXDjy1NWNCHFKU/73ZWuKz/ELpD0uiog6VHiPEs Wmba1irqOZQhM0kZAD5kQ3IM+FYJUCsfNaYv3qvJyWSiy3xch2G1b662SIXgBJ23fE ENOhCyphrONOD6T2KV4xHN7vvqVHSuX7zeUsexrBfpn+8r99ou9OvyYXZNMbFJgmo1 UxOgOIgWff/8HlBWQmPEfjO2uUUy6cu5HRQ9goF9FwQr/JTsAOfvv29iciN0smi4c0 9ThypDvEYnlIA== From: Nicolas Dufresne To: Tomasz Figa , Marek Szyprowski , Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 11/24] media: videobuf2-v4l2: Warn on holding buffers without support Date: Mon, 28 Mar 2022 15:59:23 -0400 Message-Id: <20220328195936.82552-12-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Sebastian Fricke Using V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF flag without specifying the subsystem flag VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, results in silently ignoring it. Warn the user via a debug print when the flag is requested but ignored by the videobuf2 framework. Signed-off-by: Sebastian Fricke Signed-off-by: Nicolas Dufresne --- drivers/media/common/videobuf2/videobuf2-v4l2.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index 6edf4508c636..812c8d1962e0 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -329,8 +329,13 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b */ vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE; vbuf->field = b->field; - if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) + if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) { + if (vbuf->flags & V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF) + dprintk(q, 1, + "Request holding buffer (%d), unsupported on output queue\n", + b->index); vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; + } } else { /* Zero any output buffer flags as this is a capture buffer */ vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS; From patchwork Mon Mar 28 19:59:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FF33C4332F for ; Mon, 28 Mar 2022 20:00:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344217AbiC1UCF (ORCPT ); Mon, 28 Mar 2022 16:02:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344234AbiC1UB6 (ORCPT ); Mon, 28 Mar 2022 16:01:58 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD3E13B3E3; Mon, 28 Mar 2022 13:00:06 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 8DFC11F43871 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497605; bh=m44B+xryaDUBDC4OQZH1svm4TvbuzUk06rZQFsjsWkM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dd0806z1QOOqDJRf/6nckgUYaYCCCq1RZvmymLye4MrEVbi/GLzoVD/98K8xpdfpd OOaBuahZWQqWJFxiGJsE1E24+2+QPUgfsWbegMOGbgE7oQ3IodaVuMENc6XqD7whtI RWTqn0EinA3/ORAu0d9Ov8Zkr7eM7+HZMgfvPrr3smH7Dv5rRtNpY34ypTVM6zRLVa LnoF9n9Xb8b903lw7Fj2I4G7gRqV90tOvH0uOQUKTxWWE7ZVUvzupw5sUOtKHjQCEe DrwhIK95JOVt9WxEySFw9DmEwC4A0tZVSH/8ay9040P/sk2PbBgVaXYnJZ0pesC1m/ +YcQ6pe5ShX9A== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 12/24] media: rkvdec: Stop overclocking the decoder Date: Mon, 28 Mar 2022 15:59:24 -0400 Message-Id: <20220328195936.82552-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org While this overclock hack seems to works on some implementation (some ChromeBooks, RockPi4) it is also causing instability on other implementation (notably LibreComputer Renegade, but saw more reports in the LibreELEC project, were this is already removed). While performance is indeed affectied (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed synchronously decode each frames, so this hack was simply compensating for their code being less efficienti. This hack should in my opinion have stayed downstream and I'm removing it to ensure stability across all RK3399 variants. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Mon Mar 28 19:59:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B39EC433EF for ; Mon, 28 Mar 2022 20:00:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344318AbiC1UCV (ORCPT ); Mon, 28 Mar 2022 16:02:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344252AbiC1UCB (ORCPT ); Mon, 28 Mar 2022 16:02:01 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA2133BA7C; Mon, 28 Mar 2022 13:00:08 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 359211F43874 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497607; bh=Pb+CkJFTFCox4aOeKaVWm1HNnI9nYoASnBQM3lzgOEY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nTUduKXw1HouhCvW/Gy+aJmHH6nF/jyB0cwwEPV6jAWA35xHLQXGqidD26RIhIOlw JZMquk0QFevH4YHJ+f8Vv5I6rdcV3SBm9EDQyplYLbC/FXI5WqFKeDgSpVdzMtcw8V 2g8gTxRcZuc9LpQbg8E3vYKY/OMQEhgdwBDUo0fNQkMLi9w41b3jmvIxwvb05jcPt0 UtiOLD3nauFSK6vxXNMHvoVYWcMC1tREK+TQXFIGD8SYeIXvhNpU6yV6SlUDGR6+ro PijOsp8nb4HovgDIq9hnkG7AotFwRc5qqSNcvhCWjmw8ISiWJinaFJxMqVAvM9Rp60 PnQtAyaXK9CKQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Ezequiel Garcia , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 13/24] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Mon, 28 Mar 2022 15:59:25 -0400 Message-Id: <20220328195936.82552-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this cause issues to decode the second field in a complementary field pair. Fix this by using inclusive comparison, less than or equal. Signed-off-by: Jonas Karlman Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index dff89732ddd0..842d8cd80e90 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -752,7 +752,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Mon Mar 28 19:59:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F3FC433EF for ; Mon, 28 Mar 2022 20:00:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344260AbiC1UCY (ORCPT ); Mon, 28 Mar 2022 16:02:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344262AbiC1UCC (ORCPT ); Mon, 28 Mar 2022 16:02:02 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EC7B3BFBF; Mon, 28 Mar 2022 13:00:10 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 26A8D1F43887 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497609; bh=Ddp9raQj8UeMQfeTi2igopokBtztJ648lvZgYhxyu0w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nmQHtGgp4cTLxB5/7cfkka6D2p/R8j+k0fG5AE62PwiZjhbYMp+XBvVXYf6MAMBEF gtpf/XPG0eQ/TtjkZL9Ws602REUe23MVKQFmYJQSxE9ePgeVbWbPZwSxBjhmsTccES c7o+P2K5q4AemDt88R8TIu+24NG68zJHoZ79An5i+i5bSTtKKBCa0jiaOtK9Heh881 nEfYfD26TdGGbbmfNgAweFFT7Cn0x4w6/h5hX5KrcScaXoi5f0oA9n1JlFpEMQqGA0 FsXccF9DGL8xmsN7T43zZyGFix0+NnOZuAqXyn/tF1jKwoegVlHYVTn+8ry+cZREsP AfOu/ePtN/hlg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 14/24] media: rkvdec: h264: Fix dpb_valid implementation Date: Mon, 28 Mar 2022 15:59:26 -0400 Message-Id: <20220328195936.82552-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ref builder only provided reference that are marked as valid in the dpb. Thus the current implementation of dpb_valid would always set the flag to 1. This is not representing missing frames (this is called 'non-existing' pictures in the spec). In some context, these non-existing pictures still need occupy a slot in the reference list according to the spec. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 842d8cd80e90..db1e762baee5 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -112,6 +112,7 @@ struct rkvdec_h264_run { const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES]; }; struct rkvdec_h264_ctx { @@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, } } +static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, + struct rkvdec_h264_run *run) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) { + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; + struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; + int buf_idx = -1; + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + buf_idx = vb2_find_timestamp(cap_q, + dpb[i].reference_ts, 0); + + run->ref_buf_idx[i] = buf_idx; + } +} + static void assemble_hw_rps(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { @@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = 0; + u8 dpb_valid = run->ref_buf_idx[i] >= 0; u8 idx = 0; switch (j) { @@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (idx >= ARRAY_SIZE(dec_params->dpb)) continue; - dpb_valid = !!(dpb[idx].flags & - V4L2_H264_DPB_ENTRY_FLAG_ACTIVE); set_ps_field(hw_rps, DPB_INFO(i, j), idx | dpb_valid << 4); @@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run, unsigned int dpb_idx) { struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; - const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; - int buf_idx = -1; - - if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - buf_idx = vb2_find_timestamp(cap_q, - dpb[dpb_idx].reference_ts, 0); + int buf_idx = run->ref_buf_idx[dpb_idx]; /* * If a DPB entry is unused or invalid, address of current destination @@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); + lookup_ref_buf_idx(ctx, &run); assemble_hw_rps(ctx, &run); config_registers(ctx, &run); From patchwork Mon Mar 28 19:59:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6845DC433EF for ; Mon, 28 Mar 2022 20:00:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344355AbiC1UCa (ORCPT ); Mon, 28 Mar 2022 16:02:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344216AbiC1UCC (ORCPT ); Mon, 28 Mar 2022 16:02:02 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 381A33C713; Mon, 28 Mar 2022 13:00:12 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 062B51F43877 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497611; bh=Lmej6mvr+yQMSxRRbj/l26Dv68B/G/B0rXWK3YrsMjQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NV+GhHpYnjMTzN1hs6yJQSwtbapQ3u3ipCkqaruCnWaRBKOX6wpA+9sdVbOnVVA7W XbykfN1mo93fX2n4O1yO0dGBLJG75otXwHXqq+oJklmAippFbWtHQh5QiaESJKStC6 NRakJe2iit1+dSJUvmOe1nuYUV2BzFF9WDQ32TJtpojG0Zi+reYJjLL6k2wirrtxfY ZN2Fmcmr+htAVoXAy+x6k+fscpoSJFZAIWnocQWnBw9fDX/97Bf4nI1S1UUa8JSs8B 5wBi3ZLfwJV6UgwnRChOoTtH8xIYLW/8j6kjEhYAVUeo67bClTHK10JcPbsigf5w5F RBWgqWq9CTqlQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 15/24] media: rkvdec: Enable capture buffer holding for H264 Date: Mon, 28 Mar 2022 15:59:27 -0400 Message-Id: <20220328195936.82552-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In order to support interlaced video decoding, the driver must allow holding the capture buffer so that the second field can be decoded into it. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 4 ++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2df8cf4883e2..05824f1997f7 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -138,6 +138,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { .ops = &rkvdec_h264_fmt_ops, .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts), .decoded_fmts = rkvdec_h264_vp9_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, }, { .fourcc = V4L2_PIX_FMT_VP9_FRAME, @@ -394,6 +395,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + /* Enable format specific queue feature */ + vq->subsystem_flags |= desc->subsystem_flags; + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2f4ea1786b93..e37f1a015fa0 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -81,6 +81,7 @@ struct rkvdec_coded_fmt_desc { const struct rkvdec_coded_fmt_ops *ops; unsigned int num_decoded_fmts; const u32 *decoded_fmts; + u32 subsystem_flags; }; struct rkvdec_dev { From patchwork Mon Mar 28 19:59:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72529C433FE for ; Mon, 28 Mar 2022 20:00:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344277AbiC1UC0 (ORCPT ); Mon, 28 Mar 2022 16:02:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344273AbiC1UCD (ORCPT ); Mon, 28 Mar 2022 16:02:03 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 417F853A45; Mon, 28 Mar 2022 13:00:13 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 963391F4388C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497612; bh=Y/EFmW32eD4KkKmhvAOQSjuBN39ky/laGXZQlpMupCc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bSgtdbGw4tYbD69RdtJrdkBRkEuQp47VMh2KzTtJdxoliJAInHWpAHa19MWxP+ef7 8+GEh/+RSK3v3aF5PfKNrNUqd8rGHpodqKNHRdiTz1R8P9f4Ov4nFjly0Wk7J0y2KL ebN8TluDyF/oyUDQjHticp2yjGFipQF4zAyFzAqMG8CfzypxRAqXv+br4WHoTNwEjv P6RrkSI/k+dhLbroD+xeNFri+pu9XFzpF62MkVHgPTQ4nnYpVB6DfMiWOJSEZ4+rGJ ydKvI6bNa+vrgIfidR0R1xXfsNdXWfyLSjqgVojDmoT2bhEoooxxac6gjk4LUS5x4u htRTV+HcbUkHg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 16/24] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Mon, 28 Mar 2022 15:59:28 -0400 Message-Id: <20220328195936.82552-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OPTUPT buffer resolution. Signed-off-by: Jonas Karlman Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 05824f1997f7..22c0382c579e 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -269,6 +269,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Mon Mar 28 19:59:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1FA2C433EF for ; Mon, 28 Mar 2022 20:00:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344297AbiC1UC1 (ORCPT ); Mon, 28 Mar 2022 16:02:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344290AbiC1UCT (ORCPT ); Mon, 28 Mar 2022 16:02:19 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1C013A711; Mon, 28 Mar 2022 13:00:15 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 68EF71F43899 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497614; bh=3DAbnv2Hbe0DYkbYL/MjN1MxBLh+/5IkhTs/7B4fPDM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fiOYY4dPfmIhjc9ZKM4zGNG7GZZ3WDWBteWGWUX3JAGNO16UF2Jft9PWQSIHOe2CL N9fGGOy400n3fnHyzv0i/jssKEkkaCRIlhaJT7/gFbeXpooSMhYlzZVqLt9nJ3sQFS jqndN/sbXcMxRY+hdz7SKIx6n5O9e9tYIccJHaUg+/iCAilRoY0UaVovTgwZl1qZHl 0c1uuYByvU/tvMvvrbKqAap6WWJ+scZCd0376Hv6+ZuIU4xymClf/yleuqzOz5sX2t k/Budtpbddo1aEV6oqaqldBS1/vxDE+2KIqgzxBgX99VbDp9Qisj1kw8QXqeRNuUGe 91qnuaOLnUe6g== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 17/24] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Mon, 28 Mar 2022 15:59:29 -0400 Message-Id: <20220328195936.82552-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The width and height in mbs is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. Signed-off-by: Jonas Karlman Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++++++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index db1e762baee5..847b8957dad3 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 22c0382c579e..67539f4bf382 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,8 +29,11 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; + unsigned int width, height; /* * TODO: The hardware supports 10-bit and 4:2:2 profiles, * but it's currently broken in the driver. @@ -45,6 +48,13 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) if (sps->bit_depth_luma_minus8 != 0) /* Only 8-bit is supported */ return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; } return 0; } From patchwork Mon Mar 28 19:59:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E25DC433FE for ; Mon, 28 Mar 2022 20:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344234AbiC1UC3 (ORCPT ); Mon, 28 Mar 2022 16:02:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343979AbiC1UCT (ORCPT ); Mon, 28 Mar 2022 16:02:19 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C24259A77; Mon, 28 Mar 2022 13:00:17 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 394D21F438A2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497616; bh=PqdQnTESUHQAXRfjK8M+MyHsl8WocJ3A7cMRoX/7vwc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IFTmVD0qqy2AA0++iIq5Y4/vmDXOq4qBb9aPdbD0Rv36lYHxr+O/H+EN9aTHBQk66 EOhqW1TUr8j+S5Meu+XfMG6ROIgibKAf/fxXGzfbYIDutmlnlBwHclIR0e9vt+7J7c H8cjgxv1muMhwijlvLK+/Rtl+Rx3yRJrlzu4Ej6ySdXodtRoJleMzt3sSt1KsfZybn rm1/2BJmizUymlb3kbaQLIgcD90zpqGjnl6wgO2LQlZF05kbUqdPoDslAgNkcBvLTc SvFAo7+mmkTjWHRTFSKh3xqzokOJ1KH0qoBEo53dd8M8n/o0w25MlVU43rrRL+74Ug FjPikprYubBQw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Ezequiel Garcia , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 18/24] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Mon, 28 Mar 2022 15:59:30 -0400 Message-Id: <20220328195936.82552-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet is 3 bits wide. 8 is wrongly added to the bit depth value written to these 3-bit fields. Because only the 3 LSB is written the hardware is configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Signed-off-by: Jonas Karlman Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 847b8957dad3..ec52b195bbd7 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Mon Mar 28 19:59:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79D5BC4332F for ; Mon, 28 Mar 2022 20:00:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344245AbiC1UCd (ORCPT ); Mon, 28 Mar 2022 16:02:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344305AbiC1UCU (ORCPT ); Mon, 28 Mar 2022 16:02:20 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 427385A0B9; Mon, 28 Mar 2022 13:00:19 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 161CD1F438A5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497618; bh=QCJPsQYSB9YDo1HOMH1IpCQstEyUl6W5GERVwRBYIVI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hj+rYXI+luMx73+JuzZBTXXnWHQXQReI7Hmoowk6HECNyL91XfOpxSp+iPVs3sSEF kLwD9FS6EoNZlcjXcTpzmBkdWuZr2sdXe7IeIYTi3YvDwV11sSSjT9VkWmARn8vO6J OYqyP7LlEBmJ/Grnl15beX/50yUYKLVEt5/jnOxcrf+Gbp+arRScCBp7hv3DMfXkfL 5b3/veO2PCCjdbswsGMcqMMJU3db673BHkHw5HPfvZbX+WJYubzwRgXm2ZXUViDYSW RICc+J7t1abNgIbqCo8yc7L3/xXZpBGVrRfm3ASp78XcVdNjsiZftovttUY2V7oaiB BHsOhEdOI5oZA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 19/24] media: rkvdec-h264: Add field decoding support Date: Mon, 28 Mar 2022 15:59:31 -0400 Message-Id: <20220328195936.82552-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This makes use of the new feature in the reference builder to program up to 32 references when doing field decoding. It also signals the parity (top of bottom) of the field to the hardware. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------ 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index ec52b195bbd7..891c48bf6a51 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl { u8 err_info[RKV_ERROR_INFO_SIZE]; }; -#define RKVDEC_H264_DPB_SIZE 16 - struct rkvdec_h264_reflists { struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; - u8 num_valid; }; struct rkvdec_h264_run { @@ -738,23 +735,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; int buf_idx = -1; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) { buf_idx = vb2_find_timestamp(cap_q, dpb[i].reference_ts, 0); + if (buf_idx < 0) + pr_debug("No buffer for reference_ts %llu", + dpb[i].reference_ts); + } run->ref_buf_idx[i] = buf_idx; } } static void assemble_hw_rps(struct rkvdec_ctx *ctx, + struct v4l2_h264_reflist_builder *builder, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; - const struct v4l2_ctrl_h264_sps *sps = run->sps; struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu; - u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4); u32 *hw_rps = priv_tbl->rps; u32 i, j; @@ -772,37 +772,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num <= dec_params->frame_num) { - p[i] = dpb[i].frame_num; - continue; - } - - p[i] = dpb[i].frame_num - max_frame_num; + p[i] = builder->refs[i].frame_num; } for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { - for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = run->ref_buf_idx[i] >= 0; - u8 idx = 0; + for (i = 0; i < builder->num_valid; i++) { + struct v4l2_h264_reference *ref; + u8 dpb_valid; + u8 bottom; switch (j) { case 0: - idx = h264_ctx->reflists.p[i].index; + ref = &h264_ctx->reflists.p[i]; break; case 1: - idx = h264_ctx->reflists.b0[i].index; + ref = &h264_ctx->reflists.b0[i]; break; case 2: - idx = h264_ctx->reflists.b1[i].index; + ref = &h264_ctx->reflists.b1[i]; break; } - if (idx >= ARRAY_SIZE(dec_params->dpb)) + if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb))) continue; + dpb_valid = run->ref_buf_idx[ref->index] >= 0; + bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF; + set_ps_field(hw_rps, DPB_INFO(i, j), - idx | dpb_valid << 4); + ref->index | dpb_valid << 4); + set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom); } } } @@ -990,10 +989,6 @@ static void config_registers(struct rkvdec_ctx *ctx, rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15); } - /* - * Since support frame mode only - * top_field_order_cnt is the same as bottom_field_order_cnt - */ reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt); writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0); @@ -1109,7 +1104,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params, run.sps, run.decode_params->dpb); - h264_ctx->reflists.num_valid = reflist_builder.num_valid; v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); @@ -1117,7 +1111,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); lookup_ref_buf_idx(ctx, &run); - assemble_hw_rps(ctx, &run); + assemble_hw_rps(ctx, &reflist_builder, &run); config_registers(ctx, &run); rkvdec_run_postamble(ctx, &run.base); From patchwork Mon Mar 28 19:59:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E45FCC4332F for ; Mon, 28 Mar 2022 20:00:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344366AbiC1UCg (ORCPT ); Mon, 28 Mar 2022 16:02:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344253AbiC1UCU (ORCPT ); Mon, 28 Mar 2022 16:02:20 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2019F5B3E1; Mon, 28 Mar 2022 13:00:21 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id AEA9B1F438B1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497619; bh=BQgi4WXn9GkaikKLBxnmXBJ9Op1ly9b7u96QvOVoYaY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nQy6Ul9u9ZNQiDPpimTjQGgDKJ4dMDUQ4QHsaH6Vtmdydoas4nbQmvrWD/vVrf/6p Y8vXWtMlaovLs7s5bcR2TGA55jiDeiHt7oE5atDSNWmLCPtAvU9dAAE8oy0uGCbJAK eORjbvoA3MGKEmj1cQ9EBllZ52/2Or5tpTLLIwClMzg2CrZ0CZV5FDkJJV2g+9oOBb 2YKnVer0ahF5asTI39s2dK0CAI756v9o04AvIxR2jUPP4FSJ1vxow6dUBx24PH/ZsI 6/j03Rhfs4mas3bS5A/U3P0UNrfJZYUhQEcG0bqtakc6I2n8i5/vuxc3a0b6kzqI/e 5Eb+8DiJGY0gQ== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 20/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Mon, 28 Mar 2022 15:59:32 -0400 Message-Id: <20220328195936.82552-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is needed to optimizing field decoding. Each field will be decoded in the same capture buffer, so to make use of the queues we need to be able to ask the driver to keep the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 67148ba346f5..50d636678ff3 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n", From patchwork Mon Mar 28 19:59:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1B29C433FE for ; Mon, 28 Mar 2022 20:00:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344324AbiC1UCi (ORCPT ); Mon, 28 Mar 2022 16:02:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344319AbiC1UCV (ORCPT ); Mon, 28 Mar 2022 16:02:21 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC9F53B3C2; Mon, 28 Mar 2022 13:00:22 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 6107D1F438B3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497621; bh=qASp9j35h86HZHqACbW4hS8pRQcR68hjlS1LdUain6U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bSYBFsly9GrPfSwBI9lPzlksp9cEBsdRIyvQw2RwExw4t9cHPSfmpMTol+4t9Z2RU j3QxeXh0CDyalwcffVXx4lzqWFskW6LvhdiGU6/f/U6aoMhbw10J6KNiJlBMIt34Vf 6TdZjf7ouduq5K7Pt9Ycmpw/MAjAuQFQtlsOFIzIXiTU1unRooRT5vrlpMau3Wb9Q8 7gHQ9mYwOpa140DlGQVjpz00gfy8tTVhUXBHeGu0DK3mqncCtAmwzApZdFTOqn4dYC pfSbmR0rwINKx3pzQUCTmyLuYdhp2fXJrNk7QfryzTqKFzBp2uidnWwtBJTj42CSjg UM9DR4nZgn8GA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 21/24] media: hantro: Stop using H.264 parameter pic_num Date: Mon, 28 Mar 2022 15:59:33 -0400 Message-Id: <20220328195936.82552-22-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The hardware expects FrameNumWrap or long_term_frame_idx. Picture numbers are per field, and are mostly used during the memory management process, which is done in userland. This fixes two ITU conformance tests: - MR6_BT_B - MR8_BT_B Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_h264.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 0b4d2491be3b..228629fb3cdf 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) return 0; - if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - return dpb->pic_num; return dpb->frame_num; } From patchwork Mon Mar 28 19:59:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BBD3C433FE for ; Mon, 28 Mar 2022 20:01:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344326AbiC1UCj (ORCPT ); Mon, 28 Mar 2022 16:02:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344255AbiC1UCV (ORCPT ); Mon, 28 Mar 2022 16:02:21 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A61A5C34B; Mon, 28 Mar 2022 13:00:24 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 2C2AF1F438C4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497623; bh=Og2wNbsGzxzouALoYhFoXVFpDemGtbhl+ssP9/EEn08=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NQ8pTF5N12gMnd+ao3DJztqsVvAZXfgkH3/36NVWu1VlrUL2QKRyzP3+n9F2SRvvS NWT28qm9AbEx2JhJiRHOkaWU30WY6mQMHXQcsHyhvo5Gb+fsk/y0jvWSTiUAfpSfN+ TdPxbOuq+77LsI05gd/INB5I/KjbFuPUxc6ilHPFqAamcBKY3f2/jtD/7NugF274lq 9OsmTwu0HedVjbtwOvEc8AC+fW1uxBJI8lhMSG3+OX/pGiGOSSpV5FhKgxjbAVYsdL QADmAXLcOOhvpbEdRQ1FslUWrbZbUp1dqJtS5p8D6bg94ppj/xTy5qI50Vux3c6zx1 dOW2ZBNutLEXg== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 22/24] media: hantro: h264: Make dpb entry management more robust Date: Mon, 28 Mar 2022 15:59:34 -0400 Message-Id: <20220328195936.82552-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The driver maintains stable slot location for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb; From patchwork Mon Mar 28 19:59:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E276C433EF for ; Mon, 28 Mar 2022 20:01:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344267AbiC1UCu (ORCPT ); Mon, 28 Mar 2022 16:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344184AbiC1UCY (ORCPT ); Mon, 28 Mar 2022 16:02:24 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1D2F5C858; Mon, 28 Mar 2022 13:00:26 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 3FA4D1F438C2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497625; bh=8ZY/TY5wUQG0UMY3VQ/LrDrnewxXRhJm8xn75yuBk5Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LSDm5ErzEA5Fbw1ws4czjOsu3fDm4pT7JTICNQb9Fxicx6vsUkjVibEJRKm5wdL1O RBQz0zvf0GFTHerfTot93aA7pF8puNqK0jfkRs9hc6KNv9c0YQ/snt2M+kxyvVWs+S 3EBf7R+5vYaeovxq0AC+6izjekHkVTooXRWLjT7GU7RtNgNCmzqDnta4gia2cQ1DoH TG4bC9yYVGWQsG3EzJrtji98AFV+6ZbCgyA0PWPcYOHg/UAQfX/9jawDNdU2juVia5 Ft73gjEr8k+CaHrjhTKr7JNAoe8Ku9MUs/j0HW1rub/lD3a9+Xd9eIlAtko1TX8l1T 9C9nhjLJIUDBA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 23/24] media: hantro: Add H.264 field decoding support Date: Mon, 28 Mar 2022 15:59:35 -0400 Message-Id: <20220328195936.82552-24-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This adds the required code to support field decoding. While most of the code is derived from Rockchip and VSI reference code, the reduction of the reference list to 16 entries has been found by trial and errors. The list consist of all the references with the opposite field parity. The strategy being to deduplicate the reference picture that points to the same storage (same index). The choice of opposite parity has been made to keep the other field or a field pair to be kept in the list. This method may not be robust if a field was lost. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++--- drivers/staging/media/hantro/hantro_hw.h | 1 + 2 files changed, 94 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 7377fc26f780..f6fc939aa726 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -22,6 +22,11 @@ #define POC_BUFFER_SIZE 34 #define SCALING_LIST_SIZE (6 * 16 + 2 * 64) +/* For valid and long term reference marking, index are reversed, so bit 31 + * indicates the status of the picture 0. + */ +#define REF_BIT(i) BIT(32 - 1 - (i)) + /* Data structure describing auxiliary buffer format. */ struct hantro_h264_dec_priv_tbl { u32 cabac_table[CABAC_INIT_BUFFER_SIZE]; @@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx) { const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls; const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode; + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps; struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu; const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; u32 dpb_longterm = 0; @@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx) tbl->poc[i * 2] = dpb[i].top_field_order_cnt; tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt; + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + /* * Set up bit maps of valid and long term DPBs. - * NOTE: The bits are reversed, i.e. MSb is DPB 0. + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame + * decoding, bit 31 to 15 are used, while for field decoding, + * all bits are used, with bit 31 being a top field, 30 a bottom + * field and so on. */ - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) { + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) + dpb_valid |= REF_BIT(i * 2); + + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) + dpb_valid |= REF_BIT(i * 2 + 1); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) { + dpb_longterm |= REF_BIT(i * 2); + dpb_longterm |= REF_BIT(i * 2 + 1); + } + } else { + dpb_valid |= REF_BIT(i); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) + dpb_longterm |= REF_BIT(i); + } + } + ctx->h264_dec.dpb_valid = dpb_valid; + ctx->h264_dec.dpb_longterm = dpb_longterm; + + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) || + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) { + tbl->poc[32] = ctx->h264_dec.cur_poc; + tbl->poc[33] = 0; + } else { + tbl->poc[32] = dec_param->top_field_order_cnt; + tbl->poc[33] = dec_param->bottom_field_order_cnt; } - ctx->h264_dec.dpb_valid = dpb_valid << 16; - ctx->h264_dec.dpb_longterm = dpb_longterm << 16; - - tbl->poc[32] = dec_param->top_field_order_cnt; - tbl->poc[33] = dec_param->bottom_field_order_cnt; assemble_scaling_list(ctx); } @@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, { struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; dma_addr_t dma_addr = 0; + s32 cur_poc = ctx->h264_dec.cur_poc; + u32 flags; if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts); @@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, dma_addr = hantro_get_dec_buf_addr(ctx, buf); } - return dma_addr; + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0; + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) < + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ? + 0x1 : 0; + + return dma_addr | flags; } u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) @@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) return dpb->frame_num; } +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + int write_idx = 0; + int i; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + write_idx = b->num_valid; + goto done; + } + + for (i = 0; i < b->num_valid; i++) { + if (!(b->cur_pic_fields == reflist[i].fields)) { + reflist[write_idx++] = reflist[i]; + continue; + } + } + +done: + /* Should not happen unless we have a bug in the reflist builder. */ + if (WARN_ON(write_idx > 16)) + write_idx = 16; + + /* Clear the remaining, some streams fails otherwise */ + for (; write_idx < 16; write_idx++) + reflist[write_idx].index = 15; +} + int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) { struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec; @@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) /* Update the DPB with new refs. */ update_dpb(ctx); - /* Prepare data in memory. */ - prepare_table(ctx); - /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode, ctrls->sps, ctx->h264_dec.dpb); + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count; + + /* Prepare data in memory. */ + prepare_table(ctx); + v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); + + /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce + * the actual picture lists in field through the dpb_valid, + * dpb_longterm bitmap along with the current frame parity. + */ + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) { + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1); + } + return 0; } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 292aaaabaf24..fd869369fb97 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx { struct hantro_h264_dec_ctrls ctrls; u32 dpb_longterm; u32 dpb_valid; + s32 cur_poc; }; /** From patchwork Mon Mar 28 19:59:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12794195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95852C433F5 for ; Mon, 28 Mar 2022 20:01:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344289AbiC1UCm (ORCPT ); Mon, 28 Mar 2022 16:02:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344340AbiC1UCZ (ORCPT ); Mon, 28 Mar 2022 16:02:25 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BFF75D1AC; Mon, 28 Mar 2022 13:00:28 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 173C11F438BD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648497627; bh=5bMWt8FIlhATFg8dWZpSwD375jbwZ9dZXKe/ctzMbWc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HgE8I+mDkxkQ2X9yVMvzFDQ4jmEqoeYUkBg3IBrZElVU6jyB8GKIFEdYDfaA5E7Nt VsYfXmZ7jIXrWHsBn57QQHjFOKAl1GAF5icpLR08XgBFPqgxXS9XG7uySm7sNh7bcK d3/7xeYZ3V/t6VinCbny8rAAZYBZ9TTaMEYUW1E+KTFs0+cQKnaPCTHEGvD3G40bBk gTiAW3vZmHVzewLprpcQJTIwAXwFY+kphUECS2Ci6zv+k0HIzAHe2YAxl/LnRClop9 PEIrQU7bIWEKMK3Cq2pri/rmqXODlitsSCd2gAiS3y54w45iHyx+n+ae/sLMeutUl5 BYlnDrYQVnITw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Alex Bee , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v1 24/24] media: rkvdec-h264: Don't hardcode SPS/PPS parameters Date: Mon, 28 Mar 2022 15:59:36 -0400 Message-Id: <20220328195936.82552-25-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220328195936.82552-1-nicolas.dufresne@collabora.com> References: <20220328195936.82552-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Alex Bee Some SPS/PPS parameters are currently hardcoded in the driver even though so do exist in the uapi which is stable by now. Use them instead of hardcoding them. Conformance tests have shown there is no difference, but it might increase decoder performance. Signed-off-by: Alex Bee --- drivers/staging/media/rkvdec/rkvdec-h264.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 891c48bf6a51..91f65d78e453 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -655,13 +655,14 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, #define WRITE_PPS(value, field) set_ps_field(hw_ps->info, field, value) /* write sps */ - WRITE_PPS(0xf, SEQ_PARAMETER_SET_ID); - WRITE_PPS(0xff, PROFILE_IDC); - WRITE_PPS(1, CONSTRAINT_SET3_FLAG); + WRITE_PPS(sps->seq_parameter_set_id, SEQ_PARAMETER_SET_ID); + WRITE_PPS(sps->profile_idc, PROFILE_IDC); + WRITE_PPS((sps->constraint_set_flags & 1 << 3) ? 1 : 0, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); - WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); + WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS), + QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); WRITE_PPS(sps->pic_order_cnt_type, PIC_ORDER_CNT_TYPE); @@ -679,8 +680,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, DIRECT_8X8_INFERENCE_FLAG); /* write pps */ - WRITE_PPS(0xff, PIC_PARAMETER_SET_ID); - WRITE_PPS(0x1f, PPS_SEQ_PARAMETER_SET_ID); + WRITE_PPS(pps->pic_parameter_set_id, PIC_PARAMETER_SET_ID); + WRITE_PPS(pps->seq_parameter_set_id, PPS_SEQ_PARAMETER_SET_ID); WRITE_PPS(!!(pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE), ENTROPY_CODING_MODE_FLAG); WRITE_PPS(!!(pps->flags & V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT),