From patchwork Thu Mar 31 19:37:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55048C433EF for ; Thu, 31 Mar 2022 19:37:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240768AbiCaTjZ (ORCPT ); Thu, 31 Mar 2022 15:39:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234059AbiCaTjY (ORCPT ); Thu, 31 Mar 2022 15:39:24 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 790901F42C3; Thu, 31 Mar 2022 12:37:36 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A02F61F47247 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755455; bh=p/CNjMUurjHMQVHA7WJucC4cA80P6l/szurKGX62Y7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LsmrQncNO5q6QMqipTARVmMkEJWMwJ3VbtzG3UmZFAiIDj+6GkDJHD5Bwp6Cl+iVG mkOyrlkZ6WickSz13aX1M52J85u7N4PCYiFz6Y7IFBWRLtZj5XAAPK//VCgCYv41V6 Z8jMLtXblEOvIiubnO8ZYuL42Ca6uaCgsbTsfa5DnG47j5reg7o1kwEJTQUSuZw5vU gVO5s4Mtj4HENvkURrU0xPHtYEfMzkQJ9Mb0h18GHZKdgNGiVr7Y64VP/jZuGsqajB TWxUOjgqrJ91Wg719kHxgBAN/fnk7zUxbZFhVc6fnNuL3d/HQHzFGqreIo2HusOtw+ TsOY69JC/1CUQ== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 01/23] media: doc: Document dual use of H.264 pic_num/frame_num Date: Thu, 31 Mar 2022 15:37:03 -0400 Message-Id: <20220331193726.289559-2-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org These two fields need documentation as they have dual meaning. It is also confusing since pic_num is a derived value from frame_num, so this should help application developers. If we ever need to make a V2 of this API, I would suggest to remove pic_num entirely. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- .../media/v4l/ext-ctrls-codec-stateless.rst | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst index 6541e4c32b26..49f89b702068 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst @@ -649,10 +649,16 @@ Stateless Codec Control ID :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64. * - __u32 - ``pic_num`` - - + - For short term references, this should match the derived value PicNum + (8-28) and for long term references it should match the derived value + LongTermPicNum (8-29). Note that pic_num is the same as FrameNumWrap + for frame decoding. * - __u16 - ``frame_num`` - - + - For short term references, this should match the frame_num value from + the slice header syntax (the driver will wrap the value if neeeded). For + long term references, this should be set to the value of + long_term_frame_idx described in the dec_ref_pic_marking() syntax. * - __u8 - ``fields`` - Specifies how the DPB entry is referenced. See :ref:`Reference Fields ` From patchwork Thu Mar 31 19:37:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2FEAC433EF for ; Thu, 31 Mar 2022 19:37:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240795AbiCaTjc (ORCPT ); Thu, 31 Mar 2022 15:39:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240780AbiCaTj2 (ORCPT ); Thu, 31 Mar 2022 15:39:28 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE4351F160C; Thu, 31 Mar 2022 12:37:37 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E0A2B1F4724B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755456; bh=RzISFmgzdgSdtW0+/gjxICIzsCqQWUTem2X0Qdl07rI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P8JxZJhCDdx9Qxk6/3g0ArBQAL6zq/4nDv4o4se2D/r8x9FqDSTG1Gmclg/ipJvOd QVlZiS09IZSS6knovbP88HF2IV7HrZE1F3rhavI2rX4E0451TzvLMBrUwygzg1GjGI irNuFE3v7ose8uwUbKL2ZQSLKbjzcElTY/HxaeP2BDXzjK85i+gTlt8+YOVA47lGj6 7d+S7kJOkvLXM5m7rWGukhvKJ9TAGX/mCC6sFzvZCD+1PsYD2j/I2gdHM/oH4ueAsy aKVOeo7LNLz2HOR4LEw51uhN5GKXsZa77yUZ14AZ1Bgg11PvsIE/kVhKqYEApsz9Cg DE6LjQf5fnoCw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 02/23] media: v4l2-mem2mem: Trace on implicit un-hold Date: Thu, 31 Mar 2022 15:37:04 -0400 Message-Id: <20220331193726.289559-3-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org If the timestamp of the src buffer differs from the timestamp of a held dst buffer, the held buffer is implicitly removed and marked as done. Add a trace to help debugging if someone hits that case. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-mem2mem.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 675e22895ebe..fbcd79763e8f 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -336,6 +336,7 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, if (src && dst && dst->is_held && dst->vb2_buf.copied_timestamp && dst->vb2_buf.timestamp != src->vb2_buf.timestamp) { + dprintk("src and dst timestamp mismatch, removing held capture buffer.\n"); dst->is_held = false; v4l2_m2m_dst_buf_remove(m2m_ctx); v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE); From patchwork Thu Mar 31 19:37:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54243C433FE for ; Thu, 31 Mar 2022 19:37:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240802AbiCaTjd (ORCPT ); Thu, 31 Mar 2022 15:39:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240779AbiCaTj2 (ORCPT ); Thu, 31 Mar 2022 15:39:28 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43D9B2102AA; Thu, 31 Mar 2022 12:37:39 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 300941F4724D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755458; bh=RrVHdlTOtkGYblUxkaOs8WkHp5+WDfAbFJ5ZboXk3tI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zi8EIAxPwcX4aRMfL7d91NXXfjHGmzyE7d5YlfyJFimNZvfte0AFF3kdVetq7/jQ8 SHDzIqgSlp2U00OIVmZUryvMOarEIl6hNXGTTe2rsUlfHFyKM/TNiv83qUAwxYWitU e0LshldRfnvfBzvDTNH2D6HtkKqnSlgfIkvieLX9ue+aomb27kFQtzbw5J2A5nc5rm 6zWLF3OsMNlBEw0g17XjwClK2DV1qlRv98JTZ8JLxGM78tky9EN9A94slez3cccAIf gZCq/VpQ35UM2JUiwA+BLBZDMqCGulg1UDpMOd+2bXLBVmefG71lYvC/ycd8+j/5x7 cIjSRNUvOnj3w== From: Nicolas Dufresne To: Tomasz Figa , Marek Szyprowski , Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 03/23] media: videobuf2-v4l2: Warn on holding buffers without support Date: Thu, 31 Mar 2022 15:37:05 -0400 Message-Id: <20220331193726.289559-4-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Sebastian Fricke Using V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF flag without specifying the subsystem flag VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, results in silently ignoring it. Warn the user via a debug print when the flag is requested but ignored by the videobuf2 framework. Signed-off-by: Sebastian Fricke Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/media/common/videobuf2/videobuf2-v4l2.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index 6edf4508c636..812c8d1962e0 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -329,8 +329,13 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b */ vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE; vbuf->field = b->field; - if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) + if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) { + if (vbuf->flags & V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF) + dprintk(q, 1, + "Request holding buffer (%d), unsupported on output queue\n", + b->index); vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; + } } else { /* Zero any output buffer flags as this is a capture buffer */ vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS; From patchwork Thu Mar 31 19:37:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 047D3C433F5 for ; Thu, 31 Mar 2022 19:37:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240815AbiCaTjf (ORCPT ); Thu, 31 Mar 2022 15:39:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240781AbiCaTj2 (ORCPT ); Thu, 31 Mar 2022 15:39:28 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70D3222451D; Thu, 31 Mar 2022 12:37:40 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A47591F47247 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755459; bh=XWSCMrc+NrLm1p3fYYJooFLeLhu8K+wqylIt0iyj7Vc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ci2A27WG3nn3hVwxc9jwIR3E4CPWTQcVTJX69lc7p4gFqCHtA6JBUl2Zc403FpXq5 LkQ1jV3EkM9Dn964TVruUiU4v9lkrRH9Fb7J0VIytyFFZtFeOIktrZzNfcxOPviNDp o0jMoLjQioZWZclguu1x7RvVwgXbWK1O6VGJRcseqdEWPy8sEYg/hqZUdFQ2x9J9oq g5RbA6c821YYmIh0tZ95dqRwo/xZCZW2B5tcxonaUAFoGB/d/vG4+jCtUYEX65iuB6 1l7ebu99zlOQkx1eBuyjPSfSIpK3Oy/QWqai+EVzKRIlJnavqmkwAlyP/y7mQs+0mr IYSHXnNduxHAw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 04/23] media: h264: Avoid wrapping long_term_frame_idx Date: Thu, 31 Mar 2022 15:37:06 -0400 Message-Id: <20220331193726.289559-5-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org For long term references, frame_num is set to long_term_frame_idx which does not require wrapping. This is fixed by observation, no directly related issue have been found yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-h264.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 5633a242520a..ac47519a9fbe 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -57,8 +57,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * '8.2.4.1 Decoding process for picture numbers' of the spec. * TODO: This logic will have to be adjusted when we start * supporting interlaced content. + * For long term references, frame_num is set to + * long_term_frame_idx which requires no wrapping. */ - if (dpb[i].frame_num > cur_frame_num) + if (!b->refs[i].longterm && dpb[i].frame_num > cur_frame_num) b->refs[i].frame_num = (int)dpb[i].frame_num - max_frame_num; else From patchwork Thu Mar 31 19:37:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA49FC433F5 for ; Thu, 31 Mar 2022 19:37:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240856AbiCaTjm (ORCPT ); Thu, 31 Mar 2022 15:39:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240796AbiCaTjc (ORCPT ); Thu, 31 Mar 2022 15:39:32 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCB0F1F160C; Thu, 31 Mar 2022 12:37:43 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id B24EA1F4724C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755462; bh=I1NV/FqMIg1TwxGn1zzJ4HMiOapNHncuGBbSoCDFeG4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g8NEE8ifGEKkPJ0Frxvbsfm4Te7+QxImekdoGu2Xl+Tn8jLiOBhBvQsQrsFg3XqPR o60seeCQ6NCmp0nfbcsYJf2AiSw4yItX9Bs4lWJXeC7fxbwwGTKmf9qxzoyJFiZ2M9 0O8DitUvnP4tqmQ19AaXGKXPt28w3laZc4PMasWJ3sqYAw3d/haefwk68YKnFAhzQi SMct/WWX7gJ8tfBktXYTs0Y4U5PDehlRI/UfXzu18KY1pGQP0v4BOckl1x08f0IHCJ d+wKsaYzlYCwe9CXYZ3aFBtfDHQq9kvULLXZBP9ZEBmrMyqIPGKEejev+8UisGGWTe RzkvC+l0wpcOw== From: Nicolas Dufresne To: Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman , Heiko Stuebner , Matthias Brugger Cc: kernel@collabora.com, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org Subject: [PATCH v2 05/23] media: h264: Use v4l2_h264_reference for reflist Date: Thu, 31 Mar 2022 15:37:07 -0400 Message-Id: <20220331193726.289559-6-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In preparation for adding field decoding support, convert the byte arrays for reflist into array of struct v4l2_h264_reference. That struct will allow us to mark which field of the reference picture is being referenced. Signed-off-by: Nicolas Dufresne --- .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++- drivers/media/v4l2-core/v4l2-h264.c | 33 ++++--- .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++---- drivers/staging/media/hantro/hantro_hw.h | 6 +- .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++---------- drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +-- include/media/v4l2-h264.h | 19 ++-- 7 files changed, 116 insertions(+), 107 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c index 43542de11e9c..72c599e05a47 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param, /* * The firmware expects unused reflist entries to have the value 0x20. */ -static void fixup_ref_list(u8 *ref_list, size_t num_valid) +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b) { - memset(&ref_list[num_valid], 0x20, 32 - num_valid); + u32 i; + + /* FIXME mark the reference parity */ + for (i = 0; i < b->num_valid; i++) + ref_list[i] = b->index; + + for (; i < 32; i++) + ref_list[i] = 0x20; } static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); /* Adapt the built lists to the firmware's expectations */ - fixup_ref_list(p0_reflist, reflist_builder.num_valid); - fixup_ref_list(b0_reflist, reflist_builder.num_valid); - fixup_ref_list(b1_reflist, reflist_builder.num_valid); + get_ref_list(p0_reflist, reflist_builder); + get_ref_list(b0_reflist, reflist_builder); + get_ref_list(b1_reflist, reflist_builder); memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, sizeof(inst->vsi_ctx.h264_slice_params)); diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index ac47519a9fbe..afbfcf78efe4 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, pic_order_count = dpb[i].top_field_order_cnt; b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid] = i; + b->unordered_reflist[b->num_valid].index = i; b->num_valid++; } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) - b->unordered_reflist[i] = i; + b->unordered_reflist[i].index = i; } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const struct v4l2_h264_reflist_builder *builder = data; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist) + struct v4l2_h264_reference *reflist) { memcpy(reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist) + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist) { memcpy(b0_reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c index f49dbfb8a843..9de7f05eff2a 100644 --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; int reg_num; u32 reg; @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 0; i < 15; i += 3) { - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]); + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++)); } @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx) * of forward and backward reference picture lists and first 4 entries * of P forward picture list. */ - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) | - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]); + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) | + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC); /* @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) { - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) | - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) | - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) | - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) | - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) | - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]); + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index); vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++)); } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index ed018e293ba0..2bc6b8f088f5 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - u8 p[HANTRO_H264_DPB_SIZE]; - u8 b0[HANTRO_H264_DPB_SIZE]; - u8 b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; }; /** diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c index 64a6330475eb..46c1a83bcc4e 100644 --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; u32 reg; int i; @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx) b1_reflist = ctx->h264_dec.reflists.b1; p_reflist = ctx->h264_dec.reflists.p; - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) | - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) | - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) | - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) | - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) | - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]); + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) | + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) | + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) | + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) | + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) | + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74)); - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) | - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) | - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) | - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) | - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) | - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]); + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) | + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) | + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) | + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) | + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) | + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75)); reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) | @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx) VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83)); - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) | - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) | - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) | - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) | - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) | - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) | + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) | + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) | + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) | + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) | + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100)); - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) | - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) | - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) | - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) | - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) | - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) | + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) | + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) | + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) | + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) | + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101)); - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) | - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) | - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) | - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) | + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) | + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) | + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102)); - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) | - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) | - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) | - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) | - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) | - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) | + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) | + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) | + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) | + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) | + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103)); - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) | - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) | - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) | - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) | - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) | - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) | + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) | + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) | + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) | + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) | + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104)); - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) | - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) | - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) | - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) | + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) | + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) | + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105)); - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) | - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) | - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) | - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]); + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) | + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) | + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) | + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106)); reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm); diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 951e19231da2..3c7f3d87fab4 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - u8 p[RKVDEC_H264_DPB_SIZE]; - u8 b0[RKVDEC_H264_DPB_SIZE]; - u8 b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; u8 num_valid; }; @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, switch (j) { case 0: - idx = h264_ctx->reflists.p[i]; + idx = h264_ctx->reflists.p[i].index; break; case 1: - idx = h264_ctx->reflists.b0[i]; + idx = h264_ctx->reflists.b0[i].index; break; case 2: - idx = h264_ctx->reflists.b1[i]; + idx = h264_ctx->reflists.b1[i].index; break; } diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4b1c71c935e0..ef9a894e3c32 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; u8 num_valid; }; @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist); + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist); /** * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist); + struct v4l2_h264_reference *reflist); #endif /* _MEDIA_V4L2_H264_H */ From patchwork Thu Mar 31 19:37:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65C53C4332F for ; Thu, 31 Mar 2022 19:37:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240858AbiCaTjn (ORCPT ); Thu, 31 Mar 2022 15:39:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240805AbiCaTjd (ORCPT ); Thu, 31 Mar 2022 15:39:33 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1C3923B3E8; Thu, 31 Mar 2022 12:37:45 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 4C41B1F4724D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755464; bh=VjX+liG+SlI3iuDWs50MzK4K6rmwEP4Ktzmj7Beu+Mo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i1Up/5emswsszUr/IL82EU9CA2ojTsAugslRJT9YMSbyEf30Ti5BXZws52rF6LNiV y5+cLWg7C5SqyiIXBAY2YUtyVYPQVQ0q/bQalp4u+V1GMeduCKVwTa1UYz2JGXe2/C WafKxJ5Y4tsT9UbfNjuahk3E1SccJN388ZRPx+KytlH1ngeNwZ2lLs6XZ8Risfa/FK qBtN4be/+0m8Hzg3xmNvmH9zRQPftUBsc+WvrSTyMOyB/+WlCd0Eofe5CV2R77f6yN F7nLcd0c4r8+gO7wek8hFboqHt5R4ZY5gR22Z5omOh24PqR6akhbS2/G+USGX5dCq0 xfEvo4aX5tcTg== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v2 06/23] media: h264: Increase reference lists size to 32 Date: Thu, 31 Mar 2022 15:37:08 -0400 Message-Id: <20220331193726.289559-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is to accommodate support for field decoding, which splits the top and the bottom references into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index afbfcf78efe4..4b46b36526c0 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2bc6b8f088f5..292aaaabaf24 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Thu Mar 31 19:37:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36630C433EF for ; Thu, 31 Mar 2022 19:38:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240839AbiCaTjw (ORCPT ); Thu, 31 Mar 2022 15:39:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240835AbiCaTjk (ORCPT ); Thu, 31 Mar 2022 15:39:40 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D5E923C0E7; Thu, 31 Mar 2022 12:37:47 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 2FB0A1F4724F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755465; bh=TG6URAwoxuOMsxmElkXZN+3kr6++zE3/TmJ6jTIeCc0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JGVnr/FtZT3zuIJdqcifkyNee2EAFLxBcAh+IS7LZnNsmfdiXWV3PlyKjTB83WI5+ 1UH7iWvOuh0Qpnv7Tqj05AehCHHAEDRGyX2VgERLv5QdIt3pGYli65+SqHXLPzl6R8 wxWCwO5QGvy9ZWt+uCn/nrhUwd8XYh6dBy7jFjh3AHMB5GVhKpsQx2TDmWrOVCjd+t +wZfeujPGizeL248Q6yn/vi6HFViB56RF59I4A1vEVtZD2BihPdO+TLhLzrlHZhnPZ NP99ppZHKcdsTPxyPywN1cB8FLhzi9ILL5ru2xWgTy9IHb8F56fNP8t1iV8DrENQ5v 4rgHUVr4YZMzw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 07/23] media: h264: Store current picture fields Date: Thu, 31 Mar 2022 15:37:09 -0400 Message-Id: <20220331193726.289559-8-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This information, also called picture structure, is required in field decoding mode to construct reference lists. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 10 +++++++--- include/media/v4l2-h264.h | 4 ++++ 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 4b46b36526c0..58f18bb0afb6 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -34,13 +34,17 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, cur_frame_num = dec_params->frame_num; memset(b, 0, sizeof(*b)); - if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) + if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) { b->cur_pic_order_count = min(dec_params->bottom_field_order_cnt, dec_params->top_field_order_cnt); - else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) + b->cur_pic_fields = V4L2_H264_FRAME_REF; + } else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) { b->cur_pic_order_count = dec_params->bottom_field_order_cnt; - else + b->cur_pic_fields = V4L2_H264_BOTTOM_FIELD_REF; + } else { b->cur_pic_order_count = dec_params->top_field_order_cnt; + b->cur_pic_fields = V4L2_H264_TOP_FIELD_REF; + } for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { u32 pic_order_count; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index e282fb16ac58..e165a54c68fa 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -21,6 +21,7 @@ * @refs.longterm: set to true for a long term reference * @refs: array of references * @cur_pic_order_count: picture order count of the frame being decoded + * @cur_pic_fields: fields present in the frame being decoded * @unordered_reflist: unordered list of references. Will be used to generate * ordered P/B0/B1 lists * @num_valid: number of valid references in the refs array @@ -36,7 +37,10 @@ struct v4l2_h264_reflist_builder { u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; + s32 cur_pic_order_count; + u8 cur_pic_fields; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; From patchwork Thu Mar 31 19:37:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10E53C433F5 for ; Thu, 31 Mar 2022 19:38:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240854AbiCaTkE (ORCPT ); Thu, 31 Mar 2022 15:40:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240849AbiCaTjm (ORCPT ); Thu, 31 Mar 2022 15:39:42 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C87123D477; Thu, 31 Mar 2022 12:37:48 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 9836A1F47251 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755467; bh=es6kMBJ8LPORvT6E2TfIJklYinEGihlPlvjF8xMQCJ0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TA6pAi7lKPceVbWIaTog0bZFid73sxywGren5s6+Ipi3zjAXuKeU6llQVM1RN3On5 wxSyqnwjG96YXCYcltX8rn/TvazKhwz/G0opT8mlr5J5iZLVjacQZXJCtl6+brHQm4 mzYH4OVfhtR/tCwekS4OYkF7ISyp2HKObw0QU4hbY3jfQkE7W5XYHxdl5lAFdzcicU nzhm0W5/4W/PQ/hgYNvNtrTiZCO55L0QxcA0QW3aiN87bwl44tYx6M7OBI/SsvYPbS 9Ph6q+2Rc/EeEosf+57vAC/Xx2BNCXjqNojJPoExoK+YNsFq9U0MuXKto4FtgzuFEF KQ0UuuSrAX77A== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 08/23] media: h264: Store all fields into the unordered list Date: Thu, 31 Mar 2022 15:37:10 -0400 Message-Id: <20220331193726.289559-9-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When the current picture is a field, store each field into the unordered_list and preserve both top and bottom picture order count. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 65 +++++++++++++++++++++-------- include/media/v4l2-h264.h | 6 ++- 2 files changed, 51 insertions(+), 20 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 58f18bb0afb6..38d8dbda0045 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -47,8 +47,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, } for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { - u32 pic_order_count; - if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; @@ -59,8 +57,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, /* * Handle frame_num wraparound as described in section * '8.2.4.1 Decoding process for picture numbers' of the spec. - * TODO: This logic will have to be adjusted when we start - * supporting interlaced content. * For long term references, frame_num is set to * long_term_frame_idx which requires no wrapping. */ @@ -70,17 +66,33 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, else b->refs[i].frame_num = dpb[i].frame_num; - if (dpb[i].fields == V4L2_H264_FRAME_REF) - pic_order_count = min(dpb[i].top_field_order_cnt, - dpb[i].bottom_field_order_cnt); - else if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) - pic_order_count = dpb[i].bottom_field_order_cnt; - else - pic_order_count = dpb[i].top_field_order_cnt; + b->refs[i].top_field_order_cnt = dpb[i].top_field_order_cnt; + b->refs[i].bottom_field_order_cnt = dpb[i].bottom_field_order_cnt; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + u8 fields = V4L2_H264_FRAME_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + continue; + } + + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) { + u8 fields = V4L2_H264_TOP_FIELD_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + } - b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid].index = i; - b->num_valid++; + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) { + u8 fields = V4L2_H264_BOTTOM_FIELD_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + } } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) @@ -88,6 +100,23 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); +static s32 v4l2_h264_get_poc(const struct v4l2_h264_reflist_builder *b, + const struct v4l2_h264_reference *ref) +{ + switch (ref->fields) { + case V4L2_H264_FRAME_REF: + return min(b->refs[ref->index].top_field_order_cnt, + b->refs[ref->index].bottom_field_order_cnt); + case V4L2_H264_TOP_FIELD_REF: + return b->refs[ref->index].top_field_order_cnt; + case V4L2_H264_BOTTOM_FIELD_REF: + return b->refs[ref->index].bottom_field_order_cnt; + } + + /* not reached */ + return 0; +} + static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const void *data) { @@ -150,8 +179,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, builder->refs[idxb].pic_num ? -1 : 1; - poca = builder->refs[idxa].pic_order_count; - pocb = builder->refs[idxb].pic_order_count; + poca = v4l2_h264_get_poc(builder, ptra); + pocb = v4l2_h264_get_poc(builder, ptrb); /* * Short term pics with POC < cur POC first in POC descending order @@ -195,8 +224,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, builder->refs[idxb].pic_num ? -1 : 1; - poca = builder->refs[idxa].pic_order_count; - pocb = builder->refs[idxb].pic_order_count; + poca = v4l2_h264_get_poc(builder, ptra); + pocb = v4l2_h264_get_poc(builder, ptrb); /* * Short term pics with POC > cur POC first in POC ascending order diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index e165a54c68fa..4cef717b3f18 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -15,7 +15,8 @@ /** * struct v4l2_h264_reflist_builder - Reference list builder object * - * @refs.pic_order_count: reference picture order count + * @refs.top_field_order_cnt: top field order count + * @refs.bottom_field_order_cnt: bottom field order count * @refs.frame_num: reference frame number * @refs.pic_num: reference picture number * @refs.longterm: set to true for a long term reference @@ -32,7 +33,8 @@ */ struct v4l2_h264_reflist_builder { struct { - s32 pic_order_count; + s32 top_field_order_cnt; + s32 bottom_field_order_cnt; int frame_num; u32 pic_num; u16 longterm : 1; From patchwork Thu Mar 31 19:37:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BDAEC433EF for ; Thu, 31 Mar 2022 19:38:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240943AbiCaTjy (ORCPT ); Thu, 31 Mar 2022 15:39:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240876AbiCaTjv (ORCPT ); Thu, 31 Mar 2022 15:39:51 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C25423D5A7; Thu, 31 Mar 2022 12:37:49 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id EDC501F47255 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755468; bh=CCCLzz5lUZJ9ab0CExOOh36ojMkZJq6/0fFDYiktq9Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GfK+mou3ezaFhxxu8/w8QbXkSgyKRVrLhN8TxjPchHNoIzz8UPuu3BOrImGYWjp5+ AVkO7OlfcbSm4bLrpRdYp239qZWS8qPgVLBwDYB8D4jQg7l2HZSLgqbiFsDgJ35VdJ Dmhev3I0nQq4tL4A4f0rVlKuskyPGproDz/lCkZQ8P9Xjpzo/g7h4+TBiE18t7UizI k6eX8P1BhtbLKt5L+PJ2876KCQsIuBwwrfmwx/2K6mt6LPQdPFFFMPYLVaWHzLoWoc vwoLJxuVO0RgJdSiCiGyD5Cu8ieL7vcteIvN2Vs4f+LQ1ujCm1CIA0qLzfLDCG4u15 soQpkjVpVqFIg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 09/23] media: v4l2: Trace calculated p/b0/b1 initial reflist Date: Thu, 31 Mar 2022 15:37:11 -0400 Message-Id: <20220331193726.289559-10-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add debug print statements to print the content of P & B reference lists, to verify that the ordering of the generated reference lists is correct. This is especially important for the field decoding mode, where sorting is more complex. Signed-off-by: Nicolas Dufresne Tested-by: Sebastian Fricke Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 38d8dbda0045..bcf9b7774560 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +static char ref_type_to_char (u8 ref_type) +{ + switch (ref_type) { + case V4L2_H264_FRAME_REF: + return 'f'; + case V4L2_H264_TOP_FIELD_REF: + return 't'; + case V4L2_H264_BOTTOM_FIELD_REF: + return 'b'; + } + + return '?'; +} + +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char *out_str, const int len) +{ + int n = 0, i; + + n += snprintf(out_str + n, len - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + /* this is pic_num for frame and frame_num (wrapped) for field, + * but for frame pic_num is equal to frame_num (wrapped). + */ + int frame_num = builder->refs[reflist[i].index].frame_num; + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(out_str + n, len - n, "%i%c%c|", + frame_num, longterm ? 'l' : 's', + ref_type_to_char (reflist[i].fields)); + } + + return out_str; +} + +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist) +{ + char buf[1024]; + + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n", + builder->cur_pic_order_count, + ref_type_to_char(builder->cur_pic_fields), + format_ref_list_p(builder, reflist, buf, sizeof(buf))); +} + +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char *out_str, const int len) +{ + int n = 0, i; + + n += snprintf(out_str + n, len - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + int frame_num = builder->refs[reflist[i].index].frame_num; + u32 poc = v4l2_h264_get_poc(builder, reflist + i); + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(out_str + n, len - n, "%i%c%c|", + longterm ? frame_num : poc, + longterm ? 'l' : 's', + ref_type_to_char(reflist[i].fields)); + } + + return out_str; +} + +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, u8 list_num) +{ + char buf[1024]; + + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s", + list_num, builder->cur_pic_order_count, + ref_type_to_char (builder->cur_pic_fields), + format_ref_list_b(builder, reflist, buf, sizeof(buf))); +} + /** * v4l2_h264_build_p_ref_list() - Build the P reference list * @@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sizeof(builder->unordered_reflist[0]) * builder->num_valid); sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); + + print_ref_list_b(builder, b0_reflist, 0); + print_ref_list_b(builder, b1_reflist, 1); } EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists); From patchwork Thu Mar 31 19:37:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797642 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44B64C43217 for ; Thu, 31 Mar 2022 19:38:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240977AbiCaTjz (ORCPT ); Thu, 31 Mar 2022 15:39:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240882AbiCaTjw (ORCPT ); Thu, 31 Mar 2022 15:39:52 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 447C223D47F; Thu, 31 Mar 2022 12:37:51 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 5D5281F47259 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755470; bh=vFiboGbkkLNbdZn5zzoPbZK4i0E+JBLCLUcpwqULJmM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hnpstaFXXxNbuSgYUwq0ocufEaPGPNGgWmRvN2gfIxU1xF6h07+wQ5fBYvN37Oihs vzPkDrjawQEtSS66RhDHJz0Ybeub3i837Rb/mOKq/v/CTkjmdu8pYDY9qnIeL9QILh DqMVOyTEpBxmPhgpBBCoGd151fDri1A7Ha15QpBbyHBv2ffRjgcaLvjlnFjPWfDwtg IqWKmfnO3gG9vybMWIsREzjFQt+1lyxmpEqiIqFUow4cza0m65xNirQfDrl3taFOHj CcbCmOQiDii8shRh74pMNVRtqfEmlJknWiAi1S3O3M1w/8HfHwyztHRuahmj7gTx19 au5714raBFmig== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 10/23] media: h264: Sort p/b reflist using frame_num Date: Thu, 31 Mar 2022 15:37:12 -0400 Message-Id: <20220331193726.289559-11-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In the reference list builder, frame_num refers to FrameNumWrap in the spec, which is the same as the pic_num for frame decoding. The same applies for long_term_pic_num and long_term_frame_idx. Sort all type of references by frame_num so the sort can be reused for fields reflist were the sorting is done using frame_num instead. In short, pic_num is never actually used for building reference lists. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 23 +++++++++++++---------- include/media/v4l2-h264.h | 2 -- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index bcf9b7774560..7e1eba03099a 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -50,7 +50,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - b->refs[i].pic_num = dpb[i].pic_num; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) b->refs[i].longterm = true; @@ -139,15 +138,19 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, } /* - * Short term pics in descending pic num order, long term ones in - * ascending order. + * For frames, short term pics are in descending pic num order and long + * term ones in ascending order. For fields, the same direction is used + * but with frame_num (wrapped). For frames, the value of pic_num and + * frame_num are the same (see formula (8-28) and (8-29)). For this + * reason we can use frame_num only and share this funciton between + * frames and fields reflist. */ if (!builder->refs[idxa].longterm) return builder->refs[idxb].frame_num < builder->refs[idxa].frame_num ? -1 : 1; - return builder->refs[idxa].pic_num < builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < builder->refs[idxb].frame_num ? -1 : 1; } @@ -173,10 +176,10 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, return 1; } - /* Long term pics in ascending pic num order. */ + /* Long term pics in ascending frame num order. */ if (builder->refs[idxa].longterm) - return builder->refs[idxa].pic_num < - builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < + builder->refs[idxb].frame_num ? -1 : 1; poca = v4l2_h264_get_poc(builder, ptra); @@ -218,10 +221,10 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return 1; } - /* Long term pics in ascending pic num order. */ + /* Long term pics in ascending frame num order. */ if (builder->refs[idxa].longterm) - return builder->refs[idxa].pic_num < - builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < + builder->refs[idxb].frame_num ? -1 : 1; poca = v4l2_h264_get_poc(builder, ptra); diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4cef717b3f18..0d9eaa956123 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -18,7 +18,6 @@ * @refs.top_field_order_cnt: top field order count * @refs.bottom_field_order_cnt: bottom field order count * @refs.frame_num: reference frame number - * @refs.pic_num: reference picture number * @refs.longterm: set to true for a long term reference * @refs: array of references * @cur_pic_order_count: picture order count of the frame being decoded @@ -36,7 +35,6 @@ struct v4l2_h264_reflist_builder { s32 top_field_order_cnt; s32 bottom_field_order_cnt; int frame_num; - u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; From patchwork Thu Mar 31 19:37:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9824C433F5 for ; Thu, 31 Mar 2022 19:38:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240990AbiCaTj5 (ORCPT ); Thu, 31 Mar 2022 15:39:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240897AbiCaTjw (ORCPT ); Thu, 31 Mar 2022 15:39:52 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89F3223D770; Thu, 31 Mar 2022 12:37:52 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A6A121F47253 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755471; bh=SqPD7oTR+3FuYunP0khMcdvMcGz4bOvgAm9j9vzmvSk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iv777jy1D9CAI9gb1UAFfxR2k7QirrCUKilgDl8VU9WTv+FByMkFfG5Ua4MFWhsU5 YIcdCPJVM1KT7RTtBvuPriBFyRWCt4ldLG7HF/rCh/Opjtuz/5iFKnfoU+Ug1aBTps 3IQXnC0bPymS3XjP0Z9kkPLaVrLcWASjASJ19lZwTyJjs62ZLlJbJr1UZ+0HeUHJ33 QcBUZf3My8+xWSH0UCh1z2dAfrehA1KnPAVK20mYreggE5bl8QTW722Ph9n0SM+VeS ayvPRtQvJNqe7QGbh0TUS+2uWMKjA46XMmBCP8KrWvhmZpozHjRQ/EB9VRBXrg1UYb LoFeblkB9CqVg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 11/23] media: v4l2: Reorder field reflist Date: Thu, 31 Mar 2022 15:37:13 -0400 Message-Id: <20220331193726.289559-12-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As per spec, the field reflist requires interleaving top and bottom field in a specific way that does not fit inside the sort operation. The process consist of alternating references parity, starting with a reference of the same parity as the current picture. This processs is done twice, once for short term references and a second time for the long term references. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 7e1eba03099a..d2d1f3bed7e8 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -244,6 +244,40 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +/* + * The references need to be reordered so that references are alternating + * between top and bottom field references starting with the current picture + * parity. This has to be * done for short term and long term references + * separately. + */ +static void reorder_field_reflist(const struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + struct v4l2_h264_reference tmplist[V4L2_H264_REF_LIST_LEN]; + u8 lt, i = 0, j = 0, k = 0; + + memcpy(tmplist, reflist, sizeof(tmplist[0]) * b->num_valid); + + for (lt = 0; lt <= 1; lt++) { + do { + for (; i < b->num_valid && b->refs[tmplist[i].index].longterm == lt; i++) { + if (tmplist[i].fields == b->cur_pic_fields) { + reflist[k++] = tmplist[i++]; + break; + } + } + + for (; j < b->num_valid && b->refs[tmplist[j].index].longterm == lt; j++) { + if (tmplist[j].fields != b->cur_pic_fields) { + reflist[k++] = tmplist[j++]; + break; + } + } + } while ((i < b->num_valid && b->refs[tmplist[i].index].longterm == lt) || + (j < b->num_valid && b->refs[tmplist[j].index].longterm == lt)); + } +} + static char ref_type_to_char (u8 ref_type) { switch (ref_type) { @@ -346,6 +380,9 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) + reorder_field_reflist(builder, reflist); + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -379,6 +416,11 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, sort_r(b1_reflist, builder->num_valid, sizeof(*b1_reflist), v4l2_h264_b1_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) { + reorder_field_reflist(builder, b0_reflist); + reorder_field_reflist(builder, b1_reflist); + } + if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); From patchwork Thu Mar 31 19:37:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8D17C433FE for ; Thu, 31 Mar 2022 19:38:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240747AbiCaTkC (ORCPT ); Thu, 31 Mar 2022 15:40:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240905AbiCaTjw (ORCPT ); Thu, 31 Mar 2022 15:39:52 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4859223DE96; Thu, 31 Mar 2022 12:37:54 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E968B1F47255 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755473; bh=jjxmbWJ+OUjcuFBbFVuZBkezVddHPzPh9O3QIqvsczU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RwGxqDjb0z4nPRaKXOOXYxg8RTqfoEsVbLNoGTKoOzPLv3hNGVnc6HLxlP8mSG/at AG+QmqMCODgEoABnKum4nYnaByGjFVNYW9dBw7IJgMjmfBhD0NhgMaJ8pAlPcT82zP Lz6kwWHhO7YdLQTjpqVWuhsRwi8Rr6Kjs+wCzmoD0UcTv4MHn5PhDy+oDM6nGn0+V/ 7OmMUW8v0byfOjG9iiJxlykixdc7+5gDkU7MslxNYsodoOqffxM6OlJAO1R0yAIqbt jSXOE3Vpj9rhLG/RDjJMIHnxY/W5m2zWng9BLDkBqpkfSBD3ZApYKpfDusZaPnYtcs hjn1fjNYR0I2A== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 12/23] media: rkvdec: Stop overclocking the decoder Date: Thu, 31 Mar 2022 15:37:14 -0400 Message-Id: <20220331193726.289559-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org While this overclock hack seems to work on some implementations (some ChromeBooks, RockPi4) it also causes instability on other implementations (notably LibreComputer Renegade, but there were more reports in the LibreELEC project, where this has been removed). While performance is indeed affected (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed used to synchronously decode each frames, so this hack was simply compensating for their code being less efficient. In my opinion, this hack should not have been included upstream. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Thu Mar 31 19:37:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797644 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 324EAC433EF for ; Thu, 31 Mar 2022 19:38:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240916AbiCaTkC (ORCPT ); Thu, 31 Mar 2022 15:40:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240910AbiCaTjx (ORCPT ); Thu, 31 Mar 2022 15:39:53 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4E4C239324; Thu, 31 Mar 2022 12:37:55 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A363D1F47259 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755474; bh=3kRlflMZKylSksBlTGyBHGjQWwONuPlHkkyiaE6iWjY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mBFH7UXO/BWTgn0mge7OZnNiQGKEOEaL6P/pQiX4Ig4krhPmp8vindah+nakwuOEm g+w5iw0mm/dh9cG9ldqAZkBPPHwfVAHf7eghaXlkbDB6P9WzjNNvhoYlIYATxTyi// yhDiFcfMo4iNJQxxm2trUN8PC3/l1iP/Xhw2Duv345c9/sMRSg2T34tFM0w7CgIEeF guIual9HLdo8qKc2EEpkdNbPxMhPeGJk6GPHvXfvEzkfI7+jhlVQ9fh2XOEh+wj11x W2OX0h/f8jGe+PUhf2bK2h490khW4JkfIbzcWNqsNXNdNRRiZw1EnafTLN76t5F6sv JFZMjSWr1QZwg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 13/23] media: rkvdec: h264: Fix dpb_valid implementation Date: Thu, 31 Mar 2022 15:37:15 -0400 Message-Id: <20220331193726.289559-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ref builder only provided references that are marked as valid in the dpb. Thus the current implementation of dpb_valid would always set the flag to 1. This is not representing missing frames (this is called 'non-existing' pictures in the spec). In some context, these non-existing pictures still need to occupy a slot in the reference list according to the spec. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index dff89732ddd0..bcde37d72244 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -112,6 +112,7 @@ struct rkvdec_h264_run { const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES]; }; struct rkvdec_h264_ctx { @@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, } } +static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, + struct rkvdec_h264_run *run) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) { + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; + struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; + int buf_idx = -1; + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + buf_idx = vb2_find_timestamp(cap_q, + dpb[i].reference_ts, 0); + + run->ref_buf_idx[i] = buf_idx; + } +} + static void assemble_hw_rps(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { @@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = 0; + bool dpb_valid = run->ref_buf_idx[i] >= 0; u8 idx = 0; switch (j) { @@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (idx >= ARRAY_SIZE(dec_params->dpb)) continue; - dpb_valid = !!(dpb[idx].flags & - V4L2_H264_DPB_ENTRY_FLAG_ACTIVE); set_ps_field(hw_rps, DPB_INFO(i, j), idx | dpb_valid << 4); @@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run, unsigned int dpb_idx) { struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; - const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; - int buf_idx = -1; - - if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - buf_idx = vb2_find_timestamp(cap_q, - dpb[dpb_idx].reference_ts, 0); + int buf_idx = run->ref_buf_idx[dpb_idx]; /* * If a DPB entry is unused or invalid, address of current destination @@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); + lookup_ref_buf_idx(ctx, &run); assemble_hw_rps(ctx, &run); config_registers(ctx, &run); From patchwork Thu Mar 31 19:37:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C0ABC433EF for ; Thu, 31 Mar 2022 19:38:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241021AbiCaTkG (ORCPT ); Thu, 31 Mar 2022 15:40:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240933AbiCaTjy (ORCPT ); Thu, 31 Mar 2022 15:39:54 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD03423F3A6; Thu, 31 Mar 2022 12:37:57 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 538C01F4725C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755476; bh=s3TCwg9L0AAOtv/6Hb2IdzZi6+nwv8qK9bwTi/Shc+k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jfGbAELsD56jkeFy74WurVTn19hU0NwWH0mqm7jGdhva3aRlObO+CSaQ05wC3Hl9b IJQP7wTfoGK5IW9uOdJIaRJ+a/Sza1KTG7p7mjnINjGQY8W9oPoW60r3ggaWLSGvc2 7I1uswmzE5ZPvQ5z0eFTKiqmH73zPPp02kgpQ2MsIVe7XC8WEhSijExUFXdxuOrfHO aFPDih9mmHf5LouncQ1S0Bktk5Pcg5ivSp6CWHMCpIr/GKuznpcvrAb7Gl0nXrb9KL HpAWLFCO7LniNhwiGb9LtS6yp8pTqgzEKDczRcS1fCLZ/uuGIb4FmNuO5sgnfTU6N7 9rRK4jd5T4Vnw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Ezequiel Garcia , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 14/23] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Thu, 31 Mar 2022 15:37:16 -0400 Message-Id: <20220331193726.289559-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet are 3 bits wide. 8 is wrongly added to the bit depth values written to these 3 bit fields. Because only the 3 LSB are written, the hardware was configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index bcde37d72244..8d44a884a52e 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Thu Mar 31 19:37:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797648 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E18E3C433FE for ; Thu, 31 Mar 2022 19:38:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241049AbiCaTkP (ORCPT ); Thu, 31 Mar 2022 15:40:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240887AbiCaTjy (ORCPT ); Thu, 31 Mar 2022 15:39:54 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7BC623F3ED; Thu, 31 Mar 2022 12:37:59 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 422C41F4725B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755478; bh=n6MPrL7fQ9yMzjnP11oTkFqivourvSLa0aWHcN/kENg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jkEylmlesJkAj/v1WszIOWWnTqVvM5yv/nbStUOKJ7IsZuMzohiE12OqgETjnrjws 0sO7cTfSLluqql9ZvlMIAG+O+mLQsHVOh+A30WQ6sz0JXqICdTNXwM1SIQSV8pnVeU hZCtn5pSTa6++6wSEcsyyFGzt9xjp+qEtTo9syicqb4eBoI+9gSIKIb4ubcGmTqWWe SOhW1vW9cWFx90iSef9jrGTYd6+yV826nWnRkxVRcd6sxXBb/Z/14wSZUAe/32v6bz p6PTyc9uUEbB9bpyK+Gth3D10QNCf5b2LXC56dsKWaBQ1IiMpzmpXj642yOyrynz3V XbUiJSobUUpZg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 15/23] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Thu, 31 Mar 2022 15:37:17 -0400 Message-Id: <20220331193726.289559-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The width and height in macroblocks is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++++++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 8d44a884a52e..a42cf19bcc6d 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2df8cf4883e2..1b805710e195 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,8 +29,11 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; + unsigned int width, height; /* * TODO: The hardware supports 10-bit and 4:2:2 profiles, * but it's currently broken in the driver. @@ -45,6 +48,13 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) if (sps->bit_depth_luma_minus8 != 0) /* Only 8-bit is supported */ return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; } return 0; } From patchwork Thu Mar 31 19:37:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A00BC433EF for ; Thu, 31 Mar 2022 19:38:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241074AbiCaTkW (ORCPT ); Thu, 31 Mar 2022 15:40:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240889AbiCaTjz (ORCPT ); Thu, 31 Mar 2022 15:39:55 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 883CA23FF01; Thu, 31 Mar 2022 12:38:01 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 08D9D1F4725F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755480; bh=HmMBg5b8YH4LzW0sBzAl4u/8f+BCfQenxWj/fruvkP8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KbVTEhI17KkBlxrzHM4EuGkf7QIYl4hZ/hOfcIcrzBe13cx9Xk5TLnQilPRIkCk/u +RAY0swuwiXIxoC7+k0vRmZm036mvACh+quaET/cKJOI1pwgQV9em1k9c0K7ymuKsf /Q7biHye8EnmdIsduVy61i1tLrDezx2l9quW2m4+1MKZOo/JAiSUAXxQnx3emifXBL 8COXL3t4Q7KArtzscrvkyquK+0+O2W+4FznSZTULfF5040+lllV7hyjwDLsz/z77P2 O3gWwpXQDdPv8zkLNX/NAufkBj5R0d8RXWFUa8vfYn7pR9ZDFaRx+fBm4Bc5cxyEGx fuwF0A10DIMuA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Ezequiel Garcia , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 16/23] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Thu, 31 Mar 2022 15:37:18 -0400 Message-Id: <20220331193726.289559-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this cause issues to decode the second field in a complementary field pair. Fix this by using inclusive comparison, less than or equal. Signed-off-by: Jonas Karlman Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index a42cf19bcc6d..730f8ebf7f58 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -773,7 +773,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Thu Mar 31 19:37:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD758C433EF for ; Thu, 31 Mar 2022 19:38:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241099AbiCaTkY (ORCPT ); Thu, 31 Mar 2022 15:40:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240973AbiCaTjz (ORCPT ); Thu, 31 Mar 2022 15:39:55 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52546240581; Thu, 31 Mar 2022 12:38:03 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 006D21F47261 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755482; bh=L/IEg8/gRB+ksIPpqGEbacLI5hKb4dkPkUIdTUf313k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W/QXCvRIJgWwkAyOdtpt3s+KdDjWzN0fnPFot3fGjyCOCG30P5KLMKziHPG2YxxPu DCoIhPqN3ATybHM3LGenjmN6zIDzeqdiPnoipRBZ/XkCZKVB2azOdDKNRk3MjVN99i CVw8tH6n6PiMGaBeQOpGOtIGhigO8Ec5KFKEk//TOgFm99TVrRqxl1wB4+W4Xb01JF g85nox2wUHoGKIVDmso4qQPZJcCvzKO85vJmaLJRxLuNvbSXjg3f6r03C/Ns/7edvC daZGOIkiSguY/SqYhgNZ7N18UdZ97Y0Kso0OcjYhay8Wz7oC36QCaQnt8y7+bk83OK 8r1MuPnkntV7Q== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 17/23] media: rkvdec: Enable capture buffer holding for H264 Date: Thu, 31 Mar 2022 15:37:19 -0400 Message-Id: <20220331193726.289559-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In order to support interlaced video decoding, the driver must allow holding the capture buffer so that the second field can be decoded into it. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 4 ++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 1b805710e195..b6376eaa92d7 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -148,6 +148,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { .ops = &rkvdec_h264_fmt_ops, .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts), .decoded_fmts = rkvdec_h264_vp9_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, }, { .fourcc = V4L2_PIX_FMT_VP9_FRAME, @@ -404,6 +405,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + /* Enable format specific queue features */ + vq->subsystem_flags |= desc->subsystem_flags; + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2f4ea1786b93..e37f1a015fa0 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -81,6 +81,7 @@ struct rkvdec_coded_fmt_desc { const struct rkvdec_coded_fmt_ops *ops; unsigned int num_decoded_fmts; const u32 *decoded_fmts; + u32 subsystem_flags; }; struct rkvdec_dev { From patchwork Thu Mar 31 19:37:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C07EBC4332F for ; Thu, 31 Mar 2022 19:38:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241000AbiCaTkZ (ORCPT ); Thu, 31 Mar 2022 15:40:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240982AbiCaTj4 (ORCPT ); Thu, 31 Mar 2022 15:39:56 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EE0D23D46B; Thu, 31 Mar 2022 12:38:04 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A94421F47263 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755483; bh=77EXBZHZKJfw+bMSetijgv88NZsVwqAJYYAOhg0Ep10=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nTv+anF6T6CZSDOWJpi/e7JMPAiQ+OiqDq+zoXcivpfWj7RiIAT/QPwgP5BPFCfv5 gsIO6FqhhxY2Ww7s8nunjbRmJ7/l517nuWJls4QZgKjjY0P/hRvGxLO/F7Te92Lst6 FWM6R7qfFqcEtMT2V1TzcnkeNbQHvtGGd35gaKbqaGQlZgehqCyFaENvVuQLvYE5RI sEx0D+LS5k7zGfAk96yg3bm//7/ufqz6jkvHRMxAezteFD/K3ZJc97vMbwMXsC2TmH awsoiK0OybnrqBRvZLR/Wbl8SNZ7MUkVd79MskUJ7f0OjxV4xGxeg/3iiPepvnZ0mG UihnBY7ifC5cQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 18/23] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Thu, 31 Mar 2022 15:37:20 -0400 Message-Id: <20220331193726.289559-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OUTPUT buffer resolution. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index b6376eaa92d7..21b6e7dc5181 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -279,6 +279,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Thu Mar 31 19:37:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797652 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5A39C433EF for ; Thu, 31 Mar 2022 19:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240920AbiCaTk1 (ORCPT ); Thu, 31 Mar 2022 15:40:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240995AbiCaTj5 (ORCPT ); Thu, 31 Mar 2022 15:39:57 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E3E523D5A7; Thu, 31 Mar 2022 12:38:06 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 8BB5D1F47265 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755485; bh=tJkYBPUI53S8Pz6DtM2kRVN+d+Rk5Laqr934vgpA+2I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cKJkcilysUe2ZqBlmd0qmiPyaByTjRoOqgM8QM6waeOnmKgNPBJ49UC8Eb7vp43nf XwqKXb6Dh/fbUplGRaEP7PI7Xragk94lRZvW5hSdghupE3qcMeDI+XV/X9gMVq+oWU 0Zo+XxjEymadF53qEha91MRqLjsakUc1UhXZMnV6l6MbeL5tIzgx0j6ofyxTyW7rs7 cVvPNaTlMbPVBuPfoeDinFZCOWwTwMprbic7m3MxPmEenTjuta56sLmGdidW1+UyKq kdKU/OpiEvRcGE5cGrtM8IRV5PLl5abX2Of+tXpHm1QLj4h84CqKd0QpRN7HiFyTl7 NyYZNV3cIbfvA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 19/23] media: rkvdec-h264: Add field decoding support Date: Thu, 31 Mar 2022 15:37:21 -0400 Message-Id: <20220331193726.289559-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This makes use of the new feature in the reference builder to program up to 32 references when doing field decoding. It also signals the parity (top or bottom) of the field to the hardware. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------ 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 730f8ebf7f58..d50da72933bc 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl { u8 err_info[RKV_ERROR_INFO_SIZE]; }; -#define RKVDEC_H264_DPB_SIZE 16 - struct rkvdec_h264_reflists { struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; - u8 num_valid; }; struct rkvdec_h264_run { @@ -738,23 +735,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; int buf_idx = -1; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) { buf_idx = vb2_find_timestamp(cap_q, dpb[i].reference_ts, 0); + if (buf_idx < 0) + pr_debug("No buffer for reference_ts %llu", + dpb[i].reference_ts); + } run->ref_buf_idx[i] = buf_idx; } } static void assemble_hw_rps(struct rkvdec_ctx *ctx, + struct v4l2_h264_reflist_builder *builder, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; - const struct v4l2_ctrl_h264_sps *sps = run->sps; struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu; - u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4); u32 *hw_rps = priv_tbl->rps; u32 i, j; @@ -772,37 +772,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num <= dec_params->frame_num) { - p[i] = dpb[i].frame_num; - continue; - } - - p[i] = dpb[i].frame_num - max_frame_num; + p[i] = builder->refs[i].frame_num; } for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { - for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - bool dpb_valid = run->ref_buf_idx[i] >= 0; - u8 idx = 0; + for (i = 0; i < builder->num_valid; i++) { + struct v4l2_h264_reference *ref; + bool dpb_valid; + bool bottom; switch (j) { case 0: - idx = h264_ctx->reflists.p[i].index; + ref = &h264_ctx->reflists.p[i]; break; case 1: - idx = h264_ctx->reflists.b0[i].index; + ref = &h264_ctx->reflists.b0[i]; break; case 2: - idx = h264_ctx->reflists.b1[i].index; + ref = &h264_ctx->reflists.b1[i]; break; } - if (idx >= ARRAY_SIZE(dec_params->dpb)) + if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb))) continue; + dpb_valid = run->ref_buf_idx[ref->index] >= 0; + bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF; + set_ps_field(hw_rps, DPB_INFO(i, j), - idx | dpb_valid << 4); + ref->index | dpb_valid << 4); + set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom); } } } @@ -990,10 +989,6 @@ static void config_registers(struct rkvdec_ctx *ctx, rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15); } - /* - * Since support frame mode only - * top_field_order_cnt is the same as bottom_field_order_cnt - */ reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt); writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0); @@ -1109,7 +1104,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params, run.sps, run.decode_params->dpb); - h264_ctx->reflists.num_valid = reflist_builder.num_valid; v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); @@ -1117,7 +1111,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); lookup_ref_buf_idx(ctx, &run); - assemble_hw_rps(ctx, &run); + assemble_hw_rps(ctx, &reflist_builder, &run); config_registers(ctx, &run); rkvdec_run_postamble(ctx, &run.base); From patchwork Thu Mar 31 19:37:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 777F4C433FE for ; Thu, 31 Mar 2022 19:38:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240929AbiCaTk3 (ORCPT ); Thu, 31 Mar 2022 15:40:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240919AbiCaTkF (ORCPT ); Thu, 31 Mar 2022 15:40:05 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC68823D776; Thu, 31 Mar 2022 12:38:08 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 3B01D1F47267 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755487; bh=OT93d1YFbCsPl7b/q8Q9Ng7KBwuAX7su/L0Pt2JnS6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BLW245FpFm1U2UG9Q60ps8Hy5eaR5MstdHCKZC9iaLqioT1JxP0Ik2h47Oz0cI5q+ nymV4c/TmmiMxn74T88xYmfybIfMYgJkv7o77E3jpk70cL23DhF7aRtguFzaLrnqoh dHiZNVYcZUOHsGjrnr6NITnlqQ6ObBxMZw3arQiNXg8cbPw49iMuSJwXkUE08cy0/+ wZyuoDSxWdCWvM3BZvad4t/+1W0pN6HEs7UWwzbMTzr4PkUwwp9hyRbQP1JYDxc0GM a04j+gWFgGlMgLmR/4lgoYf0zjTgy5IjANtDtQFDLV5J+K13RE4wF6N8SQaxZ8sHkO diWgQU/zMSnQQ== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 20/23] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Thu, 31 Mar 2022 15:37:22 -0400 Message-Id: <20220331193726.289559-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is needed to optimize field decoding. Each field will be decoded into the same capture buffer. To be able to queue multiple buffers, we need to be able to ask the driver to hold the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 67148ba346f5..50d636678ff3 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n", From patchwork Thu Mar 31 19:37:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BAA0C433F5 for ; Thu, 31 Mar 2022 19:38:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241012AbiCaTkc (ORCPT ); Thu, 31 Mar 2022 15:40:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241026AbiCaTkG (ORCPT ); Thu, 31 Mar 2022 15:40:06 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C103D23F9F5; Thu, 31 Mar 2022 12:38:10 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 092A01F47266 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755489; bh=rqMU1/6TJ46L9lH5r2+bP74ysDX3ywT+LO6k1I+k3Bg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A9lbdHIAvXLk56s++dH+TARP3iPn8nxqBL43DHyHw063RYlN3TCEBrYyFCaE0LiP9 rGMCV/SOgX8KwJWC0Ihv8exLfF4D8ER9ORtY2rTYGBsSItbcvqnQ5llHSkSUZVXoMS qHB12chJCWDhbw6Zk4uEgSXS/d5TG3qw8AcPV1Xms2x0QfbvlRddxzfwndEv4IRobN HbrG8GRarM6+xbaMW2Y11TsETJMU5FD6WJXUYvJZgavz2rdNrCpgElXJ5loFoJHXFM 14rhrTNPNloz9KXCVzwnj+l+7fT+t3D/KrlFV4mjrMDvLo1udPFCKEpk41R+gSXuLA qfu1iCs6k1FhQ== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 21/23] media: hantro: Stop using H.264 parameter pic_num Date: Thu, 31 Mar 2022 15:37:23 -0400 Message-Id: <20220331193726.289559-22-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The hardware expects FrameNumWrap or long_term_frame_idx. Picture numbers are per field, and are mostly used during the memory management process, which is done in userland. This fixes two ITU conformance tests: - MR6_BT_B - MR8_BT_B Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_h264.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 0b4d2491be3b..228629fb3cdf 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) return 0; - if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - return dpb->pic_num; return dpb->frame_num; } From patchwork Thu Mar 31 19:37:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E83CC433EF for ; Thu, 31 Mar 2022 19:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240949AbiCaTkj (ORCPT ); Thu, 31 Mar 2022 15:40:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241060AbiCaTkP (ORCPT ); Thu, 31 Mar 2022 15:40:15 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 413492414D1; Thu, 31 Mar 2022 12:38:12 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C572A1F47264 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755491; bh=XEgENTYjCMT2Anghn2qsNUIMLssrmzXSWUAtGOk2EIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E2gxoqJYVViah1bnCh0Ns/I6N5AWWkUGJ5pwWF1M2iGOkatQdu7HYSfXKLCPR6Vjs CIOdDSZTD8YGL/O0L0HIIUraQV/W+PS2aZqG0ebyX3gYmT0hnDmMfbqpGI4DIX6HgC +CXc0sS+dZYb/4sDd8TzDpDaTyiBaREh+L3XbxjKYDjZTHJGx8q+UWB290VIDhPwvO bqKYSgcQr5KZ7ycx3sB7ZR4Cp+0a9HHvql1YJ7trdLRBLGv6ZvzHqSXVGH3FI4qSNC kq2I4PWNBc6HPvyPsvZvKya5e2900uSShQ/0cLwh2RVgLekIsUANMF8sKfvUT1yfBK HeSQf8SQMqmMg== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 22/23] media: hantro: h264: Make dpb entry management more robust Date: Thu, 31 Mar 2022 15:37:24 -0400 Message-Id: <20220331193726.289559-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The driver maintains stable slot locations for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb; From patchwork Thu Mar 31 19:37:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12797656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB890C433F5 for ; Thu, 31 Mar 2022 19:39:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241066AbiCaTko (ORCPT ); Thu, 31 Mar 2022 15:40:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241062AbiCaTkV (ORCPT ); Thu, 31 Mar 2022 15:40:21 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B579241A06; Thu, 31 Mar 2022 12:38:14 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id B7BB71F47268 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755493; bh=QXCqhwGgS71Plij0LYpMswWc72NGt82a9mKy6xwpDRk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f3tdOpbrfmBJM9E1eieNv3lSTG8VF2jict14/WplVjzSJWPvrdt1WljxliR/iwYJN gouvm328JQH/r5kJA06kVD4ZZuWct+FNxxAQVshBE4W9lhnd1nTQxwmQs5nlmDzXO/ yILZ0nAmysFMKoG3LgfSI3fYs84oTRdx1IfrnSafmC1hKX+J0fTesRCckW2C9UelpW 555/F3ldoPVZHM3cZxnibOaUc3pxF1USjL8EXbh77casUQ67cVyWEbXaIosgPN+CQg 70notWdfPUeDqt0ROnEIBqGBaCvI6gG+7btIN6l/vMRPDIWd2i+/rmBeiKIeun25Qw CqjzRYpRU9f3w== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 23/23] media: hantro: Add H.264 field decoding support Date: Thu, 31 Mar 2022 15:37:25 -0400 Message-Id: <20220331193726.289559-24-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This adds the required code to support field decoding. While most of the code is derived from Rockchip and VSI reference code, the reduction of the reference list to 16 entries was found by trial and errors. The list consists of all the references with the opposite field parity. The strategy is to deduplicate the reference picture that points to the same storage (same index). The choice of opposite parity has been made to keep the other field of the current field pair in the list. This method may not be robust if a field was lost. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++--- drivers/staging/media/hantro/hantro_hw.h | 1 + 2 files changed, 94 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 7377fc26f780..f6fc939aa726 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -22,6 +22,11 @@ #define POC_BUFFER_SIZE 34 #define SCALING_LIST_SIZE (6 * 16 + 2 * 64) +/* For valid and long term reference marking, index are reversed, so bit 31 + * indicates the status of the picture 0. + */ +#define REF_BIT(i) BIT(32 - 1 - (i)) + /* Data structure describing auxiliary buffer format. */ struct hantro_h264_dec_priv_tbl { u32 cabac_table[CABAC_INIT_BUFFER_SIZE]; @@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx) { const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls; const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode; + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps; struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu; const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; u32 dpb_longterm = 0; @@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx) tbl->poc[i * 2] = dpb[i].top_field_order_cnt; tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt; + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + /* * Set up bit maps of valid and long term DPBs. - * NOTE: The bits are reversed, i.e. MSb is DPB 0. + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame + * decoding, bit 31 to 15 are used, while for field decoding, + * all bits are used, with bit 31 being a top field, 30 a bottom + * field and so on. */ - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) { + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) + dpb_valid |= REF_BIT(i * 2); + + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) + dpb_valid |= REF_BIT(i * 2 + 1); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) { + dpb_longterm |= REF_BIT(i * 2); + dpb_longterm |= REF_BIT(i * 2 + 1); + } + } else { + dpb_valid |= REF_BIT(i); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) + dpb_longterm |= REF_BIT(i); + } + } + ctx->h264_dec.dpb_valid = dpb_valid; + ctx->h264_dec.dpb_longterm = dpb_longterm; + + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) || + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) { + tbl->poc[32] = ctx->h264_dec.cur_poc; + tbl->poc[33] = 0; + } else { + tbl->poc[32] = dec_param->top_field_order_cnt; + tbl->poc[33] = dec_param->bottom_field_order_cnt; } - ctx->h264_dec.dpb_valid = dpb_valid << 16; - ctx->h264_dec.dpb_longterm = dpb_longterm << 16; - - tbl->poc[32] = dec_param->top_field_order_cnt; - tbl->poc[33] = dec_param->bottom_field_order_cnt; assemble_scaling_list(ctx); } @@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, { struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; dma_addr_t dma_addr = 0; + s32 cur_poc = ctx->h264_dec.cur_poc; + u32 flags; if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts); @@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, dma_addr = hantro_get_dec_buf_addr(ctx, buf); } - return dma_addr; + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0; + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) < + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ? + 0x1 : 0; + + return dma_addr | flags; } u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) @@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) return dpb->frame_num; } +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + int write_idx = 0; + int i; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + write_idx = b->num_valid; + goto done; + } + + for (i = 0; i < b->num_valid; i++) { + if (!(b->cur_pic_fields == reflist[i].fields)) { + reflist[write_idx++] = reflist[i]; + continue; + } + } + +done: + /* Should not happen unless we have a bug in the reflist builder. */ + if (WARN_ON(write_idx > 16)) + write_idx = 16; + + /* Clear the remaining, some streams fails otherwise */ + for (; write_idx < 16; write_idx++) + reflist[write_idx].index = 15; +} + int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) { struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec; @@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) /* Update the DPB with new refs. */ update_dpb(ctx); - /* Prepare data in memory. */ - prepare_table(ctx); - /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode, ctrls->sps, ctx->h264_dec.dpb); + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count; + + /* Prepare data in memory. */ + prepare_table(ctx); + v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); + + /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce + * the actual picture lists in field through the dpb_valid, + * dpb_longterm bitmap along with the current frame parity. + */ + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) { + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1); + } + return 0; } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 292aaaabaf24..fd869369fb97 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx { struct hantro_h264_dec_ctrls ctrls; u32 dpb_longterm; u32 dpb_valid; + s32 cur_poc; }; /**