From patchwork Tue Apr 26 12:57:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827056 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40615C433F5 for ; Tue, 26 Apr 2022 12:58:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350329AbiDZNBP (ORCPT ); Tue, 26 Apr 2022 09:01:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349620AbiDZNBJ (ORCPT ); Tue, 26 Apr 2022 09:01:09 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BDEAE177D42; Tue, 26 Apr 2022 05:58:01 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E17851F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977880; bh=9s/jwY2vdLbXyetN/42aw11t+0XH2cY1iw+Tw9pwgjM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SD2dBeBEPnw4SkKggKXX/LgBYHxa0D6QIbxvHQ8/1wQl/7UOkD6cR0nng3rxHTK4G n7TTUOrZIfzBJtD567KrT5hA1/ysn1AD9ftnij4IGMB74FEfkJY628zjIOAiPXvc2r SP4xE6HTr9ibOgOhPGCaXkm79eH2w5A7+VjhJOwzZB8TcsGE5juocOfezPv1eOHO4o fR2Cs1deoXfdrCXcPkP86lfYe1L0gy82ntgKY65xgtPKF/ZgNRkQmxg5rlU13xEBOx KeUDK5Qba1zszzOL2QfBobU9Ah45QbrHelGSBaTW4W672qEaMl2ELZOvSxr8oKFXNC Xl/rppY7ZHQDA== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org Subject: [PATCH v4 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Date: Tue, 26 Apr 2022 08:57:27 -0400 Message-Id: <20220426125751.108293-2-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org These two fields need documentation as they have dual meaning. It is also confusing since pic_num is a derived value from frame_num, so this should help application developers. If we ever need to make a V2 of this API, I would suggest to remove pic_num entirely. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- .../media/v4l/ext-ctrls-codec-stateless.rst | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst index 6541e4c32b26..bee73065e993 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst @@ -649,10 +649,16 @@ Stateless Codec Control ID :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64. * - __u32 - ``pic_num`` - - + - For short term references, this must match the derived value PicNum + (8-28) and for long term references it must match the derived value + LongTermPicNum (8-29). When decoding frames (as opposed to fields) + pic_num is the same as FrameNumWrap. * - __u16 - ``frame_num`` - - + - For short term references, this must match the frame_num value from + the slice header syntax (the driver will wrap the value if needed). For + long term references, this must be set to the value of + long_term_frame_idx described in the dec_ref_pic_marking() syntax. * - __u8 - ``fields`` - Specifies how the DPB entry is referenced. See :ref:`Reference Fields ` From patchwork Tue Apr 26 12:57:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6580FC433F5 for ; Tue, 26 Apr 2022 12:58:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349651AbiDZNBR (ORCPT ); Tue, 26 Apr 2022 09:01:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350315AbiDZNBO (ORCPT ); Tue, 26 Apr 2022 09:01:14 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E38417E21D; Tue, 26 Apr 2022 05:58:03 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 357771F4398F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977882; bh=CR0Uld+xP65yR8nRZHFwyooNDEbFE3R+SnqsqiS63yI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YfUx0XBD1y7yy1+Nfm9IpGmZ9onovXxyqFs5+/7/NUnbcF92hMSndIS3Bw49UATy3 MiDj9S6OLVmUMPZOmD5YMFCChSuuDUnaV3nOtlmdrptn6WlGZMQUXCGVcnxi1a0QYx TjdKAhI5PkBqat5/wCZfnKT3vvvYNYMpxCnDeuKOfBldx7Djsyo1ulZybWEU4NLt1N KMMQUrp55POy6wnaFDscb7yiMUJYe6Ny68u/rDSv3qhUZIVYpPZg4jNVvQEkXv/z/a eI3NGaO5BK+yf4XWoaErYP+zpvEbtmZcjYmCoIsbCBIkd+sZdbtVBre7WihsX0yYAh wjaaZAMkzOxPA== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Ezequiel Garcia , linux-kernel@vger.kernel.org Subject: [PATCH v4 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Date: Tue, 26 Apr 2022 08:57:28 -0400 Message-Id: <20220426125751.108293-3-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org If the timestamp of the src buffer differs from the timestamp of a held dst buffer, the held buffer is implicitly removed and marked as done. Add a trace to help debugging if someone hits that case. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-mem2mem.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 675e22895ebe..d1d29d6ab024 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -336,6 +336,7 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, if (src && dst && dst->is_held && dst->vb2_buf.copied_timestamp && dst->vb2_buf.timestamp != src->vb2_buf.timestamp) { + dprintk("Timestamp mismatch, returning held capture buffer\n"); dst->is_held = false; v4l2_m2m_dst_buf_remove(m2m_ctx); v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE); From patchwork Tue Apr 26 12:57:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4612DC433F5 for ; Tue, 26 Apr 2022 12:58:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350361AbiDZNB0 (ORCPT ); Tue, 26 Apr 2022 09:01:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350322AbiDZNBP (ORCPT ); Tue, 26 Apr 2022 09:01:15 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC5C317F112; Tue, 26 Apr 2022 05:58:04 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 986A21F43991 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977883; bh=BQPoxOAZCWnxNCOeZo2D/Nd5xmmLaOzP4aXydtQmwEk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lCMq/XYI1rT//EaxIr4tzjZMIi9SfAN1J+rxbo0nX1o4jG4rFJlV21qrb532VgoPi EEausR1Wl1NSrzYd+jfzayy0oZQjE5BI7jV38EemoScgGvtutOjtfDOghZfLlOdrFD a/4lzfJCPQH8WFiU/NWsOdcSzvcXMBcGxuLi/QUOTJUzgjNzTm1kbRZ/5/Z6gp4imz Z4JHsvPFxFNCK/NvXo6GVxqx5P3XPJMDaiRVBjtiaqHoH+0YGlzIrtBfU4nyLOF0Xo KRJ+jm1IBdnAFnMDJups+8ryQ8pIHfokEdpjIqk7b4h59bl8YB0PQw7s6Q6lqWjX9q wMAE8nt1ra+QQ== From: Nicolas Dufresne To: Tomasz Figa , Marek Szyprowski , Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, Sebastian Fricke , linux-media@vger.kernel.org, Ezequiel Garcia , linux-kernel@vger.kernel.org Subject: [PATCH v4 03/24] media: videobuf2-v4l2: Warn on holding buffers without support Date: Tue, 26 Apr 2022 08:57:29 -0400 Message-Id: <20220426125751.108293-4-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Sebastian Fricke Using V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF flag without specifying the subsystem flag VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, results in silently ignoring it. Warn the user via a debug print when the flag is requested but ignored by the videobuf2 framework. Signed-off-by: Sebastian Fricke Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/media/common/videobuf2/videobuf2-v4l2.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index 6edf4508c636..812c8d1962e0 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -329,8 +329,13 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b */ vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE; vbuf->field = b->field; - if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) + if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) { + if (vbuf->flags & V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF) + dprintk(q, 1, + "Request holding buffer (%d), unsupported on output queue\n", + b->index); vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; + } } else { /* Zero any output buffer flags as this is a capture buffer */ vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS; From patchwork Tue Apr 26 12:57:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D38DC433F5 for ; Tue, 26 Apr 2022 12:58:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350368AbiDZNB0 (ORCPT ); Tue, 26 Apr 2022 09:01:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350325AbiDZNBP (ORCPT ); Tue, 26 Apr 2022 09:01:15 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E8B5177D47; Tue, 26 Apr 2022 05:58:06 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 335911F43982 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977885; bh=fNRZoSK/0DILeV16LVHwTQzonwtkLN2LiUy6JGrnOQU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fkv6n+WV8+Zvr4P0dEKD5P40Rzhb7kMRK/4rDVPQKGKHqJsnpzsyEMzxwq8KktH1z wwQHVcadl8QyVax+00Cu2Rl+wRQfeNxeXCY8uZ8zZwNp+PmhuXG9TGmFklOLFRPe0L S1bhRAvjg8j1ffbRifxo9SwaQBCE1XFBT7ShUuppUVgU1SJRtsnn+hr1SNEu11jHPr uRcPq0ko3CuNfFIDnsmCVkt47n/126jrtqZEOgQ6vxQjJXjeIUOhESata2ickcrjKa TM+Fka8HxR3fe4l85obJmuvMCSIVAZloo1pncuTTiyhwuJwWK3s9c03sbYsSsvBRaH +UTgKxu7t5CHg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Ezequiel Garcia , linux-kernel@vger.kernel.org Subject: [PATCH v4 04/24] media: h264: Avoid wrapping long_term_frame_idx Date: Tue, 26 Apr 2022 08:57:30 -0400 Message-Id: <20220426125751.108293-5-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org For long term references, frame_num is set to long_term_frame_idx which does not require wrapping. This is fixed by observation, no directly related issue have been found yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-h264.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 5633a242520a..ac47519a9fbe 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -57,8 +57,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * '8.2.4.1 Decoding process for picture numbers' of the spec. * TODO: This logic will have to be adjusted when we start * supporting interlaced content. + * For long term references, frame_num is set to + * long_term_frame_idx which requires no wrapping. */ - if (dpb[i].frame_num > cur_frame_num) + if (!b->refs[i].longterm && dpb[i].frame_num > cur_frame_num) b->refs[i].frame_num = (int)dpb[i].frame_num - max_frame_num; else From patchwork Tue Apr 26 12:57:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18CC6C433F5 for ; Tue, 26 Apr 2022 12:58:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350339AbiDZNBh (ORCPT ); Tue, 26 Apr 2022 09:01:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350244AbiDZNBS (ORCPT ); Tue, 26 Apr 2022 09:01:18 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D056417B983; Tue, 26 Apr 2022 05:58:09 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 861891F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977888; bh=6spd5zDDXQhVBNuvmuT3DFsMJlwSvquZ1zvEha0LOEI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TqWzgHDY9Ak5+G5HgVpoyVMfBXHYr75AcBGNmAOztGJoSAIQxxLMMNBvAa0VdbZg7 PwqrtSiuXgNOa0qKhke4XVyj0B+XN9fbeDhKoNd5+VuJj9dQTU3gl+YaP6iyUpszT6 gb1Mx1iQmQV1aZ7jiv8kT3p0kLqwCcgB8TWzltYbHYUWHBFvOd8ey+ziDLB8B3JJpC Zv0r99IMt0ZZ60+ovHe9BkZHnrReClurTJg3EMiBwlukxp7U//ZJQ22UNEWA1AIRjd gbJ+i5e4jv0OKEmg5nnVZBv5iseBAwAUDGRB/IA0GDDrd/KuIUal1HIikQP48oBeFq vgbDSyV8+okqw== From: Nicolas Dufresne To: Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Dmitry Osipenko , Thierry Reding , Jonathan Hunter , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman , Heiko Stuebner , Matthias Brugger Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-tegra@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org Subject: [PATCH v4 05/24] media: h264: Use v4l2_h264_reference for reflist Date: Tue, 26 Apr 2022 08:57:31 -0400 Message-Id: <20220426125751.108293-6-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In preparation for adding field decoding support, convert the byte arrays for reflist into array of struct v4l2_h264_reference. That struct will allow us to mark which field of the reference picture is being referenced. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Tested-by: Dmitry Osipenko --- .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 36 +++++-- .../media/platform/nvidia/tegra-vde/h264.c | 19 ++-- drivers/media/v4l2-core/v4l2-h264.c | 33 ++++--- .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++---- drivers/staging/media/hantro/hantro_hw.h | 6 +- .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++---------- drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +-- include/media/v4l2-h264.h | 19 ++-- 8 files changed, 143 insertions(+), 118 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c index 43542de11e9c..8e59cf5c05a0 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c @@ -439,11 +439,24 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param, } } -/* - * The firmware expects unused reflist entries to have the value 0x20. - */ -static void fixup_ref_list(u8 *ref_list, size_t num_valid) +static void get_ref_list(u8 *ref_list, + const struct v4l2_h264_reference *v4l2_ref_list, + int num_valid) { + u32 i; + + /* + * TODO The firmware does not support field decoding. Future + * implementation must use v4l2_ref_list[i].fields to obtain + * the reference field parity. + */ + + for (i = 0; i < num_valid; i++) + ref_list[i] = v4l2_ref_list[i].index; + + /* + * The firmware expects unused reflist entries to have the value 0x20. + */ memset(&ref_list[num_valid], 0x20, 32 - num_valid); } @@ -459,6 +472,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_SCALING_MATRIX); struct mtk_h264_dec_slice_param *slice_param = &inst->h264_slice_param; struct v4l2_h264_reflist_builder reflist_builder; + struct v4l2_h264_reference v4l2_p0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b1_reflist[V4L2_H264_REF_LIST_LEN]; u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0; u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0; u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1; @@ -475,12 +491,14 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) /* Build the reference lists */ v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps, inst->dpb); - v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); - v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); + v4l2_h264_build_p_ref_list(&reflist_builder, v4l2_p0_reflist); + v4l2_h264_build_b_ref_lists(&reflist_builder, v4l2_b0_reflist, + v4l2_b1_reflist); + /* Adapt the built lists to the firmware's expectations */ - fixup_ref_list(p0_reflist, reflist_builder.num_valid); - fixup_ref_list(b0_reflist, reflist_builder.num_valid); - fixup_ref_list(b1_reflist, reflist_builder.num_valid); + get_ref_list(p0_reflist, v4l2_p0_reflist, reflist_builder.num_valid); + get_ref_list(b0_reflist, v4l2_b0_reflist, reflist_builder.num_valid); + get_ref_list(b1_reflist, v4l2_b1_reflist, reflist_builder.num_valid); memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, sizeof(inst->vsi_ctx.h264_slice_params)); diff --git a/drivers/media/platform/nvidia/tegra-vde/h264.c b/drivers/media/platform/nvidia/tegra-vde/h264.c index d8e5534e80c8..acc1b97f06ad 100644 --- a/drivers/media/platform/nvidia/tegra-vde/h264.c +++ b/drivers/media/platform/nvidia/tegra-vde/h264.c @@ -45,9 +45,9 @@ struct tegra_vde_h264_decoder_ctx { }; struct h264_reflists { - u8 p[V4L2_H264_NUM_DPB_ENTRIES]; - u8 b0[V4L2_H264_NUM_DPB_ENTRIES]; - u8 b1[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference p[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference b0[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference b1[V4L2_H264_NUM_DPB_ENTRIES]; }; static int tegra_vde_wait_mbe(struct tegra_vde *vde) @@ -768,7 +768,7 @@ static int tegra_vde_h264_setup_frames(struct tegra_ctx *ctx, struct h264_reflists reflists; struct vb2_buffer *ref; unsigned int i; - u8 *dpb_id; + struct v4l2_h264_reference *dpb_id; int err; /* @@ -811,14 +811,16 @@ static int tegra_vde_h264_setup_frames(struct tegra_ctx *ctx, } for (i = 0; i < b.num_valid; i++) { - ref = get_ref_buf(ctx, dst, dpb_id[i]); + int dpb_idx = dpb_id[i].index; - err = tegra_vde_h264_setup_frame(ctx, h264, &b, ref, dpb_id[i], + ref = get_ref_buf(ctx, dst, dpb_idx); + + err = tegra_vde_h264_setup_frame(ctx, h264, &b, ref, dpb_idx, h264->dpb_frames_nb++); if (err) return err; - if (b.refs[dpb_id[i]].pic_order_count < b.cur_pic_order_count) + if (b.refs[dpb_idx].top_field_order_cnt < b.cur_pic_order_count) h264->dpb_ref_frames_with_earlier_poc_nb++; } @@ -880,6 +882,9 @@ static int tegra_vde_h264_setup_context(struct tegra_ctx *ctx, if (h->pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) return -EOPNOTSUPP; + if (h->decode_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) + return -EOPNOTSUPP; + if (h->sps->profile_idc == 66) h264->baseline_profile = 1; diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index ac47519a9fbe..afbfcf78efe4 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, pic_order_count = dpb[i].top_field_order_cnt; b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid] = i; + b->unordered_reflist[b->num_valid].index = i; b->num_valid++; } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) - b->unordered_reflist[i] = i; + b->unordered_reflist[i].index = i; } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const struct v4l2_h264_reflist_builder *builder = data; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist) + struct v4l2_h264_reference *reflist) { memcpy(reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist) + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist) { memcpy(b0_reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c index f49dbfb8a843..9de7f05eff2a 100644 --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; int reg_num; u32 reg; @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 0; i < 15; i += 3) { - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]); + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++)); } @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx) * of forward and backward reference picture lists and first 4 entries * of P forward picture list. */ - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) | - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]); + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) | + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC); /* @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) { - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) | - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) | - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) | - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) | - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) | - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]); + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index); vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++)); } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index ed018e293ba0..2bc6b8f088f5 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - u8 p[HANTRO_H264_DPB_SIZE]; - u8 b0[HANTRO_H264_DPB_SIZE]; - u8 b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; }; /** diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c index 64a6330475eb..46c1a83bcc4e 100644 --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; u32 reg; int i; @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx) b1_reflist = ctx->h264_dec.reflists.b1; p_reflist = ctx->h264_dec.reflists.p; - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) | - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) | - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) | - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) | - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) | - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]); + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) | + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) | + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) | + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) | + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) | + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74)); - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) | - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) | - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) | - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) | - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) | - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]); + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) | + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) | + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) | + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) | + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) | + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75)); reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) | @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx) VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83)); - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) | - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) | - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) | - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) | - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) | - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) | + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) | + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) | + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) | + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) | + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100)); - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) | - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) | - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) | - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) | - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) | - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) | + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) | + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) | + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) | + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) | + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101)); - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) | - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) | - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) | - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) | + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) | + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) | + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102)); - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) | - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) | - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) | - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) | - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) | - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) | + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) | + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) | + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) | + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) | + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103)); - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) | - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) | - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) | - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) | - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) | - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) | + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) | + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) | + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) | + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) | + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104)); - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) | - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) | - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) | - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) | + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) | + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) | + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105)); - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) | - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) | - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) | - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]); + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) | + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) | + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) | + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106)); reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm); diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 951e19231da2..3c7f3d87fab4 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - u8 p[RKVDEC_H264_DPB_SIZE]; - u8 b0[RKVDEC_H264_DPB_SIZE]; - u8 b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; u8 num_valid; }; @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, switch (j) { case 0: - idx = h264_ctx->reflists.p[i]; + idx = h264_ctx->reflists.p[i].index; break; case 1: - idx = h264_ctx->reflists.b0[i]; + idx = h264_ctx->reflists.b0[i].index; break; case 2: - idx = h264_ctx->reflists.b1[i]; + idx = h264_ctx->reflists.b1[i].index; break; } diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4b1c71c935e0..ef9a894e3c32 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; u8 num_valid; }; @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist); + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist); /** * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist); + struct v4l2_h264_reference *reflist); #endif /* _MEDIA_V4L2_H264_H */ From patchwork Tue Apr 26 12:57:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC143C433FE for ; Tue, 26 Apr 2022 12:58:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350400AbiDZNBi (ORCPT ); Tue, 26 Apr 2022 09:01:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350334AbiDZNBT (ORCPT ); Tue, 26 Apr 2022 09:01:19 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7E5517D4A1; Tue, 26 Apr 2022 05:58:11 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 91AFA1F43982 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977890; bh=vuftnHCEWwJdNLAqyrxNwOkREV8boF39Sd5EZvT7GwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dtPFBWG6icCFN7/tfl63OERpTPpFK1cwwNlfKMgDXOOU9TLmeK1S/ki6qAOudgkTV vQ2HDDwuRi9QvOTEoVjdKTyyvMlLCTzWETNVUfZB5rozmycH1v6s9SXzgsLJti/+Vd AK4+rkEH7j8lmAzHwSOxR0PIN8tPMNKJ25ammNbGrlEOgwO/0nj7vB5jFMqXRPQyLM VgeWaOv2r6UjnNc0fMvCugMQWzlhhP1RGoBWRE/4MzM41pieC0mg0DnwNP4Fbb4Znw SmfWBxVj7f1jycI4SblMk54J7k8C+qRbZQhhRnlpASecDA42/sGOlYVt4djFaJ9J/L 5sNqDy13qx5eg== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v4 06/24] media: h264: Increase reference lists size to 32 Date: Tue, 26 Apr 2022 08:57:32 -0400 Message-Id: <20220426125751.108293-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is to accommodate support for field decoding, which splits the top and the bottom references into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index afbfcf78efe4..4b46b36526c0 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2bc6b8f088f5..292aaaabaf24 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Tue Apr 26 12:57:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8F12C433EF for ; Tue, 26 Apr 2022 12:58:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350382AbiDZNBj (ORCPT ); Tue, 26 Apr 2022 09:01:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350340AbiDZNBU (ORCPT ); Tue, 26 Apr 2022 09:01:20 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9190617E21D; Tue, 26 Apr 2022 05:58:13 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7DA501F4398F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977892; bh=TG6URAwoxuOMsxmElkXZN+3kr6++zE3/TmJ6jTIeCc0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LclxaSjPDfOpC1UByITD5g0TcRn93XHlXMEJxfu6eBw6NAIFotmhiFn865sKV37A2 BylfSgeLLOjvU7FE4GgWjZI4fw1BVHHEYqQE+Q9pQf4IxXdbe29xsHmNUZ0CIYG73W uaSH7XY7FZycGmEv2UScGuDjLhF5e40uFrm/lGBYsvC9Q+9sPUUogaMTT7fkdL2DQ7 gD0qR6bUJvCLxpDzLwXR0mo0Xid1GU/b4JM7nGbuaGnVxJD6gpQQBYjsp7tNRw8Wdb 8AZKC0WQmiCfyjTX/rjOS9PKUp1IKg4/btz4G5XS6279dHAIJj9qttd6rJ/Hi8Q2x1 koC8n4xkKSpGQ== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org Subject: [PATCH v4 07/24] media: h264: Store current picture fields Date: Tue, 26 Apr 2022 08:57:33 -0400 Message-Id: <20220426125751.108293-8-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This information, also called picture structure, is required in field decoding mode to construct reference lists. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 10 +++++++--- include/media/v4l2-h264.h | 4 ++++ 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 4b46b36526c0..58f18bb0afb6 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -34,13 +34,17 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, cur_frame_num = dec_params->frame_num; memset(b, 0, sizeof(*b)); - if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) + if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) { b->cur_pic_order_count = min(dec_params->bottom_field_order_cnt, dec_params->top_field_order_cnt); - else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) + b->cur_pic_fields = V4L2_H264_FRAME_REF; + } else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) { b->cur_pic_order_count = dec_params->bottom_field_order_cnt; - else + b->cur_pic_fields = V4L2_H264_BOTTOM_FIELD_REF; + } else { b->cur_pic_order_count = dec_params->top_field_order_cnt; + b->cur_pic_fields = V4L2_H264_TOP_FIELD_REF; + } for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { u32 pic_order_count; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index e282fb16ac58..e165a54c68fa 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -21,6 +21,7 @@ * @refs.longterm: set to true for a long term reference * @refs: array of references * @cur_pic_order_count: picture order count of the frame being decoded + * @cur_pic_fields: fields present in the frame being decoded * @unordered_reflist: unordered list of references. Will be used to generate * ordered P/B0/B1 lists * @num_valid: number of valid references in the refs array @@ -36,7 +37,10 @@ struct v4l2_h264_reflist_builder { u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; + s32 cur_pic_order_count; + u8 cur_pic_fields; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; From patchwork Tue Apr 26 12:57:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77814C433F5 for ; Tue, 26 Apr 2022 12:58:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350399AbiDZNBl (ORCPT ); Tue, 26 Apr 2022 09:01:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350350AbiDZNBZ (ORCPT ); Tue, 26 Apr 2022 09:01:25 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE2D417F125; Tue, 26 Apr 2022 05:58:14 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id EB7521F43991 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977893; bh=es6kMBJ8LPORvT6E2TfIJklYinEGihlPlvjF8xMQCJ0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D/PVhK8Onnpb//BfkbB4XspS8oVIFucs0SkfAwzVfXYTx4n6ZUM9vdyti2RJ89XHn fI5baLZklPl1OmS1A5t5h04Qrfy7MdK33vjHELfMpvYLPo9fxbDbvCJYK0bNrIbdjH D0i2DWTDGxn0JVMJ1mSul2n4XlVkjFP19o7bTeRwLqGCyouY3CSjq+N3qDOSvcj05t zxMFWHaZx7j3dJ6ZHdW1EpWVfr62IjC28tOtaHB3Zm2qobcgaiKZ9Mp3uIezL1I156 4bqmu/gq2wyGwt7E4empEFFmJ+fmUX8vy4QnhbvTPFWroO+4sf0YeZeK9OrE5Y8N3j AvCHR32H4qLdA== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org Subject: [PATCH v4 08/24] media: h264: Store all fields into the unordered list Date: Tue, 26 Apr 2022 08:57:34 -0400 Message-Id: <20220426125751.108293-9-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When the current picture is a field, store each field into the unordered_list and preserve both top and bottom picture order count. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 65 +++++++++++++++++++++-------- include/media/v4l2-h264.h | 6 ++- 2 files changed, 51 insertions(+), 20 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 58f18bb0afb6..38d8dbda0045 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -47,8 +47,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, } for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { - u32 pic_order_count; - if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; @@ -59,8 +57,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, /* * Handle frame_num wraparound as described in section * '8.2.4.1 Decoding process for picture numbers' of the spec. - * TODO: This logic will have to be adjusted when we start - * supporting interlaced content. * For long term references, frame_num is set to * long_term_frame_idx which requires no wrapping. */ @@ -70,17 +66,33 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, else b->refs[i].frame_num = dpb[i].frame_num; - if (dpb[i].fields == V4L2_H264_FRAME_REF) - pic_order_count = min(dpb[i].top_field_order_cnt, - dpb[i].bottom_field_order_cnt); - else if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) - pic_order_count = dpb[i].bottom_field_order_cnt; - else - pic_order_count = dpb[i].top_field_order_cnt; + b->refs[i].top_field_order_cnt = dpb[i].top_field_order_cnt; + b->refs[i].bottom_field_order_cnt = dpb[i].bottom_field_order_cnt; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + u8 fields = V4L2_H264_FRAME_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + continue; + } + + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) { + u8 fields = V4L2_H264_TOP_FIELD_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + } - b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid].index = i; - b->num_valid++; + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) { + u8 fields = V4L2_H264_BOTTOM_FIELD_REF; + + b->unordered_reflist[b->num_valid].index = i; + b->unordered_reflist[b->num_valid].fields = fields; + b->num_valid++; + } } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) @@ -88,6 +100,23 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); +static s32 v4l2_h264_get_poc(const struct v4l2_h264_reflist_builder *b, + const struct v4l2_h264_reference *ref) +{ + switch (ref->fields) { + case V4L2_H264_FRAME_REF: + return min(b->refs[ref->index].top_field_order_cnt, + b->refs[ref->index].bottom_field_order_cnt); + case V4L2_H264_TOP_FIELD_REF: + return b->refs[ref->index].top_field_order_cnt; + case V4L2_H264_BOTTOM_FIELD_REF: + return b->refs[ref->index].bottom_field_order_cnt; + } + + /* not reached */ + return 0; +} + static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const void *data) { @@ -150,8 +179,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, builder->refs[idxb].pic_num ? -1 : 1; - poca = builder->refs[idxa].pic_order_count; - pocb = builder->refs[idxb].pic_order_count; + poca = v4l2_h264_get_poc(builder, ptra); + pocb = v4l2_h264_get_poc(builder, ptrb); /* * Short term pics with POC < cur POC first in POC descending order @@ -195,8 +224,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, builder->refs[idxb].pic_num ? -1 : 1; - poca = builder->refs[idxa].pic_order_count; - pocb = builder->refs[idxb].pic_order_count; + poca = v4l2_h264_get_poc(builder, ptra); + pocb = v4l2_h264_get_poc(builder, ptrb); /* * Short term pics with POC > cur POC first in POC ascending order diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index e165a54c68fa..4cef717b3f18 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -15,7 +15,8 @@ /** * struct v4l2_h264_reflist_builder - Reference list builder object * - * @refs.pic_order_count: reference picture order count + * @refs.top_field_order_cnt: top field order count + * @refs.bottom_field_order_cnt: bottom field order count * @refs.frame_num: reference frame number * @refs.pic_num: reference picture number * @refs.longterm: set to true for a long term reference @@ -32,7 +33,8 @@ */ struct v4l2_h264_reflist_builder { struct { - s32 pic_order_count; + s32 top_field_order_cnt; + s32 bottom_field_order_cnt; int frame_num; u32 pic_num; u16 longterm : 1; From patchwork Tue Apr 26 12:57:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4464C433EF for ; Tue, 26 Apr 2022 12:58:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350392AbiDZNBm (ORCPT ); Tue, 26 Apr 2022 09:01:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350352AbiDZNBZ (ORCPT ); Tue, 26 Apr 2022 09:01:25 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27FC317F12E; Tue, 26 Apr 2022 05:58:16 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 4B5B21F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977895; bh=dscra9aQy+OFrcD4i/Eei0e+7LpQWDlVC/URKqG2tgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P8KPWohhHtPMfo5ZDHvEBRpE5RfRlbI8ZEfc8l5GyboDgpVkJIaiqDaqpSrmndrn+ EVE6Ha4ffWfANezr5G6auwR0yo0W3yl+9dICKCSkpkao0PLtDxd4KmJXLKN8pFZ4zy RzWRLVH0QiHXtJdUYuXDVvOc6Gg/vpNzKtMf3YYvz/tlZFpPg/CJ1TnQtgJ1xC5gv+ 1lGRvTN1P77AbyxqNyBY6phqMApzfSbB7YOCMk+sRCenCi0q6r+7KHRtWtpl+t0CuF 7N/Zc7ac1pze0BaS7FcWyGlvHxiH1WOp2FcZi14C4CxeFeYTi43RNVrTghsOlKOPml P6ZwQpWaQWPiw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org Subject: [PATCH v4 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist Date: Tue, 26 Apr 2022 08:57:35 -0400 Message-Id: <20220426125751.108293-10-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add debug print statements to print the content of P & B reference lists, to verify that the ordering of the generated reference lists is correct. This is especially important for the field decoding mode, where sorting is more complex. Signed-off-by: Nicolas Dufresne Tested-by: Sebastian Fricke Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 100 ++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 38d8dbda0045..853f54e0fe67 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -12,6 +12,12 @@ #include +/* + * Size of the tempory buffer allocated when printing reference lists. The + * output will be truncated if the size is too small. + */ +static const int tmp_str_size = 1024; + /** * v4l2_h264_init_reflist_builder() - Initialize a P/B0/B1 reference list * builder @@ -241,6 +247,95 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +static char ref_type_to_char(u8 ref_type) +{ + switch (ref_type) { + case V4L2_H264_FRAME_REF: + return 'f'; + case V4L2_H264_TOP_FIELD_REF: + return 't'; + case V4L2_H264_BOTTOM_FIELD_REF: + return 'b'; + } + + return '?'; +} + +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char **out_str) +{ + int n = 0, i; + + *out_str = kmalloc(tmp_str_size, GFP_KERNEL); + + n += snprintf(*out_str + n, tmp_str_size - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + /* this is pic_num for frame and frame_num (wrapped) for field, + * but for frame pic_num is equal to frame_num (wrapped). + */ + int frame_num = builder->refs[reflist[i].index].frame_num; + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(*out_str + n, tmp_str_size - n, "%i%c%c|", + frame_num, longterm ? 'l' : 's', + ref_type_to_char(reflist[i].fields)); + } + + return *out_str; +} + +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist) +{ + char *buf = NULL; + + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n", + builder->cur_pic_order_count, + ref_type_to_char(builder->cur_pic_fields), + format_ref_list_p(builder, reflist, &buf)); + + kfree(buf); +} + +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char **out_str) +{ + int n = 0, i; + + *out_str = kmalloc(tmp_str_size, GFP_KERNEL); + + n += snprintf(*out_str + n, tmp_str_size - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + int frame_num = builder->refs[reflist[i].index].frame_num; + u32 poc = v4l2_h264_get_poc(builder, reflist + i); + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(*out_str + n, tmp_str_size - n, "%i%c%c|", + longterm ? frame_num : poc, + longterm ? 'l' : 's', + ref_type_to_char(reflist[i].fields)); + } + + return *out_str; +} + +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, u8 list_num) +{ + char *buf = NULL; + + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s", + list_num, builder->cur_pic_order_count, + ref_type_to_char(builder->cur_pic_fields), + format_ref_list_b(builder, reflist, &buf)); + + kfree(buf); +} + /** * v4l2_h264_build_p_ref_list() - Build the P reference list * @@ -261,6 +356,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sizeof(builder->unordered_reflist[0]) * builder->num_valid); sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -296,6 +393,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); + + print_ref_list_b(builder, b0_reflist, 0); + print_ref_list_b(builder, b1_reflist, 1); } EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists); From patchwork Tue Apr 26 12:57:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40B34C433F5 for ; Tue, 26 Apr 2022 12:58:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350430AbiDZNB4 (ORCPT ); Tue, 26 Apr 2022 09:01:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350320AbiDZNBZ (ORCPT ); Tue, 26 Apr 2022 09:01:25 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8504517B399; Tue, 26 Apr 2022 05:58:17 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id AE6D01F4398F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977896; bh=952H20dFKMDGDG2AyTcdBsEiBoEPkmvH/b/LrPIdq6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ivwEH58V6mC1MYYJ4WS3Fo3nzVJjOsapx5RN+72kpIWPAA/9mJYTv9AvPV9SoCU7t vbomIyuebGGZ7qsih9ITtUvNwe2/rUDE/BOeGdjS+pJfehE7S/ALAOB8bJUK0Gv7xo 9C6kenOhkDskpU48m3KexD7pQji4KmqfsOIPpmHe5BTJK40h1c4r8irPIQOwN30SAQ b9/mJSD6FW73vkcPvfy+AhkzcdY28j43lVhlDIYmQM13LDyBvpeuJBN+lIsbPuF/JR QW7jhCWN5Pe+sNbPaLDcfY/YILrMSWKP9HN5+1CJSC98Zxfj/MyuYWDrovf+ACb09c AtvB0I6M706bw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org Subject: [PATCH v4 10/24] media: h264: Sort p/b reflist using frame_num Date: Tue, 26 Apr 2022 08:57:36 -0400 Message-Id: <20220426125751.108293-11-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In the reference list builder, frame_num refers to FrameNumWrap in the spec, which is the same as the pic_num for frame decoding. The same applies for long_term_pic_num and long_term_frame_idx. Sort all type of references by frame_num so the sort can be reused for fields reflist were the sorting is done using frame_num instead. In short, pic_num is never actually used for building reference lists. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 23 +++++++++++++---------- include/media/v4l2-h264.h | 2 -- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 853f54e0fe67..fe215035d9e8 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -56,7 +56,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - b->refs[i].pic_num = dpb[i].pic_num; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) b->refs[i].longterm = true; @@ -145,15 +144,19 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, } /* - * Short term pics in descending pic num order, long term ones in - * ascending order. + * For frames, short term pics are in descending pic num order and long + * term ones in ascending order. For fields, the same direction is used + * but with frame_num (wrapped). For frames, the value of pic_num and + * frame_num are the same (see formula (8-28) and (8-29)). For this + * reason we can use frame_num only and share this function between + * frames and fields reflist. */ if (!builder->refs[idxa].longterm) return builder->refs[idxb].frame_num < builder->refs[idxa].frame_num ? -1 : 1; - return builder->refs[idxa].pic_num < builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < builder->refs[idxb].frame_num ? -1 : 1; } @@ -179,10 +182,10 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, return 1; } - /* Long term pics in ascending pic num order. */ + /* Long term pics in ascending frame num order. */ if (builder->refs[idxa].longterm) - return builder->refs[idxa].pic_num < - builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < + builder->refs[idxb].frame_num ? -1 : 1; poca = v4l2_h264_get_poc(builder, ptra); @@ -224,10 +227,10 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return 1; } - /* Long term pics in ascending pic num order. */ + /* Long term pics in ascending frame num order. */ if (builder->refs[idxa].longterm) - return builder->refs[idxa].pic_num < - builder->refs[idxb].pic_num ? + return builder->refs[idxa].frame_num < + builder->refs[idxb].frame_num ? -1 : 1; poca = v4l2_h264_get_poc(builder, ptra); diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4cef717b3f18..0d9eaa956123 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -18,7 +18,6 @@ * @refs.top_field_order_cnt: top field order count * @refs.bottom_field_order_cnt: bottom field order count * @refs.frame_num: reference frame number - * @refs.pic_num: reference picture number * @refs.longterm: set to true for a long term reference * @refs: array of references * @cur_pic_order_count: picture order count of the frame being decoded @@ -36,7 +35,6 @@ struct v4l2_h264_reflist_builder { s32 top_field_order_cnt; s32 bottom_field_order_cnt; int frame_num; - u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; From patchwork Tue Apr 26 12:57:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43041C433EF for ; Tue, 26 Apr 2022 12:58:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346751AbiDZNBy (ORCPT ); Tue, 26 Apr 2022 09:01:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350359AbiDZNB0 (ORCPT ); Tue, 26 Apr 2022 09:01:26 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCC9217B98F; Tue, 26 Apr 2022 05:58:18 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 07FAE1F43991 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977897; bh=wzijXmDhK09ZXKvIar9T0QlhLA6ISJRmeorKyXSGYdY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NSTisUst3NacQiRj7gch2pbnT2Re8cPYFzZPsrWHLHAJMmghZ0xh6ZeOOyPcwYSSl 7vz5nXSqO6Wg0p3D7JQWCu56zvaJLuo/7P0tux/xKckJPCVCQ0Io41jCyoQ41APkkb rS3xR7slfgdU1WBMIB/mm6PqBbrZLsa4giL1R30oWMBYEM+meffq9H3ZErU4GfcMDs ty/gAgP18LnJNRGnaX9ybWTf95jsPGcMPMRqvqikgZjMGi/2vDtVKp+FHa2W+8FehQ 5n4AwihlSlRk+pqq98w1cPiPkTYrNAFQiGwFv3C0bClyxwUknsDWtU/5wtiFoBAFYR znuy8LdVNhDSw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org Subject: [PATCH v4 11/24] media: v4l2: Reorder field reflist Date: Tue, 26 Apr 2022 08:57:37 -0400 Message-Id: <20220426125751.108293-12-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As per spec, the field reflist requires interleaving top and bottom field in a specific way that does not fit inside the sort operation. The process consist of alternating references parity, starting with a reference of the same parity as the current picture. This processs is done twice, once for short term references and a second time for the long term references. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index fe215035d9e8..72bd64f65198 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -250,6 +250,40 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +/* + * The references need to be reordered so that references are alternating + * between top and bottom field references starting with the current picture + * parity. This has to be done for short term and long term references + * separately. + */ +static void reorder_field_reflist(const struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + struct v4l2_h264_reference tmplist[V4L2_H264_REF_LIST_LEN]; + u8 lt, i = 0, j = 0, k = 0; + + memcpy(tmplist, reflist, sizeof(tmplist[0]) * b->num_valid); + + for (lt = 0; lt <= 1; lt++) { + do { + for (; i < b->num_valid && b->refs[tmplist[i].index].longterm == lt; i++) { + if (tmplist[i].fields == b->cur_pic_fields) { + reflist[k++] = tmplist[i++]; + break; + } + } + + for (; j < b->num_valid && b->refs[tmplist[j].index].longterm == lt; j++) { + if (tmplist[j].fields != b->cur_pic_fields) { + reflist[k++] = tmplist[j++]; + break; + } + } + } while ((i < b->num_valid && b->refs[tmplist[i].index].longterm == lt) || + (j < b->num_valid && b->refs[tmplist[j].index].longterm == lt)); + } +} + static char ref_type_to_char(u8 ref_type) { switch (ref_type) { @@ -360,6 +394,9 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) + reorder_field_reflist(builder, reflist); + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -393,6 +430,11 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, sort_r(b1_reflist, builder->num_valid, sizeof(*b1_reflist), v4l2_h264_b1_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) { + reorder_field_reflist(builder, b0_reflist); + reorder_field_reflist(builder, b1_reflist); + } + if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); From patchwork Tue Apr 26 12:57:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B894C43217 for ; Tue, 26 Apr 2022 12:58:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350420AbiDZNBy (ORCPT ); Tue, 26 Apr 2022 09:01:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350380AbiDZNBc (ORCPT ); Tue, 26 Apr 2022 09:01:32 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3F0E17F11B; Tue, 26 Apr 2022 05:58:20 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 61F8F1F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977899; bh=gu3hWa9l9RwIqYfPgXJTq2hXyJdb/X3UYi37Vl+bXOc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A9PPhaws+0UJKFGVd1hph8rKqPW0f16roDicNJFfCdGnt2qPN466cQ22WLaQfg4U5 +Jxnnc1YkNht2e50RoYOjFvb/wNAKG1eu+UGyCbR/EhBVNTOiBmGazcu+7DO0/G5ql WnI0BtliPH3K5k0AyymLQZNm8otEZaMpTYQsCZo5HAI7ZBydI7UbBYsjEizKhU0c4c JTiYZEDPyYpzkw2cpXNWb1S/C+OrHDHktRpzO2Or1om6MilkX5eHtIzxOxPweZ9ioC WIyhM23PqBPJS0uf9kApXCI2runbW5D6iZ2Uyj/LxM4r2W/haPC6ILBcRTnprAIRMr u0V8WQkrdXY/Q== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 12/24] media: rkvdec: Stop overclocking the decoder Date: Tue, 26 Apr 2022 08:57:38 -0400 Message-Id: <20220426125751.108293-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org While this overclock hack seems to work on some implementations (some ChromeBooks, RockPi4) it also causes instability on other implementations (notably LibreComputer Renegade, but there were more reports in the LibreELEC project, where this has been removed). While performance is indeed affected (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed used to synchronously decode each frame, so this hack was simply compensating for their code being less efficient. In my opinion, this hack should not have been included upstream. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Tue Apr 26 12:57:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C817EC43217 for ; Tue, 26 Apr 2022 12:58:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350334AbiDZNBo (ORCPT ); Tue, 26 Apr 2022 09:01:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349636AbiDZNBf (ORCPT ); Tue, 26 Apr 2022 09:01:35 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B49717FBCE; Tue, 26 Apr 2022 05:58:23 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 70C1F1F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977901; bh=qqGKTdK+BoJvGXzu5rDFgeoUxEBT3Njqv5q7sTj6RAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UWJo24Y3WFgGCpdd4zZo3nChuheimOm8+x1xB7rfBR4tcpQbgBd5XeyIgafDQP77Z zswC30ioFQ2xggDqfWxE28q2BI4HwrlPX9L4ZTJdEzT5UmiLmjLKTlOtSmseXz6l/K aabCPa3lyhyFNCAZHmeyenUAwEkbp5/ktJR1A7NlS1IpGmxxtfFTyyyM85grd9qBOD q06Wul8cQjnx8TVI8wWUD350iIcI//IzY5t1vwZcKAGkgffWqsyxuv+GsFjBRDoVcn 2YtZat4CUTXCYVEmQP0idVgjZX/eEyp3qMCxNZSNtS2iKyoYw0HpXqnQo/NtblBbtm btlZS7Eq962BA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Boris Brezillon , Hans Verkuil Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 13/24] media: rkvdec: h264: Fix dpb_valid implementation Date: Tue, 26 Apr 2022 08:57:39 -0400 Message-Id: <20220426125751.108293-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ref builder only provided references that are marked as valid in the dpb. Thus the current implementation of dpb_valid would always set the flag to 1. This is not representing missing frames (this is called 'non-existing' pictures in the spec). In some context, these non-existing pictures still need to occupy a slot in the reference list according to the spec. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index dff89732ddd0..bcde37d72244 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -112,6 +112,7 @@ struct rkvdec_h264_run { const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES]; }; struct rkvdec_h264_ctx { @@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, } } +static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, + struct rkvdec_h264_run *run) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) { + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; + struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; + int buf_idx = -1; + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + buf_idx = vb2_find_timestamp(cap_q, + dpb[i].reference_ts, 0); + + run->ref_buf_idx[i] = buf_idx; + } +} + static void assemble_hw_rps(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { @@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = 0; + bool dpb_valid = run->ref_buf_idx[i] >= 0; u8 idx = 0; switch (j) { @@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (idx >= ARRAY_SIZE(dec_params->dpb)) continue; - dpb_valid = !!(dpb[idx].flags & - V4L2_H264_DPB_ENTRY_FLAG_ACTIVE); set_ps_field(hw_rps, DPB_INFO(i, j), idx | dpb_valid << 4); @@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run, unsigned int dpb_idx) { struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; - const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; - int buf_idx = -1; - - if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - buf_idx = vb2_find_timestamp(cap_q, - dpb[dpb_idx].reference_ts, 0); + int buf_idx = run->ref_buf_idx[dpb_idx]; /* * If a DPB entry is unused or invalid, address of current destination @@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); + lookup_ref_buf_idx(ctx, &run); assemble_hw_rps(ctx, &run); config_registers(ctx, &run); From patchwork Tue Apr 26 12:57:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA202C4332F for ; Tue, 26 Apr 2022 12:58:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350350AbiDZNBx (ORCPT ); Tue, 26 Apr 2022 09:01:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350383AbiDZNBg (ORCPT ); Tue, 26 Apr 2022 09:01:36 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28C21177D47; Tue, 26 Apr 2022 05:58:25 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 762DA1F43982 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977904; bh=FmxP3Of2pm4gkbqbbaer5qaBCfIvVbPhhE5DeBELz4c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hBcUdsG1vo3Ibv1T3PQzLkVRTO0BNO/MVGkjQjQwI+Cq1k4fW3YqyXc+hxVDtQf/o MJuP+asy+HuX8zNfZEw/5amOnVG+lj3zPobSzVPtm7WuekupyjirBzpIGRu0Cq98GL unLEetrzkGph3I6wU4USp8dMV6FRCHAqo9gjaEOKEuSHlDMgn9f17RAdUro6jJ1Wqu uPEsQMUgSnXfrEjpk5ebf2GLQCIv418eOVG79hBN5E7Zv/6+6Asg/D3t2WXaURMCbn 8JaCNhHnsPh6AgM3V5WfTkzw1p2TWjl9L9muaea9yujNEXRbpLnSD2BpZQNgkLwFT1 bsPyAFMV8nknw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Ezequiel Garcia , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 14/24] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Tue, 26 Apr 2022 08:57:40 -0400 Message-Id: <20220426125751.108293-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet are 3 bits wide. 8 is wrongly added to the bit depth values written to these 3 bit fields. Because only the 3 LSB are written, the hardware was configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index bcde37d72244..8d44a884a52e 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Tue Apr 26 12:57:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A59FEC433EF for ; Tue, 26 Apr 2022 12:58:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350385AbiDZNBs (ORCPT ); Tue, 26 Apr 2022 09:01:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350321AbiDZNBg (ORCPT ); Tue, 26 Apr 2022 09:01:36 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02D7317FBFA; Tue, 26 Apr 2022 05:58:26 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C07D41F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977905; bh=Zdyv7lj5Zwk1trozSl5/ToeVKlMaPGRdeC6c5HT1zS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Od0WtrC9/3w+VoxRpi3zn10sJ0Br+Gyxb8aoVslheETwwKNg/iGAv2aeShXCkitp/ Mu4Hk9C93KBfpJpPhx+gFja3MN2vX+NB0+0n75rpbpgc71IN++FRPYYTja1zrDaY/n 3kUPquAsCscegcf3N7IYdBYQloUa83CIBpVs5ZxT7WEvn2yM/NBLYZKjrNrE7pWiTJ ZHOiflQlrvYlhRoSitzL1RDhm9rQ++Mj2oFRrdWePsCXJcFPMU9H15LspCuU6jmBLa vpwvQbkapiOUjG2v5o0kcEh5tZLdyUH6uhfMZ5GWU0SHZQBPMBsPa4/rAIxwk0kvQo xpd9x7iSfaPsw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264 Date: Tue, 26 Apr 2022 08:57:41 -0400 Message-Id: <20220426125751.108293-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org No functional change, this moves H264 specific validation into the H264 specific code. This is in preparation of improving this validation and reusing it when VIDIOC_STREAMON is called. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++---------------- drivers/staging/media/rkvdec/rkvdec.h | 1 + 3 files changed, 30 insertions(+), 17 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 8d44a884a52e..0dcbcb1bac80 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) return 0; } +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + } + return 0; +} + const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { .adjust_fmt = rkvdec_h264_adjust_fmt, .start = rkvdec_h264_start, .stop = rkvdec_h264_stop, .run = rkvdec_h264_run, + .try_ctrl = rkvdec_h264_try_ctrl, }; diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2df8cf4883e2..e3d44d5b35f3 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,23 +29,12 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; + + if (desc->ops->try_ctrl) + return desc->ops->try_ctrl(ctx, ctrl); + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2f4ea1786b93..9df0fba799a4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops { void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf, struct vb2_v4l2_buffer *dst_buf, enum vb2_buffer_state result); + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl); }; struct rkvdec_coded_fmt_desc { From patchwork Tue Apr 26 12:57:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E087BC433FE for ; Tue, 26 Apr 2022 12:58:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350413AbiDZNBw (ORCPT ); Tue, 26 Apr 2022 09:01:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350388AbiDZNBg (ORCPT ); Tue, 26 Apr 2022 09:01:36 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A044D17B39C; Tue, 26 Apr 2022 05:58:28 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 67CE51F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977907; bh=0J+46psvSTuyS1wejcCrG3atoNuCwZAAyNkbTjDMFsU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aeiO2ufbvhFNYb7zVWgxzT+dtqF3tl7RyjAsaf8KVsW60NFm4QkISfL9a12zsEYle jA5BgytAjoyMdKIVIBVxxaJCwyvW+Doi+iLHXdGa8ZOsDMwxLXq5Dw27QRO9EyRnCu i6IPmh8a6W26TEeWalrLgOoVyFvNBD6Na9c+tWGE90XED+IfZGudpMYlKko4q1dVuw TwKI2T194dTmEohVKjaaRS5uwYzF/3zpslBwroYj1IByMcEFDoidltpPOkXEV2wFgH FqpjjuNIEWv4xLOOb2EcmI0gKiO7+Gg23oICPHN2qO5sv8vIjpnkJ5WaHUmnn9qt0j poaAaDreu9YVQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Tue, 26 Apr 2022 08:57:42 -0400 Message-Id: <20220426125751.108293-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The width and height in macroblocks is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. The validation is happening in both try_ctrt() and start() since it is otherwise possible to trick the driver during initialization by changing the OUTPUT format after having set a valid control. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 81 +++++++++++++++++----- 1 file changed, 62 insertions(+), 19 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 0dcbcb1bac80..d5ac0285f36f 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,17 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + + /* + * Use the SPS values since they are already in macroblocks + * dimensions, height can be field height (halved) if + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows + * decoding smaller images into larger allocation which can be used + * to implementing SVC spatial layer support. + */ + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); + WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), @@ -1035,13 +1044,61 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx, return 0; } +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx, + const struct v4l2_ctrl_h264_sps *sps) +{ + unsigned int width, height; + + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + /* + * when frame_mbs_only_flag is not set, this is field height, + * which is half the final height (see (7-18) in the + * specification) + */ + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) + height *= 2; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; + + return 0; +} + static int rkvdec_h264_start(struct rkvdec_ctx *ctx) { struct rkvdec_dev *rkvdec = ctx->dev; struct rkvdec_h264_priv_tbl *priv_tbl; struct rkvdec_h264_ctx *h264_ctx; + struct v4l2_ctrl *ctrl; int ret; + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, + V4L2_CID_STATELESS_H264_SPS); + if (!ctrl) + return -EINVAL; + + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + if (ret) + return ret; + h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL); if (!h264_ctx) return -ENOMEM; @@ -1139,23 +1196,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + return 0; } From patchwork Tue Apr 26 12:57:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B60EAC433F5 for ; Tue, 26 Apr 2022 12:58:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350406AbiDZNBp (ORCPT ); Tue, 26 Apr 2022 09:01:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350394AbiDZNBh (ORCPT ); Tue, 26 Apr 2022 09:01:37 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8727017B98D; Tue, 26 Apr 2022 05:58:30 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1C6751F43996 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977909; bh=Obrg+kliljhQBVbkjHH9eqyMF1KK1PscTQElBfjHqLA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M/Su31vKdsLjJViMlDV+oZDM2lWGxhUOrM0xafZj4XX9GRff9fgOXPSG4OMAc1QAs B5qSVl8w3PC38ceehO0XjfkGqsCHvh4k940A679PTb+LHLTOx3rrilS9Sp2Exzrkoq ij85Gh4X8ovJXGy0mCiByNyHOjNK0/xSni3J/R+PkETo4Hy0Yqi1agSZBU8bzvqRff gBE+vPFCxcz2wg3bZN0kte+59S6ty4xTRoc04YP6VwCQSuRwFbDEKWby7uO2+0SfCE OLgllG/q6s3WF7qFY103nMu47ObqYHVano4sby24zZQCO/SzPVbi64lxCNU86vSSC9 Hm2TfZqLLoO/w== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Ezequiel Garcia , Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Tue, 26 Apr 2022 08:57:43 -0400 Message-Id: <20220426125751.108293-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this caused issues decoding the second field in a complementary field pair. Fix this by using inclusive comparison: 'less than or equal'. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index d5ac0285f36f..7011b66c1aab 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -782,7 +782,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Tue Apr 26 12:57:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A1EC433FE for ; Tue, 26 Apr 2022 12:58:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350340AbiDZNBr (ORCPT ); Tue, 26 Apr 2022 09:01:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350244AbiDZNBj (ORCPT ); Tue, 26 Apr 2022 09:01:39 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D184220C3; Tue, 26 Apr 2022 05:58:32 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id F359F1F43999 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977911; bh=mlmVguJcq18q2C5FsZYpROZPrzKGkqiR80YOa10U0rQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R4DhZD/ZO/ZdN80EgBWl3CS/bbfNmq0mGNkaXtVKCew0b+SnrGkmV0GdkFy2zQZzj /J1QimWglAQnyxDyECg5tgducpujBvM7Jgn9I+pVNQ17xK0KLQKLq1NyZ2LjHJZizs 0IxLhE91L+gsPFTfB4BrsOoanpy/oB9V+TZ76QqbIX2qZ2FeSc3Hikz3TUBtukHukW gGactiPcwUwSMCxq6Fg2Ryr+iijk6a/spxFCBeAF9sv0CLB+igDPaGCO+51/9V0shg NIOuXwDbLBdLSIN6xt60BecfrAGk4xZHH0HggzrLiGP4//WTfbNVeNSYTmRwhrpE0b GDecRSL1kCfpQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 18/24] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Tue, 26 Apr 2022 08:57:44 -0400 Message-Id: <20220426125751.108293-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OUTPUT buffer resolution. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index e3d44d5b35f3..2bc4b1a40989 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -257,6 +257,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Tue Apr 26 12:57:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2F43C433EF for ; Tue, 26 Apr 2022 12:58:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350454AbiDZNCE (ORCPT ); Tue, 26 Apr 2022 09:02:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350401AbiDZNBm (ORCPT ); Tue, 26 Apr 2022 09:01:42 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CFF217B39C; Tue, 26 Apr 2022 05:58:34 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id CDA921F4399D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977913; bh=dZLY5A5SJIWQWpJ/gOaUMK4w2rzl/JljG8yfzL7NyHg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U+8pjgSzvqCzEEZreu22RuqA2NH9VJNkmR7y6PYxSdjioGUS+WP7LoQIgWgysclSJ shE+gSMTTqRSucz2lQni/PQQ02CQ1yyNWaZ/l2o1Q65kIIBrvruTKcauIJXLFHPYDf v82GLbYC5ca7DfVCj6HJKy6LZDYCsfYP14D7UjjH90s9EA7P/mGEXh9ZKNItEzCo5W hGMVC/BaX6c/bZY+n+PAPKk52I0TOZExvDIm4J+3Ujl/JvFvYn6eqJsyUVrNszZ4h9 GHW2APLlSJaOQBGfLZuURGPj8w7+UPzUXiXri1VJoc7WZSTBFvB22VS7xpUJO/hFsi W8GPnwn2/JBXg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 19/24] media: rkvdec-h264: Add field decoding support Date: Tue, 26 Apr 2022 08:57:45 -0400 Message-Id: <20220426125751.108293-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This makes use of the new feature in the reference builder to program up to 32 references when doing field decoding. It also signals the parity (top or bottom) of the field to the hardware. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------ 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 7011b66c1aab..fbe23d598720 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl { u8 err_info[RKV_ERROR_INFO_SIZE]; }; -#define RKVDEC_H264_DPB_SIZE 16 - struct rkvdec_h264_reflists { struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; - u8 num_valid; }; struct rkvdec_h264_run { @@ -747,23 +744,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; int buf_idx = -1; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) { buf_idx = vb2_find_timestamp(cap_q, dpb[i].reference_ts, 0); + if (buf_idx < 0) + pr_debug("No buffer for reference_ts %llu", + dpb[i].reference_ts); + } run->ref_buf_idx[i] = buf_idx; } } static void assemble_hw_rps(struct rkvdec_ctx *ctx, + struct v4l2_h264_reflist_builder *builder, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; - const struct v4l2_ctrl_h264_sps *sps = run->sps; struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu; - u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4); u32 *hw_rps = priv_tbl->rps; u32 i, j; @@ -781,37 +781,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num <= dec_params->frame_num) { - p[i] = dpb[i].frame_num; - continue; - } - - p[i] = dpb[i].frame_num - max_frame_num; + p[i] = builder->refs[i].frame_num; } for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { - for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - bool dpb_valid = run->ref_buf_idx[i] >= 0; - u8 idx = 0; + for (i = 0; i < builder->num_valid; i++) { + struct v4l2_h264_reference *ref; + bool dpb_valid; + bool bottom; switch (j) { case 0: - idx = h264_ctx->reflists.p[i].index; + ref = &h264_ctx->reflists.p[i]; break; case 1: - idx = h264_ctx->reflists.b0[i].index; + ref = &h264_ctx->reflists.b0[i]; break; case 2: - idx = h264_ctx->reflists.b1[i].index; + ref = &h264_ctx->reflists.b1[i]; break; } - if (idx >= ARRAY_SIZE(dec_params->dpb)) + if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb))) continue; + dpb_valid = run->ref_buf_idx[ref->index] >= 0; + bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF; + set_ps_field(hw_rps, DPB_INFO(i, j), - idx | dpb_valid << 4); + ref->index | dpb_valid << 4); + set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom); } } } @@ -999,10 +998,6 @@ static void config_registers(struct rkvdec_ctx *ctx, rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15); } - /* - * Since support frame mode only - * top_field_order_cnt is the same as bottom_field_order_cnt - */ reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt); writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0); @@ -1166,7 +1161,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params, run.sps, run.decode_params->dpb); - h264_ctx->reflists.num_valid = reflist_builder.num_valid; v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); @@ -1174,7 +1168,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); lookup_ref_buf_idx(ctx, &run); - assemble_hw_rps(ctx, &run); + assemble_hw_rps(ctx, &reflist_builder, &run); config_registers(ctx, &run); rkvdec_run_postamble(ctx, &run.base); From patchwork Tue Apr 26 12:57:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9650CC4332F for ; Tue, 26 Apr 2022 12:58:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241804AbiDZNCA (ORCPT ); Tue, 26 Apr 2022 09:02:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350402AbiDZNBm (ORCPT ); Tue, 26 Apr 2022 09:01:42 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCED617B983; Tue, 26 Apr 2022 05:58:35 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 9A1961F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977914; bh=MTyE9QSy2RUy2J+DqczMQkzQ2U7CksfjNGFIrOd9SV8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J9X8SraGQdLyW+/JGUWxPCHGxezEXLlPh5u0O88RM7vcGfKUZ1NvV8FhDg9p3TbfV KJqToHGy8OAloKMz37PAL5AAe2VNPEHwmLhGX1wqvA/fzDZrsPArcgOmPca5aeyllJ yW+FyC4MMSs1gDzXPzSdUVfwT77sU4oFCs0WXV5EwbguRS4Ui8Pi6Kj7y/J1o9chV7 msxtO1hRbODKHyxxlAdKBGrKxvEB39UoulTByyp54H/9METlccF8u1d7hHlsRTjJ2B o5oXWPrhNtvvt2WbIcpxXa+m/j4mFD4P/OlEDp0QagtTCgp7DrlMWjtEPOpKAJz0KA jV/SE88oopK7w== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 20/24] media: rkvdec: Enable capture buffer holding for H264 Date: Tue, 26 Apr 2022 08:57:46 -0400 Message-Id: <20220426125751.108293-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In order to support interlaced video decoding, the driver must allow holding the capture buffer so that the second field can be decoded into it. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 4 ++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2bc4b1a40989..7bab7586918c 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -127,6 +127,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { .ops = &rkvdec_h264_fmt_ops, .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts), .decoded_fmts = rkvdec_h264_vp9_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, }, { .fourcc = V4L2_PIX_FMT_VP9_FRAME, @@ -385,6 +386,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + /* Enable format specific queue features */ + vq->subsystem_flags |= desc->subsystem_flags; + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 9df0fba799a4..633335ebb9c4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -82,6 +82,7 @@ struct rkvdec_coded_fmt_desc { const struct rkvdec_coded_fmt_ops *ops; unsigned int num_decoded_fmts; const u32 *decoded_fmts; + u32 subsystem_flags; }; struct rkvdec_dev { From patchwork Tue Apr 26 12:57:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD852C433EF for ; Tue, 26 Apr 2022 12:58:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350376AbiDZNB5 (ORCPT ); Tue, 26 Apr 2022 09:01:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350381AbiDZNBp (ORCPT ); Tue, 26 Apr 2022 09:01:45 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 982D117B3BB; Tue, 26 Apr 2022 05:58:37 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 498941F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977916; bh=rqMU1/6TJ46L9lH5r2+bP74ysDX3ywT+LO6k1I+k3Bg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NN38d68dmHTqhw4X0vJpCFONROr///7o0gkQVxpcwDpQj8Jz3KwUV+lRZ6Gi7dbRj gbmNVtVshNHarIw1F26EsLUgxGPzqHidqyagAW0FfA06AVfEwlcjDMef4Te3Kx/UDO eh5Xc4y3fhdtPBW2jrxfjzpBJWAwoJcc8lSliupb+DA0kRPI9sYJMvmxUycl4sQJkl ZnWY/7oUZxfN7qUeJYl5eBXwuZbozC0bufbyGQt6TaU8dAkNVTZ57fYuuL6gUsULQT GpPwN9luugkjVokP+4YYhj0OYebaC7hMtmTrcfPw+qmIGzhtpShgaq34Xea3VHQEbN hARecf+vWDilw== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 21/24] media: hantro: Stop using H.264 parameter pic_num Date: Tue, 26 Apr 2022 08:57:47 -0400 Message-Id: <20220426125751.108293-22-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The hardware expects FrameNumWrap or long_term_frame_idx. Picture numbers are per field, and are mostly used during the memory management process, which is done in userland. This fixes two ITU conformance tests: - MR6_BT_B - MR8_BT_B Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_h264.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 0b4d2491be3b..228629fb3cdf 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) return 0; - if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - return dpb->pic_num; return dpb->frame_num; } From patchwork Tue Apr 26 12:57:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C4C0C433EF for ; Tue, 26 Apr 2022 12:59:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350459AbiDZNCI (ORCPT ); Tue, 26 Apr 2022 09:02:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350410AbiDZNBq (ORCPT ); Tue, 26 Apr 2022 09:01:46 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CE48177D42; Tue, 26 Apr 2022 05:58:39 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1B85C1F43999 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977918; bh=XEgENTYjCMT2Anghn2qsNUIMLssrmzXSWUAtGOk2EIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WPDkJ3gJEAFrvwPz4kOJCVGDXM69w7LquNYCOmkDxy8hKiZhuHv8vwhLVFR32zL4a GZFenh67Yc8514goV0s8OFXXMF1zLMrphjl9JlgC4cqg4k9HbcsCmO+q93q27VylJ9 MUPV1OyWv4X/5Ny9Xyn3EL9f13FZhfpDaMqdSGa69D7uGqlbGb0WWOIeZnaOihf6fN GDHh0QjC6IsvTHlCUySvLox9sKEr2wUuqQq1jIGLQhgWnpuipLrCZXeNqCiAyCg+Ya 5yae5ut0kQN5j2mtL/wsZwKIzGRe7n6VgZuF/iNEkf5tmJ+G8/1O5A40i7E+yy1Jcu ZqsQp8Dvl7YzA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 22/24] media: hantro: h264: Make dpb entry management more robust Date: Tue, 26 Apr 2022 08:57:48 -0400 Message-Id: <20220426125751.108293-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The driver maintains stable slot locations for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb; From patchwork Tue Apr 26 12:57:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BE3CC433F5 for ; Tue, 26 Apr 2022 12:59:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350461AbiDZNCG (ORCPT ); Tue, 26 Apr 2022 09:02:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350429AbiDZNB4 (ORCPT ); Tue, 26 Apr 2022 09:01:56 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A2CF17DA2F; Tue, 26 Apr 2022 05:58:41 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 165331F4399E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977920; bh=pVl8tjC66altjb15PP079+w75QgIf0TKWek+rq3jsWY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LXCJPO6QrlKwvHosSiRMLPaOQ9AFK/j3CKMJ3oDEVEAuOIEpoHuepVI48IN9MMswv xQhVumfQMc6WzxJTwsz0XL7QuZgAEl4ffOOVljx6BxBdEu6SoM/vmhX0glj8wuCHcM b2bCStl/OHiGD6TXCi8Ha7QwKYfbG+TtvxS6ThO0s/n69E9zWAva+WRXD7nrbWjInU 6DQUWRFY/y8WwWdzvQejVOJxkIgAVqiNqHb4Su69fVwwfmiynRJa5o0QFQ05roxmp5 +2EnYp78w9c9HyuUZOexRLse1c+lpgtGGllcbGVE60qYZfsscOljJu5uxsXdh6lcpP QIFWv5CiLJM1g== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Jonas Karlman , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 23/24] media: hantro: Add H.264 field decoding support Date: Tue, 26 Apr 2022 08:57:49 -0400 Message-Id: <20220426125751.108293-24-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This adds the required code to support field decoding. While most of the code is derived from Rockchip and VSI reference code, the reduction of the reference list to 16 entries was found by trial and errors. The list consists of all the references with the opposite field parity. The strategy is to deduplicate the reference picture that points to the same storage (same index). The choice of opposite parity has been made to keep the other field of the current field pair in the list. This method may not be robust if a field was lost. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 122 ++++++++++++++++++--- drivers/staging/media/hantro/hantro_hw.h | 1 + 2 files changed, 109 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 7377fc26f780..7502dddb324c 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -22,6 +22,12 @@ #define POC_BUFFER_SIZE 34 #define SCALING_LIST_SIZE (6 * 16 + 2 * 64) +/* + * For valid and long term reference marking, index are reversed, so bit 31 + * indicates the status of the picture 0. + */ +#define REF_BIT(i) BIT(32 - 1 - (i)) + /* Data structure describing auxiliary buffer format. */ struct hantro_h264_dec_priv_tbl { u32 cabac_table[CABAC_INIT_BUFFER_SIZE]; @@ -227,6 +233,7 @@ static void prepare_table(struct hantro_ctx *ctx) { const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls; const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode; + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps; struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu; const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; u32 dpb_longterm = 0; @@ -237,20 +244,45 @@ static void prepare_table(struct hantro_ctx *ctx) tbl->poc[i * 2] = dpb[i].top_field_order_cnt; tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt; + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + /* * Set up bit maps of valid and long term DPBs. - * NOTE: The bits are reversed, i.e. MSb is DPB 0. + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame + * decoding, bit 31 to 15 are used, while for field decoding, + * all bits are used, with bit 31 being a top field, 30 a bottom + * field and so on. */ - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) { + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) + dpb_valid |= REF_BIT(i * 2); + + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) + dpb_valid |= REF_BIT(i * 2 + 1); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) { + dpb_longterm |= REF_BIT(i * 2); + dpb_longterm |= REF_BIT(i * 2 + 1); + } + } else { + dpb_valid |= REF_BIT(i); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) + dpb_longterm |= REF_BIT(i); + } + } + ctx->h264_dec.dpb_valid = dpb_valid; + ctx->h264_dec.dpb_longterm = dpb_longterm; + + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) || + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) { + tbl->poc[32] = ctx->h264_dec.cur_poc; + tbl->poc[33] = 0; + } else { + tbl->poc[32] = dec_param->top_field_order_cnt; + tbl->poc[33] = dec_param->bottom_field_order_cnt; } - ctx->h264_dec.dpb_valid = dpb_valid << 16; - ctx->h264_dec.dpb_longterm = dpb_longterm << 16; - - tbl->poc[32] = dec_param->top_field_order_cnt; - tbl->poc[33] = dec_param->bottom_field_order_cnt; assemble_scaling_list(ctx); } @@ -326,6 +358,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, { struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; dma_addr_t dma_addr = 0; + s32 cur_poc = ctx->h264_dec.cur_poc; + u32 flags; if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts); @@ -343,7 +377,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, dma_addr = hantro_get_dec_buf_addr(ctx, buf); } - return dma_addr; + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0; + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) < + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ? + 0x1 : 0; + + return dma_addr | flags; } u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) @@ -355,6 +394,47 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) return dpb->frame_num; } +/* + * Removes all references with he same parity as current picture from the + * reference list. The remaining list will have references with the opposite + * parity. This is effectively a deduplication of references since each buffer + * stores two fields. For this eason, each buffer are found twice in the + * reference list. + * + * This technique has been chosen through trial and error. This simple approach + * resulted in the highest conformance score. Note that this method may suffer + * worse quality in the case an opposite reference frame has been lost. If this + * becomes a problem in the future, it should be possible to add a preprocessing + * to identify un-paired fields and avoid removing them. + */ +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + int write_idx = 0; + int i; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + write_idx = b->num_valid; + goto done; + } + + for (i = 0; i < b->num_valid; i++) { + if (!(b->cur_pic_fields == reflist[i].fields)) { + reflist[write_idx++] = reflist[i]; + continue; + } + } + +done: + /* Should not happen unless we have a bug in the reflist builder. */ + if (WARN_ON(write_idx > 16)) + write_idx = 16; + + /* Clear the remaining, some streams fails otherwise */ + for (; write_idx < 16; write_idx++) + reflist[write_idx].index = 15; +} + int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) { struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec; @@ -386,15 +466,29 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) /* Update the DPB with new refs. */ update_dpb(ctx); - /* Prepare data in memory. */ - prepare_table(ctx); - /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode, ctrls->sps, ctx->h264_dec.dpb); + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count; + + /* Prepare data in memory. */ + prepare_table(ctx); + v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); + + /* + * Reduce ref lists to at most 16 entries, Hantro hardware will deduce + * the actual picture lists in field through the dpb_valid, + * dpb_longterm bitmap along with the current frame parity. + */ + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) { + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1); + } + return 0; } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 292aaaabaf24..fd869369fb97 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx { struct hantro_h264_dec_ctrls ctrls; u32 dpb_longterm; u32 dpb_valid; + s32 cur_poc; }; /** From patchwork Tue Apr 26 12:57:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9E78C433F5 for ; Tue, 26 Apr 2022 12:59:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350441AbiDZNCP (ORCPT ); Tue, 26 Apr 2022 09:02:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350446AbiDZNB5 (ORCPT ); Tue, 26 Apr 2022 09:01:57 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3379617F130; Tue, 26 Apr 2022 05:58:47 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id DCF2E1F439A0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977922; bh=OT93d1YFbCsPl7b/q8Q9Ng7KBwuAX7su/L0Pt2JnS6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cBZN0vuWa0CQYiFTtznCmJ69cnWWpBJjUsOTglusEeiqSVayUvm4THtei/qrnz+XU sxKfjpwT+fa/HTO2lmpV7wL724s3r+9mMdahR1qJH2P8t1iXck7NuGbaxaJly5vKqm +CnjtVkTT3iaegGldibqCE7Vq/khasZDDfEUwlJxfHfFV8bcR5uN90Z9I3HPpzsrwr H5DMc5649WTvQ9vq9OrnHgzL20sr2Z3Y9i6F2AjFq/IVQ6dGlbdqjLCg5i3ua4vQp1 94j5HRwXA2ySzWD6Zqm1P5Cu2nEMfyBeoAfYWvwy7OQ8HM7PJJY/Mr1lI+iWwHPo40 nEC1lF3RULBxw== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 24/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Tue, 26 Apr 2022 08:57:50 -0400 Message-Id: <20220426125751.108293-25-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is needed to optimize field decoding. Each field will be decoded into the same capture buffer. To be able to queue multiple buffers, we need to be able to ask the driver to hold the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 67148ba346f5..50d636678ff3 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n",