From patchwork Tue Apr 5 20:44:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84CD4C433F5 for ; Tue, 5 Apr 2022 20:45:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0J5IeIdFAMo7HIQnXk6tAlm85LtZl1XKxroUNY2d8kc=; b=mO8xFyaiJ35MSI zsRNjr4VX11T2FAfDtY7msAZsqwsbBJHcl/DqdlJbL/LWNS+0QmUP719oqUPZYJ02titKaKl24PT/ /BixkRhInEwANb5gipl9Fv1Jivpn+IcDJbqS9UOICj2Urpy7h53zOu5BmG07Fiv/eBpdi4/mWxymN 1MI+mJJOyptRA/TgqcM/0KK6YU5Elj+wEloBDPWAy8DN7HWbfjm+GMkijevi5ozO+Uis58Nr+tWIh bjCKe9sFiTJ169mSfGw4cotk4WD9KK5romcQQ3lnY31awychMrjvQxisRhwfJGo0qzbDySJWUlIv4 hTiI8hB6q/SqO+DrwBoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq2z-002d3L-FQ; Tue, 05 Apr 2022 20:45:05 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq2m-002cxk-04; Tue, 05 Apr 2022 20:44:54 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 0C8071F4480B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191489; bh=/4x2dOVmlW5X3SMWtdjUn/oKHye1Q/xNZ6Y9N4e0NSw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xmm0t+cgocv3IS/j0VcDnBLz+QifOxbtn4lixay51yEGNHxYIfdj3MKHye/T4jYBX UD1ogaW4x/oD5yU7KN6ABNUO7ZCPTYYuLGWxpB7SI9+42PqqbQ+uYiT9Yq09sURyh+ OglDRu+3MbToFilinIBoKNyJdN01qi0CZB11xyz12mOGXq7C9+Xce/lCPErvSP8eOD 8ltLV0b6ARXus8vGhk0ZTMSrAQsfSamyei1rKoZTI8LP96tPnsz8Ez9OU7bFlesNH7 EILkJdHMNHJsYNik1kaNFohvu2xAg8ukkDMd5vE4IWIaHjljm+XHmGLX/VfKEP/TaG 0cDlqQmf+diXQ== From: Nicolas Dufresne To: Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman , Heiko Stuebner , Matthias Brugger Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org Subject: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist Date: Tue, 5 Apr 2022 16:44:06 -0400 Message-Id: <20220405204426.259074-6-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134452_350027_5FDA3A95 X-CRM114-Status: GOOD ( 17.50 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org In preparation for adding field decoding support, convert the byte arrays for reflist into array of struct v4l2_h264_reference. That struct will allow us to mark which field of the reference picture is being referenced. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++- drivers/media/v4l2-core/v4l2-h264.c | 33 ++++--- .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++---- drivers/staging/media/hantro/hantro_hw.h | 6 +- .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++---------- drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +-- include/media/v4l2-h264.h | 19 ++-- 7 files changed, 116 insertions(+), 107 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c index 43542de11e9c..72c599e05a47 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param, /* * The firmware expects unused reflist entries to have the value 0x20. */ -static void fixup_ref_list(u8 *ref_list, size_t num_valid) +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b) { - memset(&ref_list[num_valid], 0x20, 32 - num_valid); + u32 i; + + /* FIXME mark the reference parity */ + for (i = 0; i < b->num_valid; i++) + ref_list[i] = b->index; + + for (; i < 32; i++) + ref_list[i] = 0x20; } static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); /* Adapt the built lists to the firmware's expectations */ - fixup_ref_list(p0_reflist, reflist_builder.num_valid); - fixup_ref_list(b0_reflist, reflist_builder.num_valid); - fixup_ref_list(b1_reflist, reflist_builder.num_valid); + get_ref_list(p0_reflist, reflist_builder); + get_ref_list(b0_reflist, reflist_builder); + get_ref_list(b1_reflist, reflist_builder); memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, sizeof(inst->vsi_ctx.h264_slice_params)); diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index ac47519a9fbe..afbfcf78efe4 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, pic_order_count = dpb[i].top_field_order_cnt; b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid] = i; + b->unordered_reflist[b->num_valid].index = i; b->num_valid++; } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) - b->unordered_reflist[i] = i; + b->unordered_reflist[i].index = i; } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const struct v4l2_h264_reflist_builder *builder = data; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist) + struct v4l2_h264_reference *reflist) { memcpy(reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist) + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist) { memcpy(b0_reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c index f49dbfb8a843..9de7f05eff2a 100644 --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; int reg_num; u32 reg; @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 0; i < 15; i += 3) { - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]); + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++)); } @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx) * of forward and backward reference picture lists and first 4 entries * of P forward picture list. */ - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) | - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]); + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) | + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC); /* @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) { - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) | - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) | - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) | - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) | - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) | - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]); + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index); vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++)); } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index ed018e293ba0..2bc6b8f088f5 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - u8 p[HANTRO_H264_DPB_SIZE]; - u8 b0[HANTRO_H264_DPB_SIZE]; - u8 b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; }; /** diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c index 64a6330475eb..46c1a83bcc4e 100644 --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; u32 reg; int i; @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx) b1_reflist = ctx->h264_dec.reflists.b1; p_reflist = ctx->h264_dec.reflists.p; - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) | - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) | - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) | - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) | - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) | - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]); + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) | + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) | + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) | + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) | + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) | + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74)); - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) | - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) | - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) | - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) | - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) | - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]); + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) | + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) | + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) | + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) | + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) | + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75)); reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) | @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx) VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83)); - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) | - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) | - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) | - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) | - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) | - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) | + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) | + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) | + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) | + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) | + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100)); - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) | - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) | - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) | - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) | - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) | - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) | + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) | + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) | + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) | + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) | + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101)); - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) | - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) | - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) | - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) | + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) | + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) | + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102)); - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) | - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) | - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) | - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) | - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) | - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) | + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) | + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) | + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) | + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) | + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103)); - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) | - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) | - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) | - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) | - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) | - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) | + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) | + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) | + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) | + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) | + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104)); - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) | - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) | - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) | - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) | + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) | + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) | + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105)); - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) | - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) | - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) | - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]); + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) | + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) | + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) | + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106)); reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm); diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 951e19231da2..3c7f3d87fab4 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - u8 p[RKVDEC_H264_DPB_SIZE]; - u8 b0[RKVDEC_H264_DPB_SIZE]; - u8 b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; u8 num_valid; }; @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, switch (j) { case 0: - idx = h264_ctx->reflists.p[i]; + idx = h264_ctx->reflists.p[i].index; break; case 1: - idx = h264_ctx->reflists.b0[i]; + idx = h264_ctx->reflists.b0[i].index; break; case 2: - idx = h264_ctx->reflists.b1[i]; + idx = h264_ctx->reflists.b1[i].index; break; } diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4b1c71c935e0..ef9a894e3c32 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; u8 num_valid; }; @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist); + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist); /** * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist); + struct v4l2_h264_reference *reflist); #endif /* _MEDIA_V4L2_H264_H */ From patchwork Tue Apr 5 20:44:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EDD0CC4332F for ; Tue, 5 Apr 2022 20:45:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VMSfzZ4HQ83kDwxV7jiUD5iY7yyfY5TMqQjlPgElKYQ=; b=e+t0n/RQ8exExQ OL61udBVehPeAxRD+wc2YcwjcaWzvPoIJ+7jduDMYt0qBsdwkQFAmK71FqvRNueWDZlOVnad7/WRN HPJeJkZFmmn9eBJUzCdkNv+60klUagZ5bnax9VByBPdock5JrhdWPPIc3ui8qCF1N2F7BghXcbWKs reYS+80SElyFu+DOEI8tEogg99yY65fDi8kEnEaAY3yLjzeSd4wnE6Klp2x5er2pevh1NSG7GzAMr tF+SgPeeqFjK4/wxvlL4vczWgd1znoE1xk1NpB3LpXI1cGuhnx1U7M2TKgtGRkN+ekOQtGGQXfJFv UZ0gwClrBGojYbxbHKcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq30-002d3o-10; Tue, 05 Apr 2022 20:45:06 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq2o-002cy3-3B for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:44:55 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id B0DB01F44811 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191491; bh=vuftnHCEWwJdNLAqyrxNwOkREV8boF39Sd5EZvT7GwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Oph2TQqdeOHtBGxUitduvPvOdm4kEaS4JrYa+6iVs1dwf1kqV5QiMPHbB/hMMH8ZU PCT0MWiySSYbypHk1L0driXE6wusYJa/D99tsnS90oAgQYsJqKpcA1CXdQL3XJTx/w aQ4A8NzuVG88MnN0WzF3uB4tkp+03FXSjOOyYHK8JmXFl7G/EZbWvm9p5wb2JS//Xp Dzrob0ewp4S+Rhig2heeHVJ1Q7EpSWZFEqHfbVqNNW8nRSKSabdu6HJQKDUJdjWr/M 67vKP6lZyqbMWAMirB9zkf3G5JltoVGCVje+B7xyEZ1MJ2C84v070xO19DMmiOb/4g rccH9hzwLe2Vw== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 06/24] media: h264: Increase reference lists size to 32 Date: Tue, 5 Apr 2022 16:44:07 -0400 Message-Id: <20220405204426.259074-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134454_435206_96A551F8 X-CRM114-Status: GOOD ( 12.19 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This is to accommodate support for field decoding, which splits the top and the bottom references into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index afbfcf78efe4..4b46b36526c0 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2bc6b8f088f5..292aaaabaf24 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Tue Apr 5 20:44:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45B7EC433EF for ; Tue, 5 Apr 2022 20:45:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=65e3e0TEfmaWy+/yj+k9Tm5AG+qCaUQbi4ED/K4KRdc=; b=feFNNnD0m4jv9Z ZIhYeCvd5totGSTSTYeqnGiCORlmy8iD4KrslrKoVzPGePpOblrGDFW7T/4NxtV0WmKKGcOoyDTt6 EYGGV8biFt2AvTLF7jkK5z5/M1OinzUEeWtHHl2zccfCuVXq6cZuipf9HdfHd90wv/5Lp2RS7Vs50 O9iIVRS37eNnoWulxEMDWqvWJ/t7XuCA9sbuwNAuiWQf3pjKWDcEFDQdTbpJSuFfXOJzy0AI/OwP2 2OedY10ccxcYoBUh3pZj3Pgm7dsaLWrphJ3EZx/Artb0HiM5QSR2UMUQwrfTO/5ZfSiZbIGb3qT0q bn4QbZymVhFjGOTb7eyg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq30-002d4f-PW; Tue, 05 Apr 2022 20:45:06 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq2v-002d1Q-V0 for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:03 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 6DE231F4483B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191500; bh=8oDcE7AuYklwa6l0DYob/jQnR/CVHnofWCWQW3y5gNM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DeZjdI7imYZZvg+kDC9pPL5dARx83mv65wa5gmDuf9DQMxc84BSLeXsvjKegBDTEG QxDtbC7egcV8QQKplljF4VoJ48UeaFvL6qeYM2UwGF4RtA9iybZkC1BvpSBDZdinR8 XxhklC8SPZdW0mJ8roUeqszepUv4NLbJAoyUtWfjfFP2x2WiV/DvXCmF5x8lkF+Mqi JGQtSRra6PkqVtB3wI/QaJsCcRCziJg+Z+MWX6gMs7O5Of5xgTwg6MakmbmtSRFRNv GqK7eR6CPoHbDtM8mPIih6JGxLihDTd0T+yEh5l0yphROBYovOiL0BUYmP696QUvoo MgKhyNr2bBJww== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Boris Brezillon , Hans Verkuil Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 12/24] media: rkvdec: Stop overclocking the decoder Date: Tue, 5 Apr 2022 16:44:13 -0400 Message-Id: <20220405204426.259074-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134502_172760_BDDBD6EB X-CRM114-Status: GOOD ( 10.59 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org While this overclock hack seems to work on some implementations (some ChromeBooks, RockPi4) it also causes instability on other implementations (notably LibreComputer Renegade, but there were more reports in the LibreELEC project, where this has been removed). While performance is indeed affected (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed used to synchronously decode each frames, so this hack was simply compensating for their code being less efficient. In my opinion, this hack should not have been included upstream. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Tue Apr 5 20:44:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D72D5C433FE for ; Tue, 5 Apr 2022 20:45:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zc4yfmXX3hqlkZtx48WALALNxnp0n3T7mSv+rKRJ1W8=; b=jtRuq6QqLVV0Ds TkG260XiEMx2nhHJRZcYjL2IbGeZ4L7STNDk+s3JNRqM/ou41tvB623SmQMjgqTzci5bI04xe8nQr P7KQ0bgRPAebN25z4Omr//3cw9RFRqrpH8xfrWCf/mLFiDYdqLuvDtcff1PW+5WKazZVUw+Q6bXpU h3NzUyeWlI+oR5G6lCozXlpdxFDXINRg2SmR3pIx0JFeLr/tjypv8pqNY4ZvXmRmKIffjKKTsernP +rE/TA6xaFEyGj4sySmrpolTMEUu/5X1rNGrx0XrIGsKNOTjtS/sAn7MdfAhETDC3SO7IN76puP4+ Akshm2Q7vmsqjKTf15dA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq31-002d53-E8; Tue, 05 Apr 2022 20:45:07 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq2y-002d2J-1O for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:05 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7D5031F44840 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191502; bh=qqGKTdK+BoJvGXzu5rDFgeoUxEBT3Njqv5q7sTj6RAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nTBbsEIN+ODGAbyDvj4hk9G4suUl/XG2DaUFATQ6ssrFhqtSjFBWvcK+lJeQsEWXz uovYAnTDm+UVwBIBYJnHOCoD1UxniLw3i8PdiluH0S+NcL1VlDueL5SdFY1xCs1OXL ecWsv+tlLJl11xCmzpZ4Pq0IR2Qp9+8T+osmj8X2FypbyuXa8KLhYhyK9U/If4YpdO 9/2k5nQ1o2lPG3bGYXc6itCe2YUV60P85qnAPEl+Qa+DrKrCdg5ox+k68MEVE6+qFh dyVbbxtSOti4xekLHuwv1uWC6CMeDVexz84vAtV6/UTlzH5VJrgtDKqhT6Np1dNUUF l/yl6hvvmi/8g== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 13/24] media: rkvdec: h264: Fix dpb_valid implementation Date: Tue, 5 Apr 2022 16:44:14 -0400 Message-Id: <20220405204426.259074-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134504_288562_FF52F080 X-CRM114-Status: GOOD ( 17.47 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org The ref builder only provided references that are marked as valid in the dpb. Thus the current implementation of dpb_valid would always set the flag to 1. This is not representing missing frames (this is called 'non-existing' pictures in the spec). In some context, these non-existing pictures still need to occupy a slot in the reference list according to the spec. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index dff89732ddd0..bcde37d72244 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -112,6 +112,7 @@ struct rkvdec_h264_run { const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES]; }; struct rkvdec_h264_ctx { @@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, } } +static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, + struct rkvdec_h264_run *run) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) { + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; + struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; + int buf_idx = -1; + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + buf_idx = vb2_find_timestamp(cap_q, + dpb[i].reference_ts, 0); + + run->ref_buf_idx[i] = buf_idx; + } +} + static void assemble_hw_rps(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { @@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = 0; + bool dpb_valid = run->ref_buf_idx[i] >= 0; u8 idx = 0; switch (j) { @@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (idx >= ARRAY_SIZE(dec_params->dpb)) continue; - dpb_valid = !!(dpb[idx].flags & - V4L2_H264_DPB_ENTRY_FLAG_ACTIVE); set_ps_field(hw_rps, DPB_INFO(i, j), idx | dpb_valid << 4); @@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run, unsigned int dpb_idx) { struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; - const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; - int buf_idx = -1; - - if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - buf_idx = vb2_find_timestamp(cap_q, - dpb[dpb_idx].reference_ts, 0); + int buf_idx = run->ref_buf_idx[dpb_idx]; /* * If a DPB entry is unused or invalid, address of current destination @@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); + lookup_ref_buf_idx(ctx, &run); assemble_hw_rps(ctx, &run); config_registers(ctx, &run); From patchwork Tue Apr 5 20:44:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3322BC433EF for ; Tue, 5 Apr 2022 20:45:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=S5lZbopukreAxcM8xGXSCitvly1dJA3e5jSJvE/7w+U=; b=y1I2Wp8ms8mqW+ 3P263za+zEc6scuQV3uhgZBnkz8mgK1sbvfLs96zlWOqw6SZmfR9Y/GtBcEHPns5z9/Qt+IZksX02 s6n8JVvtvrGx9AYgGnBYrWz/oS2MXx+yOy5YXkyKJaR1zkPWngaGe4pQM1pHQ9cHA8zvOsXdfpca2 BVJknEKRGf7m2OPh/Vmf1mos0mVGrbHK9WCNv0lQ1dpzCUqGYCSqwPcifLFCbSW3dguNrFMzTjUEd SWgzBbBujlLv6N+mONucnJb1EJK9o5CNTDYOrgXGzuPrTM1AXfpbHCEf4d564BMwvQXTv2il1IrPy KP8dz9+FNjGGY8OKbSKA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq34-002d82-C3; Tue, 05 Apr 2022 20:45:10 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq30-002d3A-3k for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:08 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7BD5E1F44865 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191505; bh=FmxP3Of2pm4gkbqbbaer5qaBCfIvVbPhhE5DeBELz4c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LJhLobEreEIBQIAqgI5fPU5cdPQvn9SeAwIF1icf4FStyj8Ne3xA2JAXJT0xwDJnR 2Y/rTcEnJbFveYy6xO1foPNJuFpQTIFO9K2xV1cyU7SdWwKkOWFrCRP3QLNJM2x/b6 9HK+aKOnmHf9DkKK3+oYWX6pvu8TPtUYSXN9qgEbJAslJmy+R6c+g1OaRawJorYNW3 +y7yayfl/q7neKlj9LK1qtNvPlXHGlJ5GaPU5awD/XrKLk1T2G9EXBeYSuSWG7wJH1 kMKjOlbJGXbWBiu6kqp5s/S8u/l7iRuYeJ34A7JHoO73wC3OSq3RhGEKNEFHut2U0t rIp57lJK9utDw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Jonas Karlman , Ezequiel Garcia , Mauro Carvalho Chehab , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 14/24] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Tue, 5 Apr 2022 16:44:15 -0400 Message-Id: <20220405204426.259074-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134506_330632_3520D373 X-CRM114-Status: GOOD ( 11.32 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet are 3 bits wide. 8 is wrongly added to the bit depth values written to these 3 bit fields. Because only the 3 LSB are written, the hardware was configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index bcde37d72244..8d44a884a52e 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Tue Apr 5 20:44:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 04EAAC433EF for ; Tue, 5 Apr 2022 20:45:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QmWr8twgRuQ5gIHbyVZ+dfr5e0uTH47QtmJuUUvueMQ=; b=uFUzEJzbH1XImX QPVq1r+tZXSZjAJgLTuTiN8nGXWO9OCmB/WDzgz3jMHs+0mFQSpTwW17vkjF8BzuHTaCbli0lGmXP dOzHAHihPcyqtKc88qfupaZ6gyqw9KnLXu4uzW2tT1uts2HL2l0OvfZlgGYg4vqU+CGjAaDQUVsCQ pFM22zylhHhEu7DNUKzhKa847fpn29POF3i6Cd+Iyfjp5llBKCzLumWtILPRWejfS1vGOy8O/5gTD jPDtgFLejr2B4HvkmYbxqa6VXik65tyliMmX/K/b6gUR7AMUUnC+ki+cW+tzRa+dILUTpR3Yqbeck JNnXdbNJKXWF3+PPCeKQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq36-002dAu-JP; Tue, 05 Apr 2022 20:45:12 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq31-002d4w-V5 for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:09 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C469B1F44840 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191506; bh=sDJRbDHtwMEd4Z1AC3fnth9aVik8zbzUMGexAc+afw0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DPvz1ZbvaFugvYyOfrcTyPIuHXXuFXkDY7tNFLOh6LnAEsLHeMoMJkDDqTI3gi17U EHrVNOHmlXL+3kshMEXe/cWztY9kf6lIUtLvjFdi8afLlam5jFnF3hBss3bE4iGXqB Cpn5V0Sf8udrgkU6uvEfixu5gljp5r7VZ7Cn1TddHlQx3YqPyXwOFbm/YmEJsbpjzy wr7pFKSl1KVPuxX8xu4HHk6D73iThiRqBBXwsolwF31hrYJEWzDdkad2yAPkCOhEW2 Ea67FGWiWHf5BKs4tOlVeU9xcl1xEhb+vjkJVs3GhcZK7uZjpYjVwJFRZ5btbgXQXR UEZ+WS8AgXYyw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264 Date: Tue, 5 Apr 2022 16:44:16 -0400 Message-Id: <20220405204426.259074-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134508_161511_C58E7486 X-CRM114-Status: GOOD ( 16.52 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org No function change, this moves H264 specific validation into the H264 specific code. This is in preparation of improving this validation and reusing at streamone. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++---------------- drivers/staging/media/rkvdec/rkvdec.h | 1 + 3 files changed, 30 insertions(+), 17 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 8d44a884a52e..0dcbcb1bac80 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) return 0; } +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + } + return 0; +} + const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { .adjust_fmt = rkvdec_h264_adjust_fmt, .start = rkvdec_h264_start, .stop = rkvdec_h264_stop, .run = rkvdec_h264_run, + .try_ctrl = rkvdec_h264_try_ctrl, }; diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2df8cf4883e2..e3d44d5b35f3 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,23 +29,12 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; + + if (desc->ops->try_ctrl) + return desc->ops->try_ctrl(ctx, ctrl); + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2f4ea1786b93..9df0fba799a4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops { void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf, struct vb2_v4l2_buffer *dst_buf, enum vb2_buffer_state result); + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl); }; struct rkvdec_coded_fmt_desc { From patchwork Tue Apr 5 20:44:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10006C433F5 for ; Tue, 5 Apr 2022 20:45:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=K9BF2mT/F43mDhy6gVo4JtAyUjcfafJMQi7662osSFg=; b=d1OgPaRFfgAmXN TTmEThRV+30hqytduOAxD0hNyanvGHi4z8e8p2szYKei9E14kGM8PITPW0Ys7Z1cnEab1G/feHQBx 1dp8wbvnALQnb3T/bKn41PpbanX2BQTqw0Vq/I0kYA0SACWTE4EgtELnJE0ZJkx2+3lL+smCflVvq WNi2EvvNPVpWOJ3BxJiILj9bzlpA4MgLkzrQnHgDVgKu8gryjeipZpHjoa7/mtFQqTPx3HNbc60Ci D6wLoozRG3WeUtc79eRbs6WKzVKj4jVEvBPuOZhxptH9w3Rh8hYFlRA9NS18MHP5uPtWthABsqJvL +yIE0vRwOSSpkonhAOXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq39-002dEc-B7; Tue, 05 Apr 2022 20:45:15 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq33-002d6Z-SO for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:13 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 6DA311F44871 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191508; bh=RVect5wnawuQWmXyzNYDAw2lNjPQxUXn0VLgDqyjlvI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D8r5g4fh0PKqWeTQYhm6WDFxxPXXEEVPlOcQ3DSJm7pvN3JL9i8oTw/NPWXj/nRd5 YWO3Th47EwJzElmgsYOoGCqsFtqThHlpOFd1WdLps3lDxmzh1bkHmjC1xj/x9YSOZO GHlznuMdW1ROVg6uiLiRtqV4OU7Bz17k6ds2qCT+sNzXb+GGQ4nQoA03WI34P4kC2Y i7f/MX0Fv/lLMLVsJGlB3l1ZNkKRoh1QkYycKaG9xi4Fp4eJ0ea6wOkCSGZQSlHbPm macYKvHBpv02ArdI6dVl8ypu5YdYEDFCspot1KKJwHjEDo7zZzyhA+N11f/WtLxKC/ x/7ogl1um56rg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Tue, 5 Apr 2022 16:44:17 -0400 Message-Id: <20220405204426.259074-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134510_246867_F0D379F7 X-CRM114-Status: GOOD ( 22.00 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The width and height in macroblocks is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. The validation is happening in both try_ctrt() and start() since it is otherwise possible to trick the driver during initialization by changing the OUTPUT format after having set a valid control. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------ 1 file changed, 59 insertions(+), 19 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 0dcbcb1bac80..f081b476340f 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + + /* Use the SPS values since they are already in macroblocks + * dimensions, height can be field height (halved) if + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows + * decoding smaller images into larger allocation which can be used + * to implementing SVC spatial layer support. + */ + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); + WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx, return 0; } +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx, + const struct v4l2_ctrl_h264_sps *sps) +{ + unsigned int width, height; + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + /* when frame_mbs_only_flag is not set, this is field height, + * which is half the final height (see (7-18) in the + * specification) + */ + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) + height *= 2; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; + + return 0; +} + static int rkvdec_h264_start(struct rkvdec_ctx *ctx) { struct rkvdec_dev *rkvdec = ctx->dev; struct rkvdec_h264_priv_tbl *priv_tbl; struct rkvdec_h264_ctx *h264_ctx; + struct v4l2_ctrl *ctrl; int ret; + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, + V4L2_CID_STATELESS_H264_SPS); + if (!ctrl) + return -EINVAL; + + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + if (ret) + return ret; + h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL); if (!h264_ctx) return -ENOMEM; @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + return 0; } From patchwork Tue Apr 5 20:44:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801938 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59137C433EF for ; Tue, 5 Apr 2022 20:45:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ppuUOnHEfELHAIkkpSn36MxB1B/S6mBkuWMi4hW2vvE=; b=XxBntHLYKI3Y1w Jo+lgAJOPH6VdqF5861D+068TuTzNt6W9CwVPZnybcjQ/ZHvJawrz2DA4XHZ7TYCgD9vhCRKX7giQ URQ3zu74EUgLwNVLolGLAVtSvC20WQtcycLNM2nQBQld+qJr6nnqHHTMOF0NJfSCyjMSjAPPhqbv5 AFf/t9f+8Jpt1ph9d4bu66qpG60fMujUj4y+oCI+ZYQdGXgXMxkdjVDfbhy1N3T0KnIXFY5m6D0XF XCgNia97wvNTocCWU58RFSGoMQT8X+sTz6I/EY3/bC3D1UEawqTAzNw/bU440V9DmzjQmuqnXX/nW O0ODlNlBMzKUyN1rpEzw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3B-002dGr-Mh; Tue, 05 Apr 2022 20:45:17 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq35-002d9D-Rp for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:13 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 47B301F448A1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191510; bh=t6w6zh2urP7UBHlWH5GWukchWArbLK9mbtqizbdraAY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TkK+aVQN462OFzg3kXJMyyDzO9T5YZP4amtnyD+IC682s9lJt9HfFWJIqL5686Ljz /JF9E3x+CLnwgJ8byoIc1gLKjhAYVf5LLJvnskfXyFbE0nNpttYv4gxOg1s9CsRTNs AwdTGwxXdcxGIx/YlfnIu7+3Za8p8RI0he0Yu3Pa62uJ73r6KAHWnd+2NyYoKq+xfR vULVmlkefvGmBpcWiLzWObKsI1+YnxZxTF0muf+/GqNSWumbYdHjzwdySXF38cSXGe eWSRYf7IRpZi0MWDeZfFszhSmt7gZWD2MD7j0ZuG/oZyTGQAwwt26oDn+Fis6PS2GV ylea4nyVQ09Pw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Jonas Karlman , Ezequiel Garcia , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Tue, 5 Apr 2022 16:44:18 -0400 Message-Id: <20220405204426.259074-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134512_104582_86664DA1 X-CRM114-Status: GOOD ( 11.06 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this cause issues to decode the second field in a complementary field pair. Fix this by using inclusive comparison, less than or equal. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index f081b476340f..60eaf06b6e25 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Tue Apr 5 20:44:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 57F55C433EF for ; Tue, 5 Apr 2022 20:45:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GHZ5f+fUUVNEJZM3gonoS/rsoBSs6dTXw4VF2XoHKwM=; b=Q9u/AUTDRLo6FW k2tzYZQuFVwBqf2yx/jqNVK6pxtth5NXYMO9+Ms1Qr9H+1/IcxWKyWdUpkdWIbJn/5/meXLSJspmO RT+FQ7d32Y1GgRBv7IlmcbFrnHYd+ZnWvXzZh4CEvogM0aQWBYL9b3M3m/lkk5GjCgYtWza/4pDF3 LHMAgFOoTyIQNhXBaCgW+P2f4wnOkKvoKpzEOMc5y9WInPhLx9wQ5o5XJ1NpeRIjhoEAEm/d/n9ER XHeAPa9zBkCIiZTjyfBRK9/eDA2dvkq01yOXOMmgD89ZHhgCfY8sEi2hcaUXJOMcEbB0NT/1MeINS uZ1FveLq/DmVIDVJcVZg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3F-002dLa-GW; Tue, 05 Apr 2022 20:45:21 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq37-002dBO-JR for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:15 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 44DB71F448AC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191512; bh=mlmVguJcq18q2C5FsZYpROZPrzKGkqiR80YOa10U0rQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cuG1CSJZKgn64YomMpEzZ8tvvSWpKLfGOmMVqaMxRdcSC280sF3qjfiRtm8SZfnsc oDIyMMunNmiYNk+GRcGtaiwuUk/jTZkO1ErSmKQnvA66wdhT2cCvOV9HSD+J2RHWBt TSLnRP2fwVjaK5Zt/5P4WDqJAxX1USTkyI88cGP5cXewlSGKUBzpuiyPnpFbJgr6H5 e505TFfynD0h6YBztLgGCQB64ot1b+E28/yzdzHGH28aAbxMa7eZXAz6oTErCH1vH4 vnOEjZk5+e2J++7gbnEU4t841y1GAZfwoTAxZvv4ImcRoq8U+WGYt2OaOf8fMPziZQ ar0q8v8A2R/DA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Jonas Karlman , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 18/24] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Tue, 5 Apr 2022 16:44:19 -0400 Message-Id: <20220405204426.259074-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134513_860909_D347AF14 X-CRM114-Status: UNSURE ( 8.25 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OUTPUT buffer resolution. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index e3d44d5b35f3..2bc4b1a40989 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -257,6 +257,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Tue Apr 5 20:44:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801940 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EEB83C433EF for ; Tue, 5 Apr 2022 20:45:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l08GbY938rRxQe9S9QFL9RG7vnsgiPElEHecfiOZ5c8=; b=h9Iq/VHzo4DYN5 JNxj/DAvdC1jI/7KnDGydqjtyKEQqGTPqrlKvD7MS6pj7pfHfF2ngYhAZqM030pRCb4eDx054B4SY S7u3f5g/fB4Eqfz7JTfCp28yR1IrSNleGM34sEwKhSZHGv4phGEXn7UX4jDpEPxc8ak/l7gKuCuR1 vTPIDd7hJwbHDKBUuFKsXSDAEScpjHDaQD1tMerT8V22Cya5gaSJ+XwOUbSFetAMbXdwUzeMdUUEp KmYIKOs+RJAMEPg9gaYd/p4vLOikxUANS7ARX46i99+rZSV9RvTxRVAP7H2BPnkW8Syh0nxyWKOLZ 9wk6J3BZUkjpIqbKyjVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3J-002dQh-JV; Tue, 05 Apr 2022 20:45:25 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq39-002dDS-AE for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:18 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1B9041F448BD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191514; bh=EBiSjqkXo2c3hkPGX7BwCqg++QBc8XyeV1CCcGY09D4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bvjReUCSoTw3rNiyYOT/Yl7uSz/RBAsB3McArZ2SLHmvnF4URblCp7apfeW3nYIEH amW3JIT/zr6iSrSn/Fc1BADdhSRYEfASnj4UpswB6SC7M5DRS6l13Geuo6qqleq8GW Gaxjasc09W77XYG9306A347ET5BIbuB/H4LZLvwhOtHhE27aThMH6oYni1A9hmLDY9 GVaOawFo/itXFrIZ21g6q5iXqzl5LjL5Ttrwj1cSdn2BA8j9RGN/n9bPkugeCm9ibW zgEYF5NjT1yUlhzemN6hanC5rQzDJR8PjIhRLILB1GxujvwPgbbtQa46QaM6nE9D2I 7w+xJyGzpAuaA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 19/24] media: rkvdec-h264: Add field decoding support Date: Tue, 5 Apr 2022 16:44:20 -0400 Message-Id: <20220405204426.259074-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134515_717440_F42FD829 X-CRM114-Status: GOOD ( 17.54 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This makes use of the new feature in the reference builder to program up to 32 references when doing field decoding. It also signals the parity (top or bottom) of the field to the hardware. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------ 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 60eaf06b6e25..ef74a556c916 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl { u8 err_info[RKV_ERROR_INFO_SIZE]; }; -#define RKVDEC_H264_DPB_SIZE 16 - struct rkvdec_h264_reflists { struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; - u8 num_valid; }; struct rkvdec_h264_run { @@ -746,23 +743,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; int buf_idx = -1; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) { buf_idx = vb2_find_timestamp(cap_q, dpb[i].reference_ts, 0); + if (buf_idx < 0) + pr_debug("No buffer for reference_ts %llu", + dpb[i].reference_ts); + } run->ref_buf_idx[i] = buf_idx; } } static void assemble_hw_rps(struct rkvdec_ctx *ctx, + struct v4l2_h264_reflist_builder *builder, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; - const struct v4l2_ctrl_h264_sps *sps = run->sps; struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu; - u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4); u32 *hw_rps = priv_tbl->rps; u32 i, j; @@ -780,37 +780,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num <= dec_params->frame_num) { - p[i] = dpb[i].frame_num; - continue; - } - - p[i] = dpb[i].frame_num - max_frame_num; + p[i] = builder->refs[i].frame_num; } for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { - for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - bool dpb_valid = run->ref_buf_idx[i] >= 0; - u8 idx = 0; + for (i = 0; i < builder->num_valid; i++) { + struct v4l2_h264_reference *ref; + bool dpb_valid; + bool bottom; switch (j) { case 0: - idx = h264_ctx->reflists.p[i].index; + ref = &h264_ctx->reflists.p[i]; break; case 1: - idx = h264_ctx->reflists.b0[i].index; + ref = &h264_ctx->reflists.b0[i]; break; case 2: - idx = h264_ctx->reflists.b1[i].index; + ref = &h264_ctx->reflists.b1[i]; break; } - if (idx >= ARRAY_SIZE(dec_params->dpb)) + if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb))) continue; + dpb_valid = run->ref_buf_idx[ref->index] >= 0; + bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF; + set_ps_field(hw_rps, DPB_INFO(i, j), - idx | dpb_valid << 4); + ref->index | dpb_valid << 4); + set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom); } } } @@ -998,10 +997,6 @@ static void config_registers(struct rkvdec_ctx *ctx, rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15); } - /* - * Since support frame mode only - * top_field_order_cnt is the same as bottom_field_order_cnt - */ reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt); writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0); @@ -1163,7 +1158,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params, run.sps, run.decode_params->dpb); - h264_ctx->reflists.num_valid = reflist_builder.num_valid; v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); @@ -1171,7 +1165,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); lookup_ref_buf_idx(ctx, &run); - assemble_hw_rps(ctx, &run); + assemble_hw_rps(ctx, &reflist_builder, &run); config_registers(ctx, &run); rkvdec_run_postamble(ctx, &run.base); From patchwork Tue Apr 5 20:44:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74327C433F5 for ; Tue, 5 Apr 2022 20:45:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YQw5/4VCCvtNKTqT0MH1c1tOK9oTh2kKI2pVgCdk04Q=; b=c1Qftp+L3F7N29 A7h4IW23nbGulAOm0rXUy1AqeAAQbnVHknPNRrRclacYvQqbpABXdAfIf5Iw+8bIwXDLEMb0f97+N PEBIfh9WJfcdiHquI6BVBJRvY+b7/gDpqYDFWYfPxyJIjz1AoeWxdHZ6Gj2XgGQDm5B35yi4IWavw ML/RWF3upxFhPZ7p7DGzmQUSls4lA8HWjU/aT0cLf94yL/LM4DI6Fy0xiRYc1Iic251wyYwKORJeZ wKBgmu3MGF6fy1bQbhbk/T1TwUHlWBfxUH2w4Wax+bHvDOvDK4F6hP1byUP724SRPTbqBhHYJCEF5 HR94sBmQgH9BN3WpsY6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3M-002dUe-Vd; Tue, 05 Apr 2022 20:45:29 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3B-002dG0-6t for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:19 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id D871B1F448CB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191516; bh=MTyE9QSy2RUy2J+DqczMQkzQ2U7CksfjNGFIrOd9SV8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MTXu3z2Sz1/ihUk9LZChVIPAu71KKvYLI63WUFsq4V+t+dI6Wag5rJC/4VfrlWEAt SHCFbLtM0ElNJ6q/LTy8Fka4/R7QxDjtFb5b2ys/ylpAgiNd09Dthhs3i6RlAzF2+8 KkrLLA3Vc5wQaRElF9X6hTJ3mZagl5aRNxB+Pse30ELEthfHEuCEyA6Vtb81yvHYcB KDLbKeGdvxSwFIel+bibpclOtNwo47xv/L+eLMFcyCiaBxeUMKoYJUzLRZdnZQg2l2 QE35nAKfteTRgyw5P75CyRJkYNcCG2otWffSBriRi2X6QkJrsNNUQDZnl8v22Q8s6z aQ2K4uCfvjT+A== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 20/24] media: rkvdec: Enable capture buffer holding for H264 Date: Tue, 5 Apr 2022 16:44:21 -0400 Message-Id: <20220405204426.259074-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134517_441084_E1FD0EE5 X-CRM114-Status: GOOD ( 11.65 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org In order to support interlaced video decoding, the driver must allow holding the capture buffer so that the second field can be decoded into it. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 4 ++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2bc4b1a40989..7bab7586918c 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -127,6 +127,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { .ops = &rkvdec_h264_fmt_ops, .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts), .decoded_fmts = rkvdec_h264_vp9_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, }, { .fourcc = V4L2_PIX_FMT_VP9_FRAME, @@ -385,6 +386,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + /* Enable format specific queue features */ + vq->subsystem_flags |= desc->subsystem_flags; + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 9df0fba799a4..633335ebb9c4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -82,6 +82,7 @@ struct rkvdec_coded_fmt_desc { const struct rkvdec_coded_fmt_ops *ops; unsigned int num_decoded_fmts; const u32 *decoded_fmts; + u32 subsystem_flags; }; struct rkvdec_dev { From patchwork Tue Apr 5 20:44:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801942 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26269C433FE for ; Tue, 5 Apr 2022 20:45:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JqGPZFUtPTIgrt3qpOCihFL5P1KRK+O0dbzkpDNSbsw=; b=Rpos8OiNfHVZi3 LT4vyQbaYtaB/j1CjqB8XFkSf4lmBn2rBVWBWNWMrIEJ1cfhCVn/be4R8GvAj0kn0deo82V9tLLPX rc2NlSQXyw3fXDl93DhSN8dqWcj27+kWazzM0z727nTunoRYXDo3uN1/vZ0NQjPF2OPrKPaqZhZCq PPegMThKE3BN7J3ap+qrWx/hZX2iFzIlmhXP6GkQ0roc+uQWDG/1QXebyZjyKo41WehRj5N60d5wU qJ/np72t+vUuGsa6lS0wixloS55+TxsPHzVHTluPtg2k+LBlnyn0C+YuZzjtDXkG9xu6N7nSWLSKW 5/fMjBIQKcvkewuHwRQg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3O-002dWy-WC; Tue, 05 Apr 2022 20:45:31 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3C-002dHt-UJ for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:20 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 8AE231F448D6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191517; bh=rqMU1/6TJ46L9lH5r2+bP74ysDX3ywT+LO6k1I+k3Bg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e0xQeerm22/DL9eaBXTFzKpNmAFPoq+WBaI7dSn6jnuIgTrIWvI/lo4J7spQbFoKq Cos7uP0Jy4wKHvWiN6o8mH8HZ4A2QUjKDw4EG9pYcAfTuaD+Q2Fz4HfSCBn0mw9rqb /wSjUUUNcjRJJl8WwuWtruH4MiXh41HjlAPFPg9u4zWPodmfTMx1JRHILNQMIZUpOT Uzgigm0SbC7g1UHGb7X0bCpvo/shMpj49NjkeMilgOELJ6yB4G5IUdnm/0o/g9vk2G OgudDuSyHR6f2nrHuqtGLvs4cgtFVaWV+WxlKyrH6GjmwZXC5CBXCmCL/KMZJ8AWiF MrFNqFeuAl3cA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 21/24] media: hantro: Stop using H.264 parameter pic_num Date: Tue, 5 Apr 2022 16:44:22 -0400 Message-Id: <20220405204426.259074-22-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134519_172679_F602F826 X-CRM114-Status: UNSURE ( 8.54 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org The hardware expects FrameNumWrap or long_term_frame_idx. Picture numbers are per field, and are mostly used during the memory management process, which is done in userland. This fixes two ITU conformance tests: - MR6_BT_B - MR8_BT_B Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_h264.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 0b4d2491be3b..228629fb3cdf 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) return 0; - if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - return dpb->pic_num; return dpb->frame_num; } From patchwork Tue Apr 5 20:44:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF52EC433EF for ; Tue, 5 Apr 2022 20:45:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fiByLGfoKqBnur1KkCUlqe9zDHU+LppUZaG7F7jSSIU=; b=zjUbjres91VpdE 3js+zcrmA7ms16UeZQgcIaVbT4accRcHpV+iiFK3iXw3TchlpXPPVfYi6MNXVXhdS24MzttLUPBM0 iJgWc7labHOX5avWRjtaCly28SAMs41ppy5jhLG/E+hbH2oYZu3GMLXSXXpPLQEBoXdTmZScpXlox GulZTcm6Y9Nz+5NPaZwSezA0C0P/T5VzGeh+7HLuB4iLWi6U4l32m6Yh0xb9hfyZG/snCq3UeIW9x 7W/euSvZrKtzaZvdm0beR0vP4wlqGLi1KlCDQ1d2gUUTIaqRLJRigFJVA+uYvTktN9Oq+1gHtphSZ pAw5mUTvV99A85yKk0dg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3Q-002dYb-Da; Tue, 05 Apr 2022 20:45:32 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3E-002dK3-Oa for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:23 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 564691F448BD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191519; bh=XEgENTYjCMT2Anghn2qsNUIMLssrmzXSWUAtGOk2EIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K00DBOR7Hvo/9zOUg7lYZRuUhYH5gu6hpMp8+CwehK8Ew1thRyoPeXAzuCfW48OG6 ERjNW2T+FmscAkYBw4+LBX0ucFvMjMrALKymQ+f7r+zk/D+OzGNZSv8rEr1LZ1+PVS nXnhCh/EpGCqxJFtGFDm7J26poJEv2y0YMfFzr8NYw7McU3FOqMJzAl+rXkpYzgBjg xkVrnWorKFnYYHovlQlgUosTLSYopg3Nk/CQUFqQF2V0/PAMww4HO9rvzBdFnjyLUt E3yfI0Y53pG3khoL8RIpZULurg+lLoYACu1ExDNT4rllT7Y0tSS+gYUlVbXq9lzWr9 io3mWn9lF1Krg== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 22/24] media: hantro: h264: Make dpb entry management more robust Date: Tue, 5 Apr 2022 16:44:23 -0400 Message-Id: <20220405204426.259074-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134521_002902_00FF1F0A X-CRM114-Status: GOOD ( 12.46 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The driver maintains stable slot locations for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb; From patchwork Tue Apr 5 20:44:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801944 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A017C433F5 for ; Tue, 5 Apr 2022 20:45:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=T66nbeiShRdyJEb1PzZJkjczESq8OpCbZ+JM/D0M9pY=; b=xk8b8xw2oDX6j8 aHCw/DMOjFeeL/61cY48kEhFpVlL20TefjAWL0XHLCDswAdnT9wG/LpqL27pKaiu9NAEop6g2Udvy p8sxGVkY8MQgqYmDHuAozZzqACe3zngn37OI6s0ojVHGINpKsbtpG7khGvakLOqON2V23OWMy5kLj NM53y6/Nx0d6zs4MDil3q1RJ4l4IIwYKB9viA4rMgg5Z8iNPOsduUWtS6WFMwv5C0B9fkuRd6kQQa bhT5rnKf0zYHDKHPfTPsgcYbEQT1MUDin6gaQXBiMb1z40SvDiFqeKGOjwANroq8cLdISPwRBnjN3 S06vKWe6y/uOvfbCfKmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3S-002db4-VF; Tue, 05 Apr 2022 20:45:35 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3G-002dMR-MV for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:26 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 4D03D1F448CB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191521; bh=QXCqhwGgS71Plij0LYpMswWc72NGt82a9mKy6xwpDRk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q/Lm+UaoXCdYtHJ5zWUFnhkCtte0GBWhC443cG3uArOmisV5PW0Wu2yBrqAyRJmua TYgNR1o1k9Fvlo+Uv9kSp6ZJoezYQsXSpGd7xaMNMRZPh+bZNic1PqiT3DjQUpRVBC 82+cbevzAXO0CYsohlws1r+W0LVMzvI46dlByE7adLA1JaJuOz5FgX3WtLQ9glWx/Z f+6UT6G3/xsOdtlYv/JvN0Rco7TN5Ry7Qa0F/DdOdI054sglrsVKis8dI3jxMIsHBl LBadVPJouK21SdtlJCHp/tBlLeViiG+c5e5XpCtMks7+8RsX6bbolw6OIjvN1ln5C6 F7ahd0h1YMxtw== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 23/24] media: hantro: Add H.264 field decoding support Date: Tue, 5 Apr 2022 16:44:24 -0400 Message-Id: <20220405204426.259074-24-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134523_114257_3B9C0D90 X-CRM114-Status: GOOD ( 22.87 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This adds the required code to support field decoding. While most of the code is derived from Rockchip and VSI reference code, the reduction of the reference list to 16 entries was found by trial and errors. The list consists of all the references with the opposite field parity. The strategy is to deduplicate the reference picture that points to the same storage (same index). The choice of opposite parity has been made to keep the other field of the current field pair in the list. This method may not be robust if a field was lost. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++--- drivers/staging/media/hantro/hantro_hw.h | 1 + 2 files changed, 94 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 7377fc26f780..f6fc939aa726 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -22,6 +22,11 @@ #define POC_BUFFER_SIZE 34 #define SCALING_LIST_SIZE (6 * 16 + 2 * 64) +/* For valid and long term reference marking, index are reversed, so bit 31 + * indicates the status of the picture 0. + */ +#define REF_BIT(i) BIT(32 - 1 - (i)) + /* Data structure describing auxiliary buffer format. */ struct hantro_h264_dec_priv_tbl { u32 cabac_table[CABAC_INIT_BUFFER_SIZE]; @@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx) { const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls; const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode; + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps; struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu; const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; u32 dpb_longterm = 0; @@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx) tbl->poc[i * 2] = dpb[i].top_field_order_cnt; tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt; + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + /* * Set up bit maps of valid and long term DPBs. - * NOTE: The bits are reversed, i.e. MSb is DPB 0. + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame + * decoding, bit 31 to 15 are used, while for field decoding, + * all bits are used, with bit 31 being a top field, 30 a bottom + * field and so on. */ - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) { + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) + dpb_valid |= REF_BIT(i * 2); + + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) + dpb_valid |= REF_BIT(i * 2 + 1); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) { + dpb_longterm |= REF_BIT(i * 2); + dpb_longterm |= REF_BIT(i * 2 + 1); + } + } else { + dpb_valid |= REF_BIT(i); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) + dpb_longterm |= REF_BIT(i); + } + } + ctx->h264_dec.dpb_valid = dpb_valid; + ctx->h264_dec.dpb_longterm = dpb_longterm; + + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) || + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) { + tbl->poc[32] = ctx->h264_dec.cur_poc; + tbl->poc[33] = 0; + } else { + tbl->poc[32] = dec_param->top_field_order_cnt; + tbl->poc[33] = dec_param->bottom_field_order_cnt; } - ctx->h264_dec.dpb_valid = dpb_valid << 16; - ctx->h264_dec.dpb_longterm = dpb_longterm << 16; - - tbl->poc[32] = dec_param->top_field_order_cnt; - tbl->poc[33] = dec_param->bottom_field_order_cnt; assemble_scaling_list(ctx); } @@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, { struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; dma_addr_t dma_addr = 0; + s32 cur_poc = ctx->h264_dec.cur_poc; + u32 flags; if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts); @@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, dma_addr = hantro_get_dec_buf_addr(ctx, buf); } - return dma_addr; + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0; + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) < + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ? + 0x1 : 0; + + return dma_addr | flags; } u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) @@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) return dpb->frame_num; } +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + int write_idx = 0; + int i; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + write_idx = b->num_valid; + goto done; + } + + for (i = 0; i < b->num_valid; i++) { + if (!(b->cur_pic_fields == reflist[i].fields)) { + reflist[write_idx++] = reflist[i]; + continue; + } + } + +done: + /* Should not happen unless we have a bug in the reflist builder. */ + if (WARN_ON(write_idx > 16)) + write_idx = 16; + + /* Clear the remaining, some streams fails otherwise */ + for (; write_idx < 16; write_idx++) + reflist[write_idx].index = 15; +} + int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) { struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec; @@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) /* Update the DPB with new refs. */ update_dpb(ctx); - /* Prepare data in memory. */ - prepare_table(ctx); - /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode, ctrls->sps, ctx->h264_dec.dpb); + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count; + + /* Prepare data in memory. */ + prepare_table(ctx); + v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); + + /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce + * the actual picture lists in field through the dpb_valid, + * dpb_longterm bitmap along with the current frame parity. + */ + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) { + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1); + } + return 0; } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 292aaaabaf24..fd869369fb97 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx { struct hantro_h264_dec_ctrls ctrls; u32 dpb_longterm; u32 dpb_valid; + s32 cur_poc; }; /** From patchwork Tue Apr 5 20:44:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12801945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4AC99C433EF for ; Tue, 5 Apr 2022 20:45:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=R99d1t88PfHO1MavJ+lsYLpOc6iWSPg0dQY7Illl360=; b=ShhI7CBgeNYnyX xoTzPT37X9AsiWETY7tU8xeUyyYL+ElOf9/rr+3qd4Wy5PiFYotQAsPmOtderFSJlmqZx/hIGZlke U9BLLYKUToH6CrWACuA2nF/fwzCAB4opG9bWWqqwXXGuYf8JQezsTDXDV5gkoNGWAMzjLxUKLiWhd 7lW4ZtVn0Bk0bEr9FN8dxjk2pukCvA79cxYr3bMISC2QLb/zBhMDgSVdeXsScxy/ZNTQr7pjurnBr lfYaazKLG3P/hrTdhZmMdKeyy5mfNnQyEKpgy6Pjx+lmmqikXGJ9Evkod3I/mklsOM1ghmkHpU+bW XwJK75xNG/OlaJvJs8Tw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3W-002dfL-Sg; Tue, 05 Apr 2022 20:45:38 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbq3I-002dOo-FH for linux-rockchip@lists.infradead.org; Tue, 05 Apr 2022 20:45:26 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1784D1F448E3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1649191523; bh=OT93d1YFbCsPl7b/q8Q9Ng7KBwuAX7su/L0Pt2JnS6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nNvEk9FSqJ+Zr8rUxgKxyJSmv6mX4qQyAS+tDBzhdTzBvK71pbtUjfCi32itjn3dB bt32fyf4cP3I7CJmuyPKvK0UJvZFE0r4V0b9yrhHXajgEfKLUNFdiBUXnGGRmzKqjc 2Ye9J38BecODqViFHR05ROYN/bKVTU3K/+Mn9I3/cVLGPmTvB2C/hsWGWiyPWM3aQX Jst3N1NJctC3fkzp2h9zFGrHiDf432EHwN8wiv2lIORH/CW6rvAn7dt3WjTUmMfECD 4oD08X/MZoh3NhOUyTUw3kZAH1GTL74v5hAq8cDJYwQDW3wF5FXWKiHyOwkZyIUXEe /w5Rxy1mhcmyQ== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v3 24/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Tue, 5 Apr 2022 16:44:25 -0400 Message-Id: <20220405204426.259074-25-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220405204426.259074-1-nicolas.dufresne@collabora.com> References: <20220405204426.259074-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_134524_699342_35EE7052 X-CRM114-Status: GOOD ( 10.14 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This is needed to optimize field decoding. Each field will be decoded into the same capture buffer. To be able to queue multiple buffers, we need to be able to ask the driver to hold the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 67148ba346f5..50d636678ff3 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n",