From patchwork Fri Jun 18 13:15:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDF51C48BDF for ; Fri, 18 Jun 2021 13:16:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A99FB613BD for ; Fri, 18 Jun 2021 13:16:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A99FB613BD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TMcn0tLGXJAn7757vC4bLuvrnknWyFScs7bQIhFVCiw=; b=GJS8OQJxihleEL 8Cp827Ghqj1aMaC9gCg9Otkx66dUZuGCAMMdDFBE2oduFP23QEvp/vVjcngq1K4TigH5kI/O2BpsP 81Sza2iqf++WD0LSYLZbwKMgxMeblSfyloHec6gAUKStRRnh28YrSXSJiqopj3+kfOAR0sI56v76u c2FFXOxjSnci8sKG42AE5BSlCNAyA/cqsXfq7jq7SmNwuNF6xyw4cMgTn64UvQd6w50cVArYjC35/ 41ZIUJiovO+8RFqfjxEW1MojlLfvK5+y/TqHpqiuvdamaojywBornSS2XGJRCsXcW4EBzrtD7CLZO h7VwIVEki6UoN9Jf5nMw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luEMW-00E931-S8; Fri, 18 Jun 2021 13:16:44 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELS-00E8a9-Lq; Fri, 18 Jun 2021 13:15:45 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id EE6C21F448CD; Fri, 18 Jun 2021 14:15:35 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 1/8] media: hantro: Trace hevc hw cycles performance register Date: Fri, 18 Jun 2021 15:15:19 +0200 Message-Id: <20210618131526.566762-2-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061539_019085_B8A6B5AB X-CRM114-Status: GOOD ( 17.38 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org After each hevc decoded frame trace the hardware performance. It provides the number of hw cycles spend per decoded macroblock. Signed-off-by: Benjamin Gaignard --- version 3: - Change trace file name to hantro_trace.h drivers/staging/media/hantro/hantro_drv.c | 3 ++ .../staging/media/hantro/hantro_g2_hevc_dec.c | 16 ++++++++ drivers/staging/media/hantro/hantro_g2_regs.h | 1 + drivers/staging/media/hantro/hantro_hw.h | 1 + drivers/staging/media/hantro/hantro_trace.h | 39 +++++++++++++++++++ drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 + 6 files changed, 61 insertions(+) create mode 100644 drivers/staging/media/hantro/hantro_trace.h diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c index dbc69ee0b562..24a654a936ae 100644 --- a/drivers/staging/media/hantro/hantro_drv.c +++ b/drivers/staging/media/hantro/hantro_drv.c @@ -28,6 +28,9 @@ #include "hantro.h" #include "hantro_hw.h" +#define CREATE_TRACE_POINTS +#include "hantro_trace.h" + #define DRIVER_NAME "hantro-vpu" int hantro_debug; diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c index 340efb57fd18..fef16d1724da 100644 --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c @@ -7,6 +7,7 @@ #include "hantro_hw.h" #include "hantro_g2_regs.h" +#include "hantro_trace.h" #define HEVC_DEC_MODE 0xC @@ -22,6 +23,21 @@ static inline void hantro_write_addr(struct hantro_dev *vpu, vdpu_write(vpu, addr & 0xffffffff, offset); } +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx) +{ + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; + struct hantro_dev *vpu = ctx->dev; + u32 hw_cycles = 0; + u32 mbs = (sps->pic_width_in_luma_samples * + sps->pic_height_in_luma_samples) >> 8; + + if (mbs) + hw_cycles = vdpu_read(vpu, G2_HW_PERFORMANCE) / mbs; + + trace_hantro_hevc_perf(ctx, hw_cycles); +} + static void prepare_tile_info_buffer(struct hantro_ctx *ctx) { struct hantro_dev *vpu = ctx->dev; diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h index bb22fa921914..17d84ec9c5c2 100644 --- a/drivers/staging/media/hantro/hantro_g2_regs.h +++ b/drivers/staging/media/hantro/hantro_g2_regs.h @@ -177,6 +177,7 @@ #define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16) #define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17) +#define G2_HW_PERFORMANCE (G2_SWREG(63)) #define G2_ADDR_DST (G2_SWREG(65)) #define G2_REG_ADDR_REF(i) (G2_SWREG(67) + ((i) * 0x8)) #define G2_ADDR_DST_CHR (G2_SWREG(99)) diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 5737a7707944..8fa0aacb61cd 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -240,6 +240,7 @@ void hantro_h264_dec_exit(struct hantro_ctx *ctx); int hantro_hevc_dec_init(struct hantro_ctx *ctx); void hantro_hevc_dec_exit(struct hantro_ctx *ctx); int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx); +void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx); int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx); dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc); void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx); diff --git a/drivers/staging/media/hantro/hantro_trace.h b/drivers/staging/media/hantro/hantro_trace.h new file mode 100644 index 000000000000..e07d03a5ccdf --- /dev/null +++ b/drivers/staging/media/hantro/hantro_trace.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(__HANTRO_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ) +#define __HANTRO_TRACE_H__ + +#include +#include + +#include "hantro.h" + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM hantro +#define TRACE_INCLUDE_FILE hantro_trace + +TRACE_EVENT(hantro_hevc_perf, + TP_PROTO(struct hantro_ctx *ctx, u32 hw_cycles), + + TP_ARGS(ctx, hw_cycles), + + TP_STRUCT__entry( + __field(int, minor) + __field(u32, hw_cycles) + ), + + TP_fast_assign( + __entry->minor = ctx->fh.vdev->minor; + __entry->hw_cycles = hw_cycles; + ), + + TP_printk("minor = %d, %8d cycles / mb", + __entry->minor, __entry->hw_cycles) +); + +#endif /* __HANTRO_TRACE_H__ */ + +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH ../../drivers/staging/media/hantro + +/* This part must be outside protection */ +#include diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c index c3e616fd4e85..ab6ac620f0d0 100644 --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c @@ -253,6 +253,7 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = { .reset = imx8m_vpu_g2_reset, .init = hantro_hevc_dec_init, .exit = hantro_hevc_dec_exit, + .done = hantro_g2_hevc_dec_done, }, }; From patchwork Fri Jun 18 13:15:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D621C48BE8 for ; Fri, 18 Jun 2021 13:16:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F76E613EC for ; Fri, 18 Jun 2021 13:16:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F76E613EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+jMTZbdzYCwNGBd3Gs8iAvMncD9bhipJrvCl9AKqSVc=; b=Tcxfwf5wWuEKM8 sMLrShAjY3dBvfcmrUA35yfzir7btwlc/CjmUcReGZN0zSwhFahaTVxBBwQaBN3AnzS7RVHPGQ7nH GWvCkOte5gtiwT6twqEPhnwI5mrwzelXN2foH6YWAKUGyPvnZ+bYg8S+cw0RxDAtTfofjsmi0/3j0 Q8Gx4T5ygZDavzixGrwIB7vyJqz2c8N3TOB+OutuUuw8rKEcADiLZq6BO1N9q2kZCX4oCx2Y5Ig1I acpRnANA73E8mDzV8wOWrWbBnhbFL40eKzvYxT0rRjA+DAfSiyJpNaFyqmnW2e16EbcGFxG4+9Dwf I/9HeeoaFnq18vn3A1lw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELx-00E8oc-J1; Fri, 18 Jun 2021 13:16:09 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELT-00E8am-Su; Fri, 18 Jun 2021 13:15:43 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 6FB291F448CF; Fri, 18 Jun 2021 14:15:37 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 2/8] media: hantro: Add support of compressed reference buffers Date: Fri, 18 Jun 2021 15:15:20 +0200 Message-Id: <20210618131526.566762-3-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061540_237136_6D060B2C X-CRM114-Status: GOOD ( 16.11 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org Reference frame compression is a feature added in G2 decoder to compress frame buffers so that the bandwidth of storing/loading reference frames can be reduced, especially when the resolution of decoded stream is of high definition. Tested on IMX8MQ the maximun frame rate goes up to 50 fps vs 45 fps without using compressed frames. Signed-off-by: Benjamin Gaignard --- version 2: - Define the value for compression storage size - Add comments about registers usage .../staging/media/hantro/hantro_g2_hevc_dec.c | 31 +++++++++++++-- drivers/staging/media/hantro/hantro_g2_regs.h | 4 ++ drivers/staging/media/hantro/hantro_hevc.c | 39 ++++++++++++++++++- drivers/staging/media/hantro/hantro_hw.h | 2 + 4 files changed, 72 insertions(+), 4 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c index fef16d1724da..bf19825c4d96 100644 --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c @@ -382,10 +382,12 @@ static int set_ref(struct hantro_ctx *ctx) const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps; const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params; const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb; - dma_addr_t luma_addr, chroma_addr, mv_addr = 0; + dma_addr_t luma_addr, chroma_addr, mv_addr, compress_luma_addr, compress_chroma_addr = 0; struct hantro_dev *vpu = ctx->dev; size_t cr_offset = hantro_hevc_chroma_offset(sps); size_t mv_offset = hantro_hevc_motion_vectors_offset(sps); + size_t compress_luma_offset = hantro_hevc_luma_compress_offset(sps); + size_t compress_chroma_offset = hantro_hevc_chroma_compress_offset(sps); u32 max_ref_frames; u16 dpb_longterm_e; static const struct hantro_reg cur_poc[] = { @@ -458,15 +460,28 @@ static int set_ref(struct hantro_ctx *ctx) if (!luma_addr) return -ENOMEM; + /* + * An allocated frame stores Y and UV planes, motion vectors and + * compressed Y and UV planes. Compute all these offsets given + * the frame resolution and the pixel format. + */ chroma_addr = luma_addr + cr_offset; mv_addr = luma_addr + mv_offset; + compress_luma_addr = luma_addr + compress_luma_offset; + compress_chroma_addr = luma_addr + compress_chroma_offset; if (dpb[i].rps == V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR) dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i); + /* + * For each reference frame Y, UV, motion vector and + * compressed Y, UV buffers addresses must be set. + */ hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr); hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr); hantro_write_addr(vpu, G2_REG_DMV_REF(i), mv_addr); + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr); + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr); } luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val); @@ -475,7 +490,12 @@ static int set_ref(struct hantro_ctx *ctx) chroma_addr = luma_addr + cr_offset; mv_addr = luma_addr + mv_offset; + compress_luma_addr = luma_addr + compress_luma_offset; + compress_chroma_addr = luma_addr + compress_chroma_offset; + /* The next decoded frame as to be put as the last reference frame entry */ + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), compress_luma_addr); + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), compress_chroma_addr); hantro_write_addr(vpu, G2_REG_ADDR_REF(i), luma_addr); hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr); hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr); @@ -483,13 +503,18 @@ static int set_ref(struct hantro_ctx *ctx) hantro_write_addr(vpu, G2_ADDR_DST, luma_addr); hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr); hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr); + hantro_write_addr(vpu, G2_COMP_ADDR_DST, compress_luma_addr); + hantro_write_addr(vpu, G2_COMP_CHR, compress_chroma_addr); hantro_hevc_ref_remove_unused(ctx); + /* Unused reference frames entries most be cleared */ for (; i < V4L2_HEVC_DPB_ENTRIES_NUM_MAX; i++) { hantro_write_addr(vpu, G2_REG_ADDR_REF(i), 0); hantro_write_addr(vpu, G2_REG_CHR_REF(i), 0); hantro_write_addr(vpu, G2_REG_DMV_REF(i), 0); + hantro_write_addr(vpu, G2_COMP_ADDR_REF(i), 0); + hantro_write_addr(vpu, G2_COMP_CHR_REF(i), 0); } hantro_reg_write(vpu, &g2_refer_lterm_e, dpb_longterm_e); @@ -580,8 +605,8 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx) /* Don't disable output */ hantro_reg_write(vpu, &g2_out_dis, 0); - /* Don't compress buffers */ - hantro_reg_write(vpu, &g2_ref_compress_bypass, 1); + /* Compress buffers */ + hantro_reg_write(vpu, &g2_ref_compress_bypass, 0); /* use NV12 as output format */ hantro_reg_write(vpu, &g2_out_rs_e, 1); diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h index 17d84ec9c5c2..0414d92e3860 100644 --- a/drivers/staging/media/hantro/hantro_g2_regs.h +++ b/drivers/staging/media/hantro/hantro_g2_regs.h @@ -192,6 +192,10 @@ #define G2_TILE_FILTER (G2_SWREG(179)) #define G2_TILE_SAO (G2_SWREG(181)) #define G2_TILE_BSD (G2_SWREG(183)) +#define G2_COMP_ADDR_DST (G2_SWREG(190)) +#define G2_COMP_ADDR_REF(i) (G2_SWREG(192) + ((i) * 0x8)) +#define G2_COMP_CHR (G2_SWREG(224)) +#define G2_COMP_CHR_REF(i) (G2_SWREG(226) + ((i) * 0x8)) #define g2_strm_buffer_len G2_DEC_REG(258, 0, 0xffffffff) #define g2_strm_start_offset G2_DEC_REG(259, 0, 0xffffffff) diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c index 5347f5a41c2a..00400dd33a3b 100644 --- a/drivers/staging/media/hantro/hantro_hevc.c +++ b/drivers/staging/media/hantro/hantro_hevc.c @@ -27,6 +27,13 @@ #define G2_ALIGN 16 +#define CBS_SIZE 16 /* compression table size in bytes */ +#define CBS_LUMA 8 /* luminance CBS is composed of 1 8x8 coded block */ +#define CBS_CHROMA_W (8*2) /* chrominance CBS is composed of two 8x4 coded + * blocks, with Cb CB first then Cr CB following + */ +#define CBS_CHROMA_H 4 + size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps) { int bytes_per_pixel = sps->bit_depth_luma_minus8 == 0 ? 1 : 2; @@ -61,12 +68,42 @@ static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps) return mv_size; } +size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps) +{ + return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps); +} + +static size_t hantro_hevc_luma_compress_size(const struct v4l2_ctrl_hevc_sps *sps) +{ + u32 pic_width_in_cbsy = + round_up((sps->pic_width_in_luma_samples + CBS_LUMA - 1) / CBS_LUMA, CBS_SIZE); + u32 pic_height_in_cbsy = (sps->pic_height_in_luma_samples + CBS_LUMA - 1) / CBS_LUMA; + + return round_up(pic_width_in_cbsy * pic_height_in_cbsy, CBS_SIZE); +} + +size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps) +{ + return hantro_hevc_luma_compress_offset(sps) + hantro_hevc_luma_compress_size(sps); +} + +static size_t hantro_hevc_chroma_compress_size(const struct v4l2_ctrl_hevc_sps *sps) +{ + u32 pic_width_in_cbsc = + round_up((sps->pic_width_in_luma_samples + CBS_CHROMA_W - 1) / CBS_CHROMA_W, CBS_SIZE); + u32 pic_height_in_cbsc = (sps->pic_height_in_luma_samples / 2 + CBS_CHROMA_H - 1) / CBS_CHROMA_H; + + return round_up(pic_width_in_cbsc * pic_height_in_cbsc, CBS_SIZE); +} + static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx) { const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; - return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps); + return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps) + + hantro_hevc_luma_compress_size(sps) + + hantro_hevc_chroma_compress_size(sps); } static void hantro_hevc_ref_free(struct hantro_ctx *ctx) diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 8fa0aacb61cd..c5374cd74d66 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -246,6 +246,8 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc); void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx); size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps); size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps); +size_t hantro_hevc_luma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps); +size_t hantro_hevc_chroma_compress_offset(const struct v4l2_ctrl_hevc_sps *sps); static inline size_t hantro_h264_mv_size(unsigned int width, unsigned int height) From patchwork Fri Jun 18 13:15:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34CBBC48BE8 for ; Fri, 18 Jun 2021 13:16:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EA40D613D1 for ; Fri, 18 Jun 2021 13:16:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EA40D613D1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WgFvbqt4AsiO18xbjV+oRHYYc8B7KHwmD2KjKhR0uTk=; b=M7gge/S7neEU4s zjFsXjEp/+IxT0Pb+Dqea1GMWhfGn5qGxuZM88xVbdBLL3vqZ65yUH24taA5fwKMqgJuMxWk7POWt YB1qKlF+quQoBKJbtG9TTx8i708b8ZMbbIrHx3gebH4lebeFdbSkcRq15tW9zouPqATpDCdUvK4Tf L4H+RrcGATL5KDPYw5eP3lNA8Lyszdp3pH4QfqJQBoNr0MDRuqNp9P1X7GFjruHVjuw0LLqtmIuHy ZwhjMku2fiFeMHI2yYIySEawF+WpdKACkdaG9CEezTn850k/xXj+Xsx2DQ6BeM5DM34qEnPm70leM BoKKsEHUEuoIc4o1L5Lw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luEME-00E8vK-U9; Fri, 18 Jun 2021 13:16:26 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELV-00E8bS-F5; Fri, 18 Jun 2021 13:15:44 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id ECD321F448D1; Fri, 18 Jun 2021 14:15:38 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 3/8] media: hantro: hevc: Allow 10-bits encoded streams Date: Fri, 18 Jun 2021 15:15:21 +0200 Message-Id: <20210618131526.566762-4-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061541_814174_F824B552 X-CRM114-Status: GOOD ( 16.67 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org Allow to use 10-bits encoded streams but force the output to remain in 8-bits. Add a function to get chroma offset for the output since it may now not match with internal reference buffers chroma offset. Signed-off-by: Benjamin Gaignard --- drivers/staging/media/hantro/hantro_drv.c | 5 ++-- .../staging/media/hantro/hantro_g2_hevc_dec.c | 29 +++++++++++++++---- drivers/staging/media/hantro/hantro_g2_regs.h | 1 + drivers/staging/media/hantro/hantro_hevc.c | 5 ++-- 4 files changed, 31 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c index 24a654a936ae..f6635ceb5111 100644 --- a/drivers/staging/media/hantro/hantro_drv.c +++ b/drivers/staging/media/hantro/hantro_drv.c @@ -263,8 +263,9 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl) if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) /* Luma and chroma bit depth mismatch */ return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ + if (sps->bit_depth_luma_minus8 != 0 && + sps->bit_depth_luma_minus8 != 2) + /* Only 8-bit or 10-bit is supported */ return -EINVAL; if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED) /* No scaling support */ diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c index bf19825c4d96..7c4427d8e49f 100644 --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c @@ -133,6 +133,16 @@ static void prepare_tile_info_buffer(struct hantro_ctx *ctx) vpu_debug(1, "%s: no chroma!\n", __func__); } +static bool is_8bit_dst_format(struct hantro_ctx *ctx) +{ + switch (ctx->vpu_dst_fmt->fourcc) { + case V4L2_PIX_FMT_NV12: + return true; + default: + return false; + } +} + static void set_params(struct hantro_ctx *ctx) { const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; @@ -148,7 +158,8 @@ static void set_params(struct hantro_ctx *ctx) hantro_reg_write(vpu, &g2_bit_depth_y_minus8, sps->bit_depth_luma_minus8); hantro_reg_write(vpu, &g2_bit_depth_c_minus8, sps->bit_depth_chroma_minus8); - hantro_reg_write(vpu, &g2_output_8_bits, 0); + hantro_reg_write(vpu, &g2_output_8_bits, 1); + hantro_reg_write(vpu, &g2_output_format, 0); hantro_reg_write(vpu, &g2_hdr_skip_length, ctrls->hevc_hdr_skip_length); @@ -522,13 +533,21 @@ static int set_ref(struct hantro_ctx *ctx) return 0; } +static size_t hantro_hevc_output_chroma_offset(struct hantro_ctx *ctx) +{ + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; + int bytes_per_pixel = is_8bit_dst_format(ctx) ? 1 : 2; + + return sps->pic_width_in_luma_samples * + sps->pic_height_in_luma_samples * bytes_per_pixel; +} + static void set_buffers(struct hantro_ctx *ctx) { struct vb2_v4l2_buffer *src_buf, *dst_buf; struct hantro_dev *vpu = ctx->dev; - const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; - const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; - size_t cr_offset = hantro_hevc_chroma_offset(sps); + size_t output_cr_offset = hantro_hevc_output_chroma_offset(ctx); dma_addr_t src_dma, dst_dma; u32 src_len, src_buf_len; @@ -550,7 +569,7 @@ static void set_buffers(struct hantro_ctx *ctx) dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf); hantro_write_addr(vpu, G2_RASTER_SCAN, dst_dma); - hantro_write_addr(vpu, G2_RASTER_SCAN_CHR, dst_dma + cr_offset); + hantro_write_addr(vpu, G2_RASTER_SCAN_CHR, dst_dma + output_cr_offset); hantro_write_addr(vpu, G2_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma); hantro_write_addr(vpu, G2_TILE_FILTER, ctx->hevc_dec.tile_filter.dma); hantro_write_addr(vpu, G2_TILE_SAO, ctx->hevc_dec.tile_sao.dma); diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h index 0414d92e3860..941e5482d27b 100644 --- a/drivers/staging/media/hantro/hantro_g2_regs.h +++ b/drivers/staging/media/hantro/hantro_g2_regs.h @@ -77,6 +77,7 @@ #define g2_bit_depth_y_minus8 G2_DEC_REG(8, 6, 0x3) #define g2_bit_depth_c_minus8 G2_DEC_REG(8, 4, 0x3) #define g2_output_8_bits G2_DEC_REG(8, 3, 0x1) +#define g2_output_format G2_DEC_REG(8, 0, 0x7) #define g2_refidx1_active G2_DEC_REG(9, 19, 0x1f) #define g2_refidx0_active G2_DEC_REG(9, 14, 0x1f) diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c index 00400dd33a3b..112b12a84df4 100644 --- a/drivers/staging/media/hantro/hantro_hevc.c +++ b/drivers/staging/media/hantro/hantro_hevc.c @@ -202,6 +202,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx) const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; unsigned int num_tile_cols = pps->num_tile_columns_minus1 + 1; unsigned int height64 = (sps->pic_height_in_luma_samples + 63) & ~63; + unsigned int pixel_depth = sps->bit_depth_luma_minus8 == 0 ? 8 : 10; unsigned int size; if (num_tile_cols <= 1 || @@ -230,7 +231,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx) hevc_dec->tile_bsd.cpu = NULL; } - size = VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1); + size = (VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1) * pixel_depth) / 8; hevc_dec->tile_filter.cpu = dma_alloc_coherent(vpu->dev, size, &hevc_dec->tile_filter.dma, GFP_KERNEL); @@ -238,7 +239,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx) goto err_free_tile_buffers; hevc_dec->tile_filter.size = size; - size = VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1); + size = (VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1) * pixel_depth) / 8; hevc_dec->tile_sao.cpu = dma_alloc_coherent(vpu->dev, size, &hevc_dec->tile_sao.dma, GFP_KERNEL); From patchwork Fri Jun 18 13:15:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BF8FC49361 for ; Fri, 18 Jun 2021 13:17:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C5ADE613ED for ; Fri, 18 Jun 2021 13:17:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C5ADE613ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=g7f5NffxyC7UNHcNW6IvWZGJh/+U74NLTfoZ9FcuAjQ=; b=gbbkwbfHeoIbrh KUl0S0TutwO4pEXULQEfKjTJ26Ye6i/4FfOltcvAk7uoaxkaYw1lO2lLPWdHr6O30qSu/7bNPVic9 XM3FQIblqAurfGnnvlOamupSgZ6PIqpaEWZXOt2mTWI4WHTViCnvVcvS91VN6zJahV7aDEXrsvzPx y+krHQoLrNZ4cE7vy93zZ682UFPW7C84i7Cz5PJCCQ0Z6vzFxtekhKIZw5BVqJSocoh/5Vxf7gK2x IePnf1IGO4VIZWdueZ67P+EdBB1q/E4eYK6G2xqTZTOXhSwBFEM4lkpgDR79wQkB/hP2dFfFE2qD0 s6qWoFntCgwRT4VIHvzg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luEN0-00E9JO-ES; Fri, 18 Jun 2021 13:17:14 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELW-00E8cQ-UI; Fri, 18 Jun 2021 13:15:46 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 701141F448D3; Fri, 18 Jun 2021 14:15:40 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 4/8] media: Add P010 video format Date: Fri, 18 Jun 2021 15:15:22 +0200 Message-Id: <20210618131526.566762-5-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061543_338845_0E24DB07 X-CRM114-Status: GOOD ( 14.24 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org P010 is a YUV format with 10-bits per pixel with interleaved UV. Signed-off-by: Benjamin Gaignard Acked-by: Nicolas Dufresne --- version 2: - Add documentation about P010 padding - Fix the number of bits per component (16) .../media/v4l/pixfmt-yuv-planar.rst | 78 ++++++++++++++++++- drivers/media/v4l2-core/v4l2-common.c | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 4 files changed, 79 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst index 090c091affd2..af400d37c8fd 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst @@ -100,8 +100,13 @@ All components are stored with the same number of bits per component. - Cb, Cr - No - 64x32 macroblocks - - Horizontal Z order + * - V4L2_PIX_FMT_P010 + - 'P010' + - 16 + - 4:2:0 + - Cb, Cr + - No + - Linear * - V4L2_PIX_FMT_NV12MT_16X16 - 'VM12' - 8 @@ -171,6 +176,7 @@ horizontally. .. _V4L2-PIX-FMT-NV21: .. _V4L2-PIX-FMT-NV12M: .. _V4L2-PIX-FMT-NV21M: +.. _V4L2-PIX-FMT-P010: NV12, NV21, NV12M and NV21M --------------------------- @@ -470,6 +476,74 @@ number of lines as the luma plane. - Cb\ :sub:`33` - Cr\ :sub:`33` +.. _V4L2_PIX_FMT_P010: + +P010 +---- + +The number of bytes in one luminance row must be divisible by 16, +which means there will be padded 0 in the right edge when necessary. + +.. raw:: latex + + \begingroup + \small + \setlength{\tabcolsep}{2pt} + +.. tabularcolumns:: |p{2.6cm}|p{0.70cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}| + +.. flat-table:: P010 16 Bits per component + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - :cspan:`7` Byte 0 in memory + + - :cspan:`7` Byte 1 + * - + - + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * - ``V4L2_PIX_FMT_P010`` + - 'P010' + + - Y\ :sub:`9` + - Y\ :sub:`8` + - Y\ :sub:`7` + - Y\ :sub:`6` + - Y\ :sub:`5` + - Y\ :sub:`4` + - Y\ :sub:`3` + - Y\ :sub:`2` + + - Y\ :sub:`1` + - Y\ :sub:`0` + - 0 + - 0 + - 0 + - 0 + - 0 + - 0 + +.. raw:: latex + + \endgroup Fully Planar YUV Formats ======================== diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c index 04af03285a20..37b5d82359dd 100644 --- a/drivers/media/v4l2-core/v4l2-common.c +++ b/drivers/media/v4l2-core/v4l2-common.c @@ -266,6 +266,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format) { .format = V4L2_PIX_FMT_NV61, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 }, { .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 }, { .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 }, { .format = V4L2_PIX_FMT_YVU410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 4, .vdiv = 4 }, diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 2673f51aafa4..6404d5b6e350 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1282,6 +1282,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break; case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break; case V4L2_PIX_FMT_NV42: descr = "Y/CrCb 4:4:4"; break; + case V4L2_PIX_FMT_P010: descr = "10-bit Y/CrCb 4:2:0"; break; case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break; case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break; case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 9260791b8438..e5f7acde0730 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -602,6 +602,7 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */ #define V4L2_PIX_FMT_NV42 v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb 4:4:4 */ #define V4L2_PIX_FMT_HM12 v4l2_fourcc('H', 'M', '1', '2') /* 8 YUV 4:2:0 16x16 macroblocks */ +#define V4L2_PIX_FMT_P010 v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr 4:2:0 10-bit per pixel*/ /* two non contiguous planes - one Y, one Cr + Cb interleaved */ #define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */ From patchwork Fri Jun 18 13:15:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED931C48BDF for ; Fri, 18 Jun 2021 13:17:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5885610A2 for ; Fri, 18 Jun 2021 13:17:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5885610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wEn699xAQ0JTvFOj2lBabBucfhypyHUIreFYrCYQzjU=; b=NuiZP6C5syXgfG Cs18V3Sakg87t16lFsTHDmai/diovQLN1SeQNyB6WIFuMQVE7LcAyghHAY9Md0AfUUT5diNh9peAb NUh6R7HcOACDnb5aC8kWoeon5MecC1+AKgcSyvjzNhadRGZtQO+Jank0nuOqJXrh6GcdNY+X7Sl/Y BmJSLN0b6IDtVtF8vJ0RkoSHzVdaLRChq81cZ02RLfqkr4IC2b7Um+1VPOnOMnX+Uhcm4vlJzau8O mgWcTGyR5FIl/LS/W3Lvq7Ik/syr+iMOtDWYPCyxIroYJm4Dq1kSlfHCYq4Cl6bP6ZDpbf33rrYTB AzlaGCvgmaMmLQJPoUdg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luEN1-00E9Js-BR; Fri, 18 Jun 2021 13:17:15 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELY-00E8dV-Eb; Fri, 18 Jun 2021 13:15:46 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id EABA31F448D5; Fri, 18 Jun 2021 14:15:41 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 5/8] media: hantro: hevc: Allow to produce 10-bit frames Date: Fri, 18 Jun 2021 15:15:23 +0200 Message-Id: <20210618131526.566762-6-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061544_814527_1205EEB8 X-CRM114-Status: GOOD ( 17.05 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org If Hantro driver receive an 10-bit encoded bitstream allow it to produce 10-bit frames. Check that we are not try to produce 10-bit frames from a 8-bit encoded bitstream. Signed-off-by: Benjamin Gaignard --- drivers/staging/media/hantro/hantro_drv.c | 18 ++++++++++++++++++ .../staging/media/hantro/hantro_g2_hevc_dec.c | 18 ++++++++++++++---- drivers/staging/media/hantro/hantro_hevc.c | 2 +- drivers/staging/media/hantro/imx8m_vpu_hw.c | 4 ++++ 4 files changed, 37 insertions(+), 5 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c index f6635ceb5111..b6373934734e 100644 --- a/drivers/staging/media/hantro/hantro_drv.c +++ b/drivers/staging/media/hantro/hantro_drv.c @@ -243,6 +243,16 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) return vb2_queue_init(dst_vq); } +static bool hantro_is_10bit_dst_format(struct hantro_ctx *ctx) +{ + switch (ctx->vpu_dst_fmt->fourcc) { + case V4L2_PIX_FMT_P010: + return true; + default: + return false; + } +} + static int hantro_try_ctrl(struct v4l2_ctrl *ctrl) { if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { @@ -259,6 +269,10 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl) return -EINVAL; } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) { const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps; + struct hantro_ctx *ctx; + + ctx = container_of(ctrl->handler, + struct hantro_ctx, ctrl_handler); if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) /* Luma and chroma bit depth mismatch */ @@ -270,6 +284,10 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl) if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED) /* No scaling support */ return -EINVAL; + if (sps->bit_depth_luma_minus8 == 0 && + hantro_is_10bit_dst_format(ctx)) { + return -EINVAL; + } } return 0; } diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c index 7c4427d8e49f..41dc89ec926c 100644 --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c @@ -143,6 +143,16 @@ static bool is_8bit_dst_format(struct hantro_ctx *ctx) } } +static int get_dst_format(struct hantro_ctx *ctx) +{ + switch (ctx->vpu_dst_fmt->fourcc) { + case V4L2_PIX_FMT_P010: + return 0x1; + default: + return 0x0; + } +} + static void set_params(struct hantro_ctx *ctx) { const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; @@ -158,8 +168,8 @@ static void set_params(struct hantro_ctx *ctx) hantro_reg_write(vpu, &g2_bit_depth_y_minus8, sps->bit_depth_luma_minus8); hantro_reg_write(vpu, &g2_bit_depth_c_minus8, sps->bit_depth_chroma_minus8); - hantro_reg_write(vpu, &g2_output_8_bits, 1); - hantro_reg_write(vpu, &g2_output_format, 0); + hantro_reg_write(vpu, &g2_output_8_bits, is_8bit_dst_format(ctx)); + hantro_reg_write(vpu, &g2_output_format, get_dst_format(ctx)); hantro_reg_write(vpu, &g2_hdr_skip_length, ctrls->hevc_hdr_skip_length); @@ -540,7 +550,7 @@ static size_t hantro_hevc_output_chroma_offset(struct hantro_ctx *ctx) int bytes_per_pixel = is_8bit_dst_format(ctx) ? 1 : 2; return sps->pic_width_in_luma_samples * - sps->pic_height_in_luma_samples * bytes_per_pixel; + sps->pic_height_in_luma_samples * bytes_per_pixel; } static void set_buffers(struct hantro_ctx *ctx) @@ -627,7 +637,7 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx) /* Compress buffers */ hantro_reg_write(vpu, &g2_ref_compress_bypass, 0); - /* use NV12 as output format */ + /* Use raster-scan as output format */ hantro_reg_write(vpu, &g2_out_rs_e, 1); /* Bus width and max burst */ diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c index 112b12a84df4..b646bd559ffe 100644 --- a/drivers/staging/media/hantro/hantro_hevc.c +++ b/drivers/staging/media/hantro/hantro_hevc.c @@ -25,7 +25,7 @@ #define UNUSED_REF -1 -#define G2_ALIGN 16 +#define G2_ALIGN 16 #define CBS_SIZE 16 /* compression table size in bytes */ #define CBS_LUMA 8 /* luminance CBS is composed of 1 8x8 coded block */ diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c index ab6ac620f0d0..65bcf46740d7 100644 --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c @@ -135,6 +135,10 @@ static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = { .fourcc = V4L2_PIX_FMT_NV12, .codec_mode = HANTRO_MODE_NONE, }, + { + .fourcc = V4L2_PIX_FMT_P010, + .codec_mode = HANTRO_MODE_NONE, + }, { .fourcc = V4L2_PIX_FMT_HEVC_SLICE, .codec_mode = HANTRO_MODE_HEVC_DEC, From patchwork Fri Jun 18 13:15:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F8CFC48BDF for ; Fri, 18 Jun 2021 13:18:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14596613D1 for ; Fri, 18 Jun 2021 13:18:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14596613D1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7RNtK7brJOl9EK7QaqoZQDOdaTx3lh3yg5k+YnHZT84=; b=JY+BoD7RkbcM8K B2zaIkqejcIGVaFcOHBJCNM5ydCrRbNMUn6297/qAJJkYQu3HGVgB376qOzPmUzwmFPey18k/ONZC 1ReCNU4lUJJZuCb7PMRNF3PSQotb7iBdu5dkbuwC+0l2Pd0gQT2qWWr5qGRi8ASrNOP/jQLySXQnN Bqqz7BUdLyhDJs5Gz0ij7Az84sS7Qx+NyyT3ZAXDF7ck3D91hb3HbIVXXqF6zFv8aKDJxkMrdqeDd KxSUspgqv9roWa0fYzQV2x97Kyr/YOguObFMpODlc0KqYiV2ilzC+irK65GavaRzTXORmUslUiexC cAaOcd4CYRTzKop6NjHg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luENu-00E9tL-0I; Fri, 18 Jun 2021 13:18:10 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELZ-00E8eU-Uq; Fri, 18 Jun 2021 13:15:50 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 6E00D1F448CC; Fri, 18 Jun 2021 14:15:43 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 6/8] media: hantro: enumerate scaled output formats Date: Fri, 18 Jun 2021 15:15:24 +0200 Message-Id: <20210618131526.566762-7-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061546_291580_557D6331 X-CRM114-Status: GOOD ( 21.28 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org When enumerating the output formats take care of the hardware scaling capabilities. For a given input size G2 hardware block is capable of down scale the output by 2, 4 or 8 factor. When decoding 4K streams that to be could helpful to save memory bandwidth. Signed-off-by: Benjamin Gaignard --- drivers/staging/media/hantro/hantro.h | 4 ++ .../staging/media/hantro/hantro_g2_hevc_dec.c | 46 ++++++++++++++++++- drivers/staging/media/hantro/hantro_g2_regs.h | 6 +++ drivers/staging/media/hantro/hantro_hw.h | 1 + drivers/staging/media/hantro/hantro_v4l2.c | 10 ++-- drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 + 6 files changed, 63 insertions(+), 5 deletions(-) diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h index 6a21d1e95b34..ca9038b0384a 100644 --- a/drivers/staging/media/hantro/hantro.h +++ b/drivers/staging/media/hantro/hantro.h @@ -71,6 +71,9 @@ struct hantro_irq { * @reg_names: array of register range names * @num_regs: number of register range names in the array * @postproc_regs: &struct hantro_postproc_regs pointer + * @scaling: Set possible scaled output formats. + * Returns zero if OK, a negative value in error cases. + * Optional. */ struct hantro_variant { unsigned int enc_offset; @@ -92,6 +95,7 @@ struct hantro_variant { const char * const *reg_names; int num_regs; const struct hantro_postproc_regs *postproc_regs; + int (*scaling)(struct hantro_ctx *ctx, struct v4l2_frmsizeenum *fsize); }; /** diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c index 41dc89ec926c..3a8aa2ff109c 100644 --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c @@ -396,6 +396,17 @@ static void set_ref_pic_list(struct hantro_ctx *ctx) } } +static int down_scale_factor(struct hantro_ctx *ctx) +{ + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; + + if (sps->pic_width_in_luma_samples == ctx->dst_fmt.width) + return 0; + + return DIV_ROUND_CLOSEST(sps->pic_width_in_luma_samples, ctx->dst_fmt.width); +} + static int set_ref(struct hantro_ctx *ctx) { const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; @@ -409,6 +420,7 @@ static int set_ref(struct hantro_ctx *ctx) size_t mv_offset = hantro_hevc_motion_vectors_offset(sps); size_t compress_luma_offset = hantro_hevc_luma_compress_offset(sps); size_t compress_chroma_offset = hantro_hevc_chroma_compress_offset(sps); + int down_scale = down_scale_factor(ctx); u32 max_ref_frames; u16 dpb_longterm_e; static const struct hantro_reg cur_poc[] = { @@ -521,8 +533,18 @@ static int set_ref(struct hantro_ctx *ctx) hantro_write_addr(vpu, G2_REG_CHR_REF(i), chroma_addr); hantro_write_addr(vpu, G2_REG_DMV_REF(i++), mv_addr); - hantro_write_addr(vpu, G2_ADDR_DST, luma_addr); - hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr); + if (down_scale) { + chroma_addr = luma_addr + (cr_offset >> down_scale); + hantro_reg_write(vpu, &g2_down_scale_e, 1); + hantro_reg_write(vpu, &g2_down_scale_y, down_scale >> 2); + hantro_reg_write(vpu, &g2_down_scale_x, down_scale >> 2); + hantro_write_addr(vpu, G2_DS_DST, luma_addr); + hantro_write_addr(vpu, G2_DS_DST_CHR, chroma_addr); + } else { + hantro_write_addr(vpu, G2_ADDR_DST, luma_addr); + hantro_write_addr(vpu, G2_ADDR_DST_CHR, chroma_addr); + } + hantro_write_addr(vpu, G2_ADDR_DST_MV, mv_addr); hantro_write_addr(vpu, G2_COMP_ADDR_DST, compress_luma_addr); hantro_write_addr(vpu, G2_COMP_CHR, compress_chroma_addr); @@ -603,6 +625,26 @@ static void hantro_g2_check_idle(struct hantro_dev *vpu) } } +int hantro_g2_hevc_dec_scaling(struct hantro_ctx *ctx, + struct v4l2_frmsizeenum *fsize) +{ + /** + * G2 scaler can scale down by 0, 2, 4 or 8 + * use fsize->index has power of 2 diviser + **/ + if (fsize->index > 3) + return -EINVAL; + + if (!ctx->src_fmt.width || !ctx->src_fmt.height) + return -EINVAL; + + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE; + fsize->discrete.width = ctx->src_fmt.width >> fsize->index; + fsize->discrete.height = ctx->src_fmt.height >> fsize->index; + + return 0; +} + int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx) { struct hantro_dev *vpu = ctx->dev; diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h index 941e5482d27b..54f3d78ce46e 100644 --- a/drivers/staging/media/hantro/hantro_g2_regs.h +++ b/drivers/staging/media/hantro/hantro_g2_regs.h @@ -174,6 +174,10 @@ #define g2_buswidth G2_DEC_REG(58, 8, 0x7) #define g2_max_burst G2_DEC_REG(58, 0, 0xff) +#define g2_down_scale_e G2_DEC_REG(184, 7, 0x1) +#define g2_down_scale_y G2_DEC_REG(184, 2, 0x3) +#define g2_down_scale_x G2_DEC_REG(184, 0, 0x3) + #define G2_REG_CONFIG G2_SWREG(58) #define G2_REG_CONFIG_DEC_CLK_GATE_E BIT(16) #define G2_REG_CONFIG_DEC_CLK_GATE_IDLE_E BIT(17) @@ -193,6 +197,8 @@ #define G2_TILE_FILTER (G2_SWREG(179)) #define G2_TILE_SAO (G2_SWREG(181)) #define G2_TILE_BSD (G2_SWREG(183)) +#define G2_DS_DST (G2_SWREG(186)) +#define G2_DS_DST_CHR (G2_SWREG(188)) #define G2_COMP_ADDR_DST (G2_SWREG(190)) #define G2_COMP_ADDR_REF(i) (G2_SWREG(192) + ((i) * 0x8)) #define G2_COMP_CHR (G2_SWREG(224)) diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index c5374cd74d66..2edb890f10af 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -241,6 +241,7 @@ int hantro_hevc_dec_init(struct hantro_ctx *ctx); void hantro_hevc_dec_exit(struct hantro_ctx *ctx); int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx); void hantro_g2_hevc_dec_done(struct hantro_ctx *ctx); +int hantro_g2_hevc_dec_scaling(struct hantro_ctx *ctx, struct v4l2_frmsizeenum *fsize); int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx); dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc); void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx); diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index bcb0bdff4a9a..566b645f661d 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -118,7 +118,7 @@ static int vidioc_enum_framesizes(struct file *file, void *priv, struct hantro_ctx *ctx = fh_to_ctx(priv); const struct hantro_fmt *fmt; - if (fsize->index != 0) { + if (fsize->index != 0 && !ctx->dev->variant->scaling) { vpu_debug(0, "invalid frame size index (expected 0, got %d)\n", fsize->index); return -EINVAL; @@ -131,9 +131,13 @@ static int vidioc_enum_framesizes(struct file *file, void *priv, return -EINVAL; } - /* This only makes sense for coded formats */ - if (fmt->codec_mode == HANTRO_MODE_NONE) + /* For non-coded formats check if scaling is possible */ + if (fmt->codec_mode == HANTRO_MODE_NONE) { + if (ctx->dev->variant->scaling) + return ctx->dev->variant->scaling(ctx, fsize); + return -EINVAL; + } fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; fsize->stepwise = fmt->frmsize; diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c index 65bcf46740d7..d3b760569b52 100644 --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c @@ -304,4 +304,5 @@ const struct hantro_variant imx8mq_vpu_g2_variant = { .num_irqs = ARRAY_SIZE(imx8mq_g2_irqs), .clk_names = imx8mq_clk_names, .num_clocks = ARRAY_SIZE(imx8mq_clk_names), + .scaling = hantro_g2_hevc_dec_scaling, }; From patchwork Fri Jun 18 13:15:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5699C49361 for ; Fri, 18 Jun 2021 13:18:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 85210613D1 for ; Fri, 18 Jun 2021 13:18:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85210613D1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Ct7NhBjeOvw5hOfIQy84SXE5DbNYvvyQJ49CZee+eEk=; b=dRQzNaYytfxPwf LHmV+Z2h1pcID2TK7QrlAvytfenXhiSoaSEEwtb/Tul3si/+csQm8VKdmViT4rYYCud8KVS4EMQ7O 7NYx9P2NgqAI4AMmJqNPsBU8Ht/PMFiGQTpyNhRffvWPB77I5peie63lM6IVEDgbDCLpcnJrsBUah 3zZ5EApTC1MCmCe5MFwmVszrrO7eNCp+9Nl+uknmVDSpb3/Qiqpwp4+z9gzyP492qoJ6kXF8lCOMS NeEPuaPNVBlalAOcv7NBu2+JI/hPRoTUE40Es1YTkQJ228RQv9pzl2ZGeR4A+ox09PpLh1aegl3aB xIW1eq5pdVEtkJ/9/UpA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luENv-00E9uY-CS; Fri, 18 Jun 2021 13:18:11 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELb-00E8ff-Ew; Fri, 18 Jun 2021 13:15:50 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id EE6C21F448D6; Fri, 18 Jun 2021 14:15:44 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 7/8] media: hevc: Add scaling matrix control Date: Fri, 18 Jun 2021 15:15:25 +0200 Message-Id: <20210618131526.566762-8-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061547_816863_6E389BBB X-CRM114-Status: GOOD ( 16.51 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org HEVC scaling lists are used for the scaling process for transform coefficients. V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED has to set when they are encoded in the bitstream. Signed-off-by: Benjamin Gaignard --- Note: V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED is not change by this patch. There is a thread about the naming/usage of this flag here: https://lore.kernel.org/linux-arm-kernel/20210610154442.806107-8-benjamin.gaignard@collabora.com/ but that doesn't concern the scaling matrix control by itself. version 2: - Fix structure name in ext-ctrls-codec.rst .../media/v4l/ext-ctrls-codec.rst | 45 +++++++++++++++++++ .../media/v4l/vidioc-queryctrl.rst | 6 +++ drivers/media/v4l2-core/v4l2-ctrls-core.c | 6 +++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 ++ include/media/hevc-ctrls.h | 11 +++++ 5 files changed, 72 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index 8c6e2a11ed95..d4f40bb85263 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -3068,6 +3068,51 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - \normalsize +``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)`` + Specifies the HEVC scaling matrix parameters used for the scaling process + for transform coefficients. + These matrix and parameters are defined according to :ref:`hevc`. + They are described in section 7.4.5 "Scaling list data semantics" of + the specification. + +.. c:type:: v4l2_ctrl_hevc_scaling_matrix + +.. raw:: latex + + \scriptsize + +.. tabularcolumns:: |p{5.4cm}|p{6.8cm}|p{5.1cm}| + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_hevc_scaling_matrix + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``scaling_list_4x4[6][16]`` + - + * - __u8 + - ``scaling_list_8x8[6][64]`` + - + * - __u8 + - ``scaling_list_16x16[6][64]`` + - + * - __u8 + - ``scaling_list_32x32[2][64]`` + - + * - __u8 + - ``scaling_list_dc_coef_16x16[6]`` + - + * - __u8 + - ``scaling_list_dc_coef_32x32[2]`` + - + +.. raw:: latex + + \normalsize + .. c:type:: v4l2_hevc_dpb_entry .. raw:: latex diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst index f9ecf6276129..2f491c17dd5d 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst @@ -495,6 +495,12 @@ See also the examples in :ref:`control`. - n/a - A struct :c:type:`v4l2_ctrl_hevc_slice_params`, containing HEVC slice parameters for stateless video decoders. + * - ``V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_hevc_scaling_matrix`, containing HEVC + scaling matrix for stateless video decoders. * - ``V4L2_CTRL_TYPE_VP8_FRAME`` - n/a - n/a diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c index c4b5082849b6..70adfc1b9c81 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c @@ -687,6 +687,9 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, break; + case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX: + break; + case V4L2_CTRL_TYPE_AREA: area = p; if (!area->width || !area->height) @@ -1240,6 +1243,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS: elem_size = sizeof(struct v4l2_ctrl_hevc_slice_params); break; + case V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX: + elem_size = sizeof(struct v4l2_ctrl_hevc_scaling_matrix); + break; case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS: elem_size = sizeof(struct v4l2_ctrl_hevc_decode_params); break; diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c index b6344bbf1e00..cb29c2a7fabe 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c @@ -996,6 +996,7 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC Sequence Parameter Set"; case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC Picture Parameter Set"; case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters"; + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix"; case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters"; case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode"; case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code"; @@ -1488,6 +1489,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS; break; + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: + *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX; + break; case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS; break; diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h index 53c0038c792b..0e5c4a2eecff 100644 --- a/include/media/hevc-ctrls.h +++ b/include/media/hevc-ctrls.h @@ -19,6 +19,7 @@ #define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008) #define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009) #define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010) +#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011) #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012) #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015) #define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016) @@ -27,6 +28,7 @@ #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120 #define V4L2_CTRL_TYPE_HEVC_PPS 0x0121 #define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122 +#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123 #define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124 enum v4l2_mpeg_video_hevc_decode_mode { @@ -224,6 +226,15 @@ struct v4l2_ctrl_hevc_decode_params { __u64 flags; }; +struct v4l2_ctrl_hevc_scaling_matrix { + __u8 scaling_list_4x4[6][16]; + __u8 scaling_list_8x8[6][64]; + __u8 scaling_list_16x16[6][64]; + __u8 scaling_list_32x32[2][64]; + __u8 scaling_list_dc_coef_16x16[6]; + __u8 scaling_list_dc_coef_32x32[2]; +}; + /* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */ #define V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC | 0x1200) /* From patchwork Fri Jun 18 13:15:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Gaignard X-Patchwork-Id: 12331375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CB97C48BDF for ; Fri, 18 Jun 2021 13:18:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 117A6613EC for ; Fri, 18 Jun 2021 13:18:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 117A6613EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=collabora.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nj8IpGstCLwr1iA/l38s24LySLKUKtU4tTBEI5ntf94=; b=CDZOgAXgxHxW5r gYindFgW2gR1Ex71nN40XLtRStS0mH1B+QvWYawJw+7gkTEZHDBTAn4xlbPsJ+7MWebXeNbGeVsKH vaqQBIQZaGE4U3TujUGX2pDFQ86plcz5fHz/AU5G7KD6PFVQ+k79DnrnMgxA8kSoV00XSMHVq/av0 XIl4q0K737TneryO6LRnjb0x9B0MtP0Slo9qitwYcjoU5xFTWg06xM0YoE1vp13l9ngwn7eB4NsZI dA+L55KxI0wrTurRsKibiCmGrWGla3vde8+e0h94xDcUHCknRydHqyyuOBDrHZ2/7KwaB2dHS3Tq5 NaCjv94s+MS+ecTquUkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luEOa-00EAKX-24; Fri, 18 Jun 2021 13:18:52 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luELc-00E8h6-Ut; Fri, 18 Jun 2021 13:15:52 +0000 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:4cb:a870:141f:c87a:873e:7b6f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: benjamin.gaignard) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 7225A1F448CD; Fri, 18 Jun 2021 14:15:46 +0100 (BST) From: Benjamin Gaignard To: hverkuil@xs4all.nl, ezequiel@collabora.com, p.zabel@pengutronix.de, mchehab@kernel.org, shawnguo@kernel.org, s.hauer@pengutronix.de, festevam@gmail.com, gregkh@linuxfoundation.org, mripard@kernel.org, paul.kocialkowski@bootlin.com, wens@csie.org, jernej.skrabec@siol.net, emil.l.velikov@gmail.com, andrzej.p@collabora.com, jc@kynesim.co.uk, jernej.skrabec@gmail.com, nicolas@ndufresne.ca Cc: kernel@pengutronix.de, linux-imx@nxp.com, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Benjamin Gaignard Subject: [PATCH v3 8/8] media: hantro: Add scaling lists feature Date: Fri, 18 Jun 2021 15:15:26 +0200 Message-Id: <20210618131526.566762-9-benjamin.gaignard@collabora.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210618131526.566762-1-benjamin.gaignard@collabora.com> References: <20210618131526.566762-1-benjamin.gaignard@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210618_061549_310255_9C4E1CFA X-CRM114-Status: GOOD ( 20.85 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org If the bitstream embedded scaling lists allow the driver to use them for decode the frames. The scaling lists are expected to be in raster scan order (i.e. not up right diagonal scan order) Allocate the memory needed to store lists. Signed-off-by: Benjamin Gaignard --- drivers/staging/media/hantro/hantro_drv.c | 8 +-- .../staging/media/hantro/hantro_g2_hevc_dec.c | 52 +++++++++++++++++++ drivers/staging/media/hantro/hantro_hevc.c | 21 ++++++++ drivers/staging/media/hantro/hantro_hw.h | 3 ++ 4 files changed, 81 insertions(+), 3 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c index b6373934734e..bf7ca028986d 100644 --- a/drivers/staging/media/hantro/hantro_drv.c +++ b/drivers/staging/media/hantro/hantro_drv.c @@ -281,9 +281,6 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl) sps->bit_depth_luma_minus8 != 2) /* Only 8-bit or 10-bit is supported */ return -EINVAL; - if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED) - /* No scaling support */ - return -EINVAL; if (sps->bit_depth_luma_minus8 == 0 && hantro_is_10bit_dst_format(ctx)) { return -EINVAL; @@ -469,6 +466,11 @@ static const struct hantro_ctrl controls[] = { .cfg = { .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS, }, + }, { + .codec = HANTRO_HEVC_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX, + }, }, { .codec = HANTRO_HEVC_DECODER, .cfg = { diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c index 3a8aa2ff109c..9ea864ca5625 100644 --- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c +++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c @@ -608,6 +608,56 @@ static void set_buffers(struct hantro_ctx *ctx) hantro_write_addr(vpu, G2_TILE_BSD, ctx->hevc_dec.tile_bsd.dma); } +static void prepare_scaling_list_buffer(struct hantro_ctx *ctx) +{ + struct hantro_dev *vpu = ctx->dev; + const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls; + const struct v4l2_ctrl_hevc_scaling_matrix *sc = ctrls->scaling; + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps; + u8 *p = ((u8 *)ctx->hevc_dec.scaling_lists.cpu); + unsigned int scaling_list_enabled; + unsigned int i, j, k; + + scaling_list_enabled = !!(sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED); + hantro_reg_write(vpu, &g2_scaling_list_e, scaling_list_enabled); + + if (!scaling_list_enabled) + return; + + for (i = 0; i < ARRAY_SIZE(sc->scaling_list_dc_coef_16x16); i++) + *p++ = sc->scaling_list_dc_coef_16x16[i]; + + for (i = 0; i < ARRAY_SIZE(sc->scaling_list_dc_coef_32x32); i++) + *p++ = sc->scaling_list_dc_coef_32x32[i]; + + /* 128-bit boundary */ + p += 8; + + /* write scaling lists column by column */ + + for (i = 0; i < 6; i++) + for (j = 0; j < 4; j++) + for (k = 0; k < 4; k++) + *p++ = sc->scaling_list_4x4[i][4 * k + j]; + + for (i = 0; i < 6; i++) + for (j = 0; j < 8; j++) + for (k = 0; k < 8; k++) + *p++ = sc->scaling_list_8x8[i][8 * k + j]; + + for (i = 0; i < 6; i++) + for (j = 0; j < 8; j++) + for (k = 0; k < 8; k++) + *p++ = sc->scaling_list_16x16[i][8 * k + j]; + + for (i = 0; i < 2; i++) + for (j = 0; j < 8; j++) + for (k = 0; k < 8; k++) + *p++ = sc->scaling_list_32x32[i][8 * k + j]; + + hantro_write_addr(vpu, HEVC_SCALING_LIST, ctx->hevc_dec.scaling_lists.dma); +} + static void hantro_g2_check_idle(struct hantro_dev *vpu) { int i; @@ -668,6 +718,8 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx) set_buffers(ctx); prepare_tile_info_buffer(ctx); + prepare_scaling_list_buffer(ctx); + hantro_end_prepare_run(ctx); hantro_reg_write(vpu, &g2_mode, HEVC_DEC_MODE); diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c index b646bd559ffe..ec1b86e03b83 100644 --- a/drivers/staging/media/hantro/hantro_hevc.c +++ b/drivers/staging/media/hantro/hantro_hevc.c @@ -20,6 +20,8 @@ /* tile border coefficients of filter */ #define VERT_SAO_RAM_SIZE 48 /* bytes per pixel */ +#define SCALING_LIST_SIZE (16 * 64) + #define MAX_TILE_COLS 20 #define MAX_TILE_ROWS 22 @@ -294,6 +296,11 @@ int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx) if (WARN_ON(!ctrls->decode_params)) return -EINVAL; + ctrls->scaling = + hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX); + if (WARN_ON(!ctrls->scaling)) + return -EINVAL; + ctrls->sps = hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS); if (WARN_ON(!ctrls->sps)) @@ -322,6 +329,12 @@ void hantro_hevc_dec_exit(struct hantro_ctx *ctx) hevc_dec->tile_sizes.dma); hevc_dec->tile_sizes.cpu = NULL; + if (hevc_dec->scaling_lists.cpu) + dma_free_coherent(vpu->dev, hevc_dec->scaling_lists.size, + hevc_dec->scaling_lists.cpu, + hevc_dec->scaling_lists.dma); + hevc_dec->scaling_lists.cpu = NULL; + if (hevc_dec->tile_filter.cpu) dma_free_coherent(vpu->dev, hevc_dec->tile_filter.size, hevc_dec->tile_filter.cpu, @@ -365,6 +378,14 @@ int hantro_hevc_dec_init(struct hantro_ctx *ctx) hevc_dec->tile_sizes.size = size; + hevc_dec->scaling_lists.cpu = dma_alloc_coherent(vpu->dev, SCALING_LIST_SIZE, + &hevc_dec->scaling_lists.dma, + GFP_KERNEL); + if (!hevc_dec->scaling_lists.cpu) + return -ENOMEM; + + hevc_dec->scaling_lists.size = SCALING_LIST_SIZE; + hantro_hevc_ref_init(ctx); return 0; diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2edb890f10af..88add18b1bad 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -108,6 +108,7 @@ struct hantro_h264_dec_hw_ctx { */ struct hantro_hevc_dec_ctrls { const struct v4l2_ctrl_hevc_decode_params *decode_params; + const struct v4l2_ctrl_hevc_scaling_matrix *scaling; const struct v4l2_ctrl_hevc_sps *sps; const struct v4l2_ctrl_hevc_pps *pps; u32 hevc_hdr_skip_length; @@ -120,6 +121,7 @@ struct hantro_hevc_dec_ctrls { * @tile_sao: Tile SAO buffer * @tile_bsd: Tile BSD control buffer * @ref_bufs: Internal reference buffers + * @scaling_lists: Scaling lists buffer * @ref_bufs_poc: Internal reference buffers picture order count * @ref_bufs_used: Bitfield of used reference buffers * @ctrls: V4L2 controls attached to a run @@ -131,6 +133,7 @@ struct hantro_hevc_dec_hw_ctx { struct hantro_aux_buf tile_sao; struct hantro_aux_buf tile_bsd; struct hantro_aux_buf ref_bufs[NUM_REF_PICTURES]; + struct hantro_aux_buf scaling_lists; int ref_bufs_poc[NUM_REF_PICTURES]; u32 ref_bufs_used; struct hantro_hevc_dec_ctrls ctrls;