From patchwork Thu Dec 3 11:00:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42D09C63777 for ; Thu, 3 Dec 2020 11:04:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CF89C22206 for ; Thu, 3 Dec 2020 11:03:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387662AbgLCLD7 (ORCPT ); Thu, 3 Dec 2020 06:03:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730269AbgLCLD6 (ORCPT ); Thu, 3 Dec 2020 06:03:58 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 647D2C061A53 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNs-0002Km-Kn; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhV-PD; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:49 +0100 Message-Id: <20201203110106.2939463-2-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 01/18] media: allegro: extract RBSP handler from H.264 NAL generator X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The RBSP structure is the same for HEVC and H.264. In order to be able to reuse the RBSP handler for generating HEVC NAL units, extract the functions from the H.264 generator. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/Makefile | 3 +- drivers/media/platform/allegro-dvt/nal-h264.c | 336 +----------------- drivers/media/platform/allegro-dvt/nal-rbsp.c | 305 ++++++++++++++++ drivers/media/platform/allegro-dvt/nal-rbsp.h | 60 ++++ 4 files changed, 374 insertions(+), 330 deletions(-) create mode 100644 drivers/media/platform/allegro-dvt/nal-rbsp.c create mode 100644 drivers/media/platform/allegro-dvt/nal-rbsp.h diff --git a/drivers/media/platform/allegro-dvt/Makefile b/drivers/media/platform/allegro-dvt/Makefile index 8e306dcdc55c..10119e84a60f 100644 --- a/drivers/media/platform/allegro-dvt/Makefile +++ b/drivers/media/platform/allegro-dvt/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -allegro-objs := allegro-core.o nal-h264.o allegro-mail.o +allegro-objs := allegro-core.o allegro-mail.o +allegro-objs += nal-rbsp.o nal-h264.o obj-$(CONFIG_VIDEO_ALLEGRO_DVT) += allegro.o diff --git a/drivers/media/platform/allegro-dvt/nal-h264.c b/drivers/media/platform/allegro-dvt/nal-h264.c index bd48b8883572..94dd9266d850 100644 --- a/drivers/media/platform/allegro-dvt/nal-h264.c +++ b/drivers/media/platform/allegro-dvt/nal-h264.c @@ -22,6 +22,7 @@ #include #include "nal-h264.h" +#include "nal-rbsp.h" /* * See Rec. ITU-T H.264 (04/2017) Table 7-1 – NAL unit type codes, syntax @@ -33,54 +34,6 @@ enum nal_unit_type { FILLER_DATA = 12, }; -struct rbsp; - -struct nal_h264_ops { - int (*rbsp_bit)(struct rbsp *rbsp, int *val); - int (*rbsp_bits)(struct rbsp *rbsp, int n, unsigned int *val); - int (*rbsp_uev)(struct rbsp *rbsp, unsigned int *val); - int (*rbsp_sev)(struct rbsp *rbsp, int *val); -}; - -/** - * struct rbsp - State object for handling a raw byte sequence payload - * @data: pointer to the data of the rbsp - * @size: maximum size of the data of the rbsp - * @pos: current bit position inside the rbsp - * @num_consecutive_zeros: number of zeros before @pos - * @ops: per datatype functions for interacting with the rbsp - * @error: an error occurred while handling the rbsp - * - * This struct is passed around the various parsing functions and tracks the - * current position within the raw byte sequence payload. - * - * The @ops field allows to separate the operation, i.e., reading/writing a - * value from/to that rbsp, from the structure of the NAL unit. This allows to - * have a single function for iterating the NAL unit, while @ops has function - * pointers for handling each type in the rbsp. - */ -struct rbsp { - u8 *data; - size_t size; - unsigned int pos; - unsigned int num_consecutive_zeros; - struct nal_h264_ops *ops; - int error; -}; - -static void rbsp_init(struct rbsp *rbsp, void *addr, size_t size, - struct nal_h264_ops *ops) -{ - if (!rbsp) - return; - - rbsp->data = addr; - rbsp->size = size; - rbsp->pos = 0; - rbsp->ops = ops; - rbsp->error = 0; -} - /** * nal_h264_profile_from_v4l2() - Get profile_idc for v4l2 h264 profile * @profile: the profile as &enum v4l2_mpeg_video_h264_profile @@ -155,281 +108,6 @@ int nal_h264_level_from_v4l2(enum v4l2_mpeg_video_h264_level level) } } -static int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value); -static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value); - -/* - * When reading or writing, the emulation_prevention_three_byte is detected - * only when the 2 one bits need to be inserted. Therefore, we are not - * actually adding the 0x3 byte, but the 2 one bits and the six 0 bits of the - * next byte. - */ -#define EMULATION_PREVENTION_THREE_BYTE (0x3 << 6) - -static int add_emulation_prevention_three_byte(struct rbsp *rbsp) -{ - rbsp->num_consecutive_zeros = 0; - rbsp_write_bits(rbsp, 8, EMULATION_PREVENTION_THREE_BYTE); - - return 0; -} - -static int discard_emulation_prevention_three_byte(struct rbsp *rbsp) -{ - unsigned int tmp = 0; - - rbsp->num_consecutive_zeros = 0; - rbsp_read_bits(rbsp, 8, &tmp); - if (tmp != EMULATION_PREVENTION_THREE_BYTE) - return -EINVAL; - - return 0; -} - -static inline int rbsp_read_bit(struct rbsp *rbsp) -{ - int shift; - int ofs; - int bit; - int err; - - if (rbsp->num_consecutive_zeros == 22) { - err = discard_emulation_prevention_three_byte(rbsp); - if (err) - return err; - } - - shift = 7 - (rbsp->pos % 8); - ofs = rbsp->pos / 8; - if (ofs >= rbsp->size) - return -EINVAL; - - bit = (rbsp->data[ofs] >> shift) & 1; - - rbsp->pos++; - - if (bit == 1 || - (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0))) - rbsp->num_consecutive_zeros = 0; - else - rbsp->num_consecutive_zeros++; - - return bit; -} - -static inline int rbsp_write_bit(struct rbsp *rbsp, bool value) -{ - int shift; - int ofs; - - if (rbsp->num_consecutive_zeros == 22) - add_emulation_prevention_three_byte(rbsp); - - shift = 7 - (rbsp->pos % 8); - ofs = rbsp->pos / 8; - if (ofs >= rbsp->size) - return -EINVAL; - - rbsp->data[ofs] &= ~(1 << shift); - rbsp->data[ofs] |= value << shift; - - rbsp->pos++; - - if (value || - (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0))) { - rbsp->num_consecutive_zeros = 0; - } else { - rbsp->num_consecutive_zeros++; - } - - return 0; -} - -static inline int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value) -{ - int i; - int bit; - unsigned int tmp = 0; - - if (n > 8 * sizeof(*value)) - return -EINVAL; - - for (i = n; i > 0; i--) { - bit = rbsp_read_bit(rbsp); - if (bit < 0) - return bit; - tmp |= bit << (i - 1); - } - - if (value) - *value = tmp; - - return 0; -} - -static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value) -{ - int ret; - - if (n > 8 * sizeof(value)) - return -EINVAL; - - while (n--) { - ret = rbsp_write_bit(rbsp, (value >> n) & 1); - if (ret) - return ret; - } - - return 0; -} - -static int rbsp_read_uev(struct rbsp *rbsp, unsigned int *value) -{ - int leading_zero_bits = 0; - unsigned int tmp = 0; - int ret; - - while ((ret = rbsp_read_bit(rbsp)) == 0) - leading_zero_bits++; - if (ret < 0) - return ret; - - if (leading_zero_bits > 0) { - ret = rbsp_read_bits(rbsp, leading_zero_bits, &tmp); - if (ret) - return ret; - } - - if (value) - *value = (1 << leading_zero_bits) - 1 + tmp; - - return 0; -} - -static int rbsp_write_uev(struct rbsp *rbsp, unsigned int *value) -{ - int ret; - int leading_zero_bits; - - if (!value) - return -EINVAL; - - leading_zero_bits = ilog2(*value + 1); - - ret = rbsp_write_bits(rbsp, leading_zero_bits, 0); - if (ret) - return ret; - - return rbsp_write_bits(rbsp, leading_zero_bits + 1, *value + 1); -} - -static int rbsp_read_sev(struct rbsp *rbsp, int *value) -{ - int ret; - unsigned int tmp; - - ret = rbsp_read_uev(rbsp, &tmp); - if (ret) - return ret; - - if (value) { - if (tmp & 1) - *value = (tmp + 1) / 2; - else - *value = -(tmp / 2); - } - - return 0; -} - -static int rbsp_write_sev(struct rbsp *rbsp, int *value) -{ - unsigned int tmp; - - if (!value) - return -EINVAL; - - if (*value > 0) - tmp = (2 * (*value)) | 1; - else - tmp = -2 * (*value); - - return rbsp_write_uev(rbsp, &tmp); -} - -static int __rbsp_write_bit(struct rbsp *rbsp, int *value) -{ - return rbsp_write_bit(rbsp, *value); -} - -static int __rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int *value) -{ - return rbsp_write_bits(rbsp, n, *value); -} - -static struct nal_h264_ops write = { - .rbsp_bit = __rbsp_write_bit, - .rbsp_bits = __rbsp_write_bits, - .rbsp_uev = rbsp_write_uev, - .rbsp_sev = rbsp_write_sev, -}; - -static int __rbsp_read_bit(struct rbsp *rbsp, int *value) -{ - int tmp = rbsp_read_bit(rbsp); - - if (tmp < 0) - return tmp; - *value = tmp; - - return 0; -} - -static struct nal_h264_ops read = { - .rbsp_bit = __rbsp_read_bit, - .rbsp_bits = rbsp_read_bits, - .rbsp_uev = rbsp_read_uev, - .rbsp_sev = rbsp_read_sev, -}; - -static inline void rbsp_bit(struct rbsp *rbsp, int *value) -{ - if (rbsp->error) - return; - rbsp->error = rbsp->ops->rbsp_bit(rbsp, value); -} - -static inline void rbsp_bits(struct rbsp *rbsp, int n, int *value) -{ - if (rbsp->error) - return; - rbsp->error = rbsp->ops->rbsp_bits(rbsp, n, value); -} - -static inline void rbsp_uev(struct rbsp *rbsp, unsigned int *value) -{ - if (rbsp->error) - return; - rbsp->error = rbsp->ops->rbsp_uev(rbsp, value); -} - -static inline void rbsp_sev(struct rbsp *rbsp, int *value) -{ - if (rbsp->error) - return; - rbsp->error = rbsp->ops->rbsp_sev(rbsp, value); -} - -static void nal_h264_rbsp_trailing_bits(struct rbsp *rbsp) -{ - unsigned int rbsp_stop_one_bit = 1; - unsigned int rbsp_alignment_zero_bit = 0; - - rbsp_bit(rbsp, &rbsp_stop_one_bit); - rbsp_bits(rbsp, round_up(rbsp->pos, 8) - rbsp->pos, - &rbsp_alignment_zero_bit); -} - static void nal_h264_write_start_code_prefix(struct rbsp *rbsp) { u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8); @@ -767,7 +445,7 @@ ssize_t nal_h264_write_sps(const struct device *dev, nal_h264_rbsp_sps(&rbsp, sps); - nal_h264_rbsp_trailing_bits(&rbsp); + rbsp_trailing_bits(&rbsp); if (rbsp.error) return rbsp.error; @@ -814,7 +492,7 @@ ssize_t nal_h264_read_sps(const struct device *dev, nal_h264_rbsp_sps(&rbsp, sps); - nal_h264_rbsp_trailing_bits(&rbsp); + rbsp_trailing_bits(&rbsp); if (rbsp.error) return rbsp.error; @@ -859,7 +537,7 @@ ssize_t nal_h264_write_pps(const struct device *dev, nal_h264_rbsp_pps(&rbsp, pps); - nal_h264_rbsp_trailing_bits(&rbsp); + rbsp_trailing_bits(&rbsp); if (rbsp.error) return rbsp.error; @@ -896,7 +574,7 @@ ssize_t nal_h264_read_pps(const struct device *dev, nal_h264_rbsp_pps(&rbsp, pps); - nal_h264_rbsp_trailing_bits(&rbsp); + rbsp_trailing_bits(&rbsp); if (rbsp.error) return rbsp.error; @@ -942,7 +620,7 @@ ssize_t nal_h264_write_filler(const struct device *dev, void *dest, size_t n) nal_h264_write_filler_data(&rbsp); - nal_h264_rbsp_trailing_bits(&rbsp); + rbsp_trailing_bits(&rbsp); return DIV_ROUND_UP(rbsp.pos, 8); } @@ -991,7 +669,7 @@ ssize_t nal_h264_read_filler(const struct device *dev, void *src, size_t n) return -EINVAL; nal_h264_read_filler_data(&rbsp); - nal_h264_rbsp_trailing_bits(&rbsp); + rbsp_trailing_bits(&rbsp); if (rbsp.error) return rbsp.error; diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.c b/drivers/media/platform/allegro-dvt/nal-rbsp.c new file mode 100644 index 000000000000..935ba23844f2 --- /dev/null +++ b/drivers/media/platform/allegro-dvt/nal-rbsp.c @@ -0,0 +1,305 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019-2020 Pengutronix, Michael Tretter + * + * Helper functions to generate a raw byte sequence payload from values. + */ + +#include +#include +#include +#include + +#include +#include +#include + +#include "nal-rbsp.h" + +void rbsp_init(struct rbsp *rbsp, void *addr, size_t size, + struct nal_rbsp_ops *ops) +{ + if (!rbsp) + return; + + rbsp->data = addr; + rbsp->size = size; + rbsp->pos = 0; + rbsp->ops = ops; + rbsp->error = 0; +} + +static int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value); +static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value); + +/* + * When reading or writing, the emulation_prevention_three_byte is detected + * only when the 2 one bits need to be inserted. Therefore, we are not + * actually adding the 0x3 byte, but the 2 one bits and the six 0 bits of the + * next byte. + */ +#define EMULATION_PREVENTION_THREE_BYTE (0x3 << 6) + +static int add_emulation_prevention_three_byte(struct rbsp *rbsp) +{ + rbsp->num_consecutive_zeros = 0; + rbsp_write_bits(rbsp, 8, EMULATION_PREVENTION_THREE_BYTE); + + return 0; +} + +static int discard_emulation_prevention_three_byte(struct rbsp *rbsp) +{ + unsigned int tmp = 0; + + rbsp->num_consecutive_zeros = 0; + rbsp_read_bits(rbsp, 8, &tmp); + if (tmp != EMULATION_PREVENTION_THREE_BYTE) + return -EINVAL; + + return 0; +} + +static inline int rbsp_read_bit(struct rbsp *rbsp) +{ + int shift; + int ofs; + int bit; + int err; + + if (rbsp->num_consecutive_zeros == 22) { + err = discard_emulation_prevention_three_byte(rbsp); + if (err) + return err; + } + + shift = 7 - (rbsp->pos % 8); + ofs = rbsp->pos / 8; + if (ofs >= rbsp->size) + return -EINVAL; + + bit = (rbsp->data[ofs] >> shift) & 1; + + rbsp->pos++; + + if (bit == 1 || + (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0))) + rbsp->num_consecutive_zeros = 0; + else + rbsp->num_consecutive_zeros++; + + return bit; +} + +static inline int rbsp_write_bit(struct rbsp *rbsp, bool value) +{ + int shift; + int ofs; + + if (rbsp->num_consecutive_zeros == 22) + add_emulation_prevention_three_byte(rbsp); + + shift = 7 - (rbsp->pos % 8); + ofs = rbsp->pos / 8; + if (ofs >= rbsp->size) + return -EINVAL; + + rbsp->data[ofs] &= ~(1 << shift); + rbsp->data[ofs] |= value << shift; + + rbsp->pos++; + + if (value || + (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0))) { + rbsp->num_consecutive_zeros = 0; + } else { + rbsp->num_consecutive_zeros++; + } + + return 0; +} + +static inline int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value) +{ + int i; + int bit; + unsigned int tmp = 0; + + if (n > 8 * sizeof(*value)) + return -EINVAL; + + for (i = n; i > 0; i--) { + bit = rbsp_read_bit(rbsp); + if (bit < 0) + return bit; + tmp |= bit << (i - 1); + } + + if (value) + *value = tmp; + + return 0; +} + +static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value) +{ + int ret; + + if (n > 8 * sizeof(value)) + return -EINVAL; + + while (n--) { + ret = rbsp_write_bit(rbsp, (value >> n) & 1); + if (ret) + return ret; + } + + return 0; +} + +static int rbsp_read_uev(struct rbsp *rbsp, unsigned int *value) +{ + int leading_zero_bits = 0; + unsigned int tmp = 0; + int ret; + + while ((ret = rbsp_read_bit(rbsp)) == 0) + leading_zero_bits++; + if (ret < 0) + return ret; + + if (leading_zero_bits > 0) { + ret = rbsp_read_bits(rbsp, leading_zero_bits, &tmp); + if (ret) + return ret; + } + + if (value) + *value = (1 << leading_zero_bits) - 1 + tmp; + + return 0; +} + +static int rbsp_write_uev(struct rbsp *rbsp, unsigned int *value) +{ + int ret; + int leading_zero_bits; + + if (!value) + return -EINVAL; + + leading_zero_bits = ilog2(*value + 1); + + ret = rbsp_write_bits(rbsp, leading_zero_bits, 0); + if (ret) + return ret; + + return rbsp_write_bits(rbsp, leading_zero_bits + 1, *value + 1); +} + +static int rbsp_read_sev(struct rbsp *rbsp, int *value) +{ + int ret; + unsigned int tmp; + + ret = rbsp_read_uev(rbsp, &tmp); + if (ret) + return ret; + + if (value) { + if (tmp & 1) + *value = (tmp + 1) / 2; + else + *value = -(tmp / 2); + } + + return 0; +} + +static int rbsp_write_sev(struct rbsp *rbsp, int *value) +{ + unsigned int tmp; + + if (!value) + return -EINVAL; + + if (*value > 0) + tmp = (2 * (*value)) | 1; + else + tmp = -2 * (*value); + + return rbsp_write_uev(rbsp, &tmp); +} + +static int __rbsp_write_bit(struct rbsp *rbsp, int *value) +{ + return rbsp_write_bit(rbsp, *value); +} + +static int __rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int *value) +{ + return rbsp_write_bits(rbsp, n, *value); +} + +struct nal_rbsp_ops write = { + .rbsp_bit = __rbsp_write_bit, + .rbsp_bits = __rbsp_write_bits, + .rbsp_uev = rbsp_write_uev, + .rbsp_sev = rbsp_write_sev, +}; + +static int __rbsp_read_bit(struct rbsp *rbsp, int *value) +{ + int tmp = rbsp_read_bit(rbsp); + + if (tmp < 0) + return tmp; + *value = tmp; + + return 0; +} + +struct nal_rbsp_ops read = { + .rbsp_bit = __rbsp_read_bit, + .rbsp_bits = rbsp_read_bits, + .rbsp_uev = rbsp_read_uev, + .rbsp_sev = rbsp_read_sev, +}; + +void rbsp_bit(struct rbsp *rbsp, int *value) +{ + if (rbsp->error) + return; + rbsp->error = rbsp->ops->rbsp_bit(rbsp, value); +} + +void rbsp_bits(struct rbsp *rbsp, int n, int *value) +{ + if (rbsp->error) + return; + rbsp->error = rbsp->ops->rbsp_bits(rbsp, n, value); +} + +void rbsp_uev(struct rbsp *rbsp, unsigned int *value) +{ + if (rbsp->error) + return; + rbsp->error = rbsp->ops->rbsp_uev(rbsp, value); +} + +void rbsp_sev(struct rbsp *rbsp, int *value) +{ + if (rbsp->error) + return; + rbsp->error = rbsp->ops->rbsp_sev(rbsp, value); +} + +void rbsp_trailing_bits(struct rbsp *rbsp) +{ + unsigned int rbsp_stop_one_bit = 1; + unsigned int rbsp_alignment_zero_bit = 0; + + rbsp_bit(rbsp, &rbsp_stop_one_bit); + rbsp_bits(rbsp, round_up(rbsp->pos, 8) - rbsp->pos, + &rbsp_alignment_zero_bit); +} diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.h b/drivers/media/platform/allegro-dvt/nal-rbsp.h new file mode 100644 index 000000000000..90cc1a4f716d --- /dev/null +++ b/drivers/media/platform/allegro-dvt/nal-rbsp.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2019-2020 Pengutronix, Michael Tretter + */ + +#ifndef __NAL_RBSP_H__ +#define __NAL_RBSP_H__ + +#include +#include + +struct rbsp; + +struct nal_rbsp_ops { + int (*rbsp_bit)(struct rbsp *rbsp, int *val); + int (*rbsp_bits)(struct rbsp *rbsp, int n, unsigned int *val); + int (*rbsp_uev)(struct rbsp *rbsp, unsigned int *val); + int (*rbsp_sev)(struct rbsp *rbsp, int *val); +}; + +/** + * struct rbsp - State object for handling a raw byte sequence payload + * @data: pointer to the data of the rbsp + * @size: maximum size of the data of the rbsp + * @pos: current bit position inside the rbsp + * @num_consecutive_zeros: number of zeros before @pos + * @ops: per datatype functions for interacting with the rbsp + * @error: an error occurred while handling the rbsp + * + * This struct is passed around the various parsing functions and tracks the + * current position within the raw byte sequence payload. + * + * The @ops field allows to separate the operation, i.e., reading/writing a + * value from/to that rbsp, from the structure of the NAL unit. This allows to + * have a single function for iterating the NAL unit, while @ops has function + * pointers for handling each type in the rbsp. + */ +struct rbsp { + u8 *data; + size_t size; + unsigned int pos; + unsigned int num_consecutive_zeros; + struct nal_rbsp_ops *ops; + int error; +}; + +extern struct nal_rbsp_ops write; +extern struct nal_rbsp_ops read; + +void rbsp_init(struct rbsp *rbsp, void *addr, size_t size, + struct nal_rbsp_ops *ops); + +void rbsp_bit(struct rbsp *rbsp, int *value); +void rbsp_bits(struct rbsp *rbsp, int n, int *value); +void rbsp_uev(struct rbsp *rbsp, unsigned int *value); +void rbsp_sev(struct rbsp *rbsp, int *value); + +void rbsp_trailing_bits(struct rbsp *rbsp); + +#endif /* __NAL_RBSP_H__ */ From patchwork Thu Dec 3 11:00:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4474CC63777 for ; Thu, 3 Dec 2020 11:03:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D3914208FE for ; Thu, 3 Dec 2020 11:03:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728989AbgLCLDc (ORCPT ); Thu, 3 Dec 2020 06:03:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbgLCLDc (ORCPT ); Thu, 3 Dec 2020 06:03:32 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0720C061A4E for ; Thu, 3 Dec 2020 03:02:51 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNs-0002Ko-P8; Thu, 03 Dec 2020 12:02:49 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhX-Pt; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:50 +0100 Message-Id: <20201203110106.2939463-3-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 02/18] media: allegro: add helper to report unsupported fields X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Allow generators to explicitly signal an error if the C structs contain unsupported or invalid fields. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/nal-rbsp.c | 5 +++++ drivers/media/platform/allegro-dvt/nal-rbsp.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.c b/drivers/media/platform/allegro-dvt/nal-rbsp.c index 935ba23844f2..d911322d0efa 100644 --- a/drivers/media/platform/allegro-dvt/nal-rbsp.c +++ b/drivers/media/platform/allegro-dvt/nal-rbsp.c @@ -29,6 +29,11 @@ void rbsp_init(struct rbsp *rbsp, void *addr, size_t size, rbsp->error = 0; } +void rbsp_unsupported(struct rbsp *rbsp) +{ + rbsp->error = -EINVAL; +} + static int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value); static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value); diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.h b/drivers/media/platform/allegro-dvt/nal-rbsp.h index 90cc1a4f716d..c72f49fed8d3 100644 --- a/drivers/media/platform/allegro-dvt/nal-rbsp.h +++ b/drivers/media/platform/allegro-dvt/nal-rbsp.h @@ -49,6 +49,7 @@ extern struct nal_rbsp_ops read; void rbsp_init(struct rbsp *rbsp, void *addr, size_t size, struct nal_rbsp_ops *ops); +void rbsp_unsupported(struct rbsp *rbsp); void rbsp_bit(struct rbsp *rbsp, int *value); void rbsp_bits(struct rbsp *rbsp, int n, int *value); From patchwork Thu Dec 3 11:00:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 756E3C64E8A for ; Thu, 3 Dec 2020 11:04:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 061CE21D91 for ; Thu, 3 Dec 2020 11:04:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388864AbgLCLEj (ORCPT ); Thu, 3 Dec 2020 06:04:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388755AbgLCLEj (ORCPT ); Thu, 3 Dec 2020 06:04:39 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1E79C08E9AA for ; Thu, 3 Dec 2020 03:02:55 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNs-0002Kp-R4; Thu, 03 Dec 2020 12:02:54 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhc-Qo; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:51 +0100 Message-Id: <20201203110106.2939463-4-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 03/18] media: allegro: add HEVC NAL unit generator X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When encoding a video as HEVC, the allegro driver needs to generate the Non-VCL NAL units for HEVC. Do the same as for H.264 and add a module that takes C structs for the VPS/SPS/PPS and encodes the fields as RBPS as specified by "ITU-T Rec. H.265 (02/2018) high efficiency video coding". Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/Makefile | 2 +- drivers/media/platform/allegro-dvt/nal-hevc.c | 824 ++++++++++++++++++ drivers/media/platform/allegro-dvt/nal-hevc.h | 350 ++++++++ 3 files changed, 1175 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/allegro-dvt/nal-hevc.c create mode 100644 drivers/media/platform/allegro-dvt/nal-hevc.h diff --git a/drivers/media/platform/allegro-dvt/Makefile b/drivers/media/platform/allegro-dvt/Makefile index 10119e84a60f..66108a303774 100644 --- a/drivers/media/platform/allegro-dvt/Makefile +++ b/drivers/media/platform/allegro-dvt/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 allegro-objs := allegro-core.o allegro-mail.o -allegro-objs += nal-rbsp.o nal-h264.o +allegro-objs += nal-rbsp.o nal-h264.o nal-hevc.o obj-$(CONFIG_VIDEO_ALLEGRO_DVT) += allegro.o diff --git a/drivers/media/platform/allegro-dvt/nal-hevc.c b/drivers/media/platform/allegro-dvt/nal-hevc.c new file mode 100644 index 000000000000..2dcc516729d7 --- /dev/null +++ b/drivers/media/platform/allegro-dvt/nal-hevc.c @@ -0,0 +1,824 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019-2020 Pengutronix, Michael Tretter + * + * Convert NAL units between raw byte sequence payloads (RBSP) and C structs. + * + * The conversion is defined in "ITU-T Rec. H.265 (02/2018) high efficiency + * video coding". Decoder drivers may use the parser to parse RBSP from + * encoded streams and configure the hardware, if the hardware is not able to + * parse RBSP itself. Encoder drivers may use the generator to generate the + * RBSP for VPS/SPS/PPS nal units and add them to the encoded stream if the + * hardware does not generate the units. + */ + +#include +#include +#include +#include + +#include +#include +#include + +#include "nal-hevc.h" +#include "nal-rbsp.h" + +/* + * See Rec. ITU-T H.265 (02/2018) Table 7-1 – NAL unit type codes and NAL unit + * type classes + */ +enum nal_unit_type { + VPS_NUT = 32, + SPS_NUT = 33, + PPS_NUT = 34, + FD_NUT = 38, +}; + +int nal_hevc_profile_from_v4l2(enum v4l2_mpeg_video_hevc_profile profile) +{ + switch (profile) { + case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN: + return 1; + case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10: + return 2; + case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE: + return 3; + default: + return -EINVAL; + } +} +EXPORT_SYMBOL_GPL(nal_hevc_profile_from_v4l2); + +int nal_hevc_tier_from_v4l2(enum v4l2_mpeg_video_hevc_tier tier) +{ + switch (tier) { + case V4L2_MPEG_VIDEO_HEVC_TIER_MAIN: + return 0; + case V4L2_MPEG_VIDEO_HEVC_TIER_HIGH: + return 1; + default: + return -EINVAL; + } +} +EXPORT_SYMBOL_GPL(nal_hevc_tier_from_v4l2); + +int nal_hevc_level_from_v4l2(enum v4l2_mpeg_video_hevc_level level) +{ + /* + * T-Rec-H.265 p. 280: general_level_idc and sub_layer_level_idc[ i ] + * shall be set equal to a value of 30 times the level number + * specified in Table A.6. + */ + int factor = 30 / 10; + + switch (level) { + case V4L2_MPEG_VIDEO_HEVC_LEVEL_1: + return factor * 10; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2: + return factor * 20; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1: + return factor * 21; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3: + return factor * 30; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1: + return factor * 31; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4: + return factor * 40; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1: + return factor * 41; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5: + return factor * 50; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1: + return factor * 51; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2: + return factor * 52; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_6: + return factor * 60; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1: + return factor * 61; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2: + return factor * 62; + default: + return -EINVAL; + } +} +EXPORT_SYMBOL_GPL(nal_hevc_level_from_v4l2); + +void nal_hevc_write_start_code_prefix(struct rbsp *rbsp) +{ + u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8); + int i = 4; + + if (DIV_ROUND_UP(rbsp->pos, 8) + i > rbsp->size) { + rbsp->error = -EINVAL; + return; + } + + p[0] = 0x00; + p[1] = 0x00; + p[2] = 0x00; + p[3] = 0x01; + + rbsp->pos += i * 8; +} + +void nal_hevc_read_start_code_prefix(struct rbsp *rbsp) +{ + u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8); + int i = 4; + + if (DIV_ROUND_UP(rbsp->pos, 8) + i > rbsp->size) { + rbsp->error = -EINVAL; + return; + } + + if (p[0] != 0x00 || p[1] != 0x00 || p[2] != 0x00 || p[3] != 0x01) { + rbsp->error = -EINVAL; + return; + } + + rbsp->pos += i * 8; +} + +static void nal_hevc_write_filler_data(struct rbsp *rbsp) +{ + u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8); + int i; + + /* Keep 1 byte extra for terminating the NAL unit */ + i = rbsp->size - DIV_ROUND_UP(rbsp->pos, 8) - 1; + memset(p, 0xff, i); + rbsp->pos += i * 8; +} + +static void nal_hevc_read_filler_data(struct rbsp *rbsp) +{ + u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8); + + while (*p == 0xff) { + if (DIV_ROUND_UP(rbsp->pos, 8) > rbsp->size) { + rbsp->error = -EINVAL; + return; + } + + p++; + rbsp->pos += 8; + } +} + +static void nal_hevc_rbsp_profile_tier_level(struct rbsp *rbsp, + struct nal_hevc_profile_tier_level *ptl) +{ + unsigned int i; + unsigned int max_num_sub_layers_minus_1 = 0; + + rbsp_bits(rbsp, 2, &ptl->general_profile_space); + rbsp_bit(rbsp, &ptl->general_tier_flag); + rbsp_bits(rbsp, 5, &ptl->general_profile_idc); + for (i = 0; i < 32; i++) + rbsp_bit(rbsp, &ptl->general_profile_compatibility_flag[i]); + rbsp_bit(rbsp, &ptl->general_progressive_source_flag); + rbsp_bit(rbsp, &ptl->general_interlaced_source_flag); + rbsp_bit(rbsp, &ptl->general_non_packed_constraint_flag); + rbsp_bit(rbsp, &ptl->general_frame_only_constraint_flag); + if (ptl->general_profile_idc == 4 || + ptl->general_profile_compatibility_flag[4] || + ptl->general_profile_idc == 5 || + ptl->general_profile_compatibility_flag[5] || + ptl->general_profile_idc == 6 || + ptl->general_profile_compatibility_flag[6] || + ptl->general_profile_idc == 7 || + ptl->general_profile_compatibility_flag[7] || + ptl->general_profile_idc == 8 || + ptl->general_profile_compatibility_flag[8] || + ptl->general_profile_idc == 9 || + ptl->general_profile_compatibility_flag[9] || + ptl->general_profile_idc == 10 || + ptl->general_profile_compatibility_flag[10]) { + rbsp_bit(rbsp, &ptl->general_max_12bit_constraint_flag); + rbsp_bit(rbsp, &ptl->general_max_10bit_constraint_flag); + rbsp_bit(rbsp, &ptl->general_max_8bit_constraint_flag); + rbsp_bit(rbsp, &ptl->general_max_422chroma_constraint_flag); + rbsp_bit(rbsp, &ptl->general_max_420chroma_constraint_flag); + rbsp_bit(rbsp, &ptl->general_max_monochrome_constraint_flag); + rbsp_bit(rbsp, &ptl->general_intra_constraint_flag); + rbsp_bit(rbsp, &ptl->general_one_picture_only_constraint_flag); + rbsp_bit(rbsp, &ptl->general_lower_bit_rate_constraint_flag); + if (ptl->general_profile_idc == 5 || + ptl->general_profile_compatibility_flag[5] || + ptl->general_profile_idc == 9 || + ptl->general_profile_compatibility_flag[9] || + ptl->general_profile_idc == 10 || + ptl->general_profile_compatibility_flag[10]) { + rbsp_bit(rbsp, &ptl->general_max_14bit_constraint_flag); + rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_33bits); + rbsp_bits(rbsp, 33 - 32, &ptl->general_reserved_zero_33bits); + } else { + rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_34bits); + rbsp_bits(rbsp, 34 - 2, &ptl->general_reserved_zero_34bits); + } + } else if (ptl->general_profile_idc == 2 || + ptl->general_profile_compatibility_flag[2]) { + rbsp_bits(rbsp, 7, &ptl->general_reserved_zero_7bits); + rbsp_bit(rbsp, &ptl->general_one_picture_only_constraint_flag); + rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_35bits); + rbsp_bits(rbsp, 35 - 32, &ptl->general_reserved_zero_35bits); + } else { + rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_43bits); + rbsp_bits(rbsp, 43 - 32, &ptl->general_reserved_zero_43bits); + } + if ((ptl->general_profile_idc >= 1 && ptl->general_profile_idc <= 5) || + ptl->general_profile_idc == 9 || + ptl->general_profile_compatibility_flag[1] || + ptl->general_profile_compatibility_flag[2] || + ptl->general_profile_compatibility_flag[3] || + ptl->general_profile_compatibility_flag[4] || + ptl->general_profile_compatibility_flag[5] || + ptl->general_profile_compatibility_flag[9]) + rbsp_bit(rbsp, &ptl->general_inbld_flag); + else + rbsp_bit(rbsp, &ptl->general_reserved_zero_bit); + rbsp_bits(rbsp, 8, &ptl->general_level_idc); + if (max_num_sub_layers_minus_1 > 0) + rbsp_unsupported(rbsp); +} + +static void nal_hevc_rbsp_vps(struct rbsp *rbsp, struct nal_hevc_vps *vps) +{ + unsigned int i, j; + unsigned int reserved_0xffff_16bits = 0xffff; + + rbsp_bits(rbsp, 4, &vps->video_parameter_set_id); + rbsp_bit(rbsp, &vps->base_layer_internal_flag); + rbsp_bit(rbsp, &vps->base_layer_available_flag); + rbsp_bits(rbsp, 6, &vps->max_layers_minus1); + rbsp_bits(rbsp, 3, &vps->max_sub_layers_minus1); + rbsp_bits(rbsp, 1, &vps->temporal_id_nesting_flag); + rbsp_bits(rbsp, 16, &reserved_0xffff_16bits); + nal_hevc_rbsp_profile_tier_level(rbsp, &vps->profile_tier_level); + rbsp_bit(rbsp, &vps->sub_layer_ordering_info_present_flag); + for (i = vps->sub_layer_ordering_info_present_flag ? 0 : vps->max_sub_layers_minus1; + i <= vps->max_sub_layers_minus1; i++) { + rbsp_uev(rbsp, &vps->max_dec_pic_buffering_minus1[i]); + rbsp_uev(rbsp, &vps->max_num_reorder_pics[i]); + rbsp_uev(rbsp, &vps->max_latency_increase_plus1[i]); + } + rbsp_bits(rbsp, 6, &vps->max_layer_id); + rbsp_uev(rbsp, &vps->num_layer_sets_minus1); + for (i = 0; i <= vps->num_layer_sets_minus1; i++) + for (j = 0; j <= vps->max_layer_id; j++) + rbsp_bit(rbsp, &vps->layer_id_included_flag[i][j]); + rbsp_bit(rbsp, &vps->timing_info_present_flag); + if (vps->timing_info_present_flag) + rbsp_unsupported(rbsp); + rbsp_bit(rbsp, &vps->extension_flag); + if (vps->extension_flag) + rbsp_unsupported(rbsp); +} + +static void nal_hevc_rbsp_sps(struct rbsp *rbsp, struct nal_hevc_sps *sps) +{ + unsigned int i; + + rbsp_bits(rbsp, 4, &sps->video_parameter_set_id); + rbsp_bits(rbsp, 3, &sps->max_sub_layers_minus1); + rbsp_bit(rbsp, &sps->temporal_id_nesting_flag); + nal_hevc_rbsp_profile_tier_level(rbsp, &sps->profile_tier_level); + rbsp_uev(rbsp, &sps->seq_parameter_set_id); + + rbsp_uev(rbsp, &sps->chroma_format_idc); + if (sps->chroma_format_idc == 3) + rbsp_bit(rbsp, &sps->separate_colour_plane_flag); + rbsp_uev(rbsp, &sps->pic_width_in_luma_samples); + rbsp_uev(rbsp, &sps->pic_height_in_luma_samples); + rbsp_bit(rbsp, &sps->conformance_window_flag); + if (sps->conformance_window_flag) { + rbsp_uev(rbsp, &sps->conf_win_left_offset); + rbsp_uev(rbsp, &sps->conf_win_right_offset); + rbsp_uev(rbsp, &sps->conf_win_top_offset); + rbsp_uev(rbsp, &sps->conf_win_bottom_offset); + } + rbsp_uev(rbsp, &sps->bit_depth_luma_minus8); + rbsp_uev(rbsp, &sps->bit_depth_chroma_minus8); + + rbsp_uev(rbsp, &sps->log2_max_pic_order_cnt_lsb_minus4); + + rbsp_bit(rbsp, &sps->sub_layer_ordering_info_present_flag); + for (i = (sps->sub_layer_ordering_info_present_flag ? 0 : sps->max_sub_layers_minus1); + i <= sps->max_sub_layers_minus1; i++) { + rbsp_uev(rbsp, &sps->max_dec_pic_buffering_minus1[i]); + rbsp_uev(rbsp, &sps->max_num_reorder_pics[i]); + rbsp_uev(rbsp, &sps->max_latency_increase_plus1[i]); + } + rbsp_uev(rbsp, &sps->log2_min_luma_coding_block_size_minus3); + rbsp_uev(rbsp, &sps->log2_diff_max_min_luma_coding_block_size); + rbsp_uev(rbsp, &sps->log2_min_luma_transform_block_size_minus2); + rbsp_uev(rbsp, &sps->log2_diff_max_min_luma_transform_block_size); + rbsp_uev(rbsp, &sps->max_transform_hierarchy_depth_inter); + rbsp_uev(rbsp, &sps->max_transform_hierarchy_depth_intra); + + rbsp_bit(rbsp, &sps->scaling_list_enabled_flag); + if (sps->scaling_list_enabled_flag) + rbsp_unsupported(rbsp); + + rbsp_bit(rbsp, &sps->amp_enabled_flag); + rbsp_bit(rbsp, &sps->sample_adaptive_offset_enabled_flag); + rbsp_bit(rbsp, &sps->pcm_enabled_flag); + if (sps->pcm_enabled_flag) { + rbsp_bits(rbsp, 4, &sps->pcm_sample_bit_depth_luma_minus1); + rbsp_bits(rbsp, 4, &sps->pcm_sample_bit_depth_chroma_minus1); + rbsp_uev(rbsp, &sps->log2_min_pcm_luma_coding_block_size_minus3); + rbsp_uev(rbsp, &sps->log2_diff_max_min_pcm_luma_coding_block_size); + rbsp_bit(rbsp, &sps->pcm_loop_filter_disabled_flag); + } + + rbsp_uev(rbsp, &sps->num_short_term_ref_pic_sets); + if (sps->num_short_term_ref_pic_sets > 0) + rbsp_unsupported(rbsp); + + rbsp_bit(rbsp, &sps->long_term_ref_pics_present_flag); + if (sps->long_term_ref_pics_present_flag) + rbsp_unsupported(rbsp); + + rbsp_bit(rbsp, &sps->sps_temporal_mvp_enabled_flag); + rbsp_bit(rbsp, &sps->strong_intra_smoothing_enabled_flag); + rbsp_bit(rbsp, &sps->vui_parameters_present_flag); + if (sps->vui_parameters_present_flag) + rbsp_unsupported(rbsp); + + rbsp_bit(rbsp, &sps->extension_present_flag); + if (sps->extension_present_flag) { + rbsp_bit(rbsp, &sps->sps_range_extension_flag); + rbsp_bit(rbsp, &sps->sps_multilayer_extension_flag); + rbsp_bit(rbsp, &sps->sps_3d_extension_flag); + rbsp_bit(rbsp, &sps->sps_scc_extension_flag); + rbsp_bits(rbsp, 5, &sps->sps_extension_4bits); + } + if (sps->sps_range_extension_flag) + rbsp_unsupported(rbsp); + if (sps->sps_multilayer_extension_flag) + rbsp_unsupported(rbsp); + if (sps->sps_3d_extension_flag) + rbsp_unsupported(rbsp); + if (sps->sps_scc_extension_flag) + rbsp_unsupported(rbsp); + if (sps->sps_extension_4bits) + rbsp_unsupported(rbsp); +} + +static void nal_hevc_rbsp_pps(struct rbsp *rbsp, struct nal_hevc_pps *pps) +{ + unsigned int i; + + rbsp_uev(rbsp, &pps->pps_pic_parameter_set_id); + rbsp_uev(rbsp, &pps->pps_seq_parameter_set_id); + rbsp_bit(rbsp, &pps->dependent_slice_segments_enabled_flag); + rbsp_bit(rbsp, &pps->output_flag_present_flag); + rbsp_bits(rbsp, 3, &pps->num_extra_slice_header_bits); + rbsp_bit(rbsp, &pps->sign_data_hiding_enabled_flag); + rbsp_bit(rbsp, &pps->cabac_init_present_flag); + rbsp_uev(rbsp, &pps->num_ref_idx_l0_default_active_minus1); + rbsp_uev(rbsp, &pps->num_ref_idx_l1_default_active_minus1); + rbsp_sev(rbsp, &pps->init_qp_minus26); + rbsp_bit(rbsp, &pps->constrained_intra_pred_flag); + rbsp_bit(rbsp, &pps->transform_skip_enabled_flag); + rbsp_bit(rbsp, &pps->cu_qp_delta_enabled_flag); + if (pps->cu_qp_delta_enabled_flag) + rbsp_uev(rbsp, &pps->diff_cu_qp_delta_depth); + rbsp_sev(rbsp, &pps->pps_cb_qp_offset); + rbsp_sev(rbsp, &pps->pps_cr_qp_offset); + rbsp_bit(rbsp, &pps->pps_slice_chroma_qp_offsets_present_flag); + rbsp_bit(rbsp, &pps->weighted_pred_flag); + rbsp_bit(rbsp, &pps->weighted_bipred_flag); + rbsp_bit(rbsp, &pps->transquant_bypass_enabled_flag); + rbsp_bit(rbsp, &pps->tiles_enabled_flag); + rbsp_bit(rbsp, &pps->entropy_coding_sync_enabled_flag); + if (pps->tiles_enabled_flag) { + rbsp_uev(rbsp, &pps->num_tile_columns_minus1); + rbsp_uev(rbsp, &pps->num_tile_rows_minus1); + rbsp_bit(rbsp, &pps->uniform_spacing_flag); + if (!pps->uniform_spacing_flag) { + for (i = 0; i < pps->num_tile_columns_minus1; i++) + rbsp_uev(rbsp, &pps->column_width_minus1[i]); + for (i = 0; i < pps->num_tile_rows_minus1; i++) + rbsp_uev(rbsp, &pps->row_height_minus1[i]); + } + rbsp_bit(rbsp, &pps->loop_filter_across_tiles_enabled_flag); + } + rbsp_bit(rbsp, &pps->pps_loop_filter_across_slices_enabled_flag); + rbsp_bit(rbsp, &pps->deblocking_filter_control_present_flag); + if (pps->deblocking_filter_control_present_flag) { + rbsp_bit(rbsp, &pps->deblocking_filter_override_enabled_flag); + rbsp_bit(rbsp, &pps->pps_deblocking_filter_disabled_flag); + if (!pps->pps_deblocking_filter_disabled_flag) { + rbsp_sev(rbsp, &pps->pps_beta_offset_div2); + rbsp_sev(rbsp, &pps->pps_tc_offset_div2); + } + } + rbsp_bit(rbsp, &pps->pps_scaling_list_data_present_flag); + if (pps->pps_scaling_list_data_present_flag) + rbsp_unsupported(rbsp); + rbsp_bit(rbsp, &pps->lists_modification_present_flag); + rbsp_uev(rbsp, &pps->log2_parallel_merge_level_minus2); + rbsp_bit(rbsp, &pps->slice_segment_header_extension_present_flag); + rbsp_bit(rbsp, &pps->pps_extension_present_flag); + if (pps->pps_extension_present_flag) { + rbsp_bit(rbsp, &pps->pps_range_extension_flag); + rbsp_bit(rbsp, &pps->pps_multilayer_extension_flag); + rbsp_bit(rbsp, &pps->pps_3d_extension_flag); + rbsp_bit(rbsp, &pps->pps_scc_extension_flag); + rbsp_bits(rbsp, 4, &pps->pps_extension_4bits); + } + if (pps->pps_range_extension_flag) + rbsp_unsupported(rbsp); + if (pps->pps_multilayer_extension_flag) + rbsp_unsupported(rbsp); + if (pps->pps_3d_extension_flag) + rbsp_unsupported(rbsp); + if (pps->pps_scc_extension_flag) + rbsp_unsupported(rbsp); + if (pps->pps_extension_4bits) + rbsp_unsupported(rbsp); +} + +/** + * nal_hevc_write_vps() - Write PPS NAL unit into RBSP format + * @dev: device pointer + * @dest: the buffer that is filled with RBSP data + * @n: maximum size of @dest in bytes + * @pps: &struct nal_hevc_vps to convert to RBSP + * + * Convert @vps to RBSP data and write it into @dest. + * + * The size of the VPS NAL unit is not known in advance and this function will + * fail, if @dest does not hold sufficient space for the VPS NAL unit. + * + * Return: number of bytes written to @dest or negative error code + */ +ssize_t nal_hevc_write_vps(const struct device *dev, + void *dest, size_t n, struct nal_hevc_vps *vps) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit = 0; + unsigned int nal_unit_type = VPS_NUT; + unsigned int nuh_layer_id = 0; + unsigned int nuh_temporal_id_plus1 = 1; + + if (!dest) + return -EINVAL; + + rbsp_init(&rbsp, dest, n, &write); + + nal_hevc_write_start_code_prefix(&rbsp); + + /* NAL unit header */ + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + nal_hevc_rbsp_vps(&rbsp, vps); + + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_write_vps); + +/** + * nal_hevc_read_vps() - Read VPS NAL unit from RBSP format + * @dev: device pointer + * @vps: the &struct nal_hevc_vps to fill from the RBSP data + * @src: the buffer that contains the RBSP data + * @n: size of @src in bytes + * + * Read RBSP data from @src and use it to fill @vps. + * + * Return: number of bytes read from @src or negative error code + */ +ssize_t nal_hevc_read_vps(const struct device *dev, + struct nal_hevc_vps *vps, void *src, size_t n) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit; + unsigned int nal_unit_type; + unsigned int nuh_layer_id; + unsigned int nuh_temporal_id_plus1; + + if (!src) + return -EINVAL; + + rbsp_init(&rbsp, src, n, &read); + + nal_hevc_read_start_code_prefix(&rbsp); + + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + if (rbsp.error || + forbidden_zero_bit != 0 || + nal_unit_type != VPS_NUT) + return -EINVAL; + + nal_hevc_rbsp_vps(&rbsp, vps); + + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_read_vps); + +/** + * nal_hevc_write_sps() - Write SPS NAL unit into RBSP format + * @dev: device pointer + * @dest: the buffer that is filled with RBSP data + * @n: maximum size of @dest in bytes + * @sps: &struct nal_hevc_sps to convert to RBSP + * + * Convert @sps to RBSP data and write it into @dest. + * + * The size of the SPS NAL unit is not known in advance and this function will + * fail, if @dest does not hold sufficient space for the SPS NAL unit. + * + * Return: number of bytes written to @dest or negative error code + */ +ssize_t nal_hevc_write_sps(const struct device *dev, + void *dest, size_t n, struct nal_hevc_sps *sps) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit = 0; + unsigned int nal_unit_type = SPS_NUT; + unsigned int nuh_layer_id = 0; + unsigned int nuh_temporal_id_plus1 = 1; + + if (!dest) + return -EINVAL; + + rbsp_init(&rbsp, dest, n, &write); + + nal_hevc_write_start_code_prefix(&rbsp); + + /* NAL unit header */ + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + nal_hevc_rbsp_sps(&rbsp, sps); + + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_write_sps); + +/** + * nal_hevc_read_sps() - Read SPS NAL unit from RBSP format + * @dev: device pointer + * @sps: the &struct nal_hevc_sps to fill from the RBSP data + * @src: the buffer that contains the RBSP data + * @n: size of @src in bytes + * + * Read RBSP data from @src and use it to fill @sps. + * + * Return: number of bytes read from @src or negative error code + */ +ssize_t nal_hevc_read_sps(const struct device *dev, + struct nal_hevc_sps *sps, void *src, size_t n) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit; + unsigned int nal_unit_type; + unsigned int nuh_layer_id; + unsigned int nuh_temporal_id_plus1; + + if (!src) + return -EINVAL; + + rbsp_init(&rbsp, src, n, &read); + + nal_hevc_read_start_code_prefix(&rbsp); + + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + if (rbsp.error || + forbidden_zero_bit != 0 || + nal_unit_type != SPS_NUT) + return -EINVAL; + + nal_hevc_rbsp_sps(&rbsp, sps); + + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_read_sps); + +/** + * nal_hevc_write_pps() - Write PPS NAL unit into RBSP format + * @dev: device pointer + * @dest: the buffer that is filled with RBSP data + * @n: maximum size of @dest in bytes + * @pps: &struct nal_hevc_pps to convert to RBSP + * + * Convert @pps to RBSP data and write it into @dest. + * + * The size of the PPS NAL unit is not known in advance and this function will + * fail, if @dest does not hold sufficient space for the PPS NAL unit. + * + * Return: number of bytes written to @dest or negative error code + */ +ssize_t nal_hevc_write_pps(const struct device *dev, + void *dest, size_t n, struct nal_hevc_pps *pps) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit = 0; + unsigned int nal_unit_type = PPS_NUT; + unsigned int nuh_layer_id = 0; + unsigned int nuh_temporal_id_plus1 = 1; + + if (!dest) + return -EINVAL; + + rbsp_init(&rbsp, dest, n, &write); + + nal_hevc_write_start_code_prefix(&rbsp); + + /* NAL unit header */ + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + nal_hevc_rbsp_pps(&rbsp, pps); + + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_write_pps); + +/** + * nal_hevc_read_pps() - Read PPS NAL unit from RBSP format + * @dev: device pointer + * @pps: the &struct nal_hevc_pps to fill from the RBSP data + * @src: the buffer that contains the RBSP data + * @n: size of @src in bytes + * + * Read RBSP data from @src and use it to fill @pps. + * + * Return: number of bytes read from @src or negative error code + */ +ssize_t nal_hevc_read_pps(const struct device *dev, + struct nal_hevc_pps *pps, void *src, size_t n) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit; + unsigned int nal_unit_type; + unsigned int nuh_layer_id; + unsigned int nuh_temporal_id_plus1; + + if (!src) + return -EINVAL; + + rbsp_init(&rbsp, src, n, &read); + + nal_hevc_read_start_code_prefix(&rbsp); + + /* NAL unit header */ + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + nal_hevc_rbsp_pps(&rbsp, pps); + + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_read_pps); + +/** + * nal_hevc_write_filler() - Write filler data RBSP + * @dev: device pointer + * @dest: buffer to fill with filler data + * @n: size of the buffer to fill with filler data + * + * Write a filler data RBSP to @dest with a size of @n bytes and return the + * number of written filler data bytes. + * + * Use this function to generate dummy data in an RBSP data stream that can be + * safely ignored by hevc decoders. + * + * The RBSP format of the filler data is specified in Rec. ITU-T H.265 + * (02/2018) 7.3.2.8 Filler data RBSP syntax. + * + * Return: number of filler data bytes (including marker) or negative error + */ +ssize_t nal_hevc_write_filler(const struct device *dev, void *dest, size_t n) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit = 0; + unsigned int nal_unit_type = FD_NUT; + unsigned int nuh_layer_id = 0; + unsigned int nuh_temporal_id_plus1 = 1; + + if (!dest) + return -EINVAL; + + rbsp_init(&rbsp, dest, n, &write); + + nal_hevc_write_start_code_prefix(&rbsp); + + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + nal_hevc_write_filler_data(&rbsp); + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_write_filler); + +/** + * nal_hevc_read_filler() - Read filler data RBSP + * @dev: device pointer + * @src: buffer with RBSP data that is read + * @n: maximum size of src that shall be read + * + * Read a filler data RBSP from @src up to a maximum size of @n bytes and + * return the size of the filler data in bytes including the marker. + * + * This function is used to parse filler data and skip the respective bytes in + * the RBSP data. + * + * The RBSP format of the filler data is specified in Rec. ITU-T H.265 + * (02/2018) 7.3.2.8 Filler data RBSP syntax. + * + * Return: number of filler data bytes (including marker) or negative error + */ +ssize_t nal_hevc_read_filler(const struct device *dev, void *src, size_t n) +{ + struct rbsp rbsp; + unsigned int forbidden_zero_bit; + unsigned int nal_unit_type; + unsigned int nuh_layer_id; + unsigned int nuh_temporal_id_plus1; + + if (!src) + return -EINVAL; + + rbsp_init(&rbsp, src, n, &read); + + nal_hevc_read_start_code_prefix(&rbsp); + + rbsp_bit(&rbsp, &forbidden_zero_bit); + rbsp_bits(&rbsp, 6, &nal_unit_type); + rbsp_bits(&rbsp, 6, &nuh_layer_id); + rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1); + + if (rbsp.error) + return rbsp.error; + if (forbidden_zero_bit != 0 || + nal_unit_type != FD_NUT) + return -EINVAL; + + nal_hevc_read_filler_data(&rbsp); + rbsp_trailing_bits(&rbsp); + + if (rbsp.error) + return rbsp.error; + + return DIV_ROUND_UP(rbsp.pos, 8); +} +EXPORT_SYMBOL_GPL(nal_hevc_read_filler); diff --git a/drivers/media/platform/allegro-dvt/nal-hevc.h b/drivers/media/platform/allegro-dvt/nal-hevc.h new file mode 100644 index 000000000000..fc994d4242d8 --- /dev/null +++ b/drivers/media/platform/allegro-dvt/nal-hevc.h @@ -0,0 +1,350 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2019 Pengutronix, Michael Tretter + * + * Convert NAL units between raw byte sequence payloads (RBSP) and C structs. + */ + +#ifndef __NAL_HEVC_H__ +#define __NAL_HEVC_H__ + +#include +#include +#include + +struct nal_hevc_profile_tier_level { + unsigned int general_profile_space; + unsigned int general_tier_flag; + unsigned int general_profile_idc; + unsigned int general_profile_compatibility_flag[32]; + unsigned int general_progressive_source_flag; + unsigned int general_interlaced_source_flag; + unsigned int general_non_packed_constraint_flag; + unsigned int general_frame_only_constraint_flag; + union { + struct { + unsigned int general_max_12bit_constraint_flag; + unsigned int general_max_10bit_constraint_flag; + unsigned int general_max_8bit_constraint_flag; + unsigned int general_max_422chroma_constraint_flag; + unsigned int general_max_420chroma_constraint_flag; + unsigned int general_max_monochrome_constraint_flag; + unsigned int general_intra_constraint_flag; + unsigned int general_one_picture_only_constraint_flag; + unsigned int general_lower_bit_rate_constraint_flag; + union { + struct { + unsigned int general_max_14bit_constraint_flag; + unsigned int general_reserved_zero_33bits; + }; + unsigned int general_reserved_zero_34bits; + }; + }; + struct { + unsigned int general_reserved_zero_7bits; + /* unsigned int general_one_picture_only_constraint_flag; */ + unsigned int general_reserved_zero_35bits; + }; + unsigned int general_reserved_zero_43bits; + }; + union { + unsigned int general_inbld_flag; + unsigned int general_reserved_zero_bit; + }; + unsigned int general_level_idc; +}; + +/** + * struct nal_hevc_vps - Video parameter set + * + * C struct representation of the video parameter set NAL unit as defined by + * Rec. ITU-T H.265 (02/2018) 7.3.2.1 Video parameter set RBSP syntax + */ +struct nal_hevc_vps { + unsigned int video_parameter_set_id; + unsigned int base_layer_internal_flag; + unsigned int base_layer_available_flag; + unsigned int max_layers_minus1; + unsigned int max_sub_layers_minus1; + unsigned int temporal_id_nesting_flag; + struct nal_hevc_profile_tier_level profile_tier_level; + unsigned int sub_layer_ordering_info_present_flag; + struct { + unsigned int max_dec_pic_buffering_minus1[7]; + unsigned int max_num_reorder_pics[7]; + unsigned int max_latency_increase_plus1[7]; + }; + unsigned int max_layer_id; + unsigned int num_layer_sets_minus1; + unsigned int layer_id_included_flag[1024][64]; + unsigned int timing_info_present_flag; + struct { + unsigned int num_units_in_tick; + unsigned int time_scale; + unsigned int poc_proportional_to_timing_flag; + unsigned int num_ticks_poc_diff_one_minus1; + unsigned int num_hrd_parameters; + struct { + unsigned int hrd_layer_set_idx[0]; + unsigned int cprms_present_flag[0]; + }; + /* hrd_parameters( cprms_present_flag[ i ], max_sub_layers_minus1 ) */ + }; + unsigned int extension_flag; + unsigned int extension_data_flag; +}; + +struct nal_hevc_sub_layer_hrd_parameters { + unsigned int bit_rate_value_minus1[1]; + unsigned int cpb_size_value_minus1[1]; + unsigned int cbr_flag[1]; +}; + +struct nal_hevc_hrd_parameters { + unsigned int nal_hrd_parameters_present_flag; + unsigned int vcl_hrd_parameters_present_flag; + struct { + unsigned int sub_pic_hrd_params_present_flag; + struct { + unsigned int tick_divisor_minus2; + unsigned int du_cpb_removal_delay_increment_length_minus1; + unsigned int sub_pic_cpb_params_in_pic_timing_sei_flag; + unsigned int dpb_output_delay_du_length_minus1; + }; + unsigned int bit_rate_scale; + unsigned int cpb_size_scale; + unsigned int cpb_size_du_scale; + unsigned int initial_cpb_removal_delay_length_minus1; + unsigned int au_cpb_removal_delay_length_minus1; + unsigned int dpb_output_delay_length_minus1; + }; + struct { + unsigned int fixed_pic_rate_general_flag[1]; + unsigned int fixed_pic_rate_within_cvs_flag[1]; + unsigned int elemental_duration_in_tc_minus1[1]; + unsigned int low_delay_hrd_flag[1]; + unsigned int cpb_cnt_minus1[1]; + struct nal_hevc_sub_layer_hrd_parameters nal_hrd[1]; + struct nal_hevc_sub_layer_hrd_parameters vcl_hrd[1]; + }; +}; + +/** + * struct nal_hevc_vui_parameters - VUI parameters + * + * C struct representation of the VUI parameters as defined by Rec. ITU-T + * H.265 (02/2018) E.2.1 VUI parameters syntax. + */ +struct nal_hevc_vui_parameters { + unsigned int aspect_ratio_info_present_flag; + struct { + unsigned int aspect_ratio_idc; + unsigned int sar_width; + unsigned int sar_height; + }; + unsigned int overscan_info_present_flag; + unsigned int overscan_appropriate_flag; + unsigned int video_signal_type_present_flag; + struct { + unsigned int video_format; + unsigned int video_full_range_flag; + unsigned int colour_description_present_flag; + struct { + unsigned int colour_primaries; + unsigned int transfer_characteristics; + unsigned int matrix_coeffs; + }; + }; + unsigned int chroma_loc_info_present_flag; + struct { + unsigned int chroma_sample_loc_type_top_field; + unsigned int chroma_sample_loc_type_bottom_field; + }; + unsigned int neutral_chroma_indication_flag; + unsigned int field_seq_flag; + unsigned int frame_field_info_present_flag; + unsigned int default_display_window_flag; + struct { + unsigned int def_disp_win_left_offset; + unsigned int def_disp_win_right_offset; + unsigned int def_disp_win_top_offset; + unsigned int def_disp_win_bottom_offset; + }; + unsigned int vui_timing_info_present_flag; + struct { + unsigned int vui_num_units_in_tick; + unsigned int vui_time_scale; + unsigned int vui_poc_proportional_to_timing_flag; + unsigned int vui_num_ticks_poc_diff_one_minus1; + unsigned int vui_hrd_parameters_present_flag; + struct nal_hevc_hrd_parameters nal_hrd_parameters; + }; + unsigned int bitstream_restriction_flag; + struct { + unsigned int tiles_fixed_structure_flag; + unsigned int motion_vectors_over_pic_boundaries_flag; + unsigned int restricted_ref_pic_lists_flag; + unsigned int min_spatial_segmentation_idc; + unsigned int max_bytes_per_pic_denom; + unsigned int max_bits_per_min_cu_denom; + unsigned int log2_max_mv_length_horizontal; + unsigned int log2_max_mv_length_vertical; + }; +}; + +/** + * struct nal_hevc_sps - Sequence parameter set + * + * C struct representation of the video parameter set NAL unit as defined by + * Rec. ITU-T H.265 (02/2018) 7.3.2.2 Sequence parameter set RBSP syntax + */ +struct nal_hevc_sps { + unsigned int video_parameter_set_id; + unsigned int max_sub_layers_minus1; + unsigned int temporal_id_nesting_flag; + struct nal_hevc_profile_tier_level profile_tier_level; + unsigned int seq_parameter_set_id; + unsigned int chroma_format_idc; + unsigned int separate_colour_plane_flag; + unsigned int pic_width_in_luma_samples; + unsigned int pic_height_in_luma_samples; + unsigned int conformance_window_flag; + struct { + unsigned int conf_win_left_offset; + unsigned int conf_win_right_offset; + unsigned int conf_win_top_offset; + unsigned int conf_win_bottom_offset; + }; + + unsigned int bit_depth_luma_minus8; + unsigned int bit_depth_chroma_minus8; + unsigned int log2_max_pic_order_cnt_lsb_minus4; + unsigned int sub_layer_ordering_info_present_flag; + struct { + unsigned int max_dec_pic_buffering_minus1[7]; + unsigned int max_num_reorder_pics[7]; + unsigned int max_latency_increase_plus1[7]; + }; + unsigned int log2_min_luma_coding_block_size_minus3; + unsigned int log2_diff_max_min_luma_coding_block_size; + unsigned int log2_min_luma_transform_block_size_minus2; + unsigned int log2_diff_max_min_luma_transform_block_size; + unsigned int max_transform_hierarchy_depth_inter; + unsigned int max_transform_hierarchy_depth_intra; + + unsigned int scaling_list_enabled_flag; + unsigned int scaling_list_data_present_flag; + unsigned int amp_enabled_flag; + unsigned int sample_adaptive_offset_enabled_flag; + unsigned int pcm_enabled_flag; + struct { + unsigned int pcm_sample_bit_depth_luma_minus1; + unsigned int pcm_sample_bit_depth_chroma_minus1; + unsigned int log2_min_pcm_luma_coding_block_size_minus3; + unsigned int log2_diff_max_min_pcm_luma_coding_block_size; + unsigned int pcm_loop_filter_disabled_flag; + }; + + unsigned int num_short_term_ref_pic_sets; + unsigned int long_term_ref_pics_present_flag; + unsigned int sps_temporal_mvp_enabled_flag; + unsigned int strong_intra_smoothing_enabled_flag; + unsigned int vui_parameters_present_flag; + struct nal_hevc_vui_parameters vui; + unsigned int extension_present_flag; + struct { + unsigned int sps_range_extension_flag; + unsigned int sps_multilayer_extension_flag; + unsigned int sps_3d_extension_flag; + unsigned int sps_scc_extension_flag; + unsigned int sps_extension_4bits; + }; +}; + +struct nal_hevc_pps { + unsigned int pps_pic_parameter_set_id; + unsigned int pps_seq_parameter_set_id; + unsigned int dependent_slice_segments_enabled_flag; + unsigned int output_flag_present_flag; + unsigned int num_extra_slice_header_bits; + unsigned int sign_data_hiding_enabled_flag; + unsigned int cabac_init_present_flag; + unsigned int num_ref_idx_l0_default_active_minus1; + unsigned int num_ref_idx_l1_default_active_minus1; + int init_qp_minus26; + unsigned int constrained_intra_pred_flag; + unsigned int transform_skip_enabled_flag; + unsigned int cu_qp_delta_enabled_flag; + unsigned int diff_cu_qp_delta_depth; + int pps_cb_qp_offset; + int pps_cr_qp_offset; + unsigned int pps_slice_chroma_qp_offsets_present_flag; + unsigned int weighted_pred_flag; + unsigned int weighted_bipred_flag; + unsigned int transquant_bypass_enabled_flag; + unsigned int tiles_enabled_flag; + unsigned int entropy_coding_sync_enabled_flag; + struct { + unsigned int num_tile_columns_minus1; + unsigned int num_tile_rows_minus1; + unsigned int uniform_spacing_flag; + struct { + unsigned int column_width_minus1[1]; + unsigned int row_height_minus1[1]; + }; + unsigned int loop_filter_across_tiles_enabled_flag; + }; + unsigned int pps_loop_filter_across_slices_enabled_flag; + unsigned int deblocking_filter_control_present_flag; + struct { + unsigned int deblocking_filter_override_enabled_flag; + unsigned int pps_deblocking_filter_disabled_flag; + struct { + int pps_beta_offset_div2; + int pps_tc_offset_div2; + }; + }; + unsigned int pps_scaling_list_data_present_flag; + unsigned int lists_modification_present_flag; + unsigned int log2_parallel_merge_level_minus2; + unsigned int slice_segment_header_extension_present_flag; + unsigned int pps_extension_present_flag; + struct { + unsigned int pps_range_extension_flag; + unsigned int pps_multilayer_extension_flag; + unsigned int pps_3d_extension_flag; + unsigned int pps_scc_extension_flag; + unsigned int pps_extension_4bits; + }; +}; + +int nal_hevc_profile_from_v4l2(enum v4l2_mpeg_video_hevc_profile profile); +int nal_hevc_tier_from_v4l2(enum v4l2_mpeg_video_hevc_tier tier); +int nal_hevc_level_from_v4l2(enum v4l2_mpeg_video_hevc_level level); + +int nal_range_from_v4l2(enum v4l2_quantization quantization); +int nal_color_primaries_from_v4l2(enum v4l2_colorspace colorspace); +int nal_transfer_characteristics_from_v4l2(enum v4l2_colorspace colorspace, + enum v4l2_xfer_func xfer_func); +int nal_matrix_coeffs_from_v4l2(enum v4l2_colorspace colorspace, + enum v4l2_ycbcr_encoding ycbcr_encoding); + +ssize_t nal_hevc_write_vps(const struct device *dev, + void *dest, size_t n, struct nal_hevc_vps *vps); +ssize_t nal_hevc_read_vps(const struct device *dev, + struct nal_hevc_vps *vps, void *src, size_t n); + +ssize_t nal_hevc_write_sps(const struct device *dev, + void *dest, size_t n, struct nal_hevc_sps *sps); +ssize_t nal_hevc_read_sps(const struct device *dev, + struct nal_hevc_sps *sps, void *src, size_t n); + +ssize_t nal_hevc_write_pps(const struct device *dev, + void *dest, size_t n, struct nal_hevc_pps *pps); +ssize_t nal_hevc_read_pps(const struct device *dev, + struct nal_hevc_pps *pps, void *src, size_t n); + +ssize_t nal_hevc_write_filler(const struct device *dev, void *dest, size_t n); +ssize_t nal_hevc_read_filler(const struct device *dev, void *src, size_t n); + +#endif /* __NAL_HEVC_H__ */ From patchwork Thu Dec 3 11:00:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AD1AC83016 for ; Thu, 3 Dec 2020 11:04:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37E5922241 for ; Thu, 3 Dec 2020 11:04:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388840AbgLCLEO (ORCPT ); Thu, 3 Dec 2020 06:04:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387738AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAE9EC08E861 for ; Thu, 3 Dec 2020 03:02:54 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Kq-50; Thu, 03 Dec 2020 12:02:53 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhe-RW; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:52 +0100 Message-Id: <20201203110106.2939463-5-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 04/18] media: allegro: implement S_FMT for CAPTURE X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In order to support different codecs, the driver must support changing the format on CAPTURE. Therefore, the driver needs to handle S_FMT on CAPTURE. As the driver will have a different number of formats for OUTPUT and CAPTURE, split the check for the format index in ENUM_FMT into CAPTURE and OUTPUT. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 23 ++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 6b5cbee05040..a733049c8727 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -2503,13 +2503,15 @@ static int allegro_querycap(struct file *file, void *fh, static int allegro_enum_fmt_vid(struct file *file, void *fh, struct v4l2_fmtdesc *f) { - if (f->index) - return -EINVAL; switch (f->type) { case V4L2_BUF_TYPE_VIDEO_OUTPUT: + if (f->index >= 1) + return -EINVAL; f->pixelformat = V4L2_PIX_FMT_NV12; break; case V4L2_BUF_TYPE_VIDEO_CAPTURE: + if (f->index >= 1) + return -EINVAL; f->pixelformat = V4L2_PIX_FMT_H264; break; default: @@ -2557,6 +2559,21 @@ static int allegro_try_fmt_vid_cap(struct file *file, void *fh, return 0; } +static int allegro_s_fmt_vid_cap(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct allegro_channel *channel = fh_to_channel(fh); + int err; + + err = allegro_try_fmt_vid_cap(file, fh, f); + if (err) + return err; + + channel->codec = f->fmt.pix.pixelformat; + + return 0; +} + static int allegro_g_fmt_vid_out(struct file *file, void *fh, struct v4l2_format *f) { @@ -2769,7 +2786,7 @@ static const struct v4l2_ioctl_ops allegro_ioctl_ops = { .vidioc_enum_fmt_vid_out = allegro_enum_fmt_vid, .vidioc_g_fmt_vid_cap = allegro_g_fmt_vid_cap, .vidioc_try_fmt_vid_cap = allegro_try_fmt_vid_cap, - .vidioc_s_fmt_vid_cap = allegro_try_fmt_vid_cap, + .vidioc_s_fmt_vid_cap = allegro_s_fmt_vid_cap, .vidioc_g_fmt_vid_out = allegro_g_fmt_vid_out, .vidioc_try_fmt_vid_out = allegro_try_fmt_vid_out, .vidioc_s_fmt_vid_out = allegro_s_fmt_vid_out, From patchwork Thu Dec 3 11:00:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 410F4C63777 for ; Thu, 3 Dec 2020 11:04:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF69B208FE for ; Thu, 3 Dec 2020 11:04:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388760AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388316AbgLCLEM (ORCPT ); Thu, 3 Dec 2020 06:04:12 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BEF0FC08C5F2 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNs-0002Kr-W4; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhh-S7; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:53 +0100 Message-Id: <20201203110106.2939463-6-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 05/18] media: allegro: adjust channel after format change X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org A format change (i.e. the frame size or the codec) has multiple effects on a channel: - The available controls - The limits of the available controls - The default encoding options To avoid scattering the changes all over the driver, add a new function that goes over the channel and does all required adjustments. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 69 ++++++++++++++----- 1 file changed, 52 insertions(+), 17 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index a733049c8727..3253262d56f9 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -1963,7 +1963,6 @@ static int allegro_create_channel(struct allegro_channel *channel) { struct allegro_dev *dev = channel->dev; unsigned long timeout; - enum v4l2_mpeg_video_h264_level min_level; if (channel_exists(channel)) { v4l2_warn(&dev->v4l2_dev, @@ -1986,16 +1985,6 @@ static int allegro_create_channel(struct allegro_channel *channel) DIV_ROUND_UP(channel->framerate.numerator, channel->framerate.denominator)); - min_level = select_minimum_h264_level(channel->width, channel->height); - if (channel->level < min_level) { - v4l2_warn(&dev->v4l2_dev, - "user %d: selected Level %s too low: increasing to Level %s\n", - channel->user_id, - v4l2_ctrl_get_menu(V4L2_CID_MPEG_VIDEO_H264_LEVEL)[channel->level], - v4l2_ctrl_get_menu(V4L2_CID_MPEG_VIDEO_H264_LEVEL)[min_level]); - channel->level = min_level; - } - v4l2_ctrl_grab(channel->mpeg_video_h264_profile, true); v4l2_ctrl_grab(channel->mpeg_video_h264_level, true); v4l2_ctrl_grab(channel->mpeg_video_h264_i_frame_qp, true); @@ -2031,6 +2020,53 @@ static int allegro_create_channel(struct allegro_channel *channel) return channel->error; } +/** + * allegro_channel_adjust() - Adjust channel parameters to current format + * @channel: the channel to adjust + * + * Various parameters of a channel and their limits depend on the currently + * set format. Adjust the parameters after a format change in one go. + */ +static void allegro_channel_adjust(struct allegro_channel *channel) +{ + struct allegro_dev *dev = channel->dev; + struct v4l2_ctrl *ctrl; + s64 min; + s64 max; + + channel->sizeimage_encoded = + estimate_stream_size(channel->width, channel->height); + + ctrl = channel->mpeg_video_h264_level; + min = select_minimum_h264_level(channel->width, channel->height); + if (ctrl->minimum > min) + v4l2_dbg(1, debug, &dev->v4l2_dev, + "%s.minimum: %lld -> %lld\n", + v4l2_ctrl_get_name(ctrl->id), ctrl->minimum, min); + v4l2_ctrl_lock(ctrl); + __v4l2_ctrl_modify_range(ctrl, min, ctrl->maximum, + ctrl->step, ctrl->default_value); + v4l2_ctrl_unlock(ctrl); + channel->level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level); + + ctrl = channel->mpeg_video_bitrate; + max = maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level)); + if (ctrl->maximum < max) + v4l2_dbg(1, debug, &dev->v4l2_dev, + "%s: maximum: %lld -> %lld\n", + v4l2_ctrl_get_name(ctrl->id), ctrl->maximum, max); + v4l2_ctrl_lock(ctrl); + __v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max, + ctrl->step, ctrl->default_value); + v4l2_ctrl_unlock(ctrl); + + ctrl = channel->mpeg_video_bitrate_peak; + v4l2_ctrl_lock(ctrl); + __v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max, + ctrl->step, ctrl->default_value); + v4l2_ctrl_unlock(ctrl); +} + static void allegro_set_default_params(struct allegro_channel *channel) { channel->width = ALLEGRO_WIDTH_DEFAULT; @@ -2050,8 +2086,6 @@ static void allegro_set_default_params(struct allegro_channel *channel) channel->profile = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE; channel->level = select_minimum_h264_level(channel->width, channel->height); - channel->sizeimage_encoded = - estimate_stream_size(channel->width, channel->height); channel->bitrate = maximum_bitrate(channel->level); channel->bitrate_peak = maximum_bitrate(channel->level); @@ -2460,6 +2494,8 @@ static int allegro_open(struct file *file) file->private_data = &channel->fh; v4l2_fh_add(&channel->fh); + allegro_channel_adjust(channel); + return 0; error: @@ -2571,6 +2607,8 @@ static int allegro_s_fmt_vid_cap(struct file *file, void *fh, channel->codec = f->fmt.pix.pixelformat; + allegro_channel_adjust(channel); + return 0; } @@ -2641,10 +2679,7 @@ static int allegro_s_fmt_vid_out(struct file *file, void *fh, channel->quantization = f->fmt.pix.quantization; channel->xfer_func = f->fmt.pix.xfer_func; - channel->level = - select_minimum_h264_level(channel->width, channel->height); - channel->sizeimage_encoded = - estimate_stream_size(channel->width, channel->height); + allegro_channel_adjust(channel); return 0; } From patchwork Thu Dec 3 11:00:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1F0AC71156 for ; Thu, 3 Dec 2020 11:03:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 67D2821D91 for ; Thu, 3 Dec 2020 11:03:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730257AbgLCLDe (ORCPT ); Thu, 3 Dec 2020 06:03:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729085AbgLCLDd (ORCPT ); Thu, 3 Dec 2020 06:03:33 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 601B8C061A52 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNs-0002Ks-Tl; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhk-Si; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:54 +0100 Message-Id: <20201203110106.2939463-7-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 06/18] media: allegro: move encoding options to channel X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org There are several encoding options that are hard coded in the parameter that is used to configure the hardware codec. However, some of the options must be written to the SPS/PPS by the driver. Furthermore, some of the options depend on the codec that is used (i.e. H.264 or HEVC). Therefore, move options that depend on the codec to the channel and add constants for options that are independent of the codec but must be written to the SPS/PPS. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 85 ++++++++++++++----- 1 file changed, 65 insertions(+), 20 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 3253262d56f9..a2a9000bbb15 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -94,6 +94,12 @@ #define SIZE_MACROBLOCK 16 +/* Encoding options */ +#define LOG2_MAX_FRAME_NUM 4 +#define LOG2_MAX_PIC_ORDER_CNT 10 +#define BETA_OFFSET_DIV_2 -1 +#define TC_OFFSET_DIV_2 -1 + static int debug; module_param(debug, int, 0644); MODULE_PARM_DESC(debug, "Debug level (0-2)"); @@ -202,9 +208,30 @@ struct allegro_channel { struct allegro_buffer config_blob; + unsigned int log2_max_frame_num; + bool temporal_mvp_enable; + + bool enable_loop_filter_across_tiles; + bool enable_loop_filter_across_slices; + bool dbf_ovr_en; + unsigned int num_ref_idx_l0; unsigned int num_ref_idx_l1; + /* Maximum range for motion estimation */ + int b_hrz_me_range; + int b_vrt_me_range; + int p_hrz_me_range; + int p_vrt_me_range; + /* Size limits of coding unit */ + int min_cu_size; + int max_cu_size; + /* Size limits of transform unit */ + int min_tu_size; + int max_tu_size; + int max_transfo_depth_intra; + int max_transfo_depth_inter; + struct v4l2_ctrl *mpeg_video_h264_profile; struct v4l2_ctrl *mpeg_video_h264_level; struct v4l2_ctrl *mpeg_video_h264_i_frame_qp; @@ -911,32 +938,32 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->level = v4l2_level_to_mcu_level(channel->level); param->tier = 0; - param->log2_max_poc = 10; - param->log2_max_frame_num = 4; - param->temporal_mvp_enable = 1; + param->log2_max_poc = LOG2_MAX_PIC_ORDER_CNT; + param->log2_max_frame_num = channel->log2_max_frame_num; + param->temporal_mvp_enable = channel->temporal_mvp_enable; - param->dbf_ovr_en = 1; + param->dbf_ovr_en = channel->dbf_ovr_en; param->rdo_cost_mode = 1; param->custom_lda = 1; param->lf = 1; - param->lf_x_tile = 1; - param->lf_x_slice = 1; + param->lf_x_tile = channel->enable_loop_filter_across_tiles; + param->lf_x_slice = channel->enable_loop_filter_across_slices; param->src_bit_depth = 8; - param->beta_offset = -1; - param->tc_offset = -1; + param->beta_offset = BETA_OFFSET_DIV_2; + param->tc_offset = TC_OFFSET_DIV_2; param->num_slices = 1; - param->me_range[0] = 8; - param->me_range[1] = 8; - param->me_range[2] = 16; - param->me_range[3] = 16; - param->max_cu_size = ilog2(SIZE_MACROBLOCK); - param->min_cu_size = ilog2(8); - param->max_tu_size = 2; - param->min_tu_size = 2; - param->max_transfo_depth_intra = 1; - param->max_transfo_depth_inter = 1; + param->me_range[0] = channel->b_hrz_me_range; + param->me_range[1] = channel->b_vrt_me_range; + param->me_range[2] = channel->p_hrz_me_range; + param->me_range[3] = channel->p_vrt_me_range; + param->max_cu_size = channel->max_cu_size; + param->min_cu_size = channel->min_cu_size; + param->max_tu_size = channel->max_tu_size; + param->min_tu_size = channel->min_tu_size; + param->max_transfo_depth_intra = channel->max_transfo_depth_intra; + param->max_transfo_depth_inter = channel->max_transfo_depth_inter; param->prefetch_auto = 0; param->prefetch_mem_offset = 0; @@ -1266,9 +1293,9 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel, sps->constraint_set5_flag = 0; sps->level_idc = nal_h264_level_from_v4l2(channel->level); sps->seq_parameter_set_id = 0; - sps->log2_max_frame_num_minus4 = 0; + sps->log2_max_frame_num_minus4 = LOG2_MAX_FRAME_NUM - 4; sps->pic_order_cnt_type = 0; - sps->log2_max_pic_order_cnt_lsb_minus4 = 6; + sps->log2_max_pic_order_cnt_lsb_minus4 = LOG2_MAX_PIC_ORDER_CNT - 4; sps->max_num_ref_frames = 3; sps->gaps_in_frame_num_value_allowed_flag = 0; sps->pic_width_in_mbs_minus1 = @@ -2065,6 +2092,24 @@ static void allegro_channel_adjust(struct allegro_channel *channel) __v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max, ctrl->step, ctrl->default_value); v4l2_ctrl_unlock(ctrl); + + channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM; + channel->temporal_mvp_enable = true; + + channel->dbf_ovr_en = true; + channel->enable_loop_filter_across_tiles = true; + channel->enable_loop_filter_across_slices = true; + + channel->b_hrz_me_range = 8; + channel->b_vrt_me_range = 8; + channel->p_hrz_me_range = 16; + channel->p_vrt_me_range = 16; + channel->max_cu_size = ilog2(16); + channel->min_cu_size = ilog2(8); + channel->max_tu_size = ilog2(4); + channel->min_tu_size = ilog2(4); + channel->max_transfo_depth_intra = 1; + channel->max_transfo_depth_inter = 1; } static void allegro_set_default_params(struct allegro_channel *channel) From patchwork Thu Dec 3 11:00:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40870C83017 for ; Thu, 3 Dec 2020 11:04:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA19222242 for ; Thu, 3 Dec 2020 11:04:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388848AbgLCLEQ (ORCPT ); Thu, 3 Dec 2020 06:04:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388791AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4C76C08E85F for ; Thu, 3 Dec 2020 03:02:54 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Kt-5I; Thu, 03 Dec 2020 12:02:52 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhn-TF; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:55 +0100 Message-Id: <20201203110106.2939463-8-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 07/18] media: allegro: fix log2_max_poc in firmware 2019.1 X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The log2_max_poc field is used to set log2_max_pic_order_cnt_lsb_minus4 for the coded stream. It has an allowed range of 0 to 12. param contains the value without the minus4, but since firmware version 2019.1, the value has to be log2_max_pic_order_cnt_lsb - 1, presumably to fit the maximum value of 16 into a 4 bit field. The driver does not support firmware version 2019.1. Thus, change the behaviour starting from firmware version 2019.2. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-mail.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.c b/drivers/media/platform/allegro-dvt/allegro-mail.c index 993e16f06305..5dbc1c029020 100644 --- a/drivers/media/platform/allegro-dvt/allegro-mail.c +++ b/drivers/media/platform/allegro-dvt/allegro-mail.c @@ -109,8 +109,11 @@ allegro_encode_config_blob(u32 *dst, struct create_channel_param *param) val = 0; val |= param->temporal_mvp_enable ? BIT(20) : 0; - val |= FIELD_PREP(GENMASK(7, 4), param->log2_max_frame_num) | - FIELD_PREP(GENMASK(3, 0), param->log2_max_poc); + val |= FIELD_PREP(GENMASK(7, 4), param->log2_max_frame_num); + if (version >= MCU_MSG_VERSION_2019_2) + val |= FIELD_PREP(GENMASK(3, 0), param->log2_max_poc - 1); + else + val |= FIELD_PREP(GENMASK(3, 0), param->log2_max_poc); dst[i++] = val; val = 0; From patchwork Thu Dec 3 11:00:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE00AC71156 for ; Thu, 3 Dec 2020 11:04:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7002621D91 for ; Thu, 3 Dec 2020 11:04:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388745AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388501AbgLCLEM (ORCPT ); Thu, 3 Dec 2020 06:04:12 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A36EAC08E85E for ; Thu, 3 Dec 2020 03:02:54 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Ku-61; Thu, 03 Dec 2020 12:02:52 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhq-Tt; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:56 +0100 Message-Id: <20201203110106.2939463-9-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 08/18] media: allegro: use handler_setup to configure channel X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org v4l2_ctrl_handler_setup() calls s_ctrl for all controls of the handler. This ensures that the channel is initialized using the default values of the v4l2-ctrls. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index a2a9000bbb15..859a2c81c782 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -2520,6 +2520,8 @@ static int allegro_open(struct file *file) v4l2_ctrl_cluster(3, &channel->mpeg_video_bitrate_mode); + v4l2_ctrl_handler_setup(handler); + channel->mcu_channel_id = -1; channel->user_id = -1; From patchwork Thu Dec 3 11:00:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3F0DC64E8A for ; Thu, 3 Dec 2020 11:03:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8500921D91 for ; Thu, 3 Dec 2020 11:03:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730272AbgLCLD7 (ORCPT ); Thu, 3 Dec 2020 06:03:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729085AbgLCLD6 (ORCPT ); Thu, 3 Dec 2020 06:03:58 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F1EDC061A54 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Kv-1w; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKht-Ub; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:57 +0100 Message-Id: <20201203110106.2939463-10-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 09/18] media: allegro: initialize bitrate using v4l2_ctrl X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As the driver now uses the v4l2-ctrls to setup the channel, there is no need to explicitly set the bitrate. The initial bitrate is now set via the same path as if it is set from userspace using the v4l2-ctrl. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-core.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 859a2c81c782..db504c4a81f6 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -2132,8 +2132,6 @@ static void allegro_set_default_params(struct allegro_channel *channel) channel->level = select_minimum_h264_level(channel->width, channel->height); - channel->bitrate = maximum_bitrate(channel->level); - channel->bitrate_peak = maximum_bitrate(channel->level); channel->cpb_size = maximum_cpb_size(channel->level); channel->gop_size = ALLEGRO_GOP_SIZE_DEFAULT; } @@ -2421,6 +2419,8 @@ static int allegro_open(struct file *file) struct v4l2_ctrl_handler *handler; u64 mask; int ret; + unsigned int bitrate_max; + unsigned int bitrate_def; channel = kzalloc(sizeof(*channel), GFP_KERNEL); if (!channel) @@ -2486,16 +2486,17 @@ static int allegro_open(struct file *file) V4L2_CID_MPEG_VIDEO_BITRATE_MODE, V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0, V4L2_MPEG_VIDEO_BITRATE_MODE_CBR); + + bitrate_max = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + bitrate_def = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE, - 0, maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1), - 1, channel->bitrate); + 0, bitrate_max, 1, bitrate_def); channel->mpeg_video_bitrate_peak = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE_PEAK, - 0, maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1), - 1, channel->bitrate_peak); + 0, bitrate_max, 1, bitrate_def); channel->mpeg_video_cpb_size = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE, From patchwork Thu Dec 3 11:00:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80487C83019 for ; Thu, 3 Dec 2020 11:04:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3964321D91 for ; Thu, 3 Dec 2020 11:04:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388850AbgLCLEQ (ORCPT ); Thu, 3 Dec 2020 06:04:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388801AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC4F1C08E860 for ; Thu, 3 Dec 2020 03:02:54 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Kw-5H; Thu, 03 Dec 2020 12:02:52 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhw-V8; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:58 +0100 Message-Id: <20201203110106.2939463-11-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 10/18] media: allegro: implement scaling of cpb size in SPS X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The cbp_size_value_minus1 in the SPS can scaled using cpb_size_scale to reduce the number of bits necessary to encode the value. For simplicity, the scaling was set to 1. Restructure to the code to make it easier to drop the cbp_size from the channel and as we are at it, also properly implement the scaling. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-core.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index db504c4a81f6..4e5590e8f875 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -1279,6 +1279,8 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel, /* Calculation of crop units in Rec. ITU-T H.264 (04/2017) p. 76 */ unsigned int crop_unit_x = 2; unsigned int crop_unit_y = 2; + unsigned int cpb_size; + unsigned int cpb_size_scale; sps = kzalloc(sizeof(*sps), GFP_KERNEL); if (!sps) @@ -1336,13 +1338,15 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel, sps->vui.vcl_hrd_parameters_present_flag = 1; sps->vui.vcl_hrd_parameters.cpb_cnt_minus1 = 0; sps->vui.vcl_hrd_parameters.bit_rate_scale = 0; - sps->vui.vcl_hrd_parameters.cpb_size_scale = 1; /* See Rec. ITU-T H.264 (04/2017) p. 410 E-53 */ sps->vui.vcl_hrd_parameters.bit_rate_value_minus1[0] = channel->bitrate_peak / (1 << (6 + sps->vui.vcl_hrd_parameters.bit_rate_scale)) - 1; /* See Rec. ITU-T H.264 (04/2017) p. 410 E-54 */ + cpb_size = channel->cpb_size; + cpb_size_scale = ffs(cpb_size) - 4; + sps->vui.vcl_hrd_parameters.cpb_size_scale = cpb_size_scale; sps->vui.vcl_hrd_parameters.cpb_size_value_minus1[0] = - (channel->cpb_size * 1000) / (1 << (4 + sps->vui.vcl_hrd_parameters.cpb_size_scale)) - 1; + (cpb_size * 1000) / (1 << (4 + cpb_size_scale)) - 1; sps->vui.vcl_hrd_parameters.cbr_flag[0] = !v4l2_ctrl_g_ctrl(channel->mpeg_video_frame_rc_enable); sps->vui.vcl_hrd_parameters.initial_cpb_removal_delay_length_minus1 = 31; From patchwork Thu Dec 3 11:00:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2996EC71156 for ; Thu, 3 Dec 2020 11:04:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C732822201 for ; Thu, 3 Dec 2020 11:04:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388847AbgLCLEP (ORCPT ); Thu, 3 Dec 2020 06:04:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388810AbgLCLEO (ORCPT ); Thu, 3 Dec 2020 06:04:14 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77AF1C08E864 for ; Thu, 3 Dec 2020 03:02:55 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Kx-72; Thu, 03 Dec 2020 12:02:53 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNr-00CKhz-Vh; Thu, 03 Dec 2020 12:02:47 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:00:59 +0100 Message-Id: <20201203110106.2939463-12-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 11/18] media: allegro: remove cpb_size and gop_size from channel X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The cpb_size and the gop_size are straight copies of the values in the v4l2-ctrls. To avoid this duplication, directly get the values from the v4l2-ctrls and remove the copies. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 30 +++++++------------ 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 4e5590e8f875..8cabc6ba9f7e 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -203,8 +203,6 @@ struct allegro_channel { bool frame_rc_enable; unsigned int bitrate; unsigned int bitrate_peak; - unsigned int cpb_size; - unsigned int gop_size; struct allegro_buffer config_blob; @@ -925,6 +923,7 @@ static int fill_create_channel_param(struct allegro_channel *channel, int p_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp); int b_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp); int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode); + unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size); param->width = channel->width; param->height = channel->height; @@ -972,8 +971,7 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->rate_control_mode = channel->frame_rc_enable ? v4l2_bitrate_mode_to_mcu_mode(bitrate_mode) : 0; - param->cpb_size = v4l2_cpb_size_to_mcu(channel->cpb_size, - channel->bitrate_peak); + param->cpb_size = v4l2_cpb_size_to_mcu(cpb_size, channel->bitrate_peak); /* Shall be ]0;cpb_size in 90 kHz units]. Use maximum value. */ param->initial_rem_delay = param->cpb_size; param->framerate = DIV_ROUND_UP(channel->framerate.numerator, @@ -996,10 +994,10 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->max_pixel_value = 255; param->gop_ctrl_mode = 0x00000002; - param->freq_idr = channel->gop_size; + param->freq_idr = v4l2_ctrl_g_ctrl(channel->mpeg_video_gop_size); param->freq_lt = 0; param->gdr_mode = 0x00000000; - param->gop_length = channel->gop_size; + param->gop_length = v4l2_ctrl_g_ctrl(channel->mpeg_video_gop_size); param->subframe_latency = 0x00000000; param->lda_factors[0] = 51; @@ -1342,7 +1340,7 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel, sps->vui.vcl_hrd_parameters.bit_rate_value_minus1[0] = channel->bitrate_peak / (1 << (6 + sps->vui.vcl_hrd_parameters.bit_rate_scale)) - 1; /* See Rec. ITU-T H.264 (04/2017) p. 410 E-54 */ - cpb_size = channel->cpb_size; + cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size); cpb_size_scale = ffs(cpb_size) - 4; sps->vui.vcl_hrd_parameters.cpb_size_scale = cpb_size_scale; sps->vui.vcl_hrd_parameters.cpb_size_value_minus1[0] = @@ -2135,9 +2133,6 @@ static void allegro_set_default_params(struct allegro_channel *channel) channel->profile = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE; channel->level = select_minimum_h264_level(channel->width, channel->height); - - channel->cpb_size = maximum_cpb_size(channel->level); - channel->gop_size = ALLEGRO_GOP_SIZE_DEFAULT; } static int allegro_queue_setup(struct vb2_queue *vq, @@ -2394,12 +2389,6 @@ static int allegro_s_ctrl(struct v4l2_ctrl *ctrl) v4l2_ctrl_activate(channel->mpeg_video_bitrate_peak, ctrl->val == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR); break; - case V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE: - channel->cpb_size = ctrl->val; - break; - case V4L2_CID_MPEG_VIDEO_GOP_SIZE: - channel->gop_size = ctrl->val; - break; case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP: case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP: case V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP: @@ -2425,6 +2414,8 @@ static int allegro_open(struct file *file) int ret; unsigned int bitrate_max; unsigned int bitrate_def; + unsigned int cpb_size_max; + unsigned int cpb_size_def; channel = kzalloc(sizeof(*channel), GFP_KERNEL); if (!channel) @@ -2493,6 +2484,8 @@ static int allegro_open(struct file *file) bitrate_max = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); bitrate_def = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + cpb_size_max = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + cpb_size_def = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE, @@ -2504,13 +2497,12 @@ static int allegro_open(struct file *file) channel->mpeg_video_cpb_size = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE, - 0, maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1), - 1, channel->cpb_size); + 0, cpb_size_max, 1, cpb_size_def); channel->mpeg_video_gop_size = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_GOP_SIZE, 0, ALLEGRO_GOP_SIZE_MAX, - 1, channel->gop_size); + 1, ALLEGRO_GOP_SIZE_DEFAULT); v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, From patchwork Thu Dec 3 11:01:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72DF4C83012 for ; Thu, 3 Dec 2020 11:04:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E7F221D91 for ; Thu, 3 Dec 2020 11:04:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388781AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387772AbgLCLEM (ORCPT ); Thu, 3 Dec 2020 06:04:12 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AE09C061A56 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Ky-1s; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKi2-00; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:00 +0100 Message-Id: <20201203110106.2939463-13-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 12/18] media: allegro: remove profile and level from channel X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The profile and level are straight copies from the v4l2-ctrls. Avoid duplication and directly read the value of the v4l2-ctrl. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 8cabc6ba9f7e..c0519f7a67b8 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -195,8 +195,6 @@ struct allegro_channel { unsigned int osequence; u32 codec; - enum v4l2_mpeg_video_h264_profile profile; - enum v4l2_mpeg_video_h264_level level; unsigned int sizeimage_encoded; unsigned int csequence; @@ -924,6 +922,8 @@ static int fill_create_channel_param(struct allegro_channel *channel, int b_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp); int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode); unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size); + enum v4l2_mpeg_video_h264_profile profile; + enum v4l2_mpeg_video_h264_level level; param->width = channel->width; param->height = channel->height; @@ -931,11 +931,13 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->colorspace = v4l2_colorspace_to_mcu_colorspace(channel->colorspace); param->src_mode = 0x0; - param->profile = v4l2_profile_to_mcu_profile(channel->profile); - param->constraint_set_flags = BIT(1); + param->codec = channel->codec; - param->level = v4l2_level_to_mcu_level(channel->level); - param->tier = 0; + profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile); + level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level); + param->profile = v4l2_profile_to_mcu_profile(profile); + param->constraint_set_flags = BIT(1); + param->level = v4l2_level_to_mcu_level(level); param->log2_max_poc = LOG2_MAX_PIC_ORDER_CNT; param->log2_max_frame_num = channel->log2_max_frame_num; @@ -1277,6 +1279,8 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel, /* Calculation of crop units in Rec. ITU-T H.264 (04/2017) p. 76 */ unsigned int crop_unit_x = 2; unsigned int crop_unit_y = 2; + enum v4l2_mpeg_video_h264_profile profile; + enum v4l2_mpeg_video_h264_level level; unsigned int cpb_size; unsigned int cpb_size_scale; @@ -1284,14 +1288,17 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel, if (!sps) return -ENOMEM; - sps->profile_idc = nal_h264_profile_from_v4l2(channel->profile); + profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile); + level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level); + + sps->profile_idc = nal_h264_profile_from_v4l2(profile); sps->constraint_set0_flag = 0; sps->constraint_set1_flag = 1; sps->constraint_set2_flag = 0; sps->constraint_set3_flag = 0; sps->constraint_set4_flag = 0; sps->constraint_set5_flag = 0; - sps->level_idc = nal_h264_level_from_v4l2(channel->level); + sps->level_idc = nal_h264_level_from_v4l2(level); sps->seq_parameter_set_id = 0; sps->log2_max_frame_num_minus4 = LOG2_MAX_FRAME_NUM - 4; sps->pic_order_cnt_type = 0; @@ -2076,7 +2083,6 @@ static void allegro_channel_adjust(struct allegro_channel *channel) __v4l2_ctrl_modify_range(ctrl, min, ctrl->maximum, ctrl->step, ctrl->default_value); v4l2_ctrl_unlock(ctrl); - channel->level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level); ctrl = channel->mpeg_video_bitrate; max = maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level)); @@ -2130,9 +2136,6 @@ static void allegro_set_default_params(struct allegro_channel *channel) channel->sizeimage_raw = channel->stride * channel->height * 3 / 2; channel->codec = V4L2_PIX_FMT_H264; - channel->profile = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE; - channel->level = - select_minimum_h264_level(channel->width, channel->height); } static int allegro_queue_setup(struct vb2_queue *vq, @@ -2377,9 +2380,6 @@ static int allegro_s_ctrl(struct v4l2_ctrl *ctrl) "s_ctrl: %s = %d\n", v4l2_ctrl_get_name(ctrl->id), ctrl->val); switch (ctrl->id) { - case V4L2_CID_MPEG_VIDEO_H264_LEVEL: - channel->level = ctrl->val; - break; case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE: channel->frame_rc_enable = ctrl->val; break; From patchwork Thu Dec 3 11:01:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 027B9C64E8A for ; Thu, 3 Dec 2020 11:04:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A79D821D91 for ; Thu, 3 Dec 2020 11:04:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388786AbgLCLEN (ORCPT ); Thu, 3 Dec 2020 06:04:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387738AbgLCLEM (ORCPT ); Thu, 3 Dec 2020 06:04:12 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EBEDC061A55 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002Kz-3E; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKi5-0T; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:01 +0100 Message-Id: <20201203110106.2939463-14-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 13/18] media: allegro: use accessor functions for QP values X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org V4L2 specifies different controls for the QP values of different codecs. Simplify users that just care for the QP values by providing accessor function that return the correct control based on the currently set codec. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 40 ++++++++++++++++--- 1 file changed, 35 insertions(+), 5 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index c0519f7a67b8..004df6ea2430 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -264,6 +264,36 @@ struct allegro_channel { unsigned int error; }; +static inline int +allegro_channel_get_i_frame_qp(struct allegro_channel *channel) +{ + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp); +} + +static inline int +allegro_channel_get_p_frame_qp(struct allegro_channel *channel) +{ + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp); +} + +static inline int +allegro_channel_get_b_frame_qp(struct allegro_channel *channel) +{ + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp); +} + +static inline int +allegro_channel_get_min_qp(struct allegro_channel *channel) +{ + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp); +} + +static inline int +allegro_channel_get_max_qp(struct allegro_channel *channel) +{ + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp); +} + struct allegro_m2m_buffer { struct v4l2_m2m_buffer buf; struct list_head head; @@ -917,9 +947,9 @@ static s16 get_qp_delta(int minuend, int subtrahend) static int fill_create_channel_param(struct allegro_channel *channel, struct create_channel_param *param) { - int i_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp); - int p_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp); - int b_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp); + int i_frame_qp = allegro_channel_get_i_frame_qp(channel); + int p_frame_qp = allegro_channel_get_p_frame_qp(channel); + int b_frame_qp = allegro_channel_get_b_frame_qp(channel); int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode); unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size); enum v4l2_mpeg_video_h264_profile profile; @@ -982,8 +1012,8 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->target_bitrate = channel->bitrate; param->max_bitrate = channel->bitrate_peak; param->initial_qp = i_frame_qp; - param->min_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp); - param->max_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp); + param->min_qp = allegro_channel_get_min_qp(channel); + param->max_qp = allegro_channel_get_max_qp(channel); param->ip_delta = get_qp_delta(i_frame_qp, p_frame_qp); param->pb_delta = get_qp_delta(p_frame_qp, b_frame_qp); param->golden_ref = 0; From patchwork Thu Dec 3 11:01:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60AF8C83013 for ; Thu, 3 Dec 2020 11:04:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A66621D91 for ; Thu, 3 Dec 2020 11:04:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388823AbgLCLEO (ORCPT ); Thu, 3 Dec 2020 06:04:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388805AbgLCLEO (ORCPT ); Thu, 3 Dec 2020 06:04:14 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A14FC08E862 for ; Thu, 3 Dec 2020 03:02:55 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002L0-4c; Thu, 03 Dec 2020 12:02:53 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKi8-0r; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:02 +0100 Message-Id: <20201203110106.2939463-15-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 14/18] media: allegro: add helper to get entropy mode X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The driver only supports CAVLC for H.264, but HEVC only uses CABAC. As the driver has to explicitly tell the MCU to use CABAC for HEVC, add a helper function to get the entropy mode. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-core.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 004df6ea2430..016c31655467 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -944,6 +944,14 @@ static s16 get_qp_delta(int minuend, int subtrahend) return minuend - subtrahend; } +static u32 allegro_channel_get_entropy_mode(struct allegro_channel *channel) +{ +#define ALLEGRO_ENTROPY_MODE_CAVLC 0 +#define ALLEGRO_ENTROPY_MODE_CABAC 1 + + return ALLEGRO_ENTROPY_MODE_CAVLC; +} + static int fill_create_channel_param(struct allegro_channel *channel, struct create_channel_param *param) { @@ -974,6 +982,7 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->temporal_mvp_enable = channel->temporal_mvp_enable; param->dbf_ovr_en = channel->dbf_ovr_en; + param->entropy_mode = allegro_channel_get_entropy_mode(channel); param->rdo_cost_mode = 1; param->custom_lda = 1; param->lf = 1; From patchwork Thu Dec 3 11:01:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74D0AC83012 for ; Thu, 3 Dec 2020 11:03:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 299A6208FE for ; Thu, 3 Dec 2020 11:03:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730255AbgLCLDd (ORCPT ); Thu, 3 Dec 2020 06:03:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728779AbgLCLDd (ORCPT ); Thu, 3 Dec 2020 06:03:33 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 360A1C061A51 for ; Thu, 3 Dec 2020 03:02:53 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002L1-3Y; Thu, 03 Dec 2020 12:02:51 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKiB-1L; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:03 +0100 Message-Id: <20201203110106.2939463-16-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 15/18] media: allegro: rename codec specific functions X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The maximum bitrate and the size of the CPB are dependent on the level. As the levels differ between the different codecs, the functions to get the maximum bitrate and CPB must be codec specific, too. Rename the functions to make it obvious that the function are only valid for H.264. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-core.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 016c31655467..6a8196735eaf 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -507,7 +507,7 @@ select_minimum_h264_level(unsigned int width, unsigned int height) return level; } -static unsigned int maximum_bitrate(enum v4l2_mpeg_video_h264_level level) +static unsigned int h264_maximum_bitrate(enum v4l2_mpeg_video_h264_level level) { switch (level) { case V4L2_MPEG_VIDEO_H264_LEVEL_1_0: @@ -546,7 +546,7 @@ static unsigned int maximum_bitrate(enum v4l2_mpeg_video_h264_level level) } } -static unsigned int maximum_cpb_size(enum v4l2_mpeg_video_h264_level level) +static unsigned int h264_maximum_cpb_size(enum v4l2_mpeg_video_h264_level level) { switch (level) { case V4L2_MPEG_VIDEO_H264_LEVEL_1_0: @@ -2124,7 +2124,7 @@ static void allegro_channel_adjust(struct allegro_channel *channel) v4l2_ctrl_unlock(ctrl); ctrl = channel->mpeg_video_bitrate; - max = maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level)); + max = h264_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level)); if (ctrl->maximum < max) v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: maximum: %lld -> %lld\n", @@ -2521,10 +2521,10 @@ static int allegro_open(struct file *file) V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0, V4L2_MPEG_VIDEO_BITRATE_MODE_CBR); - bitrate_max = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); - bitrate_def = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); - cpb_size_max = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); - cpb_size_def = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + bitrate_max = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + bitrate_def = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + cpb_size_max = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + cpb_size_def = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE, From patchwork Thu Dec 3 11:01:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D4C1C83014 for ; Thu, 3 Dec 2020 11:04:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E092B21D91 for ; Thu, 3 Dec 2020 11:04:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388818AbgLCLEO (ORCPT ); Thu, 3 Dec 2020 06:04:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388809AbgLCLEO (ORCPT ); Thu, 3 Dec 2020 06:04:14 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6041AC08E863 for ; Thu, 3 Dec 2020 03:02:55 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002L2-6y; Thu, 03 Dec 2020 12:02:53 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKiE-1s; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:04 +0100 Message-Id: <20201203110106.2939463-17-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 16/18] media: allegro: increase offset in CAPTURE buffer X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The HEVC Non-VCL NAL units require more space than the H.264 Non-VCL NAL units. Therefore, the driver needs to reserve more space in front of the actual coded data that is written by the hardware codec. Increase the offset that shall be used by the hardware codec from 64 bytes to 128 bytes which is as arbitrary as before. Signed-off-by: Michael Tretter --- drivers/media/platform/allegro-dvt/allegro-core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 6a8196735eaf..4eacad3d53cb 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -90,7 +90,7 @@ * because it needs to write SPS/PPS NAL units. The encoder writes the actual * frame data after the offset. */ -#define ENCODER_STREAM_OFFSET SZ_64 +#define ENCODER_STREAM_OFFSET SZ_128 #define SIZE_MACROBLOCK 16 From patchwork Thu Dec 3 11:01:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB84FC63777 for ; Thu, 3 Dec 2020 11:04:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 51E6A22206 for ; Thu, 3 Dec 2020 11:04:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388888AbgLCLEj (ORCPT ); Thu, 3 Dec 2020 06:04:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388620AbgLCLEj (ORCPT ); Thu, 3 Dec 2020 06:04:39 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B982FC08ED7E for ; Thu, 3 Dec 2020 03:02:55 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002L3-59; Thu, 03 Dec 2020 12:02:54 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKiH-2J; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:05 +0100 Message-Id: <20201203110106.2939463-18-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 17/18] media: allegro: activate v4l2-ctrls only for current codec X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org These controls are specific to H.264 and shall only be activated, if the coded format is H.264. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 4eacad3d53cb..5ff4509be13d 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -2105,6 +2105,7 @@ static int allegro_create_channel(struct allegro_channel *channel) static void allegro_channel_adjust(struct allegro_channel *channel) { struct allegro_dev *dev = channel->dev; + u32 codec = channel->codec; struct v4l2_ctrl *ctrl; s64 min; s64 max; @@ -2140,6 +2141,21 @@ static void allegro_channel_adjust(struct allegro_channel *channel) ctrl->step, ctrl->default_value); v4l2_ctrl_unlock(ctrl); + v4l2_ctrl_activate(channel->mpeg_video_h264_profile, + codec == V4L2_PIX_FMT_H264); + v4l2_ctrl_activate(channel->mpeg_video_h264_level, + codec == V4L2_PIX_FMT_H264); + v4l2_ctrl_activate(channel->mpeg_video_h264_i_frame_qp, + codec == V4L2_PIX_FMT_H264); + v4l2_ctrl_activate(channel->mpeg_video_h264_max_qp, + codec == V4L2_PIX_FMT_H264); + v4l2_ctrl_activate(channel->mpeg_video_h264_min_qp, + codec == V4L2_PIX_FMT_H264); + v4l2_ctrl_activate(channel->mpeg_video_h264_p_frame_qp, + codec == V4L2_PIX_FMT_H264); + v4l2_ctrl_activate(channel->mpeg_video_h264_b_frame_qp, + codec == V4L2_PIX_FMT_H264); + channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM; channel->temporal_mvp_enable = true; From patchwork Thu Dec 3 11:01:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tretter X-Patchwork-Id: 11948471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A044C64E8A for ; Thu, 3 Dec 2020 11:04:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A644421D91 for ; Thu, 3 Dec 2020 11:04:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388855AbgLCLE2 (ORCPT ); Thu, 3 Dec 2020 06:04:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388755AbgLCLE1 (ORCPT ); Thu, 3 Dec 2020 06:04:27 -0500 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E72D0C094240 for ; Thu, 3 Dec 2020 03:02:56 -0800 (PST) Received: from [2a0a:edc0:0:1101:1d::39] (helo=dude03.red.stw.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kkmNt-0002L4-7l; Thu, 03 Dec 2020 12:02:55 +0100 Received: from mtr by dude03.red.stw.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1kkmNs-00CKiK-36; Thu, 03 Dec 2020 12:02:48 +0100 From: Michael Tretter To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de, hverkuil-cisco@xs4all.nl, mchehab@kernel.org, Michael Tretter Date: Thu, 3 Dec 2020 12:01:06 +0100 Message-Id: <20201203110106.2939463-19-m.tretter@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201203110106.2939463-1-m.tretter@pengutronix.de> References: <20201203110106.2939463-1-m.tretter@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: mtr@pengutronix.de Subject: [PATCH 18/18] media: allegro: add support for HEVC encoding X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on metis.ext.pengutronix.de) X-PTX-Original-Recipient: linux-media@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The Allegro Codec supports HEVC encoding. The messages to the MCU are the same for H.264 and HEVC, but some options have to be changed. These are actually only a few options. The driver, however, must add the HEVC VPS/SPS/PPS NAL Units to the coded stream and must properly provide the HEVC format and controls to user space. Signed-off-by: Michael Tretter --- .../media/platform/allegro-dvt/allegro-core.c | 545 ++++++++++++++++-- .../media/platform/allegro-dvt/allegro-mail.c | 6 + .../media/platform/allegro-dvt/allegro-mail.h | 1 + 3 files changed, 516 insertions(+), 36 deletions(-) diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c index 5ff4509be13d..83370b0bd704 100644 --- a/drivers/media/platform/allegro-dvt/allegro-core.c +++ b/drivers/media/platform/allegro-dvt/allegro-core.c @@ -30,6 +30,7 @@ #include "allegro-mail.h" #include "nal-h264.h" +#include "nal-hevc.h" /* * Support up to 4k video streams. The hardware actually supports higher @@ -209,6 +210,8 @@ struct allegro_channel { bool enable_loop_filter_across_tiles; bool enable_loop_filter_across_slices; + bool enable_deblocking_filter_override; + bool enable_reordering; bool dbf_ovr_en; unsigned int num_ref_idx_l0; @@ -235,6 +238,16 @@ struct allegro_channel { struct v4l2_ctrl *mpeg_video_h264_min_qp; struct v4l2_ctrl *mpeg_video_h264_p_frame_qp; struct v4l2_ctrl *mpeg_video_h264_b_frame_qp; + + struct v4l2_ctrl *mpeg_video_hevc_profile; + struct v4l2_ctrl *mpeg_video_hevc_level; + struct v4l2_ctrl *mpeg_video_hevc_tier; + struct v4l2_ctrl *mpeg_video_hevc_i_frame_qp; + struct v4l2_ctrl *mpeg_video_hevc_max_qp; + struct v4l2_ctrl *mpeg_video_hevc_min_qp; + struct v4l2_ctrl *mpeg_video_hevc_p_frame_qp; + struct v4l2_ctrl *mpeg_video_hevc_b_frame_qp; + struct v4l2_ctrl *mpeg_video_frame_rc_enable; struct { /* video bitrate mode control cluster */ struct v4l2_ctrl *mpeg_video_bitrate_mode; @@ -267,31 +280,46 @@ struct allegro_channel { static inline int allegro_channel_get_i_frame_qp(struct allegro_channel *channel) { - return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp); + if (channel->codec == V4L2_PIX_FMT_HEVC) + return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_i_frame_qp); + else + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp); } static inline int allegro_channel_get_p_frame_qp(struct allegro_channel *channel) { - return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp); + if (channel->codec == V4L2_PIX_FMT_HEVC) + return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_p_frame_qp); + else + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp); } static inline int allegro_channel_get_b_frame_qp(struct allegro_channel *channel) { - return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp); + if (channel->codec == V4L2_PIX_FMT_HEVC) + return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_b_frame_qp); + else + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp); } static inline int allegro_channel_get_min_qp(struct allegro_channel *channel) { - return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp); + if (channel->codec == V4L2_PIX_FMT_HEVC) + return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_min_qp); + else + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp); } static inline int allegro_channel_get_max_qp(struct allegro_channel *channel) { - return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp); + if (channel->codec == V4L2_PIX_FMT_HEVC) + return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_max_qp); + else + return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp); } struct allegro_m2m_buffer { @@ -585,6 +613,86 @@ static unsigned int h264_maximum_cpb_size(enum v4l2_mpeg_video_h264_level level) } } +static enum v4l2_mpeg_video_hevc_level +select_minimum_hevc_level(unsigned int width, unsigned int height) +{ + unsigned int luma_picture_size = width * height; + enum v4l2_mpeg_video_hevc_level level; + + if (luma_picture_size <= 36864) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_1; + else if (luma_picture_size <= 122880) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_2; + else if (luma_picture_size <= 245760) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1; + else if (luma_picture_size <= 552960) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_3; + else if (luma_picture_size <= 983040) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1; + else if (luma_picture_size <= 2228224) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_4; + else if (luma_picture_size <= 8912896) + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_5; + else + level = V4L2_MPEG_VIDEO_HEVC_LEVEL_6; + + return level; +} + +static unsigned int hevc_maximum_bitrate(enum v4l2_mpeg_video_h264_level level) +{ + /* + * See Rec. ITU-T H.265 v5 (02/2018), A.4.2 Profile-specific level + * limits for the video profiles. + */ + switch (level) { + case V4L2_MPEG_VIDEO_HEVC_LEVEL_1: + return 128; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2: + return 1500; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1: + return 3000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3: + return 6000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1: + return 10000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4: + return 12000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1: + return 20000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5: + return 25000; + default: + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1: + return 40000; + } +} + +static unsigned int hevc_maximum_cpb_size(enum v4l2_mpeg_video_h264_level level) +{ + switch (level) { + case V4L2_MPEG_VIDEO_HEVC_LEVEL_1: + return 350; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2: + return 1500; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1: + return 3000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3: + return 6000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1: + return 10000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4: + return 12000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1: + return 20000; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5: + return 25000; + default: + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1: + return 40000; + } +} + static const struct fw_info * allegro_get_firmware_info(struct allegro_dev *dev, const struct firmware *fw, @@ -908,6 +1016,55 @@ static u16 v4l2_level_to_mcu_level(enum v4l2_mpeg_video_h264_level level) } } +static u8 hevc_profile_to_mcu_profile(enum v4l2_mpeg_video_hevc_profile profile) +{ + switch (profile) { + default: + case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN: + return 1; + case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10: + return 2; + case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE: + return 3; + } +} + +static u16 hevc_level_to_mcu_level(enum v4l2_mpeg_video_hevc_level level) +{ + switch (level) { + case V4L2_MPEG_VIDEO_HEVC_LEVEL_1: + return 10; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2: + return 20; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1: + return 21; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3: + return 30; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1: + return 31; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4: + return 40; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1: + return 41; + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5: + return 50; + default: + case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1: + return 51; + } +} + +static u8 hevc_tier_to_mcu_tier(enum v4l2_mpeg_video_hevc_tier tier) +{ + switch (tier) { + default: + case V4L2_MPEG_VIDEO_HEVC_TIER_MAIN: + return 0; + case V4L2_MPEG_VIDEO_HEVC_TIER_HIGH: + return 1; + } +} + static u32 v4l2_bitrate_mode_to_mcu_mode(enum v4l2_mpeg_video_bitrate_mode mode) { @@ -949,6 +1106,10 @@ static u32 allegro_channel_get_entropy_mode(struct allegro_channel *channel) #define ALLEGRO_ENTROPY_MODE_CAVLC 0 #define ALLEGRO_ENTROPY_MODE_CABAC 1 + /* HEVC always uses CABAC, but this has to be explicitly set */ + if (channel->codec == V4L2_PIX_FMT_HEVC) + return ALLEGRO_ENTROPY_MODE_CABAC; + return ALLEGRO_ENTROPY_MODE_CAVLC; } @@ -960,8 +1121,6 @@ static int fill_create_channel_param(struct allegro_channel *channel, int b_frame_qp = allegro_channel_get_b_frame_qp(channel); int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode); unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size); - enum v4l2_mpeg_video_h264_profile profile; - enum v4l2_mpeg_video_h264_level level; param->width = channel->width; param->height = channel->height; @@ -971,17 +1130,37 @@ static int fill_create_channel_param(struct allegro_channel *channel, param->src_mode = 0x0; param->codec = channel->codec; - profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile); - level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level); - param->profile = v4l2_profile_to_mcu_profile(profile); - param->constraint_set_flags = BIT(1); - param->level = v4l2_level_to_mcu_level(level); + if (channel->codec == V4L2_PIX_FMT_H264) { + enum v4l2_mpeg_video_h264_profile profile; + enum v4l2_mpeg_video_h264_level level; + + profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile); + level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level); + + param->profile = v4l2_profile_to_mcu_profile(profile); + param->constraint_set_flags = BIT(1); + param->level = v4l2_level_to_mcu_level(level); + } else { + enum v4l2_mpeg_video_hevc_profile profile; + enum v4l2_mpeg_video_hevc_level level; + enum v4l2_mpeg_video_hevc_tier tier; + + profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_profile); + level = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level); + tier = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_tier); + + param->profile = hevc_profile_to_mcu_profile(profile); + param->level = hevc_level_to_mcu_level(level); + param->tier = hevc_tier_to_mcu_tier(tier); + } param->log2_max_poc = LOG2_MAX_PIC_ORDER_CNT; param->log2_max_frame_num = channel->log2_max_frame_num; param->temporal_mvp_enable = channel->temporal_mvp_enable; param->dbf_ovr_en = channel->dbf_ovr_en; + param->override_lf = channel->enable_deblocking_filter_override; + param->enable_reordering = channel->enable_reordering; param->entropy_mode = allegro_channel_get_entropy_mode(channel); param->rdo_cost_mode = 1; param->custom_lda = 1; @@ -1454,6 +1633,158 @@ static void allegro_channel_eos_event(struct allegro_channel *channel) v4l2_event_queue_fh(&channel->fh, &eos_event); } +static ssize_t allegro_hevc_write_vps(struct allegro_channel *channel, + void *dest, size_t n) +{ + struct allegro_dev *dev = channel->dev; + struct nal_hevc_vps *vps; + struct nal_hevc_profile_tier_level *ptl; + ssize_t size; + unsigned int num_ref_frames = channel->num_ref_idx_l0; + s32 profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_profile); + s32 level = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level); + s32 tier = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_tier); + + vps = kzalloc(sizeof(*vps), GFP_KERNEL); + if (!vps) + return -ENOMEM; + + vps->base_layer_internal_flag = 1; + vps->base_layer_available_flag = 1; + vps->temporal_id_nesting_flag = 1; + + ptl = &vps->profile_tier_level; + ptl->general_profile_idc = nal_hevc_profile_from_v4l2(profile); + ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1; + ptl->general_tier_flag = nal_hevc_tier_from_v4l2(tier); + ptl->general_progressive_source_flag = 1; + ptl->general_frame_only_constraint_flag = 1; + ptl->general_level_idc = nal_hevc_level_from_v4l2(level); + + vps->sub_layer_ordering_info_present_flag = 0; + vps->max_dec_pic_buffering_minus1[0] = num_ref_frames; + vps->max_num_reorder_pics[0] = num_ref_frames; + + size = nal_hevc_write_vps(&dev->plat_dev->dev, dest, n, vps); + + kfree(vps); + + return size; +} + +static ssize_t allegro_hevc_write_sps(struct allegro_channel *channel, + void *dest, size_t n) +{ + struct allegro_dev *dev = channel->dev; + struct nal_hevc_sps *sps; + struct nal_hevc_profile_tier_level *ptl; + ssize_t size; + unsigned int num_ref_frames = channel->num_ref_idx_l0; + s32 profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_profile); + s32 level = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level); + s32 tier = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_tier); + + sps = kzalloc(sizeof(*sps), GFP_KERNEL); + if (!sps) + return -ENOMEM; + + sps->temporal_id_nesting_flag = 1; + + ptl = &sps->profile_tier_level; + ptl->general_profile_idc = nal_hevc_profile_from_v4l2(profile); + ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1; + ptl->general_tier_flag = nal_hevc_tier_from_v4l2(tier); + ptl->general_progressive_source_flag = 1; + ptl->general_frame_only_constraint_flag = 1; + ptl->general_level_idc = nal_hevc_level_from_v4l2(level); + + sps->seq_parameter_set_id = 0; + sps->chroma_format_idc = 1; /* Only 4:2:0 sampling supported */ + sps->pic_width_in_luma_samples = round_up(channel->width, 8); + sps->pic_height_in_luma_samples = round_up(channel->height, 8); + sps->conf_win_right_offset = + sps->pic_width_in_luma_samples - channel->width; + sps->conf_win_bottom_offset = + sps->pic_height_in_luma_samples - channel->height; + sps->conformance_window_flag = + sps->conf_win_right_offset || sps->conf_win_bottom_offset; + + sps->log2_max_pic_order_cnt_lsb_minus4 = LOG2_MAX_PIC_ORDER_CNT - 4; + + sps->sub_layer_ordering_info_present_flag = 1; + sps->max_dec_pic_buffering_minus1[0] = num_ref_frames; + sps->max_num_reorder_pics[0] = num_ref_frames; + + sps->log2_min_luma_coding_block_size_minus3 = + channel->min_cu_size - 3; + sps->log2_diff_max_min_luma_coding_block_size = + channel->max_cu_size - channel->min_cu_size; + sps->log2_min_luma_transform_block_size_minus2 = + channel->min_tu_size - 2; + sps->log2_diff_max_min_luma_transform_block_size = + channel->max_tu_size - channel->min_tu_size; + sps->max_transform_hierarchy_depth_intra = + channel->max_transfo_depth_intra; + sps->max_transform_hierarchy_depth_inter = + channel->max_transfo_depth_inter; + + sps->sps_temporal_mvp_enabled_flag = channel->temporal_mvp_enable; + sps->strong_intra_smoothing_enabled_flag = channel->max_cu_size > 4; + + size = nal_hevc_write_sps(&dev->plat_dev->dev, dest, n, sps); + + kfree(sps); + + return size; +} + +static ssize_t allegro_hevc_write_pps(struct allegro_channel *channel, + struct mcu_msg_encode_frame_response *msg, + void *dest, size_t n) +{ + struct allegro_dev *dev = channel->dev; + struct nal_hevc_pps *pps; + ssize_t size; + int i; + + pps = kzalloc(sizeof(*pps), GFP_KERNEL); + if (!pps) + return -ENOMEM; + + pps->pps_pic_parameter_set_id = 0; + pps->pps_seq_parameter_set_id = 0; + + if (msg->num_column > 1 || msg->num_row > 1) { + pps->tiles_enabled_flag = 1; + pps->num_tile_columns_minus1 = msg->num_column - 1; + pps->num_tile_rows_minus1 = msg->num_row - 1; + + for (i = 0; i < msg->num_column; i++) + pps->column_width_minus1[i] = msg->tile_width[i] - 1; + + for (i = 0; i < msg->num_row; i++) + pps->row_height_minus1[i] = msg->tile_height[i] - 1; + } + + pps->loop_filter_across_tiles_enabled_flag = + channel->enable_loop_filter_across_tiles; + pps->pps_loop_filter_across_slices_enabled_flag = + channel->enable_loop_filter_across_slices; + pps->deblocking_filter_control_present_flag = 1; + pps->deblocking_filter_override_enabled_flag = + channel->enable_deblocking_filter_override; + pps->pps_beta_offset_div2 = BETA_OFFSET_DIV_2; + pps->pps_tc_offset_div2 = TC_OFFSET_DIV_2; + + pps->lists_modification_present_flag = channel->enable_reordering; + + size = nal_hevc_write_pps(&dev->plat_dev->dev, dest, n, pps); + + kfree(pps); + + return size; +} + static u64 allegro_put_buffer(struct allegro_channel *channel, struct list_head *list, struct vb2_v4l2_buffer *buffer) @@ -1577,8 +1908,27 @@ static void allegro_channel_finish_frame(struct allegro_channel *channel, curr = vb2_plane_vaddr(&dst_buf->vb2_buf, 0); free = partition->offset; + + if (channel->codec == V4L2_PIX_FMT_HEVC && msg->is_idr) { + len = allegro_hevc_write_vps(channel, curr, free); + if (len < 0) { + v4l2_err(&dev->v4l2_dev, + "not enough space for video parameter set: %zd left\n", + free); + goto err; + } + curr += len; + free -= len; + v4l2_dbg(1, debug, &dev->v4l2_dev, + "channel %d: wrote %zd byte VPS nal unit\n", + channel->mcu_channel_id, len); + } + if (msg->is_idr) { - len = allegro_h264_write_sps(channel, curr, free); + if (channel->codec == V4L2_PIX_FMT_H264) + len = allegro_h264_write_sps(channel, curr, free); + else + len = allegro_hevc_write_sps(channel, curr, free); if (len < 0) { v4l2_err(&dev->v4l2_dev, "not enough space for sequence parameter set: %zd left\n", @@ -1593,7 +1943,10 @@ static void allegro_channel_finish_frame(struct allegro_channel *channel, } if (msg->slice_type == AL_ENC_SLICE_TYPE_I) { - len = allegro_h264_write_pps(channel, curr, free); + if (channel->codec == V4L2_PIX_FMT_H264) + len = allegro_h264_write_pps(channel, curr, free); + else + len = allegro_hevc_write_pps(channel, msg, curr, free); if (len < 0) { v4l2_err(&dev->v4l2_dev, "not enough space for picture parameter set: %zd left\n", @@ -1611,7 +1964,10 @@ static void allegro_channel_finish_frame(struct allegro_channel *channel, dst_buf->vb2_buf.planes[0].data_offset = free; free = 0; } else { - len = nal_h264_write_filler(&dev->plat_dev->dev, curr, free); + if (channel->codec == V4L2_PIX_FMT_H264) + len = nal_h264_write_filler(&dev->plat_dev->dev, curr, free); + else + len = nal_hevc_write_filler(&dev->plat_dev->dev, curr, free); if (len < 0) { v4l2_err(&dev->v4l2_dev, "failed to write %zd filler data\n", free); @@ -2011,6 +2367,16 @@ static void allegro_destroy_channel(struct allegro_channel *channel) v4l2_ctrl_grab(channel->mpeg_video_h264_min_qp, false); v4l2_ctrl_grab(channel->mpeg_video_h264_p_frame_qp, false); v4l2_ctrl_grab(channel->mpeg_video_h264_b_frame_qp, false); + + v4l2_ctrl_grab(channel->mpeg_video_hevc_profile, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_level, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_tier, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_i_frame_qp, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_max_qp, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_min_qp, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_p_frame_qp, false); + v4l2_ctrl_grab(channel->mpeg_video_hevc_b_frame_qp, false); + v4l2_ctrl_grab(channel->mpeg_video_frame_rc_enable, false); v4l2_ctrl_grab(channel->mpeg_video_bitrate_mode, false); v4l2_ctrl_grab(channel->mpeg_video_bitrate, false); @@ -2067,6 +2433,16 @@ static int allegro_create_channel(struct allegro_channel *channel) v4l2_ctrl_grab(channel->mpeg_video_h264_min_qp, true); v4l2_ctrl_grab(channel->mpeg_video_h264_p_frame_qp, true); v4l2_ctrl_grab(channel->mpeg_video_h264_b_frame_qp, true); + + v4l2_ctrl_grab(channel->mpeg_video_hevc_profile, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_level, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_tier, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_i_frame_qp, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_max_qp, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_min_qp, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_p_frame_qp, true); + v4l2_ctrl_grab(channel->mpeg_video_hevc_b_frame_qp, true); + v4l2_ctrl_grab(channel->mpeg_video_frame_rc_enable, true); v4l2_ctrl_grab(channel->mpeg_video_bitrate_mode, true); v4l2_ctrl_grab(channel->mpeg_video_bitrate, true); @@ -2113,8 +2489,13 @@ static void allegro_channel_adjust(struct allegro_channel *channel) channel->sizeimage_encoded = estimate_stream_size(channel->width, channel->height); - ctrl = channel->mpeg_video_h264_level; - min = select_minimum_h264_level(channel->width, channel->height); + if (codec == V4L2_PIX_FMT_H264) { + ctrl = channel->mpeg_video_h264_level; + min = select_minimum_h264_level(channel->width, channel->height); + } else { + ctrl = channel->mpeg_video_hevc_level; + min = select_minimum_hevc_level(channel->width, channel->height); + } if (ctrl->minimum > min) v4l2_dbg(1, debug, &dev->v4l2_dev, "%s.minimum: %lld -> %lld\n", @@ -2125,7 +2506,10 @@ static void allegro_channel_adjust(struct allegro_channel *channel) v4l2_ctrl_unlock(ctrl); ctrl = channel->mpeg_video_bitrate; - max = h264_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level)); + if (codec == V4L2_PIX_FMT_H264) + max = h264_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level)); + else + max = hevc_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level)); if (ctrl->maximum < max) v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: maximum: %lld -> %lld\n", @@ -2156,21 +2540,51 @@ static void allegro_channel_adjust(struct allegro_channel *channel) v4l2_ctrl_activate(channel->mpeg_video_h264_b_frame_qp, codec == V4L2_PIX_FMT_H264); - channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM; + v4l2_ctrl_activate(channel->mpeg_video_hevc_profile, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_level, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_tier, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_i_frame_qp, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_max_qp, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_min_qp, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_p_frame_qp, + codec == V4L2_PIX_FMT_HEVC); + v4l2_ctrl_activate(channel->mpeg_video_hevc_b_frame_qp, + codec == V4L2_PIX_FMT_HEVC); + + if (codec == V4L2_PIX_FMT_H264) + channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM; channel->temporal_mvp_enable = true; - - channel->dbf_ovr_en = true; + channel->dbf_ovr_en = (codec == V4L2_PIX_FMT_H264); + channel->enable_deblocking_filter_override = (codec == V4L2_PIX_FMT_HEVC); + channel->enable_reordering = (codec == V4L2_PIX_FMT_HEVC); channel->enable_loop_filter_across_tiles = true; channel->enable_loop_filter_across_slices = true; - channel->b_hrz_me_range = 8; - channel->b_vrt_me_range = 8; - channel->p_hrz_me_range = 16; - channel->p_vrt_me_range = 16; - channel->max_cu_size = ilog2(16); - channel->min_cu_size = ilog2(8); - channel->max_tu_size = ilog2(4); - channel->min_tu_size = ilog2(4); + if (codec == V4L2_PIX_FMT_H264) { + channel->b_hrz_me_range = 8; + channel->b_vrt_me_range = 8; + channel->p_hrz_me_range = 16; + channel->p_vrt_me_range = 16; + channel->max_cu_size = ilog2(16); + channel->min_cu_size = ilog2(8); + channel->max_tu_size = ilog2(4); + channel->min_tu_size = ilog2(4); + } else { + channel->b_hrz_me_range = 16; + channel->b_vrt_me_range = 16; + channel->p_hrz_me_range = 32; + channel->p_vrt_me_range = 32; + channel->max_cu_size = ilog2(32); + channel->min_cu_size = ilog2(8); + channel->max_tu_size = ilog2(32); + channel->min_tu_size = ilog2(4); + } channel->max_transfo_depth_intra = 1; channel->max_transfo_depth_inter = 1; } @@ -2525,6 +2939,51 @@ static int allegro_open(struct file *file) &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP, 0, 51, 1, 30); + + channel->mpeg_video_hevc_profile = + v4l2_ctrl_new_std_menu(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_PROFILE, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, 0x0, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN); + channel->mpeg_video_hevc_level = + v4l2_ctrl_new_std_menu(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1, 0x0, + V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1); + channel->mpeg_video_hevc_tier = + v4l2_ctrl_new_std_menu(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_TIER, + V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, 0x0, + V4L2_MPEG_VIDEO_HEVC_TIER_MAIN); + channel->mpeg_video_hevc_i_frame_qp = + v4l2_ctrl_new_std(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, + 0, 51, 1, 30); + channel->mpeg_video_hevc_max_qp = + v4l2_ctrl_new_std(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, + 0, 51, 1, 51); + channel->mpeg_video_hevc_min_qp = + v4l2_ctrl_new_std(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, + 0, 51, 1, 0); + channel->mpeg_video_hevc_p_frame_qp = + v4l2_ctrl_new_std(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, + 0, 51, 1, 30); + channel->mpeg_video_hevc_b_frame_qp = + v4l2_ctrl_new_std(handler, + &allegro_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP, + 0, 51, 1, 30); + channel->mpeg_video_frame_rc_enable = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, @@ -2537,10 +2996,17 @@ static int allegro_open(struct file *file) V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0, V4L2_MPEG_VIDEO_BITRATE_MODE_CBR); - bitrate_max = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); - bitrate_def = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); - cpb_size_max = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); - cpb_size_def = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + if (channel->codec == V4L2_PIX_FMT_H264) { + bitrate_max = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + bitrate_def = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + cpb_size_max = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + cpb_size_def = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1); + } else { + bitrate_max = hevc_maximum_bitrate(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1); + bitrate_def = hevc_maximum_bitrate(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1); + cpb_size_max = hevc_maximum_cpb_size(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1); + cpb_size_def = hevc_maximum_cpb_size(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1); + } channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler, &allegro_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE, @@ -2645,9 +3111,12 @@ static int allegro_enum_fmt_vid(struct file *file, void *fh, f->pixelformat = V4L2_PIX_FMT_NV12; break; case V4L2_BUF_TYPE_VIDEO_CAPTURE: - if (f->index >= 1) + if (f->index >= 2) return -EINVAL; - f->pixelformat = V4L2_PIX_FMT_H264; + if (f->index == 0) + f->pixelformat = V4L2_PIX_FMT_H264; + if (f->index == 1) + f->pixelformat = V4L2_PIX_FMT_HEVC; break; default: return -EINVAL; @@ -2686,7 +3155,10 @@ static int allegro_try_fmt_vid_cap(struct file *file, void *fh, f->fmt.pix.height = clamp_t(__u32, f->fmt.pix.height, ALLEGRO_HEIGHT_MIN, ALLEGRO_HEIGHT_MAX); - f->fmt.pix.pixelformat = V4L2_PIX_FMT_H264; + if (f->fmt.pix.pixelformat != V4L2_PIX_FMT_HEVC && + f->fmt.pix.pixelformat != V4L2_PIX_FMT_H264) + f->fmt.pix.pixelformat = V4L2_PIX_FMT_H264; + f->fmt.pix.bytesperline = 0; f->fmt.pix.sizeimage = estimate_stream_size(f->fmt.pix.width, f->fmt.pix.height); @@ -2826,6 +3298,7 @@ static int allegro_enum_framesizes(struct file *file, void *fh, struct v4l2_frmsizeenum *fsize) { switch (fsize->pixel_format) { + case V4L2_PIX_FMT_HEVC: case V4L2_PIX_FMT_H264: case V4L2_PIX_FMT_NV12: break; diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.c b/drivers/media/platform/allegro-dvt/allegro-mail.c index 5dbc1c029020..7e08c5050f2e 100644 --- a/drivers/media/platform/allegro-dvt/allegro-mail.c +++ b/drivers/media/platform/allegro-dvt/allegro-mail.c @@ -67,12 +67,16 @@ static inline u32 settings_get_mcu_codec(struct create_channel_param *param) if (version < MCU_MSG_VERSION_2019_2) { switch (pixelformat) { + case V4L2_PIX_FMT_HEVC: + return 2; case V4L2_PIX_FMT_H264: default: return 1; } } else { switch (pixelformat) { + case V4L2_PIX_FMT_HEVC: + return 1; case V4L2_PIX_FMT_H264: default: return 0; @@ -117,7 +121,9 @@ allegro_encode_config_blob(u32 *dst, struct create_channel_param *param) dst[i++] = val; val = 0; + val |= param->enable_reordering ? BIT(0) : 0; val |= param->dbf_ovr_en ? BIT(2) : 0; + val |= param->override_lf ? BIT(12) : 0; dst[i++] = val; if (version >= MCU_MSG_VERSION_2019_2) { diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.h b/drivers/media/platform/allegro-dvt/allegro-mail.h index f7485cf78c4f..2c7bc509eac3 100644 --- a/drivers/media/platform/allegro-dvt/allegro-mail.h +++ b/drivers/media/platform/allegro-dvt/allegro-mail.h @@ -65,6 +65,7 @@ struct create_channel_param { u32 temporal_mvp_enable; u32 enable_reordering; u32 dbf_ovr_en; + u32 override_lf; u32 num_ref_idx_l0; u32 num_ref_idx_l1; u32 custom_lda;