From patchwork Mon Apr 8 12:32:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philipp Zabel X-Patchwork-Id: 10889511 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90AFB14DB for ; Mon, 8 Apr 2019 12:33:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79DBC285D8 for ; Mon, 8 Apr 2019 12:33:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6DAE22862A; Mon, 8 Apr 2019 12:33:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D50828630 for ; Mon, 8 Apr 2019 12:33:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726549AbfDHMdL (ORCPT ); Mon, 8 Apr 2019 08:33:11 -0400 Received: from metis.ext.pengutronix.de ([85.220.165.71]:45191 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726592AbfDHMdK (ORCPT ); Mon, 8 Apr 2019 08:33:10 -0400 Received: from dude02.hi.pengutronix.de ([2001:67c:670:100:1d::28] helo=dude02.pengutronix.de.) by metis.ext.pengutronix.de with esmtp (Exim 4.89) (envelope-from ) id 1hDTSW-00005P-FG; Mon, 08 Apr 2019 14:33:08 +0200 From: Philipp Zabel To: linux-media@vger.kernel.org Cc: kernel@pengutronix.de Subject: [PATCH 07/10] media: coda: limit frame interval enumeration to supported frame sizes Date: Mon, 8 Apr 2019 14:32:53 +0200 Message-Id: <20190408123256.22868-7-p.zabel@pengutronix.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190408123256.22868-1-p.zabel@pengutronix.de> References: <20190408123256.22868-1-p.zabel@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:100:1d::28 X-SA-Exim-Mail-From: p.zabel@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-media@vger.kernel.org Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Let VIDIOC_ENUM_FRAMEINTERVALS return -EINVAL if userspace queries frame intervals for unsupported frame sizes. Signed-off-by: Philipp Zabel --- drivers/media/platform/coda/coda-common.c | 33 ++++++++++++++++++----- 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c index 943f003c26c4..2966eb1c4d2d 100644 --- a/drivers/media/platform/coda/coda-common.c +++ b/drivers/media/platform/coda/coda-common.c @@ -1117,7 +1117,8 @@ static int coda_enum_frameintervals(struct file *file, void *fh, struct v4l2_frmivalenum *f) { struct coda_ctx *ctx = fh_to_ctx(fh); - int i; + struct coda_q_data *q_data; + const struct coda_codec *codec; if (f->index) return -EINVAL; @@ -1126,12 +1127,32 @@ static int coda_enum_frameintervals(struct file *file, void *fh, if (!ctx->vdoa && f->pixel_format == V4L2_PIX_FMT_YUYV) return -EINVAL; - for (i = 0; i < CODA_MAX_FORMATS; i++) { - if (f->pixel_format == ctx->cvd->src_formats[i] || - f->pixel_format == ctx->cvd->dst_formats[i]) - break; + if (ctx->inst_type == CODA_INST_ENCODER) { + if (coda_format_normalize_yuv(f->pixel_format) == + V4L2_PIX_FMT_YUV420) { + q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE); + codec = coda_find_codec(ctx->dev, f->pixel_format, + q_data->fourcc); + } else { + codec = coda_find_codec(ctx->dev, V4L2_PIX_FMT_YUV420, + f->pixel_format); + } + } else { + if (coda_format_normalize_yuv(f->pixel_format) == + V4L2_PIX_FMT_YUV420) { + q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT); + codec = coda_find_codec(ctx->dev, q_data->fourcc, + f->pixel_format); + } else { + codec = coda_find_codec(ctx->dev, f->pixel_format, + V4L2_PIX_FMT_YUV420); + } } - if (i == CODA_MAX_FORMATS) + if (!codec) + return -EINVAL; + + if (f->width < MIN_W || f->width > codec->max_w || + f->height < MIN_H || f->height > codec->max_h) return -EINVAL; f->type = V4L2_FRMIVAL_TYPE_CONTINUOUS;