From patchwork Tue Mar 13 09:11:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 10277981 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0006B60231 for ; Tue, 13 Mar 2018 09:11:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0B1E28F45 for ; Tue, 13 Mar 2018 09:11:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D432028F4F; Tue, 13 Mar 2018 09:11:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B11C28F45 for ; Tue, 13 Mar 2018 09:11:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752108AbeCMJLo (ORCPT ); Tue, 13 Mar 2018 05:11:44 -0400 Received: from mail-pg0-f68.google.com ([74.125.83.68]:39975 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752052AbeCMJLm (ORCPT ); Tue, 13 Mar 2018 05:11:42 -0400 Received: by mail-pg0-f68.google.com with SMTP id g8so7695485pgv.7 for ; Tue, 13 Mar 2018 02:11:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id; bh=e9ONcc6Fd4zMXp86iHPdyHLELy/XGRrCJbCg2gcTsrM=; b=HaURFcJgd+XANkZrjkP4EuQcKF2DZXI9p0lddLvvxDPqEiP5RngBI5nmsijw3hs7tT wx96AdENj/uC6iZijXeIGKz3n/dJ3YaQP09Js85RH0T1T1B3ouFv2I778xPlGRoPdys8 4rMTzpYnC7+CwR5BmfRfrzOnNXo1DyB72AOnY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=e9ONcc6Fd4zMXp86iHPdyHLELy/XGRrCJbCg2gcTsrM=; b=cMRkQylo4aDqmDr16sKqpUbptCb3U97u6EUDTpZIFPOBV67y9cj/plkpU47VLqHHxV 71whG3uV1hybDJKX25VukptK/1S/JssPhRPj4EsH1r8ododhh8qqtLP4yAVXONOSoMvM vuXYu/R0my1ED8MBtoOWhzimD+tY88UHiX13MhGSpQcl2SEhoWA9tKOLV/UfLavVwKTk PXIThyZOwjaRBeOdi6B5djsjk0S440ivWcb34ei6LTIQ4LYQEn+A603oqN6l3HRGNYkw rfYrettkwupEZIfXt0EckoywyvCAWKyEB0qCxffXhrVvu9d6cs651lGmxqXYjwm/DNL7 FbMw== X-Gm-Message-State: AElRT7GJ4+4gH0Wj0ecqlVulQmO9DHVUIwudrgWDH/0qtg8SNVnr3+dT HcpQATARbnULMOTvUro7aoS+5A== X-Google-Smtp-Source: AG47ELvPm1uj/Y81Iguik+rL/D86u71ZkSzNaxkT/v6D1QC6yr1NwVfX2MLtEhWDZnwaJinRvyprsw== X-Received: by 10.99.117.10 with SMTP id q10mr9139570pgc.423.1520932302375; Tue, 13 Mar 2018 02:11:42 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:4:1002:a6cd:a898:e07b:a331]) by smtp.gmail.com with ESMTPSA id w129sm22702048pfd.155.2018.03.13.02.11.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Mar 2018 02:11:41 -0700 (PDT) From: Alexandre Courbot To: Stanimir Varbanov Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org, Alexandre Courbot Subject: [PATCH] venus: vdec: fix format enumeration Date: Tue, 13 Mar 2018 18:11:35 +0900 Message-Id: <20180313091135.145589-1-acourbot@chromium.org> X-Mailer: git-send-email 2.16.2.660.g709887971b-goog Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP find_format_by_index() stops enumerating formats as soon as the index matches, and returns NULL if venus_helper_check_codec() finds out that the format is not supported. This prevents formats to be properly enumerated if a non-supported format is present, as the enumeration will end with it. Fix this by moving the call to venus_helper_check_codec() into the loop, and keep enumerating when it fails. Signed-off-by: Alexandre Courbot Change-Id: I4ff66e0b85172598efa59a6f01da8cb60597a6a5 --- drivers/media/platform/qcom/venus/vdec.c | 13 +++++++------ drivers/media/platform/qcom/venus/venc.c | 9 +++++++-- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c index c9e9576bb08a..3677302cfe43 100644 --- a/drivers/media/platform/qcom/venus/vdec.c +++ b/drivers/media/platform/qcom/venus/vdec.c @@ -135,20 +135,21 @@ find_format_by_index(struct venus_inst *inst, unsigned int index, u32 type) return NULL; for (i = 0; i < size; i++) { + bool valid; + if (fmt[i].type != type) continue; - if (k == index) + valid = (type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE || + venus_helper_check_codec(inst, fmt[i].pixfmt)); + if (k == index && valid) break; - k++; + if (valid) + k++; } if (i == size) return NULL; - if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && - !venus_helper_check_codec(inst, fmt[i].pixfmt)) - return NULL; - return &fmt[i]; } diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c index e3a10a852cad..5eba4c7cd52e 100644 --- a/drivers/media/platform/qcom/venus/venc.c +++ b/drivers/media/platform/qcom/venus/venc.c @@ -120,11 +120,16 @@ find_format_by_index(struct venus_inst *inst, unsigned int index, u32 type) return NULL; for (i = 0; i < size; i++) { + bool valid; + if (fmt[i].type != type) continue; - if (k == index) + valid = (type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE || + venus_helper_check_codec(inst, fmt[i].pixfmt)); + if (k == index && valid) break; - k++; + if (valid) + k++; } if (i == size)