From patchwork Fri Sep 30 21:16:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Javier Martinez Canillas X-Patchwork-Id: 9358855 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 679246086A for ; Fri, 30 Sep 2016 21:17:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5267C2A18E for ; Fri, 30 Sep 2016 21:17:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 46E502A196; Fri, 30 Sep 2016 21:17:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB8102A195 for ; Fri, 30 Sep 2016 21:17:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933618AbcI3VRg (ORCPT ); Fri, 30 Sep 2016 17:17:36 -0400 Received: from ec2-52-27-115-49.us-west-2.compute.amazonaws.com ([52.27.115.49]:40562 "EHLO s-opensource.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933417AbcI3VRe (ORCPT ); Fri, 30 Sep 2016 17:17:34 -0400 Received: from localhost (localhost [127.0.0.1]) by s-opensource.com (Postfix) with ESMTP id D09F4A0E78; Fri, 30 Sep 2016 21:17:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osg.samsung.com Received: from s-opensource.com ([127.0.0.1]) by localhost (s-opensource.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kmBCTCIfScbX; Fri, 30 Sep 2016 21:17:39 +0000 (UTC) Received: from minerva.localdomain (unknown [181.124.104.51]) by s-opensource.com (Postfix) with ESMTPSA id 649C4A0E61; Fri, 30 Sep 2016 21:17:34 +0000 (UTC) From: Javier Martinez Canillas To: linux-kernel@vger.kernel.org Cc: Mauro Carvalho Chehab , Marek Szyprowski , Krzysztof Kozlowski , Kukjin Kim , Hans Verkuil , Inki Dae , linux-samsung-soc@vger.kernel.org, Sylwester Nawrocki , Shuah Khan , Nicolas Dufresne , linux-media@vger.kernel.org, Javier Martinez Canillas Subject: [PATCH 4/4] [media] exynos-gsc: do proper bytesperline and sizeimage calculation Date: Fri, 30 Sep 2016 17:16:44 -0400 Message-Id: <1475270204-14005-5-git-send-email-javier@osg.samsung.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1475270204-14005-1-git-send-email-javier@osg.samsung.com> References: <1475270204-14005-1-git-send-email-javier@osg.samsung.com> Sender: linux-samsung-soc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-samsung-soc@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The driver don't take into account the differences between packed, semi planar and multi planar formats when calculating the pixel format bytes per lines and image size values. This makes GStreamer to fail when the following formats are used NV12, NV21, NV16, NV61, YV12, I420 and Y42B: "gst_video_frame_map_id: failed to map video frame plane 1" Nicolas suggested to use the logic found in the Exynos FIMC v4l2 driver since does this correctly. So this patch changes the bytes per line and image size calculation according to what's done in this media driver. After this patch most supported formats work correctly. There are still issues with the NV21 and NV61 formats, but that seems to be a separate problem since NV12 and NV16 work and these formats use the same values. So this can be fixed as a follow-up and shouldn't be a blocker for this change that improves the driver's support. Suggested-by: Nicolas Dufresne Signed-off-by: Javier Martinez Canillas --- drivers/media/platform/exynos-gsc/gsc-core.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c index 8bb1d2be7234..a6c47deba3b7 100644 --- a/drivers/media/platform/exynos-gsc/gsc-core.c +++ b/drivers/media/platform/exynos-gsc/gsc-core.c @@ -451,12 +451,25 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) else /* SD */ pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; - for (i = 0; i < pix_mp->num_planes; ++i) { - int bpl = (pix_mp->width * fmt->depth[i]) >> 3; - pix_mp->plane_fmt[i].bytesperline = bpl; - pix_mp->plane_fmt[i].sizeimage = bpl * pix_mp->height; + struct v4l2_plane_pix_format *plane_fmt = &pix_mp->plane_fmt[i]; + u32 bpl = plane_fmt->bytesperline; + + if (fmt->num_comp == 1 && /* Packed */ + (bpl == 0 || (bpl * 8 / fmt->depth[i]) < pix_mp->width)) + bpl = pix_mp->width * fmt->depth[i] / 8; + + if (fmt->num_comp > 1 && /* Planar */ + (bpl == 0 || bpl < pix_mp->width)) + bpl = pix_mp->width; + + if (i != 0 && fmt->num_comp == 3) + bpl /= 2; + plane_fmt->bytesperline = bpl; + plane_fmt->sizeimage = max(pix_mp->width * pix_mp->height * + fmt->depth[i] / 8, + plane_fmt->sizeimage); pr_debug("[%d]: bpl: %d, sizeimage: %d", i, bpl, pix_mp->plane_fmt[i].sizeimage); }