From patchwork Mon Nov 25 09:58:10 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacek Anaszewski X-Patchwork-Id: 3228941 Return-Path: X-Original-To: patchwork-linux-media@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 73557C045B for ; Mon, 25 Nov 2013 09:59:05 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B5BF220221 for ; Mon, 25 Nov 2013 09:58:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 99DC0201BE for ; Mon, 25 Nov 2013 09:58:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751142Ab3KYJ6q (ORCPT ); Mon, 25 Nov 2013 04:58:46 -0500 Received: from mailout2.samsung.com ([203.254.224.25]:58478 "EHLO mailout2.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750739Ab3KYJ6o (ORCPT ); Mon, 25 Nov 2013 04:58:44 -0500 Received: from epcpsbgm1.samsung.com (epcpsbgm1 [203.254.230.26]) by mailout2.samsung.com (Oracle Communications Messaging Server 7u4-24.01(7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTP id <0MWT00JF0D1V3440@mailout2.samsung.com> for linux-media@vger.kernel.org; Mon, 25 Nov 2013 18:58:43 +0900 (KST) X-AuditID: cbfee61a-b7f316d000007e34-5d-52931f53b1fe Received: from epmmp2 ( [203.254.227.17]) by epcpsbgm1.samsung.com (EPCPMTA) with SMTP id 86.04.32308.35F13925; Mon, 25 Nov 2013 18:58:43 +0900 (KST) Received: from AMDC2362.DIGITAL.local ([106.120.53.23]) by mmp2.samsung.com (Oracle Communications Messaging Server 7u4-24.01 (7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTPA id <0MWT004BMD1ERF40@mmp2.samsung.com>; Mon, 25 Nov 2013 18:58:43 +0900 (KST) From: Jacek Anaszewski To: linux-media@vger.kernel.org Cc: sw0312.kim@samsung.com, andrzej.p@samsung.com, s.nawrocki@samsung.com, Jacek Anaszewski , Kyungmin Park Subject: [PATCH v2 03/16] s5p-jpeg: Fix erroneous condition while validating bytesperline value Date: Mon, 25 Nov 2013 10:58:10 +0100 Message-id: <1385373503-1657-4-git-send-email-j.anaszewski@samsung.com> X-Mailer: git-send-email 1.7.9.5 In-reply-to: <1385373503-1657-1-git-send-email-j.anaszewski@samsung.com> References: <1385373503-1657-1-git-send-email-j.anaszewski@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrPJMWRmVeSWpSXmKPExsVy+t9jQd1g+clBBjfm8FnMetnOYtF79Tmj xdmmN+wWPRu2slocftPOajFj8ks2BzaPvi2rGD0+b5ILYIrisklJzcksSy3St0vgyti1s5up 4AhnxboJXWwNjC/Yuxg5OSQETCR6b+5mgrDFJC7cW8/WxcjFISQwnVFi/ttHUE47k8Slm/fA OtgEDCV+vngN1iEiIC/xpPcGWBGzwFJGiYuTjoAVCQskSvy6+hrMZhFQlZj3+QYjiM0r4C4x resTcxcjB9A6BYk5k2xATE4BD4n5k1hBKoSAKpb82Mg+gZF3ASPDKkbR1ILkguKk9FxDveLE 3OLSvHS95PzcTYzgYHkmtYNxZYPFIUYBDkYlHl6LyklBQqyJZcWVuYcYJTiYlUR4T0tODhLi TUmsrEotyo8vKs1JLT7EKM3BoiTOe6DVOlBIID2xJDU7NbUgtQgmy8TBKdXA6CcbKTHDm+Fx Y5TwjWex7lP+OckeieiIDpoWZ8ws83CfzNy953xduk/bORssi9+wOUxn56u3QkcOPNya/jTg 128ppe2bTH98kz46XfS9/57nNwwdYxt+rDYI5qp6k/nB4yTfRfMi3k0v+n5e5T5/SuBL8lYu 5ZNaL233yM4Pf//6za1VVmwcjUosxRmJhlrMRcWJAI7YFfQSAgAA Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The aim of the condition is ensuring that the bytesperline value set by the user space application is proper for the given format and adjusting it if isn't. As the depth value of the format description entry is expressed in bits then the bytesperline value needs to be divided, not multiplied, by that value to get the number of bytes required to store single line of image samples. Signed-off-by: Jacek Anaszewski Signed-off-by: Kyungmin Park --- drivers/media/platform/s5p-jpeg/jpeg-core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/platform/s5p-jpeg/jpeg-core.c b/drivers/media/platform/s5p-jpeg/jpeg-core.c index 0f567c5..a1366f0 100644 --- a/drivers/media/platform/s5p-jpeg/jpeg-core.c +++ b/drivers/media/platform/s5p-jpeg/jpeg-core.c @@ -670,7 +670,7 @@ static int vidioc_try_fmt(struct v4l2_format *f, struct s5p_jpeg_fmt *fmt, bpl = pix->width; /* planar */ if (fmt->colplanes == 1 && /* packed */ - (bpl << 3) * fmt->depth < pix->width) + (bpl << 3) / fmt->depth < pix->width) bpl = (pix->width * fmt->depth) >> 3; pix->bytesperline = bpl;