From patchwork Sat May 2 17:23:06 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prashant Laddha X-Patchwork-Id: 6324811 Return-Path: X-Original-To: patchwork-linux-media@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 25ADBBEEE1 for ; Mon, 4 May 2015 10:01:29 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 4876A202E6 for ; Mon, 4 May 2015 10:01:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4A9BB202C8 for ; Mon, 4 May 2015 10:01:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751615AbbEDKBY (ORCPT ); Mon, 4 May 2015 06:01:24 -0400 Received: from rcdn-iport-3.cisco.com ([173.37.86.74]:54105 "EHLO rcdn-iport-3.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750715AbbEDKBX (ORCPT ); Mon, 4 May 2015 06:01:23 -0400 X-Greylist: delayed 574 seconds by postgrey-1.27 at vger.kernel.org; Mon, 04 May 2015 06:01:23 EDT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1928; q=dns/txt; s=iport; t=1430733684; x=1431943284; h=from:to:cc:subject:date:message-id; bh=MYwcUfIZU+9DwOH1uCViz7n+4QvPY5jgYUz8Fztmqjg=; b=ByVvQGDGcQ+t4Jrr7Zm3wgCGzCEF4tURR1SnmYpI//TzlO5GiNzZ9ZPv i1WqQx31scVgzIjSw4FzRd9noxr/0996Qtw7RyYNIs+truCYDbSTkSLRO GFCrumH8c59VEFDVKot6ciAv+9eJVuJKo9vREwITw0FmIGU+QlsA17s1N Y=; X-IronPort-AV: E=Sophos;i="5.13,365,1427760000"; d="scan'208";a="416825242" Received: from rcdn-core-9.cisco.com ([173.37.93.145]) by rcdn-iport-3.cisco.com with ESMTP; 04 May 2015 09:51:48 +0000 Received: from pla-VB.cisco.com ([10.142.186.87]) by rcdn-core-9.cisco.com (8.14.5/8.14.5) with ESMTP id t449pi1t000814; Mon, 4 May 2015 09:51:45 GMT From: Prashant Laddha To: linux-media@vger.kernel.org Cc: Hans Verkuil , Martin Bugge , Prashant Laddha Subject: [PATCH] v4l2-dv-timings: fix overflow in gtf timings calculation Date: Sat, 2 May 2015 22:53:06 +0530 Message-Id: <1430587386-28409-1-git-send-email-prladdha@cisco.com> X-Mailer: git-send-email 1.9.1 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DATE_IN_PAST_24_48, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The intermediate calculation in the expression for hblank can exceed 32 bit signed range. This overflow can lead to negative values for hblank. Typecasting intermediate variable to higher precision. Cc: Hans Verkuil Cc: Martin Bugge Signed-off-by: Prashant Laddha --- drivers/media/v4l2-core/v4l2-dv-timings.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-dv-timings.c b/drivers/media/v4l2-core/v4l2-dv-timings.c index 86e11d1..9d27f05 100644 --- a/drivers/media/v4l2-core/v4l2-dv-timings.c +++ b/drivers/media/v4l2-core/v4l2-dv-timings.c @@ -573,15 +573,15 @@ bool v4l2_detect_gtf(unsigned frame_height, /* Horizontal */ if (default_gtf) - h_blank = ((image_width * GTF_D_C_PRIME * hfreq) - - (image_width * GTF_D_M_PRIME * 1000) + - (hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000) / 2) / - (hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000); + h_blank = ((image_width * GTF_D_C_PRIME * (long long)hfreq) - + ((long long)image_width * GTF_D_M_PRIME * 1000) + + ((long long)hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000) / 2) / + ((long long)hfreq * (100 - GTF_D_C_PRIME) + GTF_D_M_PRIME * 1000); else - h_blank = ((image_width * GTF_S_C_PRIME * hfreq) - - (image_width * GTF_S_M_PRIME * 1000) + - (hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000) / 2) / - (hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000); + h_blank = ((image_width * GTF_S_C_PRIME * (long long)hfreq) - + ((long long)image_width * GTF_S_M_PRIME * 1000) + + ((long long)hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000) / 2) / + ((long long)hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000); h_blank = ((h_blank + GTF_CELL_GRAN) / (2 * GTF_CELL_GRAN)) * (2 * GTF_CELL_GRAN);