From patchwork Wed Feb 7 00:04:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gustavo A. R. Silva" X-Patchwork-Id: 10204355 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 108406020F for ; Wed, 7 Feb 2018 00:25:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F11D228D85 for ; Wed, 7 Feb 2018 00:25:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E36FE28D87; Wed, 7 Feb 2018 00:25:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B5FC28D85 for ; Wed, 7 Feb 2018 00:25:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932166AbeBGAZJ (ORCPT ); Tue, 6 Feb 2018 19:25:09 -0500 Received: from gateway21.websitewelcome.com ([192.185.45.191]:39235 "EHLO gateway21.websitewelcome.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932106AbeBGAZI (ORCPT ); Tue, 6 Feb 2018 19:25:08 -0500 X-Greylist: delayed 1242 seconds by postgrey-1.27 at vger.kernel.org; Tue, 06 Feb 2018 19:25:08 EST Received: from cm15.websitewelcome.com (cm15.websitewelcome.com [100.42.49.9]) by gateway21.websitewelcome.com (Postfix) with ESMTP id 27ADA400E07A4 for ; Tue, 6 Feb 2018 18:04:26 -0600 (CST) Received: from gator4166.hostgator.com ([108.167.133.22]) by cmsmtp with SMTP id jDDueuTAPmzEzjDDuexCap; Tue, 06 Feb 2018 18:04:26 -0600 Received: from [189.175.4.238] (port=39266 helo=embeddedgus) by gator4166.hostgator.com with esmtpa (Exim 4.89_1) (envelope-from ) id 1ejDDt-0032dU-PJ; Tue, 06 Feb 2018 18:04:25 -0600 Date: Tue, 6 Feb 2018 18:04:24 -0600 From: "Gustavo A. R. Silva" To: Bartlomiej Zolnierkiewicz Cc: dri-devel@lists.freedesktop.org, linux-fbdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Gustavo A. R. Silva" Subject: [PATCH] video: fbdev: vermilion: use 64-bit arithmetic instead of 32-bit Message-ID: <20180207000424.GA32680@embeddedgus> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - gator4166.hostgator.com X-AntiAbuse: Original Domain - vger.kernel.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - embeddedor.com X-BWhitelist: no X-Source-IP: 189.175.4.238 X-Source-L: No X-Exim-ID: 1ejDDt-0032dU-PJ X-Source: X-Source-Args: X-Source-Dir: X-Source-Sender: (embeddedgus) [189.175.4.238]:39266 X-Source-Auth: gustavo@embeddedor.com X-Email-Count: 3 X-Source-Cap: Z3V6aWRpbmU7Z3V6aWRpbmU7Z2F0b3I0MTY2Lmhvc3RnYXRvci5jb20= X-Local-Domain: yes Sender: linux-fbdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Cast _pitch_ to u64 in order to give the compiler complete information about the proper arithmetic to use. Notice that this variable is being used in a context that expects an expression of type u64 (64 bits, unsigned). The expression pitch * var->yres_virtual is currently being evaluated using 32-bit arithmetic and the result of the operation is being stored into variable mem, which is a variable of type u64. Based on that, chances are there is a potential integer overflow as a result of the operation. Addresses-Coverity-ID: 200655 ("Unintentional integer overflow") Signed-off-by: Gustavo A. R. Silva --- drivers/video/fbdev/vermilion/vermilion.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/video/fbdev/vermilion/vermilion.c b/drivers/video/fbdev/vermilion/vermilion.c index 6f8d444..5172fa5 100644 --- a/drivers/video/fbdev/vermilion/vermilion.c +++ b/drivers/video/fbdev/vermilion/vermilion.c @@ -651,7 +651,7 @@ static int vmlfb_check_var_locked(struct fb_var_screeninfo *var, } pitch = ALIGN((var->xres * var->bits_per_pixel) >> 3, 0x40); - mem = pitch * var->yres_virtual; + mem = (u64)pitch * var->yres_virtual; if (mem > vinfo->vram_contig_size) { return -ENOMEM; }