From patchwork Thu Jun 16 20:14:09 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vaibhav Hiremath X-Patchwork-Id: 889152 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter2.kernel.org (8.14.4/8.14.4) with ESMTP id p5GKFrvS023938 for ; Thu, 16 Jun 2011 20:15:54 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932742Ab1FPUOX (ORCPT ); Thu, 16 Jun 2011 16:14:23 -0400 Received: from devils.ext.ti.com ([198.47.26.153]:55656 "EHLO devils.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932722Ab1FPUOU (ORCPT ); Thu, 16 Jun 2011 16:14:20 -0400 Received: from dbdp20.itg.ti.com ([172.24.170.38]) by devils.ext.ti.com (8.13.7/8.13.7) with ESMTP id p5GKEGZ7021432 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 16 Jun 2011 15:14:18 -0500 Received: from dbde71.ent.ti.com (localhost [127.0.0.1]) by dbdp20.itg.ti.com (8.13.8/8.13.8) with ESMTP id p5GKEEux022146; Fri, 17 Jun 2011 01:44:15 +0530 (IST) Received: from dbdp31.itg.ti.com (172.24.170.98) by DBDE71.ent.ti.com (172.24.170.149) with Microsoft SMTP Server id 8.3.106.1; Fri, 17 Jun 2011 01:44:14 +0530 Received: from localhost.localdomain (dbdp20.itg.ti.com [172.24.170.38]) by dbdp31.itg.ti.com (8.13.8/8.13.8) with ESMTP id p5GKE9JC001290; Fri, 17 Jun 2011 01:44:11 +0530 (IST) From: To: CC: , , Vaibhav Hiremath Subject: [PATCH] omap_vout: Added check in reqbuf & mmap for buf_size allocation Date: Fri, 17 Jun 2011 01:44:09 +0530 Message-ID: <1308255249-18762-1-git-send-email-hvaibhav@ti.com> X-Mailer: git-send-email 1.6.2.4 In-Reply-To: References: MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter2.kernel.org [140.211.167.43]); Thu, 16 Jun 2011 20:15:54 +0000 (UTC) From: Vaibhav Hiremath The usecase where, user allocates small size of buffer through bootargs (video1_bufsize/video2_bufsize) and later from application tries to set the format which requires larger buffer size, driver doesn't check for insufficient buffer size and allows application to map extra buffer. This leads to kernel crash, when user application tries to access memory beyond the allocation size. Added check in both mmap and reqbuf call back function, and return error if the size of the buffer allocated by user through bootargs is less than the S_FMT size. Signed-off-by: Vaibhav Hiremath --- drivers/media/video/omap/omap_vout.c | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-) -- 1.6.2.4 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/media/video/omap/omap_vout.c b/drivers/media/video/omap/omap_vout.c index 3bc909a..343b50c 100644 --- a/drivers/media/video/omap/omap_vout.c +++ b/drivers/media/video/omap/omap_vout.c @@ -678,6 +678,14 @@ static int omap_vout_buffer_setup(struct videobuf_queue *q, unsigned int *count, startindex = (vout->vid == OMAP_VIDEO1) ? video1_numbuffers : video2_numbuffers; + /* Check the size of the buffer */ + if (*size > vout->buffer_size) { + v4l2_err(&vout->vid_dev->v4l2_dev, + "buffer allocation mismatch [%u] [%u]\n", + *size, vout->buffer_size); + return -ENOMEM; + } + for (i = startindex; i < *count; i++) { vout->buffer_size = *size; @@ -856,6 +864,14 @@ static int omap_vout_mmap(struct file *file, struct vm_area_struct *vma) (vma->vm_pgoff << PAGE_SHIFT)); return -EINVAL; } + /* Check the size of the buffer */ + if (size > vout->buffer_size) { + v4l2_err(&vout->vid_dev->v4l2_dev, + "insufficient memory [%lu] [%u]\n", + size, vout->buffer_size); + return -ENOMEM; + } + q->bufs[i]->baddr = vma->vm_start; vma->vm_flags |= VM_RESERVED;