From patchwork Mon Aug 20 18:09:33 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikulas Patocka X-Patchwork-Id: 1350461 Return-Path: X-Original-To: patchwork-linux-fbdev@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id A5D4A40210 for ; Mon, 20 Aug 2012 18:19:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751052Ab2HTSTN (ORCPT ); Mon, 20 Aug 2012 14:19:13 -0400 Received: from artax.karlin.mff.cuni.cz ([195.113.26.195]:39035 "EHLO artax.karlin.mff.cuni.cz" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750905Ab2HTSTM (ORCPT ); Mon, 20 Aug 2012 14:19:12 -0400 Received: by artax.karlin.mff.cuni.cz (Postfix, from userid 17421) id 5408A9806E; Mon, 20 Aug 2012 20:09:33 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by artax.karlin.mff.cuni.cz (Postfix) with ESMTP id 4FD4998018; Mon, 20 Aug 2012 20:09:33 +0200 (CEST) Date: Mon, 20 Aug 2012 20:09:33 +0200 (CEST) From: Mikulas Patocka To: Florian Tobias Schandinat , linux-fbdev@vger.kernel.org Subject: [PATCH 5/7] tgafb: fix mode setting with fbset In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) X-Personality-Disorder: Schizoid MIME-Version: 1.0 Sender: linux-fbdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org tgafb: fix mode setting with fbset Mode setting in the TGA driver is broken for these reasons: - info->fix.line_length is set just once in tgafb_init_fix function. If we change videomode, info->fix.line_length is not recalculated - so the video mode is changed but screen is corrupted because of wrong info->fix.line_length - info->fix.smem_len is set in tgafb_init_fix to the size of the default video mode (640x480). If we set a higher resolution, info->fix.smem_len is smaller than the current screen size, preventing the userspace program from mapping the framebuffer. This patch fixes it: - info->fix.line_length initialization is moved to tgafb_set_par so that it is recalculated with each mode change. - info->fix.smem_len is set to a fixed value representing the real amount of video ram (the values are taken from xfree86 driver). - add a check to tgafb_check_var to prevent us from setting a videomode that doesn't fit into videoram. - in tgafb_register, tgafb_init_fix is moved upwards, to be called before fb_find_mode (because fb_find_mode already needs the videoram size set in tgafb_init_fix). Signed-off-by: Mikulas Patocka Cc: stable@kernel.org --- drivers/video/tgafb.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux-3.4.4-fast/drivers/video/tgafb.c =================================================================== --- linux-3.4.4-fast.orig/drivers/video/tgafb.c 2012-07-02 02:15:43.000000000 +0200 +++ linux-3.4.4-fast/drivers/video/tgafb.c 2012-07-02 02:16:48.000000000 +0200 @@ -192,6 +192,8 @@ tgafb_check_var(struct fb_var_screeninfo if (var->xres_virtual != var->xres || var->yres_virtual != var->yres) return -EINVAL; + if (var->xres * var->yres * (var->bits_per_pixel >> 3) > info->fix.smem_len) + return -EINVAL; if (var->nonstd) return -EINVAL; if (1000000000 / var->pixclock > TGA_PLL_MAX_FREQ) @@ -272,6 +274,7 @@ tgafb_set_par(struct fb_info *info) par->yres = info->var.yres; par->pll_freq = pll_freq = 1000000000 / info->var.pixclock; par->bits_per_pixel = info->var.bits_per_pixel; + info->fix.line_length = par->xres * (par->bits_per_pixel >> 3); tga_type = par->tga_type; @@ -1480,6 +1483,7 @@ tgafb_init_fix(struct fb_info *info) int tga_bus_tc = TGA_BUS_TC(par->dev); u8 tga_type = par->tga_type; const char *tga_type_name = NULL; + unsigned memory_size; switch (tga_type) { case TGA_TYPE_8PLANE: @@ -1487,21 +1491,25 @@ tgafb_init_fix(struct fb_info *info) tga_type_name = "Digital ZLXp-E1"; if (tga_bus_tc) tga_type_name = "Digital ZLX-E1"; + memory_size = 2097152; break; case TGA_TYPE_24PLANE: if (tga_bus_pci) tga_type_name = "Digital ZLXp-E2"; if (tga_bus_tc) tga_type_name = "Digital ZLX-E2"; + memory_size = 8388608; break; case TGA_TYPE_24PLUSZ: if (tga_bus_pci) tga_type_name = "Digital ZLXp-E3"; if (tga_bus_tc) tga_type_name = "Digital ZLX-E3"; + memory_size = 16777216; break; default: tga_type_name = "Unknown"; + memory_size = 16777216; break; } @@ -1513,9 +1521,8 @@ tgafb_init_fix(struct fb_info *info) ? FB_VISUAL_PSEUDOCOLOR : FB_VISUAL_DIRECTCOLOR); - info->fix.line_length = par->xres * (par->bits_per_pixel >> 3); info->fix.smem_start = (size_t) par->tga_fb_base; - info->fix.smem_len = info->fix.line_length * par->yres; + info->fix.smem_len = memory_size; info->fix.mmio_start = (size_t) par->tga_regs_base; info->fix.mmio_len = 512; @@ -1640,6 +1647,9 @@ tgafb_register(struct device *dev) modedb_tga = &modedb_tc; modedbsize_tga = 1; } + + tgafb_init_fix(info); + ret = fb_find_mode(&info->var, info, mode_option ? mode_option : mode_option_tga, modedb_tga, modedbsize_tga, NULL, @@ -1657,7 +1667,6 @@ tgafb_register(struct device *dev) } tgafb_set_par(info); - tgafb_init_fix(info); if (register_framebuffer(info) < 0) { printk(KERN_ERR "tgafb: Could not register framebuffer\n");