| Message ID | 1483395842-9675-1-git-send-email-martin@kaiser.cx (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, Jan 02, 2017 at 11:24:02PM +0100, Martin Kaiser wrote: > The current code calculates the number of color map entries as > 1 << info->var.bits_per_pixel. For 32bpp modes, 1 << 32 is 0 when > written to an int variable. As a consequence, the subsequent copying > of the default (non-empty) color map into our newly allocated color map > fails and imxfb's probe function returns an error. > > On both imx1 and imx21 platforms, the color map is used only for modes > with <= 8bpp. By allocating 256 entries for the color map, we're on the > safe side. > > Signed-off-by: Martin Kaiser <martin@kaiser.cx> Acked-by: Sascha Hauer <s.hauer@pengutronix.de> Sascha > --- > Re-sending: corrected a typo in the LKML address, sorry for that. > > drivers/video/fbdev/imxfb.c | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c > index fe0c4ee..1b0faad 100644 > --- a/drivers/video/fbdev/imxfb.c > +++ b/drivers/video/fbdev/imxfb.c > @@ -985,7 +985,11 @@ static int imxfb_probe(struct platform_device *pdev) > */ > imxfb_check_var(&info->var, info); > > - ret = fb_alloc_cmap(&info->cmap, 1 << info->var.bits_per_pixel, 0); > + /* > + * For modes > 8bpp, the color map is bypassed. > + * Therefore, 256 entries are enough. > + */ > + ret = fb_alloc_cmap(&info->cmap, 256, 0); > if (ret < 0) > goto failed_cmap; > > -- > 1.7.10.4 > > >
Hi, On Monday, January 09, 2017 01:59:42 PM Sascha Hauer wrote: > On Mon, Jan 02, 2017 at 11:24:02PM +0100, Martin Kaiser wrote: > > The current code calculates the number of color map entries as > > 1 << info->var.bits_per_pixel. For 32bpp modes, 1 << 32 is 0 when > > written to an int variable. As a consequence, the subsequent copying > > of the default (non-empty) color map into our newly allocated color map > > fails and imxfb's probe function returns an error. > > > > On both imx1 and imx21 platforms, the color map is used only for modes > > with <= 8bpp. By allocating 256 entries for the color map, we're on the > > safe side. > > > > Signed-off-by: Martin Kaiser <martin@kaiser.cx> > > Acked-by: Sascha Hauer <s.hauer@pengutronix.de> Thanks, patch queued for 4.11. Best regards, -- Bartlomiej Zolnierkiewicz Samsung R&D Institute Poland Samsung Electronics > Sascha > > > --- > > Re-sending: corrected a typo in the LKML address, sorry for that. > > > > drivers/video/fbdev/imxfb.c | 6 +++++- > > 1 file changed, 5 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c > > index fe0c4ee..1b0faad 100644 > > --- a/drivers/video/fbdev/imxfb.c > > +++ b/drivers/video/fbdev/imxfb.c > > @@ -985,7 +985,11 @@ static int imxfb_probe(struct platform_device *pdev) > > */ > > imxfb_check_var(&info->var, info); > > > > - ret = fb_alloc_cmap(&info->cmap, 1 << info->var.bits_per_pixel, 0); > > + /* > > + * For modes > 8bpp, the color map is bypassed. > > + * Therefore, 256 entries are enough. > > + */ > > + ret = fb_alloc_cmap(&info->cmap, 256, 0); > > if (ret < 0) > > goto failed_cmap; -- To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c index fe0c4ee..1b0faad 100644 --- a/drivers/video/fbdev/imxfb.c +++ b/drivers/video/fbdev/imxfb.c @@ -985,7 +985,11 @@ static int imxfb_probe(struct platform_device *pdev) */ imxfb_check_var(&info->var, info); - ret = fb_alloc_cmap(&info->cmap, 1 << info->var.bits_per_pixel, 0); + /* + * For modes > 8bpp, the color map is bypassed. + * Therefore, 256 entries are enough. + */ + ret = fb_alloc_cmap(&info->cmap, 256, 0); if (ret < 0) goto failed_cmap;
The current code calculates the number of color map entries as 1 << info->var.bits_per_pixel. For 32bpp modes, 1 << 32 is 0 when written to an int variable. As a consequence, the subsequent copying of the default (non-empty) color map into our newly allocated color map fails and imxfb's probe function returns an error. On both imx1 and imx21 platforms, the color map is used only for modes with <= 8bpp. By allocating 256 entries for the color map, we're on the safe side. Signed-off-by: Martin Kaiser <martin@kaiser.cx> --- Re-sending: corrected a typo in the LKML address, sorry for that. drivers/video/fbdev/imxfb.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)