| Message ID | 407e067b-47be-e8da-848d-edb6c04f5c1c@xs4all.nl (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | vivid: fix kernel oops when enabling HFLIP and OSD | expand |
On 10/08/2018 09:08 PM, Hans Verkuil wrote: > When the OSD is on (i.e. vivid displays text on top of the test pattern), and > you enable hflip, then the driver crashes. > > The cause turned out to be a division of a negative number by an unsigned value. > You expect that -8 / 2 would be -4, but in reality it is 2147483644 :-( > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > Reported-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> > --- > diff --git a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c > index f3d9c1140ffa..e76f87dc4368 100644 > --- a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c > +++ b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c > @@ -1773,7 +1773,7 @@ typedef struct { u16 __; u8 _; } __packed x24; > pos[7] = (chr & (0x01 << 0) ? fg : bg); \ > } \ > \ > - pos += (tpg->hflip ? -8 : 8) / hdiv; \ > + pos += (tpg->hflip ? -8 : 8) / (int)hdiv; \ > } \ > } \ > } while (0) > This can be CC-ed to stable for 4.7 and up. It actually broke in 4.1, but it was called vivid-tpg.c at that time. Regards, Hans
On 10/08/2018 09:08 PM, Hans Verkuil wrote: > When the OSD is on (i.e. vivid displays text on top of the test pattern), and > you enable hflip, then the driver crashes. > > The cause turned out to be a division of a negative number by an unsigned value. > You expect that -8 / 2 would be -4, but in reality it is 2147483644 :-( > > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> > Reported-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> > --- > diff --git a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c > index f3d9c1140ffa..e76f87dc4368 100644 > --- a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c > +++ b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c > @@ -1773,7 +1773,7 @@ typedef struct { u16 __; u8 _; } __packed x24; > pos[7] = (chr & (0x01 << 0) ? fg : bg); \ > } \ > \ > - pos += (tpg->hflip ? -8 : 8) / hdiv; \ > + pos += (tpg->hflip ? -8 : 8) / (int)hdiv; \ > } \ > } \ > } while (0) > Fixes: 3e14e7a82c1ef ("vivid-tpg: add hor/vert downsampling support to tpg_gen_text") Regards, Hans
diff --git a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c index f3d9c1140ffa..e76f87dc4368 100644 --- a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c +++ b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c @@ -1773,7 +1773,7 @@ typedef struct { u16 __; u8 _; } __packed x24; pos[7] = (chr & (0x01 << 0) ? fg : bg); \ } \ \ - pos += (tpg->hflip ? -8 : 8) / hdiv; \ + pos += (tpg->hflip ? -8 : 8) / (int)hdiv; \ } \ } \ } while (0)
When the OSD is on (i.e. vivid displays text on top of the test pattern), and you enable hflip, then the driver crashes. The cause turned out to be a division of a negative number by an unsigned value. You expect that -8 / 2 would be -4, but in reality it is 2147483644 :-( Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> Reported-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> ---