diff mbox

fbcon: fix race condition between console lock and cursor timer

Message ID 1345531207-24926-1-git-send-email-airlied@redhat.com (mailing list archive)
State New, archived
Headers show

Commit Message

David Airlie Aug. 21, 2012, 6:40 a.m. UTC
So we've had a fair few reports of fbcon handover breakage between
efi/vesafb and i915 surface recently, so I dedicated a couple of
days to finding the problem.

Essentially the last thing we saw was the conflicting framebuffer
message and that was all.

So after much tracing with direct netconsole writes (printks
under console_lock not so useful), I think I found the race.

Thread A (driver load)    Thread B (timer thread)
  unbind_con_driver ->              |
  bind_con_driver ->                |
  vc->vc_sw->con_deinit ->          |
  fbcon_deinit ->                   |
  console_lock()                    |
      |                             |
      |                       fbcon_flashcursor timer fires
      |                       console_lock() <- blocked for A
      |
      |
fbcon_del_cursor_timer ->
  del_timer_sync
  (BOOM)

Of course because all of this is under the console lock,
we never see anything, also since we also just unbound the active
console guess what we never see anything.

Hopefully this fixes the problem for anyone seeing vesafb->kms
driver handoff.

Signed-off-by: David Airlie <airlied@redhat.com>
---
 drivers/video/console/fbcon.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Comments

Alan Cox Aug. 21, 2012, 9:15 a.m. UTC | #1
> So after much tracing with direct netconsole writes (printks
> under console_lock not so useful), I think I found the race.

Direct netconsole write would be a useful patch to have mainline I think
8)

> Hopefully this fixes the problem for anyone seeing vesafb->kms
> driver handoff.

Not really the proper fix but its clear and is probably the best thing to
go in initially with a cc: stable. Can you at least stick a large 

+ /* FIXME: we should sort out the unbind locking instead */

on the patch however.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
--
Peter Zijlstra Aug. 21, 2012, 12:14 p.m. UTC | #2
On Tue, 2012-08-21 at 16:40 +1000, Dave Airlie wrote:
> So after much tracing with direct netconsole writes (printks
> under console_lock not so useful) 

I always use earlyprintk on serial.. 

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
--
Peter Zijlstra Aug. 21, 2012, 12:15 p.m. UTC | #3
On Tue, 2012-08-21 at 10:15 +0100, Alan Cox wrote:
> > So after much tracing with direct netconsole writes (printks
> > under console_lock not so useful), I think I found the race.
> 
> Direct netconsole write would be a useful patch to have mainline I think
> 8) 

could we make that use the earlyprintk infrastructure?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
--
Josh Boyer Aug. 21, 2012, 1:19 p.m. UTC | #4
On Tue, Aug 21, 2012 at 2:40 AM, Dave Airlie <airlied@redhat.com> wrote:
> So we've had a fair few reports of fbcon handover breakage between
> efi/vesafb and i915 surface recently, so I dedicated a couple of
> days to finding the problem.
>
> Essentially the last thing we saw was the conflicting framebuffer
> message and that was all.
>
> So after much tracing with direct netconsole writes (printks
> under console_lock not so useful), I think I found the race.
>
> Thread A (driver load)    Thread B (timer thread)
>   unbind_con_driver ->              |
>   bind_con_driver ->                |
>   vc->vc_sw->con_deinit ->          |
>   fbcon_deinit ->                   |
>   console_lock()                    |
>       |                             |
>       |                       fbcon_flashcursor timer fires
>       |                       console_lock() <- blocked for A
>       |
>       |
> fbcon_del_cursor_timer ->
>   del_timer_sync
>   (BOOM)
>
> Of course because all of this is under the console lock,
> we never see anything, also since we also just unbound the active
> console guess what we never see anything.
>
> Hopefully this fixes the problem for anyone seeing vesafb->kms
> driver handoff.
>
> Signed-off-by: David Airlie <airlied@redhat.com>
> ---
>  drivers/video/console/fbcon.c |    6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/video/console/fbcon.c b/drivers/video/console/fbcon.c
> index 2e471c2..f8a79fc 100644
> --- a/drivers/video/console/fbcon.c
> +++ b/drivers/video/console/fbcon.c
> @@ -372,8 +372,12 @@ static void fb_flashcursor(struct work_struct *work)
>         struct vc_data *vc = NULL;
>         int c;
>         int mode;
> +       int ret;
> +
> +       ret = console_trylock();
> +       if (ret == 0)
> +               return;
>
> -       console_lock();
>         if (ops && ops->currcon != -1)
>                 vc = vc_cons[ops->currcon].d;
>

I have a Dell XPS 8300 machine with a Radeon card in it that started
showing this problem yesterday with 3.6-rc2 kernels.  I tested this
patch on top of v3.6-rc2-206-g10c63c9 this morning and the problem
seems to have been cleared up for me.  That includes making sure the
grub2 file has the gfxterm set, etc.

I know we've been seeing this quite a bit more on Fedora 17, so we'll
want to have some people test a 3.5 build with it but things are
looking better.

josh

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
--
Dave Airlie Aug. 22, 2012, 4 a.m. UTC | #5
On Tue, Aug 21, 2012 at 7:15 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> So after much tracing with direct netconsole writes (printks
>> under console_lock not so useful), I think I found the race.
>
> Direct netconsole write would be a useful patch to have mainline I think
> 8)

Well I used a one line wrapper around the netconsole write_msg, which
just passed
NULL as the first arg, then sprinkled netconsole_write_msg around the
place, not having
printf stuff could be an annoyance for some people, for this it didn't matter.

Peter I wish I had a serial port to work with :-)

>
> Not really the proper fix but its clear and is probably the best thing to
> go in initially with a cc: stable. Can you at least stick a large
>
> + /* FIXME: we should sort out the unbind locking instead */

Done, and cc stable, I'll send this to Linus via my tree as its fairly
urgent from my pov.

Dave.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
--
diff mbox

Patch

diff --git a/drivers/video/console/fbcon.c b/drivers/video/console/fbcon.c
index 2e471c2..f8a79fc 100644
--- a/drivers/video/console/fbcon.c
+++ b/drivers/video/console/fbcon.c
@@ -372,8 +372,12 @@  static void fb_flashcursor(struct work_struct *work)
 	struct vc_data *vc = NULL;
 	int c;
 	int mode;
+	int ret;
+
+	ret = console_trylock();
+	if (ret == 0)
+		return;
 
-	console_lock();
 	if (ops && ops->currcon != -1)
 		vc = vc_cons[ops->currcon].d;