Message ID | 1512724322-11678-1-git-send-email-wanpeng.li@hotmail.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
On 08.12.2017 10:12, Wanpeng Li wrote: > From: Wanpeng Li <wanpeng.li@hotmail.com> > > Reported by syzkaller: > > WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250 > CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16 > RIP: 0010:do_debug+0x222/0x250 > Call Trace: > <#DB> > debug+0x3e/0x70 > RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20 > </#DB> > _copy_from_user+0x5b/0x90 > SyS_timer_create+0x33/0x80 > entry_SYSCALL_64_fastpath+0x23/0x9a > > The syzkaller will mmap a buffer which is also the struct sigevent parameter of > timer_create(), it will also call perf_event_open() to set a BP for the buffer, > so when the implementation of timer_create() in kernel tries to get the struct > sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller > testcase also sets the debug registers for the guest, however, the kvm just > restores host debug registers when we have active breakpoints. I can observe > the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print > when running the testcase heavy multithreading. The do_debug() which is triggered > by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)). > > This patch fixes it by restoring host dr6 unconditionally before preempt/irq > enable. > > Reported-by: Dmitry Vyukov <dvyukov@google.com> > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Radim Krčmář <rkrcmar@redhat.com> > Cc: David Hildenbrand <david@redhat.com> > Cc: Dmitry Vyukov <dvyukov@google.com> > Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> > --- > arch/x86/kvm/x86.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 0c5d55c..a6370fd 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) > */ > if (hw_breakpoint_active()) > hw_breakpoint_restore(); > + else > + set_debugreg(current->thread.debugreg6, 6); > > vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); > > If you haven't seen it, I analyzed this in https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to my suggestion/question.
2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>: > On 08.12.2017 10:12, Wanpeng Li wrote: >> From: Wanpeng Li <wanpeng.li@hotmail.com> >> >> Reported by syzkaller: >> >> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250 >> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16 >> RIP: 0010:do_debug+0x222/0x250 >> Call Trace: >> <#DB> >> debug+0x3e/0x70 >> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20 >> </#DB> >> _copy_from_user+0x5b/0x90 >> SyS_timer_create+0x33/0x80 >> entry_SYSCALL_64_fastpath+0x23/0x9a >> >> The syzkaller will mmap a buffer which is also the struct sigevent parameter of >> timer_create(), it will also call perf_event_open() to set a BP for the buffer, >> so when the implementation of timer_create() in kernel tries to get the struct >> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller >> testcase also sets the debug registers for the guest, however, the kvm just >> restores host debug registers when we have active breakpoints. I can observe >> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print >> when running the testcase heavy multithreading. The do_debug() which is triggered >> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)). >> >> This patch fixes it by restoring host dr6 unconditionally before preempt/irq >> enable. >> >> Reported-by: Dmitry Vyukov <dvyukov@google.com> >> Cc: Paolo Bonzini <pbonzini@redhat.com> >> Cc: Radim Krčmář <rkrcmar@redhat.com> >> Cc: David Hildenbrand <david@redhat.com> >> Cc: Dmitry Vyukov <dvyukov@google.com> >> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> >> --- >> arch/x86/kvm/x86.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >> index 0c5d55c..a6370fd 100644 >> --- a/arch/x86/kvm/x86.c >> +++ b/arch/x86/kvm/x86.c >> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) >> */ >> if (hw_breakpoint_active()) >> hw_breakpoint_restore(); >> + else >> + set_debugreg(current->thread.debugreg6, 6); >> >> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); >> >> > > If you haven't seen it, I analyzed this in > https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to > my suggestion/question. I think it's fine to restore dr6 before preempt/irq enable. Regards, Wanpeng Li
On 10.12.2017 01:44, Wanpeng Li wrote: > 2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>: >> On 08.12.2017 10:12, Wanpeng Li wrote: >>> From: Wanpeng Li <wanpeng.li@hotmail.com> >>> >>> Reported by syzkaller: >>> >>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250 >>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16 >>> RIP: 0010:do_debug+0x222/0x250 >>> Call Trace: >>> <#DB> >>> debug+0x3e/0x70 >>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20 >>> </#DB> >>> _copy_from_user+0x5b/0x90 >>> SyS_timer_create+0x33/0x80 >>> entry_SYSCALL_64_fastpath+0x23/0x9a >>> >>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of >>> timer_create(), it will also call perf_event_open() to set a BP for the buffer, >>> so when the implementation of timer_create() in kernel tries to get the struct >>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller >>> testcase also sets the debug registers for the guest, however, the kvm just >>> restores host debug registers when we have active breakpoints. I can observe >>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print >>> when running the testcase heavy multithreading. The do_debug() which is triggered >>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)). >>> >>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq >>> enable. >>> >>> Reported-by: Dmitry Vyukov <dvyukov@google.com> >>> Cc: Paolo Bonzini <pbonzini@redhat.com> >>> Cc: Radim Krčmář <rkrcmar@redhat.com> >>> Cc: David Hildenbrand <david@redhat.com> >>> Cc: Dmitry Vyukov <dvyukov@google.com> >>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> >>> --- >>> arch/x86/kvm/x86.c | 2 ++ >>> 1 file changed, 2 insertions(+) >>> >>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >>> index 0c5d55c..a6370fd 100644 >>> --- a/arch/x86/kvm/x86.c >>> +++ b/arch/x86/kvm/x86.c >>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) >>> */ >>> if (hw_breakpoint_active()) >>> hw_breakpoint_restore(); >>> + else >>> + set_debugreg(current->thread.debugreg6, 6); >>> >>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); >>> >>> >> >> If you haven't seen it, I analyzed this in >> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to >> my suggestion/question. > > I think it's fine to restore dr6 before preempt/irq enable. That make sense, as I assume this is the first time that a trap would be delivered. Reviewed-by: David Hildenbrand <david@redhat.com> And certainly stable material? > > Regards, > Wanpeng Li >
On 10/12/2017 01:44, Wanpeng Li wrote: > 2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>: >> On 08.12.2017 10:12, Wanpeng Li wrote: >>> From: Wanpeng Li <wanpeng.li@hotmail.com> >>> >>> Reported by syzkaller: >>> >>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250 >>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16 >>> RIP: 0010:do_debug+0x222/0x250 >>> Call Trace: >>> <#DB> >>> debug+0x3e/0x70 >>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20 >>> </#DB> >>> _copy_from_user+0x5b/0x90 >>> SyS_timer_create+0x33/0x80 >>> entry_SYSCALL_64_fastpath+0x23/0x9a >>> >>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of >>> timer_create(), it will also call perf_event_open() to set a BP for the buffer, >>> so when the implementation of timer_create() in kernel tries to get the struct >>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller >>> testcase also sets the debug registers for the guest, however, the kvm just >>> restores host debug registers when we have active breakpoints. I can observe >>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print >>> when running the testcase heavy multithreading. The do_debug() which is triggered >>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)). >>> >>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq >>> enable. >>> >>> Reported-by: Dmitry Vyukov <dvyukov@google.com> >>> Cc: Paolo Bonzini <pbonzini@redhat.com> >>> Cc: Radim Krčmář <rkrcmar@redhat.com> >>> Cc: David Hildenbrand <david@redhat.com> >>> Cc: Dmitry Vyukov <dvyukov@google.com> >>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> >>> --- >>> arch/x86/kvm/x86.c | 2 ++ >>> 1 file changed, 2 insertions(+) >>> >>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >>> index 0c5d55c..a6370fd 100644 >>> --- a/arch/x86/kvm/x86.c >>> +++ b/arch/x86/kvm/x86.c >>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) >>> */ >>> if (hw_breakpoint_active()) >>> hw_breakpoint_restore(); >>> + else >>> + set_debugreg(current->thread.debugreg6, 6); >>> >>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); >>> >>> >> >> If you haven't seen it, I analyzed this in >> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to >> my suggestion/question. > > I think it's fine to restore dr6 before preempt/irq enable. If no breakpoint is active, you should be able to restore it only in sched_out. Paolo
2017-12-12 23:08 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>: > On 10/12/2017 01:44, Wanpeng Li wrote: >> 2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>: >>> On 08.12.2017 10:12, Wanpeng Li wrote: >>>> From: Wanpeng Li <wanpeng.li@hotmail.com> >>>> >>>> Reported by syzkaller: >>>> >>>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250 >>>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16 >>>> RIP: 0010:do_debug+0x222/0x250 >>>> Call Trace: >>>> <#DB> >>>> debug+0x3e/0x70 >>>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20 >>>> </#DB> >>>> _copy_from_user+0x5b/0x90 >>>> SyS_timer_create+0x33/0x80 >>>> entry_SYSCALL_64_fastpath+0x23/0x9a >>>> >>>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of >>>> timer_create(), it will also call perf_event_open() to set a BP for the buffer, >>>> so when the implementation of timer_create() in kernel tries to get the struct >>>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller >>>> testcase also sets the debug registers for the guest, however, the kvm just >>>> restores host debug registers when we have active breakpoints. I can observe >>>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print >>>> when running the testcase heavy multithreading. The do_debug() which is triggered >>>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)). >>>> >>>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq >>>> enable. >>>> >>>> Reported-by: Dmitry Vyukov <dvyukov@google.com> >>>> Cc: Paolo Bonzini <pbonzini@redhat.com> >>>> Cc: Radim Krčmář <rkrcmar@redhat.com> >>>> Cc: David Hildenbrand <david@redhat.com> >>>> Cc: Dmitry Vyukov <dvyukov@google.com> >>>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> >>>> --- >>>> arch/x86/kvm/x86.c | 2 ++ >>>> 1 file changed, 2 insertions(+) >>>> >>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >>>> index 0c5d55c..a6370fd 100644 >>>> --- a/arch/x86/kvm/x86.c >>>> +++ b/arch/x86/kvm/x86.c >>>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) >>>> */ >>>> if (hw_breakpoint_active()) >>>> hw_breakpoint_restore(); >>>> + else >>>> + set_debugreg(current->thread.debugreg6, 6); >>>> >>>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); >>>> >>>> >>> >>> If you haven't seen it, I analyzed this in >>> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to >>> my suggestion/question. >> >> I think it's fine to restore dr6 before preempt/irq enable. > > If no breakpoint is active, you should be able to restore it only in > sched_out. Do it in v2. :) Regards, Wanpeng Li
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0c5d55c..a6370fd 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) */ if (hw_breakpoint_active()) hw_breakpoint_restore(); + else + set_debugreg(current->thread.debugreg6, 6); vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());